Big Data: Can the Latest Trend Deliver?

April 25, 2014

If you track Big Data, you will want to read “Why Big Data Is Stillborn (for Now).” The write up hits the highlights of the flickering hyperbole machine that sells fancy math to the government and organizations desperate for a Silver Bullet.

The article asserts:

Most “big data” has to be moved in physical containers. Most data centers do not have excess capacity to handle petabyte level simultaneous search and pattern discovery.

Believe in real time and high speed access? Consider this statement:

Bandwidth, throughput, and how “real time” is defined all come down to the weak link in the chain and we have many ***very weak*** links across the chain and especially in Washington, D.C. The bottom line is always “who benefits?” The FCC decision to destroy net neutrality is in error. The citizen, not the corporation, is “root” in a Smart Nation.

If you wonder why your Big Data investments have yet to deliver a golden goose pumping out 24 caret eggs everyday, check out this write up. Worth reading.

Stephen E Arnold, April 25, 2014

Small Analytics Firms Reaping the Benefit of Investment Cycle

April 23, 2014

Small time analytics isn’t really as startup-y as people may think anymore. These companies are in high demand and are pulling in some serious cash. We discovered just how much and how serious from a recent Cambridge Science Park article, “Cambridge Text Analytics Linguamatics Hits $10m in Sales.”

According to the story:

Linguamatics’ sales showed strong growth and exceeded ten million dollars in 2013, it was announced today – outperforming the company’s targeted growth and expected sales figures.  The increased sales came from a boost in new customers and increased software licenses to existing customers in the pharmaceutical and healthcare sectors. This included 130 per cent growth in healthcare sales plus increased sales in professional services.

This earning potential has clearly grabbed the attention of investors. This, is feeding a cycle of growth, which is why the Linguamaticses of the world can rake in impressive numbers. Just the other day, for example, Tech Circle reported on a microscopic Mumbai big data company that landed $3m in investments. They say it takes money to make money and right now, the world of big data analytics has that cycle down pat. It won’t last forever, but it’s fun to watch as it does.

Patrick Roland, April 23, 2014

Sponsored by ArnoldIT.com, developer of Augmentext

Small Analytics Firms Reaping the Benefit of Investment Cycle

April 22, 2014

Small time analytics isn’t really as startup-y as people may think anymore. These companies are in high demand and are pulling in some serious cash. We discovered just how much and how serious from a recent Cambridge Science Park article, “Cambridge Text Analytics Linguamatics Hits $10m in Sales.”

According to the story:

Linguamatics’ sales showed strong growth and exceeded ten million dollars in 2013, it was announced today – outperforming the company’s targeted growth and expected sales figures.  The increased sales came from a boost in new customers and increased software licenses to existing customers in the pharmaceutical and healthcare sectors. This included 130 per cent growth in healthcare sales plus increased sales in professional services.

This earning potential has clearly grabbed the attention of investors. This, is feeding a cycle of growth, which is why the Linguamaticses of the world can rake in impressive numbers. Just the other day, for example, Tech Circle reported on a microscopic Mumbai big data company that landed $3m in investments. They say it takes money to make money and right now, the world of big data analytics has that cycle down pat. It won’t last forever, but it’s fun to watch as it does.

Patrick Roland, April 22, 2014

Sponsored by ArnoldIT.com, developer of Augmentext

Imagine the Internet without Search Engines

April 17, 2014

Centrifuge Systems proposes an interesting idea in “Big Data Discovery Without Link Analysis Is Like The Web Without Google.” Centrifuge Systems asks readers of the short article to imagine using the Internet without a search engine. How would we locate information? It would be similar to the librarian’s favorite description of the Internet all the contents of a library spilled on the floor. The article continues to explain that big data without link analysis works the same as the Internet without a search engine.

What is link analysis?

“You can view link analysis as a data discovery technique that reveals the structure and content of information by representing it as a set of interconnected objects. When combined with a visual representation, an investigator can quickly gain an understanding of the strength of relationships and the frequency of contacts and immediately discover new associations. Link analysis offers an intuitive alternative to the traditional relational database formats and BI tools without deep technical expertise.”

It is a convincing analogy. To increase a potential client’s interest, Centrifuge Systems offers a Data Discovery Challenge, where the client is given a free solution. In other terms, it’s a free estimate for services. Big data is full of analytics, but has anyone other than Centrifuge Systems offer rich link analysis?

Whitney Grace, April 17, 2014
Sponsored by ArnoldIT.com, developer of Augmentext

Google Glass and Predictive Analytics

April 13, 2014

I read “I Was Assaulted for Wearing Google Glass.” This is a sad commentary on our times. Will I be assaulted in Harrod’s Creek for driving my Kia Soul and wearing a T shirt that says “ Seavey’s Dog Kennel”? After I read the item, I wondered, “Why can’t Google’s predictive analytics be used to display to a Glass wearer that the assault risk level?” A color coded scheme could be used based on previous Glass users’ encounters, GPS data, and other inputs available to the Google / Recorded Future systems.

I noted this passage:

Why were people laughing at my misfortune or implying I somehow deserved it?

Beats me.

Stephen E Arnold, April 13, 2014

The Enigma App

April 1, 2014

Information can be an enigma, which is probably why the developers named their new app that. Visiting the Enigma Web site opens on a picture of either New York or London with the headline “navigate the world of public data.” It is an intriguing idea that one would think could be accomplished with search engine or academic database. Then again when you think about the process and how time consuming it is, it would be handy to have a search engine that did most of the work for you.

Enigma was built as a solution to this problem. The company says they have:

“Enigma is amassing the largest collection of public data produced by governments, universities, companies, and organizations. Concentrating all of this data provides new insights into economies, companies, places and individuals.”

Enigma’s services do come with a fee, however. They offer public data search and quick analytics for free with sign-up, but if you want API access and online support you need to upgrade to plans that start at $195/month. The data search must be gold, when you consider that many of these records are available for the public. It is worth exploring to see how the service differs from a basic search engine, but it is hard to sign up. The registration page is finicky.

Whitney Grace, April 01, 2014
Sponsored by ArnoldIT.com, developer of Augmentext

Darpa Prods Big Data Experts

March 29, 2014

I read “Darpa Calls for Advanced Big Data Ideas.” If the write up is accurate, Darpa is not on board with the marketing innovations about Big Data, whatever the term means. Darpa wants more. According to the TechRadar story:

According to V3, DARPA director Arati Prabhakar told a briefing on emerging threats with the House Armed Services Committee’s Subcommittee on Intelligence that it is looking to come up with some advanced big data ideas. She said that DARPA is creating a new set of cyber security capabilities that will ensure that networked information is trustworthy.

Address “big data” may be easier if those talking about it would define the term and the context in which the phrase is being used. Those who chant “Big Data,” including Darpa, are just empowering the sales people, the self appointed experts, and the failed middle school teachers who write “reports” for mid tier consulting firms.

Stephen E Arnold, March 29, 2014

Addiction Model Measures App’s Addictiveness Over Retention

March 28, 2014

The article on re/code.net titled Mixpanel: How Addictive Is Your App? presents a new analytic report called Addiction. Under a picture of a wrist cuffed to the smartphone it holds, the article cheerfully explains that fifty percent of social app users engage with the service for over five hours a day. Enterprise apps are used more during the business day, and messaging apps show a lesser addiction in their users, supporting the idea that people are now using social media apps for most of their communications. The article explains,

“Addiction adds an extra layer of insight that allows companies to analyze user behavior on an even deeper level. One thing that’s clear is that addiction is inextricably linked to function: If your product is a social app that people don’t use more than once a day, that’s a red flag — and not one you would have previously been able to catch if you relied solely on Retention.”

The article stipulates that the most important feature of Addiction is that it enables companies to visualize how “embedded” their service is in user’s daily schedules. This will allow them to better follow the effect of their smallest adjustments in the app and really see how their customers react. Whether or not this is a dangerous ability is not considered.

Chelsea Kerwin, March 28, 2014

Sponsored by ArnoldIT.com, developer of Augmentext

The HP View of Watson

March 19, 2014

I suppose IBM will respond with more than recipes at South by Southwest. If you enjoy big companies’ analyses of one another, you will want to gobble up “15 Reasons HP Autonomy IDOL OnDemand Beats IBM Watson.” This is not the recipe for making pals with a $100 billion outfit.

What does IBM Watson have as weaknesses? What does the reinvented (sort of) Autonomy technology have as strengths? I cannot reproduce the 15 items, but I can highlight five of the weaknesses and enjoin you to crack open the slideshow that chops up the IBM Watson PR stunt.

Here are the six weaknesses I found interesting:

  1. Reason 3. IBM Watson is a data scientist heavy platform. IDOL is not. My view is that HP paid $11 billion for Autonomy and now has to deal with the write down, legal actions related to the deal, and tossing out Mike Lynch’s revenue producing formula. Set aside the data scientists and the flip side “too few data scientists” and consider the financial mountain HP has to climb. A data scientist or two might help.
  2. Reason 4. HP has “an ultimate partner story.” I find this fascinating. Autonomy grew via acquisitions and an indirect sales model. Now HP wants to make the partner model generate enough revenue to pay off the Autonomy purchase price, grow HP’s top line faster than traditional lines of business collapse, and make partners really happy. This may be a big job. See IBM weakness 9, 11, 12, and 14. There is some overlap which suggests HP is having difficulty cooking up 15 credible weaknesses of Watson. (I can name some, by the way.)
  3. Reason 6. HP offers a “proven power platform for analytics.” I am not sure about the alliteration nor am I confident in my understanding of analytics and search. IBM Watson doesn’t have much to offer in either of these departments. IDOL, at least the pre HP incarnation, had reasonably robust security capabilities. I wonder how these will be migrated to the HP multi cloud environment. IBM Watson is doing recipes, so it too has its hands full.
  4. Reason 10. HP asserts that it offers a “potential app store.” I understand app store. Apple offers one that works well. Google is in the app store business. Amazon has poked its nose into the marketplace as well. I don’t think either HP or IBM have credible app stores for variants of the two companies’ search technologies. Oh, well, it sounds good. “Potential” is a deal breaker for me.
  5. Reason 13. HP “is focused on ramping up the innovation lifecycle.” I think this means coming up with good ideas faster. I am not sure if a service can spark a client’s innovation. Doesn’t lifecycle include death? Since IBM Watson seems a work in progress, I am not sure HP’s just released reinvention of Autonomy has a significant advantage because it too is “ramping up.”
  6. Reason 15. HP has “fired up” engineers. Okay, maybe. IBM has engineers, but I am not sure if they are fired up. My question is, “Is being fired up” a good thing. I want engineers to deliver solutions that work, are not “ramping up,” and not marketing driven.

My take on this slide deck is that it is nothing more than a marketing vehicle. I had to click multiple ads for HP products and services to view the 15 reasons. Imagine my disappointment that five of the IBM weaknesses related to partnering programs. Wow, that must be really helpful to a licensee of cloud Autonomy trying to deal with performance issues on an HP data center. HP is definitely countering IBM Watson’s recipe play with old fashioned cheerleading. Rah, rah.

Stephen E Arnold, March 19, 2014

Civic Predictive Analysis Proving Accurate

March 19, 2014

We find the field of predictive analysis fascinating (see here, here, and here, for example), and now we have more evidence of how important this work can be. Motherboard reports on “The Math that Predicted the Revolutions Sweeping the Globe Right Now.” The key component: high food prices. Writer Brian Merchant explains:

“There’s at least one common thread between the disparate nations, cultures, and people in conflict, one element that has demonstrably proven to make these uprisings more likely: high global food prices.

Just over a year ago, complex systems theorists at the New England Complex Systems Institute warned us that if food prices continued to climb, so too would the likelihood that there would be riots across the globe. Sure enough, we’re seeing them now. The paper’s author, Yaneer Bar-Yam, charted the rise in the FAO food price index—a measure the UN uses to map the cost of food over time—and found that whenever it rose above 210, riots broke out worldwide. It happened in 2008 after the economic collapse, and again in 2011, when a Tunisian street vendor who could no longer feed his family set himself on fire in protest.”

Bar-Yam’s model forewarned about the Arab Spring and the Tunisian self-immolation. Well, not those specific ways unrest would manifest, but that something big and ugly was bound to happen. Similarly, the same model divined that there would be conflicts around the world this year—as we have seen in the Ukraine, Venezuela, Brazil, Thailand, Bosnia, Syria, Spain, France, Sweden…. Last year’s global food prices were the third-highest on record; this is no coincidence. See the article for more on Bar-Yam’s methods as well as specific links between food scarcity and some of the conflicts currently shaking the world.

What can this technology do, besides hand a few of us a big bucket of “I-told-you-so”? Armed with this information, policymakers could take steps to modify the way the global marketplace is run and stop (at least some, possibly most) food shortages before they start. This means powerful people from many countries would have to work together to make major changes on a global scale for the good of humanity. With money involved. Hey, anything’s possible, right?

Cynthia Murrell, March 19, 2014

Sponsored by ArnoldIT.com, developer of Augmentext

« Previous PageNext Page »