Rich Media: Too Expensive to Store?
July 30, 2013
I saw an interesting post called “Cost of Storing All Human Audio visual Experiences.” I am no logician, but if one stores “all”, then isn’t the cost infinite? The person writing the post presents some data which pegs the cost for seven billion people at about $1 trillion a year.
Several observations:
- With the emergence of smart nanodevices with audio and video capabilities, perhaps the estimate is off the mark?
- Once the data are captured, who manages the content? Likely candidates include nation states, companies which operate as nation states, or venture funded start ups?
- How does one find a particular time segment germane to a query pertinent to a patent claim?
Interesting question if one sets aside the “all”. The next time I look for a video on YouTube or Vimeo, I will ask myself, “What type of search system is needed to deal with even larger volumes of rich media?”
Is the new Dark Ages of information access fast approaching? Yikes! Has the era already arrived?
Stephen E Arnold, July 30, 2013
Sponsored by Xenky
Dangerous Glitches Still to be Worked Out in Electronic Medical Records
July 30, 2013
Electronic medical records are part of the constantly evolving big-data landscape, and there have been some issues, according to Bloomberg‘s “Digital Health Records’ Risks Emerge as Deaths Blamed on Systems.” Not surprisingly, the period just after a new EMR system is implemented has been found to be the most dangerous time. See the article for examples of harm caused by errors. Journalist Jordan Robertson writes:
“Electronic health records are supposed to improve medical care by providing physicians quick and easy access to a patient’s history, prescriptions, lab results and other vital data. While the new computerized systems have decreased some kinds of errors, such as those caused by doctors’ illegible prescriptions, the shift away from paper has also created new problems, with sometimes dire consequences.”
Perhaps it would help if docs could access necessary data. Yet, even now, medical practices have trouble prying clinical data from their EMR systems. Apparently, a lack of data integration is the culprit, according to “Why do Docs Struggle with Population Health Data?” at Government HealthIT. That article summarizes:
“Today, with modern EHR systems, clinicians may have an easier time getting clinical data — but not all of it, which is a problem for providers pursuing population health goals. It’s also a problem as federal health officials and patient-safety organizations like the National Quality Forum try to transition from process-based quality measurements. . . to outcomes-based metrics.”
Are such digital-data challenges confined to health records alone? Unlikely, though in few (if any)other fields is big data consistently a life-and-death issue. We must remember that digital health records have also been shown to improve outcomes, but are we netting more good than bad? I suspect so, but we will probably never know for certain. One thing is sure: there’s no turning back now. Surely, mistakes will decline as systems are refined and staff acclimated. I know that is cold comfort to anyone who has suffered such a preventable loss.
Cynthia Murrell, July 30, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Snowden Controversy Will Mean More Big Data Transparency
July 29, 2013
With the recent news of the NSA’s phone records requests and deep use of data mining, we are on the cusp of a new frontier for big data and its relationship to the government. However, a little digging shows this fascinating time capsule from 2007, “Lost in the Cloud: Google in the US Government.”
According to this whitepaper:
“Consumer Watchdog investigation has found. One of the most visible signs of Google’s clout is the hangars at the National Aeronautics and Space Administration’s Moffett Air0ield, near Google’s world headquarters, where a 0leet of jets and helicopters stands ready to ferry the company’s top executives near or far, for business or pleasure. When a deal between NASA and top Google executives to use the base was 0irst disclosed in 2007, it called for only four jets to use the base.”
Not a stunner, really. The government has long been a supporter of big data and data mining technology that is sweeping the private sector these days. But, honestly, did anyone think Uncle Sam was sinking money into this field to help refine the census? We are not political one way or the other, but what we do love is transparency in big data and we suspect that’s what’s on the horizon after this affair.
Patrick Roland, July 29, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Selfless Analytic Partnerships Score Big
July 29, 2013
One would think the big data analytics field would be cutthroat and bloody with competition. Oddly, though, it is actually a mostly friendly affair. It seems like we hear more and more about disparate firms partnering up to give customers a better product. Such was the case with this recent Business Wire story, “JackBe Unveils Presto Real-Time Analytics Add On with Terracotta Big Memory.”
According to the story:
“JackBe®, the leading provider of Real-Time Actionable Intelligence software, today released its Presto Real-Time Analytics Add-On With Terracotta BigMemory (RTA Add-On), which bundles Terracotta’s enterprise-grade BigMemory in-memory data management platform for seamless high performance in-memory analytics. This supercharged combination of in-memory and analytics allows Presto to mash Big Data with live and transactional enterprise data into actionable dashboards in a fraction of the time required by traditional methods, putting real-time Big Data analytics at the fingertips of decision-makers.”
We like the way these two companies are playing to their strengths. When selfless analytics minds do that, the customers score big. It is reminiscent of the IBM big data partnership. In these cases we are seeing savvy companies partnering to do something extraordinary.
Patrick Roland, July 29, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
CIO Top 10 Big Data Startups
July 29, 2013
CIO.com has finalized their list of the top ten big data startups to watch. The results have been released. Read more in their article, “10 Top Big Data Startups to Watch–Final Rankings.”
The article states the criteria:
“After more than 4,000 votes were cast, the final Big Data startup rankings are in. Keep in mind that while voting was weighted heavily, it was not the be-all-and-end-all consideration. Other criteria included big-name end users, VC funding, the pedigree of the management team and market positioning.”
So after such thorough analysis, we were pleased to see a few familiar names on the list. Several good companies that are delivering quality products, LucidWorks among them:
“LucidWorks did well in the voting, has solid funding ($16 million) and is uniquely positioned in this roundup with their focus on Big Data search. However, they’ll need to get more on-the-record customers on the books to climb higher than this.”
LucidWorks Big Data will continue to do well among its existing customer base and expand into further markets. Their open source platform brings the strength of Apache Lucene Solr to an intuitive solution that is ready out-of-the-box.
Emily Rae Aldridge, July 29, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Hadoop Summit Makes Big Data Engaging
July 26, 2013
The Hadoop Summit was held at the end of June in San Jose, California. The event focuses primarily on end users and developers of things built on and around Apache Hadoop open source technology. However, as Hadoop is integrated into more and more software offerings, Hadoop’s reach continues to expand. SmartData Collective offers a good write-up of how Hadoop is making Big Data technology more dynamic. Read more in their article, “Hadoop Summit and Hortonworks Promise to Make Big Data More Engaging.”
The article describes the event:
“Hadoop Summit is the biggest event on the West Coast centered on Hadoop, the open source technology for large-scale data processing. The conference organizers, Hortonworks, estimated that more than 2,400 people attended, which if true would be double-digit growth from last year. Growth on the supplier side was even larger, which indicates the opportunity this market represents. Held in Silicon Valley, the event attracts enterprise customers, industry innovators, thought leaders and venture capitalists.”
The author then goes on to say that among the companies that are making big gains in information discovery with Hadoop are LucidWorks and Microsoft. LucidWorks uses Hadoop in its LucidWorks Big Data offering, which has received quite a bit of attention for its usability and effectiveness.
Emily Rae Aldridge, July 26, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
LucidWorks Makes Big Data Better
July 25, 2013
LucidWorks made its name as an open source enterprise search value-added vendor built upon Apache Lucene Solr. However, in the last 18 months, much attention has shifted to the Big Data sector and LucidWorks Big Data solution. Mark Smith with Ventana Research gives his review of the product in his article, “Big Data Search is Getting Better with LucidWorks.”
He writes:
“LucidWorks has two product offerings in the search market. LucidWorks Search provides the ability to rapidly set up search and index content using Apache Solr. . . Solr, built on top of Lucene, is an enterprise platform that provides full-text search, dynamic clustering, geospatial search and other enterprise-class capabilities. Second, LucidWorks Big Data, which was released late last year, uses the compute and storage capabilities of Hadoop to support larger-scale deployments.”
LucidWorks is known for its commercial-grade support and services. Their design has improved Solr’s usability for business use, although Solr on its own is known as one of the strongest infrastructures for open source enterprises. LucidWorks is currently expanding, adding staff, and gearing up for the next advance in enterprise technology. Keep your eyes open.
Emily Rae Aldridge, July 25, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Gartner Highlights Cool Big Data Firms
July 24, 2013
Is coolness enough to succeed in the big data analytics world? That’s a tougher question to answer than you’d think. If we’re talking about fly-by-night hipness, then probably not. However, Gartner has been taking the entire industry back to high school by deeming certain organizations cool. Shockingly, they might be on to something, as we discovered in a recent PR Web story, “Gartner Research Recognizes Centrifuge as a ‘Cool Vendor’ for Big Data.”
According to the story:
“Centrifuge Systems today announced it has been selected by Gartner as a “Cool Vendor” in the respected analyst firm’s annual report on Security Intelligence Vendors for 2013* that recognizes innovative and disruptive technology solutions in big-data analytics, particularly those that provide “contextual analysis of security intelligence across silos”.”
Interesting news on a promising company. Normally, we are pretty skeptical about such designations, but have become big fans of Gartner’s cool awards. It’s not bestowing a lot of hype on a company to blow our minds, it’s a more subtle way of saying someone is doing something good for the time being, check it out. That sounds like a recommendation we can seriously get behind.
Patrick Roland, July 24, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Powerful Partnerships Persevere in Big Data
July 23, 2013
Powerful partnerships in the analytic world are quickly becoming a daily happening. However, it’s rare that two major names in the industry link arms to build something for customers. However, that is just what happened recently, according to an InetSoft article, “InetSoft Adds SaaS BI for Quickbooks Users, Advanced Reporting and Analytics Through InetSoft’s Style Intelligence.”
According to the story:
“The Bison System is a specialized, hosted business intelligence tool with the best data connector available for millions of QuickBooks customers. The Bison System extracts customer QuickBooks data, securely transfers it to the Bison Servers, and then immediately processes it into the customer’s database. Within moments, the data is available for dashboards and customized reporting for QuickBooks business stakeholders.”
This blockbuster pairing of InetSoft and Quickbooks felt familiar. It reminded us of a recent Forbes article about Oracle’s similar pairings, which said: “three strategic partnership announcements this week, some of which have surprised even seasoned enterprise software industry observers and analysts.” That’s why companies like these two examples are on top. They are finding bold ways to partner and give clients analytic tools that are going well beyond standard big data.
Patrick Roland, July 23, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Little Big Data Analyzed to Glean DOMA Ruling Response on Twitter
July 22, 2013
The article titled What Twitter Had to Say About Doma and Prop 8 on Lexalytics attempts to break down the social media sites statistical response to the Supreme Court rulings handed down on Wednesday, June 26. According to the blog, about 180,000 tweets were posted on Twitter in response to the rulings, and of those tweets twice as many were positive (about 43%) as negative (about 19%). The article explains why this may be slightly misleading,
“As we can see more clearly here in our top themes, some of it is just the wording. Tweets including words such as “Good riddance” are scored by the software as negative, although they obviously aren’t feeling negative about the DOMA and Prop 8 rulings.
The themes “equal liberty”, “same-sex marriage”, and “marriage ruling” are swayed by more technical discussions of the Supreme Court’s actual rulings. “
Even stranger to those interested is the idea that only 180,000 tweets can count as Big Data. Isn’t Big Data supposed to be about terabytes? This seems like a junior sized portion. The article does take care to note that the sample can in no way be called representative of the general population, taking into account that only a small percentage of the general population even uses Twitter.
Chelsea Kerwin, July 22, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
-
- Subscribe to Beyond Search
Feature archive
News archive
-
