LucidWorks Gains Funding for Unlocking Big Data Opportunities
May 16, 2013
With the flood of interest in big data solutions and technology that can chop the masses of unstructured content down to size, we are also seeing much VC funding go towards startups in this area. Linux Today reports that “LucidWorks Pulls in $10M to Turn Open Source Data into ‘Business Gold’”
LucidWorks started as Lucid Imagination in 2008 and focused on provided support, training and consulting for open source search technologies Lucene and Solr. Opportunity arose when the company saw a need for open source search to become more accessible by entering into the big data market.
Providing a quick rundown on LucidWorks current technology offerings, the article tells us:
“LucidWorks product suite contains two development platforms that enable organizations to search, discover, and analyze their data. LucidWorks Search is built on top of Apache Lucene/Solr open-source search project and seeks to simplify and improve the process of building embedded search applications. The other product, LucidWorks Big Data, then helps businesses make sense of the data.”
We have been following LucidWorks since before their name changed and their exciting news comes as no surprise. As one of the leaders in open source enterprise search technology they will undoubtedly remain one of the top on our list to follow.
Megan Feil, May 16, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Holy Grail of Search and Big Data?
May 14, 2013
A Business Wire press release caught our eyes recently as it announced the distribution of LucidWorks Search with the MapR Platform for Apache Hadoop. “MapR Technologies Distributes Enterprise-Grade Search with Hadoop Platform” shares that now customers will have predictive analytics, search, discovery and advanced database operations at their fingertips on a single platform.
Integrating LucidWorks technology with MapR beefs up the added value that LucidWorks Search offers as far as security, connectivity and user management. Additionally, MapR announced that the M7 Edition is available; this combines unprecedented Hadoop and NoSQL capabilities together in one platform.
According to Ben Woo, managing director, Neuralytix:
“Integrating search capabilities into Hadoop is an important milestone for the industry and represents tremendous opportunity for customers to find new insight and derive value from Big Data. This is an enormous step forward especially in time-sensitive processes such as fraud detection where Big Data must be searched as it streams into the enterprise.”
MapR’s chief application architect tells us that using search and big data is not just about analyzing social media content and Web traffic. We wonder…big data and search: has the holy grail (or one of them) been found?
Megan Feil, May 14, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
A Fresh Look at Big Data
May 8, 2013
Next week I am doing an invited talk in London. My subject is search and Big Data. I will be digging into this notion in this month’s Honk newsletter and adding some business intelligence related comments at an Information Today conference in New York later this month. (I have chopped the number of talks I am giving this year because at my age air travel and the number of 20 somethings at certain programs makes me jumpy.)
I want to highlight one point in my upcoming London talk; namely, the financial challenge which companies face when they embrace Big Data and then want to search the information in the system and search the Big Data system’s outputs.
Here are the simplified curves:
Notice that precision and recall has not improved significantly over the last 30 years. I anticipate that many search vendors will tell me that their systems deliver excellent precision and recall. I am not convinced. The data which I have reviewed show that over a period of 10 years most systems hit the 80 to 85 percent precision and recall level for content which is about a topic. Content collections composed of scientific, technical, and medical information where the terminology is reasonably constrained can do better. I have seen scores above 90 percent. However, for general collections, precision and recall has not been improving relative to the advances in other disciplines; for example, converting structured data outputs to fancy graphics.
Hadoop Heats Up with New Startups
May 8, 2013
While there is some controversy over whether Hadoop is the only necessary tool to mine opportunities from big data, Hadoop and insights from big data seem to be synonymous according to Datamation’s recent article. They give us the rundown on “Seven Hot Hadoop Startups that Will Tame Big Data.”
According to this article, the current Hadoop ecosytem market is worth around $77 million. With growth, the value is projected to be at $813 million by 2016. The article notes that Hadoop has not been proven as completely effective in the enterprise world. Queries are still a weak point.
The article discusses seven startups that intend on seeing Hadoop through into maturity like Alpine Data Labs. The following excerpt explains why they are on this list:
“According to Alpine Data, part of the problem is that it’s much too difficult to get real insights out of Hadoop and other parallel platforms. Most companies don’t know what to do with massive datasets, and few have gotten any further with Hadoop than batch processing and basic querying. Alpine Data set out to simplify machine-learning methods and make them available on petabyte-scale datasets. Their tools make these methods available in a lightweight web application with a code-free, drag-and-drop interface.”
With the amount of attention on Hadoop over the years, Hadoop start ups are not a commodity. A list featuring a selection of the new ones to watch is much appreciated. Check out the full and useful list of hot Hadoop start ups.
Megan Feil, May 08, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
CRN Releases Big Data 100
April 25, 2013
Big Data has been quite the buzzword in 2013. In response, multiple solutions have popped up that intend to solve enterprise Big Data problems. In order to identify the strongest contenders on the market, CRN has released a list of their top 100 picks for Big Data solutions. Read all the details of the report in the CRN article, “The 2013 Big Data 100.”
The article states:
“Market researcher Gartner recently issued a report declaring 2013 to be the year of large-scale adoption of big data technologies: A survey of IT executives found that 42 percent had already invested in big data technology or planned to do so within a year. Recognizing that need, we present the inaugural Big Data 100 list, developed by the CRN editorial team, identifying vendors that have demonstrated an ability to innovate in bringing to market products and technologies that help businesses manage big data.”
Among the awardees is LucidWorks, a leading contender in open source enterprise search since 2008. It is important to pick a solution that suits your company and your data needs. Because LucidWorks is built on the open source infrastructure of Apache Lucene and Solr, their solutions are scalable and easily customizable, though fully functional out-of-the-box.
Emily Rae Aldridge, April 25, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
LucidWorks Receives High Honor
April 24, 2013
LucidWorks, a company focused on providing open source enterprise search solutions for the enterprise, is used to receiving accolades. To add to their growing collection, LucidWorks has now received a placement on the CRN 2013 Big Data 100 list. Read all the details in the press release, “LucidWorks Named to Inaugural CRN Big Data 100.”
The statement begins:
“LucidWorks, the company transforming the way people access information, announced it was recently honored by being named to UBM Tech Channel’s CRN 2013 Big Data 100 list. The inaugural list recognizes innovative technology vendors that offer products and services to help businesses manage ‘Big Data’ – the rapidly increasing volume, speed and variety of information being generated today. The list covers three categories: business analytics, data management, infrastructure and services.”
LucidWorks Big Data is changing the way that enterprises deal with their Big Data needs. However, LucidWorks Search is also an option for organizations that have a lighter data load. No matter the size of your enterprise or the type of information needs present, LucidWorks makes a fully-supported solution to meet those needs.
Emily Rae Aldridge, April 24, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Hadoop in Demand Yet Lacks Trained Professionals
April 24, 2013
Hadoop has been in the headlines lately for its major changes and how it is being integrated into more organizations. PR NewsWire takes a look at the open source database platform and what it predicts will happen for the company in the future in, “Global Hadoop Market 2012-2016-Lack Of Trained Professionals To Be A Major Challenge.” The article examines a recent TechNavio report that analyzes the Global Hadoop Market 2012-2016. TechNavio predicts Hadoop will grow at a CAGR 55.63%, mainly due to rise in big data analytics and the company offering Hadoop-as-a-service. While technology and service wise Hadoop is doing well, it faces a deficit in trained professionals who can do the work.
TechNavio said:
“’The demand for cost-effective Hadoop-based big data solutions is driving this market. Organizations understand the importance of big data solutions, but installing and hiring new professionals to deploy them is a costly affair. As a result, organizations and decision makers are adopting Hadoop-as-a-service (HDaaS) solutions that provide cost-effective big data management and analytics. HDaaS solutions offer the necessary hardware, software, and services required to support big data management at low subscription fees.’”
What seemed to be a straight shoot, Hadoop is facing a problem that might limit its growth and development. HaaS does take care of part of the problem, but someone still has to work with the software. Will Hadoop innovation shift to where the proficient professionals are? We think it is a strong possibility.
Whitney Grace, April 24, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Talking about Big Data
April 23, 2013
The Council on Foreign Relations blog Asia Unbound features the piece “Big Data: An interview with Kenneth Cukier and Viktor Mayer-Schonberger” provides insight on how the two authors feel about big data, foreign policy and China. Cukier and Mayer-Schonberger are the authors of the book “Big Data: A revolution That Will Transform How We Live, Work, and Think.” They both had some interesting comments about the future of Big Data. Cukier believes that the Council on Foreign Relations should use Big Data because it is a valuable tool that in previous years we were unable to harvest but now we can do so easily and cost efficiently. Not using it would be a critical error when understanding and practicing foreign policy. In countries such as the U.S. some officials are beginning to understand its importance and are paving the way. Mayer-Schonberger comments about China and how many start-ups are beginning to understand the importance of Big Data and view it as a great opportunity. Cukier made an interesting comment concerning the U.S. and its position in the Big Data world.
“Yes, the U.S. is at the forefront of big data. But that’s just because it tends to be at the forefront of most technology trends. And this says less about America’s inherent strengths and more about its vibrant educational, business and financial culture that encourages people to flit between disciplines and sectors and start businesses. So for the moment, the U.S. has a very healthy lead. But it is not inherently sustainable. Big data companies are cropping up everywhere. Yet the most interesting place plowing into big data is China.”
With the U.S., China as well as other areas continuing to come up with new ideas, Mayer-Schonberger believes that new niches will be born which will ultimately create new opportunities for other potential players. Though various users may have different ways of utilizing Big Data, it seems that it has definitely spread and can’t be linked to just one particular group. Big Data is knocking down barriers.
April Holmes, April 23, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
The Unknowns and Big Data Integration
April 23, 2013
John Yapaola, CEO of Kapow Software gave an interesting presentation at the Kapow.wow user summit about the three main cases for Big Data. According to the Kapow Software blog article “Winning the Unknown Unknowns with Big Data Integration” the three cases are known knowns, known unknowns and unknown unknowns (UU’s). Yapaola uses post 911 as an example, to show that the United States realized that we had very little actual data about the enemy. The current known data intelligence that we had was not good enough because there were new sources of unknown information about unknown enemies and combatants that was constantly emerging. There is a similar battle in the business world. It involves the shift from “data at rest” to “data in motion.” Data at rest describes your business data, which is physically stored in databases and mainframes tucked behind the firewall. This can be thought of as the comfort zone of the business or the “known knowns” because in this area business decisions are generally made based on historical data.
“Data in motion is all the new types of digital data, constantly changing, external and unstructured. This is where businesses are crossing the chasm into the known unknowns.”
Companies know that their competitors are making changes but they don’t know exactly what changes are occurring or when. Yapaola goes on to point out that the last area the “unknowns unknowns” is the most complicated and require much more predictive intelligence and information in order to be able to identify and ultimately react. However, on the other hand unknown unknowns can be a good thing and bring about innovation. Yapaola sums it up by saying
“You have volatility built into your business – changing market conditions, new competitors entering the market, employees’ turnover, financial markets etc. The information you need is also constantly changing, in a variety of internal and external sources, multiple formats, structured and unstructured. At the end of the day, the combination of integrating the right data, the agility that you get it with and the ability to distribute it as Big Data insights directly to business users for decision making is going to determine your business trajectory. Value creation is at the heart of this Big Data movement.”
Wouldn’t it be great to be able to break down the unknowns and unknowns and predict which horse will win the Kentucky Derby or even more important how will China decide to handle North Korea. Yapaola finds an interesting way to remind us that with things constantly changing in the business world the importance of predictive tools and understanding Big Data in order to be prepared for these changes is more important than ever before.
April Holmes, April 23, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Five Noteworthy Big Data Analytics Apps
April 22, 2013
As the relationship between Big Data and analytics continues to grow so does the number of enterprise analytics applications. There are now a variety of applications designed to help enterprises get the most from their Big Data. The Enterprise Apps Today article “5 More Buzz-worthy Big Data Analytics Apps” gives an overview of the analytic tools Concurrent, Birst, SAS, Clearstory Data and Terracotta. Concurrent Lingual is a free open source project that was designed to work with Apache Hadoop. It uses Cascading application framework which allows SQL users to use their existing skills to run application on Hadoop without needing additional training. Birst Big Data is notable because experts say it helps to eliminate a vast majority of the upfront investment that is generally required to leverage Big Data. SAS Visual Analytics uses an in-memory engine to help speed up the analytics as well as the visualization process. Mark Torr, director of the SAS Global Center of Excellence stated
“SAS Visual Analytics enables creation and dissemination of dashboards, reports and the results of investigative exploration either through the Web or to native mobile applications running on an iPad or Android tablet. We believe that we are the only vendor that has analytics in-built rather than tagged-on through call outs to other services.”
“Terracotta In-Genius sits on top of Terracotta’s BigMemory 4.0 in-memory data platform. It comes with event stream processing and messaging, is said to process one million event transactions per second, identify patterns and create action items based on predicted behavior. It is also certified to operate with Oracle, SAP Hana and Hadoop.”
ClearStory Data gives users the ability to discover, analyze as well as consume data from data sources including Hadoop and Web and social application interfaces. For me these apps were new but regardless they definitely seem to bring a little something extra to the table when it comes to Big Data and analytics.
April Holmes, April 22, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
-
- Subscribe to Beyond Search
Feature archive
News archive
-
