Apache Software Foundation Graduates New Program
April 15, 2013
The Apache Software Foundation is a strong presence in the world of open source software development, perhaps the strongest presence. Their system involves supporting fledging initiatives through the incubation process, and then graduating them into the great wide world. This is a process that has grown some of the most productive and useful software solutions that are currently tackling the issues of Big Data and unstructured data – namely Apache Lucene and Solr.
Read more about the Apache Software Foundation’s latest success and graduation ceremony in the article, “The Apache Software Foundation Announces Apache cTAKES(TM) as a Top-Level Project.”
The article begins:
“The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of nearly 150 Open Source projects and initiatives, today announced that Apache cTAKES has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the Project’s community and products have been well-governed under the ASF’s meritocratic process and principles.”
Previously mentioned, Lucene/Solr currently supports some of the most exciting initiatives in enterprise search, such as LucidWorks Search and LucidWorks Big Data. Knowing that these solutions are built upon open source Apache initiatives bring efficiency, security, and cost-effectiveness.
Emily Rae Aldridge, April 15, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
IBM Big Data Improving but Not Perfect
April 9, 2013
IBM has been gaining good ground in the last few years regarding its integration with open source technology. One reason for the move is its foray into Big Data. What we know of Big Data is that open source is really helpful in solving the problem of all sorts of unstructured data. IBM deserves some credit for their work. Read details of their progress in the article, “IBM Big Data Announcements Move in Right Direction but More Must be Done.”
The story begins:
“IBM has a tremendous portfolio of Big Data products and services, including both acquisitions and technologies developed in house like Watson, writes Jeff Kelly in his Alert “IBM Accelerates Its Big Data Portfolio”. And it has embraced the Open Source Big Data community, at least at the foundational level, particularly with its BigInsights Hadoop platform. But IBM has problems as well. That huge portfolio is fragmented and sometimes confusing, and IBM’s approach, which is heavily dependent on its services arm, is expensive, geared to large enterprises, and too costly for most mid-sized companies.”
For those small to mid-sized companies, an open source based solution might be preferred. For instance, LucidWorks is built on open source Lucene/Solr. It is not just added in; the open source infrastructure is the foundation. This brings not only scalability and flexibility, but affordability, which is essential for a smaller business.
Emily Rae Aldridge, April 9, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Ingersoll Says the Solution is Search
April 4, 2013
For companies tackling big problems related to large sets of data, Grant Ingersoll has the solution – search. At the recent GigaOm Structure: Data Conference, Ingersoll, CTO of LucidWorks, recommends that organizations take another look at search solutions. GigaOm covers the details in their story, “How Search Can Solve Big Data Problems.”
The article begins:
“There are many solutions for figuring out how to parse large amounts of data, but LucidWorks CTO Grant Ingersoll has a suggestion: use search. At GigaOM’s Structure:Data conference in New York City Thursday, Ingersoll laid out his case for why search is a big part of dealing with databases and indexes. ‘Search should be a critical part of your architecture,’ he told attendees. It is a system building block for any large problem you’re trying to solve that requires a ranked set of results. And it doesn’t have to be just text search, it can be for any type of search, he said.”
Ingersoll goes on to assert that search has changed dramatically quickly. For those organizations that have not updated their search solution in several years, there are more options on the market that are likely to serve their purposes more effectively. LucidWorks, Ingersoll’s company, is a longstanding name in the field, and yet has undergone dramatic changes even in the last few years. If your organization is exploring options for more effective search and Big Data management, LucidWorks is worth a serious look.
Emily Rae Aldridge, April 4, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Federal Paper Overload
April 4, 2013
With all of the things that are going on in the federal government world it seems that they can add paperwork overload to the list. The GCN article “How to Get on Top of the Federal Records Tidal Wave” sheds light on not only the increasing amount of paperwork that federal agencies are dealing with but also how they are exceeding their annual records management budgets by millions. The MeriTalk survey of federal records managers survey report “Federal Records Management: Navigating the Storm” found the following
“A single federal agency currently spends an average of $34.4 million per year on records management, and manages an average of 209 million records. That number is expected to increase as much as 144 percent — to 511 million records — by 2015.”
Survey participants felt that the inability for their agencies to properly manage their records posed as a hindrance to their agency operations. It was estimated that 18 percent of the annual budget is lost due to inefficient records management. The number of records is expected to grow from 8.4 billion to 20.4 billion so the lack of proper paperwork management could snowball if not gotten under control. 43 percent of survey respondents thought that records management personnel needed better training, 33 percent suggested more funding and 32 percent suggested more support from agency leadership as the best ways to improve managing records. The report recommended that records management should be a top priority and that agencies should invest in training. In addition “agency managers should adopt smart digitization methods and timely destruction of records.” The Federal Paper Work Act was supposed to be the answer to the paperwork overload but looks like big data has taken over. Only time will tell if this is really an improvement or just another shot in the dark.
April Holmes, April 04, 2012
Sponsored by ArnoldIT.com, developer of Augmentext
Update: Apology to Ventana Research
April 3, 2013
An update on Ventana. We have no reservations about recommending Ventana and its team for research projects.
The draft story “Big Claims of Analytics Progress” written by Cynthia Murrell was inadvertently published. The write up took the angle that Ventana’s research raised some questions. The wrongly published draft ran on March 29, 2013, was inadvertently posted by me. After doing some checking into this unfortunate matter, I learned that indeed I hit the incorrect button in the WordPress interface.
As a result, a draft story ran as a “ready for publication” story. I certainly do not and did not want to question the professionalism of Ventana and its consulting team.
The article in question was deleted in a routine check of posted write ups, but the links to the story are in various indexes. You may have seen a reference to the story at this link: http://news.silobreaker.com/big-claims-of-analytics-progress-5_2266711671936385024
When I checked a few moments (8 30 am Eastern) ago, the Silobreaker story was reported “not found.” There was another link to the story at http://www.i4u.com/2013/03/facebook/progress-analytics-big-claims and the story points to another 404 page.
My experience is that when a Beyond Search link goes dark, some indexes drop the link to the source.
So, I am sorry I hit the wrong button, sending a story to the publication queue and not to the “draft” queue for further revision. I apologized yesterday in this story http://arnoldit.com/wordpress/2013/04/02/ventana-benchmark-research-a-mistake-and-a-correction/ and I have been asked to make clear that I made the error myself.
To be crystal clear, I am sorry that I made the mistake.
If anyone reading Beyond Search wishes to comment or offer additional inputs, please, use the Comments section of the blog.
Stephen E Arnold, April 3, 2013
Sponsored by Augmentext.com
Ventana Benchmark Research: A Mistake and a Correction
April 2, 2013
Inadvertently by me or as a result of the same clever teens who hacked our Twitter account, a story about Ventana Research appeared which was not edited for actual release. To set the record straight, I want to quote from the Ventana news release and apologize for this unfortunate error. The item which caught our attention focused on research conducted by Ventana.
You will want to check out “Ventana Research Commences New Benchmark Research on Information Optimization.” Here’s the core of the announcement:
Ventana Research announced today that the firm is set to begin their latest Benchmark Research on Information Optimization. The firm has embarked on this benchmark research to evaluate how technology is used to support better access and utilization of information in business and IT. Information optimization is also a key driver and benefit of the use of big data technology. Information powers today’s businesses, and previous research conducted by Ventana Research indicates that organizations are aware of the challenges associated with accessing and assembling information so that it is useful and relevant. This new research on Information optimization, the latest from the leading business technology research firm, will analyze how organizations collect data, in what forms they collect it, how it is assemble and integrated and how it is best used. The research is designed to provide the building blocks for a foundation for optimizing the use and integration of information, enabling Ventana Research to provide guidance on efficiently and effectively using technology to deliver the right information to business and IT when and how it is needed. This research will also examine organizations’ people, process, information and technology competencies, maturity, trends and best practices in how they make information available to those who need it. The research aims to provide significant new insights into attitudes toward information optimization and the processes they enable. It will help provide information on best practices and practical methods to build a business case for investment. The research will also investigate the most critical information sources and how organizations provide access that critical information to decision makers in the form and cycles they need it.
For more information about Ventana, navigate to http://www.ventanaresearch.com/.
Sorry for the confusion caused by either an error made by me, one of my team, or the folks who seem to find Beyond Search worthy of spoofing. If you have a comment, feel free to use the Comments section of the blog.
Stephen E Arnold, April 2, 2013
Sponsored by Augmentext
Actian and Big Data Get Fast Answers
April 2, 2013
Actian is already a leader when it comes to big data management thanks to its analytics database Vectorwise Action Apps and it seems that they are focused on making a good thing even better. The Fort Mills Times article “Actian Announces Availability of Vectorwise 3.0 for Getting Fast Answers from Big Data” talks about Actian Corp and the introduction of its new Vectorwise 3.0 analytics database with advanced Hadoop integration. The Vectorwise 3.0 analytic database allows companies to integrate Hadoop using the Vectorwise Hadoop Connector and it helps companies to get important business information form their big data quickly and with ease. Fred Gallagher, general manger of Vectorwise stated
“We are talking about moving data at a rate of up to 3 Terabytes per hour onto a modest x86 server costing less than $15,000. That’s like downloading 20 movies in two minutes. Vectorwise 3.0 is now even faster, with a more efficient storage engine, supporting more data types and analytical SQL functions, and enhanced DDL features. It also has improved monitoring and profiling accessibility.”
Many companies rely highly on data to shape their business and Actian studied companies Hadoop usage to develop the innovative Vectorwise 3.0 analytics database. In the past Hadoop data stores were slow when it came to batch processing of analytics but the Vectorwise Hadoop Connector helps to alleviate this problem by delivering not only timely but also responsive multi-user querying and analytics processing. Industry users are already expressing their eagerness for the Hadoop Connector’s release in April 2013 and more importantly its positive impact on data analytics. As Hadoop integration and the benefits continue to make the news it looks like Hadoop’s got the data game down packed.
April Holmes, April 02, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Predictive Analytics Becoming an Important Governance Tool
March 31, 2013
Many of our cities need help right now. Those destined for default may need software to help with planning the future, and predictive analytics may be the answer. That is but one area where analytics could help our municipalities; GCN examines the relationship between such software and government agencies in “Analytics: Predicting the Future (and Past and Present).”
Though police are still a long way from the predictive power of 2002’s “Minority Report,” notes writer Rutrell Yasin, police departments in a number of places are using analytics software to forecast trouble. And the advantages are not limited to law-enforcement.
The article begins with a basic explanation of “predictive analytics,” but quickly moves on to some illustrations. Miami-Dade County, Florida, for example, uses products from IBM to manage water resources, traffic, and crime. One key advantage—interdepartmental collaboration. See the article for the details on that county’s use of this technology.
Though perhaps not the most popular of applications, predictive analysis is also now being used to enhance tax collection. So far, the IRS and the Australian Taxation Authority have embraced the tools, but certainly more tax agencies must be eyeing the possibilities. Any tax cheats out there—you have been warned.
Leave it to the CIA‘s head technology guy to capture the essence of the predictive analysis picture as we move into the future. Yasin writes:
“The real power of big data analytics will be unlocked when analytic tools are in the hands of everybody, not just among data scientists who will tell people how to use it, according to Gus Hunt, the CIA’s CTO, during a recent seminar on Big Data in Washington, D.C.
“‘We are going to have to get analytics and visualization [tools] that are so dead-simple easy to use, anybody can take advantage of them, anybody can use them,’ Hunt said.”
Are we there yet?
Cynthia Murrell, March 31, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Big Data and the New Mainframe Era
March 30, 2013
Short honk. Navigate to either your paper copy of the Wall Street Journal or the electronic version of “Demand Surges for Supercomputers.” The estimable Wall Street Journal asserts:
Sales of supercomputers priced at $500,000 and higher jumped 29% last year to $5.6 billion, research firm IDC estimated. That contrasted with demand for general-purpose servers, which fell 1.9% to $51.3 billion, the firm said.
Most folks assume that nifty cloud services are just the ticket for jobs requiring computational horsepower. Maybe not? For the cheerleaders for big data, the hyperbole to crunch bits every which way may usher in a new era of—hang on, gentle reader—the 21st century mainframe.
Amazon, Google, and Rackspace type number crunching solutions may not be enough for some applications. If big iron continues to sell along with big storage, I may dust off my old JCL reference book and brush up on DASDs.
Stephen E Arnold, March 30, 2013
IBM Goes Big On Big Data
March 25, 2013
IBM is probably the biggest name in big data when it comes to commercial, propriety vendors. The company is an established, household name and they continue to make great technological advances, the most notable being Watson the AI. When visiting the company’s Web site, IBM has gone above and beyond to set themselves apart from other Big Data companies. Take a look at the Smarter Analytics page they created. IBM is stressing the analytical aspect of big data and how their solutions cover software, research, hardware, and services:
“Big data is more than a matter of size; it is a way to uncover insights and opportunities from new and emerging internal and external sources of data and content. IBM’s big data capabilities include an enterprise-class big data platform, predictive and content analytics, and decision management to give your organization a competitive edge. IBM’s capabilities and signature solutions are designed to complement your existing information, analytics and content management infrastructure, so you can get started quickly and achieve game-changing results.”
Unlike other companies who spout what they have done, IBM provides video evidence documenting how big data has changed/helped companies. Several of those who benefited were T-Mobile, Vestas Wind Systems, NYSE Euronext, and Fiserv. IBM knows how to market itself as a viable big data solution. Unlike other companies it has the multi-generation appeal because of its longevity and new advances.
Whitney Grace, March 25, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
-
- Subscribe to Beyond Search
Feature archive
News archive
-
