Talend and Amazon Love
March 6, 2013
Talend has been collaborating with Amazon Web Services, we learn in Enhanced Online News’ post, “Talend Expands Big Data Integration Platform with Support for Amazon Redshift.” Business analysts, data scientists, and enterprise architects will soon be able to manage their Amazon Redshift data using three solutions from Talend, Talend Open Studio for Big Data, Talend Enterprise Big Data, and Talend Platform for Big Data. The write-up tells us:
“By using Talend’s connectors to Amazon Redshift, users will be able to load and extract data to and from Amazon Redshift, and also connect the Cloud-based data warehousing platform to the full spectrum of transactional, operational and analytic data sources. . . . Only Talend scales for today’s big data and cloud environments with more than 450 native connectors to relational databases, packaged applications, SaaS applications, files, legacy systems, Hadoop clusters, NoSQL databases, and more.
“In addition, Talend’s unique data quality capabilities, natively included in the Talend Platform for Big Data, eliminate inconsistent data, enforce rules and create consistent information through standardization.”
Talend says their Redshift connectors will be available within the month from Talend Exchange, the company’s community sharing platform, or directly from the Talend Studio. Future versions of the Talend Platform will have the connectors built in.
Talend was already a leader in open-source data management when its 2010 acquisition of Sopera boosted its standing in that market. It is a leading open-source vendor, providing middleware for both data management and application integration. The company takes pride in its powerful and flexible open solutions.
Cynthia Murrell, March 06, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Exclusive Interview: Mark Bennett of LucidWorks
March 5, 2013
Engineer Mark Bennett says it’s the tools that matter. Beyond Search agrees. Having tools and talking about tools are two very different things.
Mr. Bennett, co-founder of New Idea Engineering, recently brought more than twenty years’ enterprise search experience to LucidWorks, along with knowledge across major commercial search platforms, superior mathematics and physics-related disciplinary training, and a history in the search industry, including an early tenure at Verity, one of the pioneers in enterprise and large-scale information retrieval back in the 1990s.
Mark Bennett of LucidWorks, a member of their core enterprise search engineering team, recently granted an exclusive interview to the Arnold Information Technology Search Wizards Speak series to discuss the trajectory of search in 2013. LucidWorks is the leading developer of search, discovery, and analytics software based on Apache Lucene and Apache Solr technology. The full text of the interview is available at http://goo.gl/eoeuz.
He told Beyond Search:
“In a nutshell: search, analytics, and content processing vendors have to recognize that what is needed to allow developers to use the product is different from what is required to sell the product and deliver software which users embrace,” Bennett said about the immediate future of search products. “The challenge that keeps search specialists engaged is the problem of dealing with outliers—bizarre business requirements that every project seems to unearth. Outliers are the new norm.”
Bennett recalls a talk with a vendor ten years about a particularly tough search problem. Then, the vendor “ticked off a half dozen reasons why it was really very hard to solve and not worth the effort.” Years later, open source people visited the same problem, came up with a similar list, and diligently worked through those items. “LucidWorks, for instance, delivers facets, suggestions, advanced file storage, and high performance without the punishing costs of proprietary solution,” Bennett explained.
Stephen E. Arnold, Managing Director of Arnold Information Technology and publisher of the influential search industry blog Beyond Search, said:
“In my analysis of open source search, I rated LucidWorks as one of the leading vendors in enterprise search. Other firms with open source components have not yet achieved the technical critical mass of LucidWorks. Proprietary search vendors are integrating open source search technology into their systems in an effort to reduce their technology costs. At this time, LucidWorks is one of the leading vendors of enterprise and Web-centric search. Firms like Attivio (http://www.idc.com/getdoc.jsp?containerId=236514#.US9fGzBcgug) and ElasticSearch (http://www.idc.com/getdoc.jsp?containerId=237410) ElasticSearch are racing to catch up with LucidWorks’ robust technology, engineering and consulting services, and training programs.”
Bennett commented on the differences between LucidWorks and other retrieval solutions companies. “Despite all the comparisons done lately, the target audiences for most open source solutions are very different,” he explained. “If you spin up a copy of Solr you’ve got a very powerful Web user interface, and LucidWorks gives you even more of an administrative user interface. But when you fire up ElasticSearch, you’ve got a REST API.”
Bennett still often works from the Unix command prompt. “But when I watch a Windows or Mac power user for a day, and then watch a Unix command prompt guru—both get a lot of work done. My point is that each is a different type of power user. By the way, I work from the Unix command prompt myself.”
His point is that vendors need to be able to address the user interface preferences. “I do wonder what happens when an ElasticSearch developer hands off an application to a busy information technology person or an operations team to manage. Either those new owners are will need to know the ‘Web command line’ (URL and JSON syntax) extremely well, or if not, an administrative framework will be needed.”
LucidWorks is a step beyond more commercial proprietary search systems, in Bennett’s opinion, because it serves both groups of users. “Our professional services team has experience with many of other search engines. Chances are we’ve worked with many of the pieces before and know how to crack tough problems quickly. If an issue is a first time event, I am confident we can develop a solution.” He added:
“LucidWorks has delivered an open source enterprise search solution which accomplishes two things,” Arnold said. “First, it is an excellent alternative to many proprietary information retrieval systems. Second, the system takes the rough edges off some open source search solutions which add to an organization’s costs, not keeping them within budget allocations.”
Search is not a “one size fits all” solution, Bennett confirmed. “So while some engines drop features that ‘only three percent of people will ever use’, other groups realize that it’s the tools that matter.”
Visit the LucidWorks website at http://www.lucidworks.com.
Donald Anderson, March 5, 2013
Sponsored by Mediscripts, the world leader in prescription solutions for health professionals worldwide
LucidWorks Partners with MapR
March 4, 2013
MapR Technologies and LucidWorks have proven to be good partners in the past. They are again joining forces to offer the best Big Data analytics solution on the market. PR Newswire offers the press release for the joint venture in, “LucidWorks™ Teams with MapR™ Technologies to Offer Best-in-Class Big Data Analytics Solution.”
The release states:
“Existing business intelligence (BI) tools have simply not been designed to provide spontaneous search on multi-structured data in motion. Responding directly to this need, LucidWorks, the company transforming the way people access information, and MapR Technologies, the Hadoop technology leader, today announced the integration between LucidWorks Search™ and MapR. Available now, the combined solution allows organizations to easily search their MapR Distributed File System (DFS) in a natural way to discover actionable insights from information maintained in Hadoop.”
LucidWorks builds upon the strong search infrastructure of Solr. Adding this to the power of Hadoop through the MapR distribution makes it a solution that it without equal. The partnership makes it easier to put Big Data analytics into motion while combining the security strengths of both technologies.
Emily Rae Aldridge, March 4, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Choosing the Best Fit Enterprise Infrastructure
February 28, 2013
A writer from Search CIO laments that big data is not in fact rocket science in a recent article, “Data Mining Challenges on the Horizon: Got Big Data? Now What?” This writer attended a two-hour seminar which was sponsored by the Mass Technology Leadership Council and moderated by the global VP of Oracle’s Health Sciences business unit, Kris Joshi,
We learned from this article that infrastructure has been targeted as a high priority and there are several solutions:
Harvard, for example, has a new data center in Holyoke, Mass., that will be able to handle 40,000 calls. It’s the university’s last, by the way, according to Clamp, because when this baby no longer computes, the next cluster will be in the cloud. In the big data puzzle, as Broad’s Trunnell put it, infrastructure is “in many ways the easiest one to solve.” Heck, the U.S. Department of Homeland Security is using Oracle’s Exa-line products to help monitor a purported 50 billion transactions per day.
Big data may not be rocket science but many enterprise organizations have found that choosing the appropriate technology for their specific case can be a tough job. We recommend starting with the fundamental component to a big data solution and employing an enterprise infrastructure focused on data delivery.
Megan Feil, February 28, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search.
Scientists and Businesspeople Work Together for Big Data Research Solution
February 28, 2013
Ah, we’re pleased to see this real-world step with regard to big data. Science Daily informs us, “Solving Big-Data Bottleneck: Scientists Team with Business Innovators to Tackle Research Hurdles.” Researchers from Harvard Medical School, Harvard Business School, and London Business School have partnered to apply the benefits of a commercial crowdsourcing platform to a significant challenge—finding a data-analysis program that can handle the complexities of biological research analysis. The article reveals:
“Partnering with TopCoder, a crowdsourcing platform with a global community of 450,000 algorithm specialists and software developers, researchers identified a program that can analyze vast amounts of data, in this case from the genes and gene mutations that build antibodies and T cell receptors. Since the immune system takes a limited number of genes and recombines them to fight a seemingly infinite number of invaders, predicting these genetic configurations has proven a massive challenge, with few good solutions.
“The program identified through this crowdsourcing experiment succeeded with an unprecedented level of accuracy and remarkable speed.”
This is certainly a worthy big-data application, and confirmation that folks from different disciplines can effectively work together to accelerate progress. Before the project could really get going, though, the biologists had to translate their query into less-specialized language for the TopCoder community. After that, the viable suggestions came rolling in, and researchers picked their solution from an array of good choices. (Alas, the article does not disclose which software was selected.)
Researchers see more cross-discipline projects in the future; Harvard Business School’s Karim Lakhani notes that existing platforms and communities can provide a speedy alternative to the creation of custom data-analysis solutions. Yes, let’s hear it for cooperation!
Cynthia Murrell, February 28, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Data Delivery Solutions Connect Data Points from Different Apps
February 27, 2013
It is clear that the big data space is going through a huge transformation. As vendors and experts in the industry alike push to define burgeoning technologies into niche-specific solutions, there is bound to be confusion at times and complete clarity at other points. A recent Forbes article called “Big Data “A-Ha” Moment?” discusses this industry and adjacent ones in terms of both the struggles and space for opportunities.
It seems that many can agree that the big data industry has large sums of money written all over it. Estimates from Gartner to Wikibon range from $23 Billion to $3.8 Trillion. However, the closely related industry of business intelligence has been stuck at a 24% scorecard from Gartner.
According to the article:
Many reasons have been cited for this; from expensive software, to poor usability to complex integration issues. While each of the above certainly plays a role in the lack of adoption, my observations have been that, the culture of elitism generated in the world of data has done nothing but scare the most common of companies. And this trend has been amplified recently. Look, I too admire Nate Silver, Facebook and Google for the great work they do with Data.
One of the hallmarks of the niche of big data technologies that fall under the category of data delivery solutions is their intuitive user experience and the integrative abilities to connect multiple pieces of data from different applications under one roof.
Megan Feil, February 27, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search.
Elasticsearch Secures More Funding
February 27, 2013
Elasticsearch is an enterprise search option that wants a slice of the Big Data pie. It has been making headlines for its seeming fast growth. The latest news is relayed by Enterprise Apps Today in their story, “Open Source Elasticsearch Firm Raises $24M.”
The article explains Elasticsearch’s latest announcement:
“Amsterdam-based Big Data search startup Elasticsearch announced it just closed on a Series B round of funding valued at $24 million. Index Ventures headed the round, which was backed by Benchmark Capital, an existing investor, and newcomer SV Angel. The funds, which follow a $10 million capital injection in November, will be used to help transform the company from a software provider to a full-fledged services and support organization. Elasticsearch also revealed that its open source real-time search and analytics software just crossed the 2 million download mark.”
The seeming explosive growth mentioned above may be cause for alarm. There is no way to know if the growth is sustainable, especially since the only committer on the project is the founder. The two million downloads number is also cause for suspicion. Elasticsearch requires and registers a new complete download for every upgrade release. So the count is not new registrations or even the total number of users. Elasticsearch may be making some headlines, but many users will choose to stay with a stable and trustworthy solution like LucidWorks.
Emily Rae Aldridge, February 27, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
New Terminology Does Not Change Power of Data Delivery Solutions
February 26, 2013
Many venture capitalists were chomping at the bit to fund big data start ups not too long ago. However, according to a recent article from Venture Beat it is time to move on. This venture capitalist chasing news source tells us, ‘Big Data’ Is Dead. What’s Next?
The article goes on to discuss who killed it. Everyone from media sources to industry leaders to marketing experts all the way down to the vendors themselves had a hand in the death of big data, according to this piece. Instead of the big data deluge, this article warns against a big data headache. The only cure: a big data Advil.
Writer John De Goes states:
As the industry matures, there won’t be a single term that replaces the big data moniker. Instead, different tools and technologies will carve out different niches, each more narrowly focused and highly specialized than the universal sledgehammer that was big data.I’m going to talk about some of the niches you’re going to hear about again and again. Alas, some of these will be spun into buzzwords that, like big data, accumulate so much “momentum” they eventually lose meaning.
Life after big data will involve smart data, data science, NewSQL and predictive analytics according to this article. If one thing can be certain it is that there will be new terminology to apply to new vendors in the big data game as time goes on. However, big data as a term will always be able to quickly and simply capture the essence of solutions from data delivery vendors to enterprise infrastructure solutions.
Megan Feil, February 26, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search.
LucidWorks Launches New Online Service
February 26, 2013
In the growing Big Data market, education and training is at a premium. Developers would do well to learn all they can about this quickly emerging technology. In light of that need, LucidWorks has created a new online destination, Big Data Central. Virtual Strategy hosts the press release from PR Wire, “LucidWorks Launches Big Data Central.”
After detailing the growing need of Big Data, the author continues:
“To address this need, LucidWorks, the company transforming the way people access information, is pleased to announce the launch of Big Data Central. Big Data Central is an online destination that provides success stories, educational materials, and other curated content to help executives understand current Big Data trends and shape their data strategies. The go-to site offers valuable information about Big Data collected from a wide variety of reputable sources.”
Big Data Central continues the trend of investment that LucidWorks makes in the industry, and in particular open source search. This is not the first web portal they have established in the name of education and training. Check out SearchHub.org for another open resource devoted to Lucene and Solr developers. LucidWorks’ value-added software is a good solution for many applications and is built on trust, experience, and a strong open source foundation.
Emily Rae Aldridge, February 26, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Businesses Require Scalability Seen in PolySpot Solutions for Big Data Needs
February 22, 2013
Whether companies are in the exploratory phase or have already implemented their initiatives, they are talking about big data to some extent. CIO points out an interesting dilemma some companies may be facing in their recent article, “Big Data is a Solution Looking for a Problem.” With all the hype, some companies may have forgotten to ask what problem they wanted to solve by using big data.
Big Data is forecasted to drive $34 billion of IT spending in 2013 and create 4.4 million IT jobs by 2015. In light of those numbers, there must be some major problems that companies want to address.
One of the biggest variables companies are looking at is how much data they need to hold onto for storage and analysis:
A lot of other industries are simply looking for ways to manage and monetise their data assets – and Big Data is not always the answer, according to Logan. There are plenty of off-the-shelf software programs that can visualise large data sets, and in some cases the best solution may be to simply throw some of the data away.
Finding a big data solution with the capacity for scaling will help increase the company’s ROI. An organization may not have petabytes of data to undergo analytics today but they might need that capability tomorrow. Solutions like PolySpot have scaling capabilities that growing businesses require as they are increasingly driven to find more opportunities and insights from their data on hand.
Megan Feil, February 22, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search.
-
- Subscribe to Beyond Search
Feature archive
News archive
-
