Natural Language Processing Not Suited for Web Searches
August 9, 2011
There’s a new cowboy in town and he’s shaking up the search engine industry. The article, Real Language Q&A: The Next Generation of Search?, on Search Engine Journal, explores the practicality of Oren Etzioni’s recommendations for search engines in his new paper, titled, Search Needs a Shake Up, published in Nature.
According to Etzioni, current search engines have not kept up with the time. The reliance they have on old algorithms with results displayed as a list that can run into the millions is no longer practical. As the article explains,
“In Etzioni’s view, the next generation of search would abandon the “blue link” structure in favor of directly answering the questions of users. “Moving up the information food chain requires a search engine that can interpret a user’s question, extract facts from all the information on the web, and select an appropriate answer,” he states. The tricky part, though, is in finding the answer. With so many ambiguities, it’s difficult to see how most questions could be answered by a search site.”
Conveniently, Etzioni offers his own University of Washington’s Reverb program as a step in the right direction. Reverb relies on Natural Language Processing (NLP) which is an interesting direction for search engines, but depends entirely on the reliability of the user’s question.
In a world of Etzioni’s search engine, the functionally illiterate would never receive an accurate search result because the search engine would never recognize, “who be prez bamas baby mama?”
While it would be a lovely world to live in if life was as well-spoken as Jeopardy and Watson could answer our every question quickly and precisely, that is not the case and never will be. NLP works well with voice searches and should stay there. Though Etzioni poses some interesting questions and points out the while elephant in the search engine room, the answer is not as simple as NLP. At least not yet.
Catherine Lamsfuss, August 9, 2011
Sponsored by Pandia.com, publishers of The New Landscape of Enterprise Search
Oracle Snaps Up InQuira
July 29, 2011
We learned from a source in Silicon Valley that Oracle has acquired InQuira. We noted “Oracle Buys InQuira to Boost Fusion CRM”. InQuira is an interesting search company. The firm was formed in 2002 from two semi successful search companies, Answerfriend Inc. and Electric Knowledge Inc. The company hit its stride with its positioning of “natural language search” for customer support applications. InQuira hit my radar screen when it signed a deal with Yahoo to power the Yahoo customer support service. I wrote about the upside and downside of the Yahoo implementation and then looked at InQuira every few months. You can run a query in Beyond Search and get a list of the articles I wrote to track the company’s activities since 2008. InQuira has been able to move forward despite the lemmings of search rushing into the customer service market.
According to the IDG News write up:
The company has patented NLP (natural language processing) capabilities that enable it to determine the “true intent” of a customer question, according to its website. “We expect InQuira to be the centerpiece for Oracle Fusion CRM Service,” said Anthony Lye, senior vice president of Oracle CRM, in a statement.
Our view at Beyond Search is that buying InQuira is probably a reasonable move for Oracle. The company’s Secure Enterprise Search 11g is not suited for the Fusion type of application. Oracle purchased Triple Hop, but I have lost track of that firm’s Match Point innovation within the giantness of Oracle.
Will InQuira propel Oracle forward in enterprise search in its various manifestations? My hunch is that Oracle will generate additional revenue and put pressure on the incumbents in the customer support market. Oracle may need to acquire additional search and content processing companies in order to meet the needs of the big and diverse Oracle customer base. InQuira’s approach often requires significant computational horsepower. Oracle is positioned to sell InQuira’s customers the hardware required to deliver zippy performance.
We think the notion of a giant company building a “knowledge management” solution is sort of interesting. Big companies have to buy other companies to move forward. That’s why we think Oracle may still be shopping for search and content management solutions.
Stephen E Arnold, July 29, 2011
Freebie unlike products from Oracle and InQuira.
Linguamatics Revealed
July 25, 2011
David Milward, CTO of Linguamatics sat down with The Inquirer for an in-depth look at the10 year old British company’s founder. Dr. Milward insists that it’s not hart to explain what Linguamatics is all about. The write up reported Dr. Milward as saying:
“Its software extracts knowledge from unstructured text. What’s difficult is to explain why it’s different. Isn’t that what a search engine does?”
Linguamatics is individual in that traditional searches are not very ‘agile,’ you have to program specifically what you want. With his system, you can ask any question and get relevant returns.
Milward and partner Roger Hale have taken text mining to another level with the development of the Linguamatics company. Dr. Milward said:
“Organizations are becoming more and more knowledge-driven,” he says. “Similarly to scientific discovery, they build new things based on existing knowledge.”
Automation is important in the fast paced world of enterprise. Pharmaceutical companies are just one of the knowledge driven arenas that have adopted Milwards approach to business intelligence. He demonstrated the advancements of his technology in the last election when he mined Twitter reactions. We learned:
“We found that although people don’t use fully grammatical sentences, they do use grammatical constructions.” The relatively few linguistic patterns enabled them to identify what was being said.
Linguistic structure varies with the various operations and field’s humans are involved in, as do the words we use. Dr. Milward added:
“We found that although people don’t use fully grammatical sentences, they do use grammatical constructions.” The relatively few linguistic patterns enabled them to identify what was being said.
Milward said his system can see the relationship between them all. For example his system can take the words: carcinoma, tumor and neoplasm and equate it with “cancer.” He said:
“The result is the ability to ask a question like, “What genes are associated with breast cancer?” and get back a list of genes rather than a list of documents.”
That’s pretty cool, for a system that doesn’t have a human’s rationality or ability to grow and think. Linguamatics maintains that it’s not trying to replace the human element within the process. They are simply trying to aid in the development so that a job can be done more effectively and in a shorter amount of time.
What this means to the business world is that you will be able to find companies and concepts that are linked in documents without having to pour over the results for hours on end. It will save time and in turn, will save money. Another key pint was:
“There are 20 million relevant articles in the biological domain,” says Milward. “And if you’re going into social media, for example, there are one billion tweets a week. It’s huge amounts of information and what we’re trying to do typically is pull out bits of information from that.”
While in theory Linguamatics has the ability to be a useful tool that can be utilized for the greater good, there are some barriers that it will have to overcome first. The challenge of accessibility is a big one. They have yet to find a graphical interface that can create queries that all computers understand. Let’s face it, even in this age of technology, not everyone is a programmer and knows ‘techspeak.’ All in all, it’s a promising technology and something to keep an eye on. The start-up is only ten years old and has plenty of room to grow this into something big.
Stephen E Arnold, July 25, 2011
Sponsored by Pandia.com, publishers of The New Landscape of Enterprise Search.
Questioning the Voice of the Customer
July 12, 2011
With search vendors embracing customer service and customer support, here’s an interesting insight into the niche: Steve McKee at Bloomberg Businessweek declares that “The Customer Isn’t Always Right.” Yeah, customers—who needs ‘em!
Actually, McKee does acknowledge that businesses should heed consumer voices much of the time. However, he insists that the reality of competing interests limits the value of that information:
As Adam Smith pointed out: ‘It is not from the benevolence of the butcher, the brewer or the baker that we expect our dinner, but from their regard to their own self interest.’ The same is true of the people who purchase the meat, the beer, and the bread. If you ask customers to design the perfect product, they’ll rack up the features and ratchet down the price, then be thrilled to buy from you all the way through your ‘Going Out of Business’ sale.
Businesses must balance customer input with knowledge about their own needs. The sting of losing a few sales because your prices are too high is small compared to the importance of protecting your bottom line.
What’s becoming increasingly clear, search vendors who offer customer support solutions are helping companies which want to reduce their customer support costs. Helping the customer? Maybe that is a secondary or tertiary benefit?
Cynthia Murrell, July 12, 2011
Laurent Couillard, CEO, Dassault Exalead: Exclusive Interview
June 28, 2011
Exalead caught my attention many years ago. Exalead’s Cloudview approach allowed licensees to tap into Exalead’s traditional Web and enterprise functions via on premises installations, a cloud implementation, or a hybrid approach. Today, a number of companies are working to offer these options. Exalead’s approach is stable and provides a licensee with platform flexibility as well as mobile search, mash ups, and inclusion of Exalead technology into existing enterprise applications. For organizations fed up with seven figure licensing fees for content processing systems that “never seem to arrive”, Exalead has provided a fresh approach.
Exalead provides high-performance search and semantic processing to organizations worldwide. Exalead specializes in taking a company’s data “from virtually any source, in any format” and transforming it into a search-enabled application. The firm’s technology, Exalead CloudView, represents the implementation of next-generation computing technology available for on-premises installation and from hosted or cloud services. Petascale content volume and mobile support are two CloudView capabilities. Exalead’s architecture makes integration and customization almost friction-free. The reason for the firm’s surge in the last two years has been its push into the enterprise with its search-based applications.
The idea of an enterprise application built upon a framework that can seamlessly integrate structured and unstructured data is one of the most important innovations in enterprise search. Only Google, Microsoft, and Exalead can boast commercial books about their search and content processing technology.
In 2010, Exalead’s market success triggered action on the part of one of the world’s leading engineering firms, Dassault Systèmes. Instead of licensing Exalead’s technology, the firm acquired Exalead and aggressively expanded the firm’s research, development, and marketing activities. Exalead’s approach enables more than 300 organizations to break the chains of the “key word search box” and has provided Dassault with a competitive advantage in next-generation information processing. In addition to mobile and rich media processing, Exalead is working to present integrated displays of real time information that add value to a wide range of business functions. These include traditional engineering to finding a restaurant on an iPhone.
Laurent Couillard, Chief Dassault Exalead
With the purchase of Exalead, Dassault appointed Laurent Couillard as Exalead’s chief executive officer. Mr. Couillard joined Dassault Systèmes as an application engineer in 1996, most recently serving as Vice-President Sales and Distribution for Europe, the Middle East and Africa. In that post, he played a central role in the sales transformation of 3DS, establishing a powerful reseller channel for all PLM brands and contracting with more than 140 companies. As CEO of EXALEAD, his mission is to accelerate the market penetration of applications based on search technologies. Mr. Couillard holds an M.S. from Institut Supérieur de l’Aéronautique et de l’Espace, a preeminent institution in Toulouse, France.
I asked him what was capturing his attention. He told me:
We are devoting more energy to developing packaged business applications or SBAs built on this foundation. That’s a mission right up my alley. And I intend to apply all my experience in sales and partner network development to this mission as well. That’s my charge from Dassault: To use my dual technology/sales background to develop Exalead and to penetrate new markets with SBAs, while preserving all the qualities that make Exalead so unique in this market. I’m fortunate to be in a position to leverage the full knowledge, resources, geographical coverage and expertise of the Dassault group to make this happen.
I probed for the reasons behind Dassault’s purchase of Exalead in 2010, a move which caught many analysts by surprise. He said:
Dassault saw first-hand how search-based applications based on Exalead’s systems and methods solved some of its clients’ long-standing, mission-critical business challenges quickly, painlessly and inexpensively. Dassault’s management understood–based on technical, financial, and performance facts—that Exalead’s search-based applications were a prime reason why search was, and is forecast to remain, an exceptional performer in the information technology software market. Because Dassault was seeking to diversify its content processing offerings, search in general and search based applications technology in particular were obviously an appealing choice. Dassault is, therefore, developing SBAs as one of its three core activities.
We discussed the challenges facing most of the traditional key word search and content processing systems. He noted:
You have to remember Exalead’s always understood search is sometimes something you do, and other times something you consume. In other words, sometimes it’s a search text box, and sometimes it’s the silent enabler beneath a business application, or even an entire information ecosystem.
You can read the full text of my interview with Mr. Couillard in the ArnoldIT.com Search Wizards Speak collection. The interview is located at this link.
Stephen E Arnold, June 28, 2011
Freebie from the leading vertical file service for search and content processing.
InQuira 2010 Growth
March 21, 2011
Marketwire brings to our attention that “InQuira Shows Exceptional 2010 Growth, Shatters All Prior Sales Records.” InQuira is one of the long time players in natural language processing. The company made the decision to focus on customer support and self-help applications years ago. Today the company offers knowledge applications for multi-channel customer service, sales enablement, and social CRM. For 2010, the company says that its has shown exceptional customer and revenue growth. Its partnerships include SAP and Oracle. According the write up:
“Investments in established technologies like CRM and new social channels add transactional capabilities, but create new challenges for organizations to drive a consistent knowledge experience across multiple customer touch points. Our growth is being fueled by leading brands that realize the critical need to consistently knowledge-enable their service and sales processes to deliver a world-class customer experience. Only InQuira uniquely ensures that these technologies result in worthwhile investments and deliver the business value intended.”
Taking the critical approach, has InQuira, a privately held company, really shattered “all” prior sales records? The financial data isn’t disclosed, so where is the proof? The story is still interesting. Maybe Natural Language Processing is gaining momentum but we would like some hard financial data, not assurances.
Whitney Grace, March 21, 2011
Freebie
Meaning in Casual Wording
March 3, 2011
I love science. Paired with my increasing passion for language and grammar, a sweeter cocktail could hardly be imagined. “Do Casual Words Betray Warlike Intent?” was a fascinating read.
At the recent American Association for the Advancement of Science (AAAS) meeting, James Pennebaker, a University of Texas at Austin psychologist spoke about the study he and assorted colleagues along with the Department of Homeland Security have been engaged in recently. The focus of the research has been on four similar Islamic groups and the relationship between the speech they employ and the actions that follow. The collective hope is the study’s findings can be used to forecast aggressive activity.
Isolating pronouns, determiners, adjectives and prepositions, the group mines them for what Pennebaker calls “linguistic shifts”. To date they have determined that of the four, the two groups who have committed acts of violence, telegraphed said destructiveness with the use of “more personal pronouns, and words with social meaning or which convey positive or negative emotions.” Aside from differentiating between various stylistic elements of expression, Pennebaker has also scrutinized statements made by warmongers from our past, including George W. Bush, with interesting results.
Skepticism has always fueled scientific endeavors, and we must continue to ask questions, especially those that breed discomfort. This science deals with a very grey area and Pennebaker himself labels the results as only “modest probabilistic predictions”. There is no question that this information must be used responsibly, but my aforementioned appreciation for the field keeps me from seeing this as a negative.
If one can discern an opponent’s intent in a fight or a game of cards by careful observation, why is it so strange to think the same could be done from listening to what they say?
Sarah Rogers, March 3, 2011
Freebie
NLP Gets a Full Monty
February 28, 2011
Natural Language Processing (NLP) is experiencing huge growth. From handwriting recognition to foreign language translation to predictive text on your handheld, NLP is used in many different ways to help our technology recognize what we mean when we simply speak or write English (or whatever language you happen to use in life). Natural Language Processing with Python is a book available in pdf that gives a useful introduction to NLP based on the Python programming language with its shallow learning curve.
According to its own introduction:
“This book provides a highly accessible introduction to the field of NLP. It can be used for individual study or as the textbook for a course on natural language processing or computational linguistics, or as a supplement to courses in artificial intelligence, text mining, or corpus linguistics.”
The book is geared toward beginning and intermediate levels, so even if you are new, don’t be intimidated. It is full of exercises, and the authors have used entertaining examples to lighten what might otherwise be a heavy subject. The book is available for free download and the Natural Language Toolkit with open source Python modules is as well. Whether your background is arts and humanities or science and engineering, this is a recommended read.
Alice Wasielewski, February 28, 2011
What Is a Fresh Start, Alex?
February 26, 2011
With the mounting anticipation of the man versus machine episode of Jeopardy! set to air on February 14, 15, and 16, 2011, it is hard to ignore the buzz over Watson. If you’ve been locked in a closet for the last month, Watson is IBM’s supercomputing experiment in AI. Recent articles in The Pittsburgh Post-Gazette and USA Today can bring you up to speed if necessary. In previous weeks we covered Watson’s win in the game’s practice round. But trivialities aside, what does Watson actually mean for IBM?
Well, Watson won.
The head gander in Harrod’s Creek maintains that IBM is pulling a PR stunt considering the company’s long history of work in the search field without ever impacting the market in a significant way. Omnifind 9.1 for Lucene has not been met with much fanfare, at least not here at Beyond Search, largely in part to the convoluted web of features (or corresponding fixes) and lack of support.
Yes, it is all happening on a game show, so the possibility of rigging exists. But how would introducing a fraud over national airwaves benefit IBM or what appears to be a quest to remind the public of its ingenuity? Watson performs so well due to refined natural language processing (NLP) and QA technology, two facets of search that are likely to be important players in the future. So if all goes according to plan, rather than typing a query into a search engine and waiting for a list of results out of which you must dig your own answer, the quandary will automatically be resolved. That is the claim of Watson’s power, it accurately plucks answers out of information stores and the range of applications is huge. This could be the next step in search and IBM could once again be a great innovator in the foreground. Even though IBM processors are in nearly every gadget on the market, it’s been a while since IBM has had any real recognition. That is why it does not surprise me they choose to roll-out their new tech on a prime-time television show, making advanced technology more palatable and memorable to the average consumer.
Perhaps I am being naïve or am too huge a fan of the science fiction novel, but I can’t help but be in Watson’s corner. Hey, Arthur C. Clarke got satellite communication right; maybe HAL 9000 is on its way.
Now Watson is headed to health care. Stay tuned.
Sarah Rogers, February 26, 2011
Freebie
Invention Machine Embraces NLP
February 24, 2011
Natural Language Processing (NLP) is not a new science, but it has yet to be perfected. A wealth of corporate and common knowledge lies in unstructured text documents, but most NLP search retrieves piles of seemingly disconnected documents. Users need precise relevant results, but NLP has yet to get there. Goldfire claims to know How Natural Language Processing Can Solve the Knowledge Retrieval Problem. He notes:
“Goldfire’s Natural Language query interface enables the user to put a question in a free text format, which would be the same format as if the question were given to another person. And, once relevant knowledge has been retrieved, Goldfire presents the results in a way that makes their meaning readily apparent. “
Claiming to have found a way for computer-aided knowledge extraction to overcome the natural language obstacles of semantics, syntax, and context, Goldfire marries high-level concept extraction and problem-solving capabilities. Despite such improvements, the problem of how we structure and retrieve information is unlikely to be solved anytime soon.
Emily Rae Aldridge, February 24, 2011
- Subscribe to Beyond Search
Feature archive
News archive
-
