Future Microsoft SharePoint ESP Features Revealed
July 30, 2009
Stephen Bell wrote “Microsoft Fast Search to Mate with Social Networking.” The title brought a grin to this addled goose’s bill. I like the idea of Fast ESP “mating” with a social network. The metaphor brings a number of images to mind and invites a wide range of double entrendre. Maybe the ComputerWorld editors were tickling the goose’s funny bone?
The story asserts that a fellow named Steve Letford, a Microsoft wizard in New Zealand, revealed at a SharePoint conference that Fast ESP will be “mated with SharePoint in a number of ways.” What? I thought the headline said Fast ESP, and now it is SharePoint.
Which is it? Fast ESP gets social? SharePoint gets social? SharePoint is a collaborative content management platform with search available upon installation. I guess I don’t understand what is happening.
Mr. Bell reports:
In ordinary office productivity work, the Fast search tool will give more meaningful results than an ordinary search engine on unstructured data, because of its ability to recognize the semantics of such elements as a date or an address, Letford says. Its pipeline architecture enables different inquiries to be made concurrently on a stream of documents as they pass down the pipe, meaning multifaceted queries can be executed more efficiently. Documents retrieved by the search will be able to be further interrogated interactively and the documents and their interrelationships presented in graphical format.
Now that would be something. I need “meaningful results”. I am not sure if an address is a “semantics” component but entity extraction or a tagged field can be interpreted under the fuzzy wuzzy banner of semantics. Precision is not part of the toolkit of a marketer.
The article references a now long-i-the-tooth IDC study about the cost of looking for information. In the present economic climate, my hunch is that the organizations with such high costs might be road kill, but that type of thinking does not slow the rush of the Microsoft argument Mr. Bell reports.
The killer comment for me was:
Customers of the enterprise search tool will have to buy a specific license and purchase a dedicated search server. Enterprise search is not something you install on your PC or laptop, he [Letford] says. A third common use for high-end search capabilities, Microsoft predicts, will be in monetizing website offerings by guiding users to content for which they will be willing to pay, and bringing up relevant advertising messages. Letford confirmed, in answer to a question from the audience, that Microsoft’s public search-engine, Bing, will evolve to use FAST-style technology.
What does this statement mean for Powerset, the Xerox Parc plumbing, and the glue code created to create the Bing.com systems? More evolutionary change? Sounds like a major overhaul to me. If this statement is accurate (keep in mind that it comes far from the search think tanks in Redmond and Oslo, Microsoft has big plans for Fast ESP. SharePoint, and Bing.com.
Let’s do a flashback to the pre-Fast ESP sell out and before the Norwegian investigators looked into the alleged fraud at Fast Search & Transfer.
I was in a presentation given by Bjorn Olstad, then Chief Technical Officer of Fast Search & Transfer and Adjunct Professor of the Norwegian University of Science & Technology. I snagged a copy after the presentation, and I have no idea if it is still online. I want to summarize some of the points in Dr. Olstad’s presentation from the days when Fast Search & Transfer was burning up the European software scene as the “Google of Scandinavia”. I will include a handful of screenshots. The quality is not so hot, but you will be able to get the drift of the argument.
Fast Search & Transfer ESP, Pre-Microsoft Buy Out
The idea Dr. Olstad was driving home was that a search system had to pick up the wheelbarrow for traditional database systems. Database queries are useful but search engines add significant functionality to the structured information.
These screenshots are the ones I downloaded after Dr. Olstad’s presentation. The PowerPoint did not carry a copyright message but I want to make clear that these are from Fast Search and not my recreation of the graphics used by Dr. Olstad. The opinions about the information on the screenshots are mine, however.
The future was the Fast Search system applied to the data for CareerBuilder.com. Instead of running a SQL query and getting a big set of job listings from the CareerBuilder.com database, Fast ESP made it possible to get “five targeted jobs with three clicks.”
This is quite compelling and it shows that Fast ESP was able to replicate the Endeca functionality, reduce the hassle of long results lists like those generated by the Google, and delivered on point results to the CareerBuilder.com user. If you go to CareerBuilder.com today you will see a number of functions, presumably built on the Fast ESP plumbing; for example, information input forms, alerts, a recommendation engine, geographic search suggestions, an ad for E Trade, and social functions such as help with a résumé and links to job fairs. CareerBuilder has “owner affiliates”. These include Microsoft, Gannett, McClatchy, and the Tribune Company.
Dr. Olstad showed an example of Fast ESP “discovering” information germane to a technical problem. The idea was that Fast ESP was not just about text search. The system could crack the code of structured data and perform “cleansing, mining, relevance, and discovery functions. As you can see from the Fast Search slide below, the system included Web logs in its content acquisition arsenal and can answer questions such as “What is the phone umber to Will’s Barber Shop?” Remember this is a pre 2008 Microsoft buy out of Fast and before the news of the shortfall in Fast Search revenues rocked the Norwegian business community.
The points Dr. Olstad made included the Fast Search system could “search and explore focused “infinite” data sets, structured and unstructured. Because the notion of focus which means narrow and sharp and “infinite” makes no sense to this addled goose, I will show you the slide that Dr. Olstad used to make this point, which sounds quite remarkable even today. Google is not even close to making this type of assertion. As I reported earlier this week, Google is just now coming to grips with locality in sparse data tables. Focused infinite data has not cropped up in that company’s technical information to my knowledge.
Dr. Olstad showed a summary table that made clear the speed advantage Fast ESP had over traditional database systems.
The message two or three years ago seemed to be that Fast ESP runs rings about the query performance of a relational database management system like Oracle, IBM DB2, or Microsoft SQL Server. (Maybe Microsoft will replace some SQL Server query functions with the higher performance Fast ESP system?)
The importance of the Fast ESP technology is that licensees can develop search derivative applications or SDAs. The idea is that this method is a “game changer driven by extreme retrieval and on-the-fly analytics”. The functions of the approach include “contextual insight” and “user interaction.” My thought is that “user interaction” like the inclusion of Web log content implies a social content orientation prior to Fast Search’s purchase by Microsoft. Fast ESP, if this is indeed accurate, was thinking social content and social search functions before Microsoft bought the company.
The cost analysis offered by Dr. Olstad still astound me even three years after hearing his lecture. His PowerPoint slide asserted that the Fast ESP hardware cost would be $90,000 in 2006 dollars. The RDBMS hardware cost would be $320,000 for 32 CPUs on four eight way Sun servers. The dollars don’t mean much today, but the difference does. Fast requires one fourth the hardware, delivering a cost that is three or four times less than the traditional approach. Amazing if true today. With SQL Server thumping away within most SharePoint and other heavy lifting Microsoft installations, the benefit to customers is obvious. Again. The caveat is the accuracy of the data behind this assertion.
The show case for Dr. Olstad’s lecture was the Scopus database which runs on Fast technology. Scopus is an Elsevier product. I learned shortly after Dr. Olstad’s lecture that Elsevier BV, the “owner” of Scopus, was taking a hard look at Lucene, but I don’t think much came of that investigation of open source search. One of the technologists working on that project left Elsevier to found an open source search company. But the Scopus executive is certainly clear about his satisfaction with the Fast ESP system. The statement on the screenshot below is: “David Goodman standing up and declaring in public, that Scopus is the best-designed database he’s ever seen …” Strong praise and I think other publishers would have stampeded Fast’s sales office to license software that would help resolve their cost and information challenges. The statement is so strong that Fast Search would not need sales professionals, just order takers.
Dr. Olstad then showed that the Fast ESP platform could snap into virtually any technical environment. Among the functions supported are a wide range of structured and unstructured information, some information is formal and other information is informal and social. The slide in Dr. Olstad’s presentation has an IBM copyright mark and I concluded that IBM and Fast Search were working together to make integration easy. The integration vehicle in this slide is IBM’s UIMA standard which is now open source.
Dr. Olstad reviewed the semantic functions of Fast ESP. The example used was the contextual navigation for ThisIsTravel.com. (When I checked the site on July 29, 2009, I experienced latency of about 20 seconds for my test queries. Your mileage may vary. Performance of Web sites is important to me because it shows the real world characteristics of the site’s technology.)
What jumped out at me were the reader reviews and comments that the system processed and integrated into the search results.
What Did This Tell Me?
I found the ComputerWorld article interesting, but it was presenting as new some of the features and functions that I thought I heard Dr. Olstad describe two or three years ago.
One point that puzzles me is the possible dissonance between the actual market uptake for Fast ESP prior to the Microsoft buy out of the company and the subsequent alleged financial issues associated with Fast. If Fast ESP had these state-of-the-art functions a couple of years, maybe three or four years ago, why aren’t these available in SharePoint now? I am hopeful that ComputerWorld will follow the thread of this story and provide Microsoft information that makes clear when my clients and others can take advantage of Fast ESP’s capabilities.
Stephen Arnold, July 30, 2009
Comments
3 Responses to “Future Microsoft SharePoint ESP Features Revealed”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
[…] Future Microsoft SharePoint ESP Features Revealed : Beyond Search Stephen Bell wrote “Microsoft Fast Search to Mate with Social Networking.” The title brought a grin to this addled goose’s bill. I like the idea of Fast ESP “mating” with a social network. The metaphor brings a number of images to mind and invites a wide range of double entrendre. Maybe the ComputerWorld editors were tickling the goose’s funny bone? […]
my funny bones are always active coz i always crack jokes everyday .
i have so many funny bones in myself that is why i would love to be a comedian .~.