Deep Web Tech’s Abe Lederman Interviewed
June 9, 2008
Abe Lederman, one of the founders of Verity, created Deep Web Technologies to provide “one-stop access to multiple research resources.” By 1999, Deep Web Technologies offered a system that performed “federated search.” Mr. Lederman defines “federated search” as a system that “allows users to search multiple information sources in parallel.” He added in his interview with ArnoldIT.com:
Results are retrieved, aggregated, ranked and deduped. This doesn’t seem too difficult, but trust me it’s much harder than one might think. Deep Web started out building federated search solutions for the Federal government. We run some highly visible public sites such as Science.gov, WorldWideScience.org and Scitopia.org. We have expanded our market in the last few years and sell to corporate libraries as well as academic libraries.
believes that Google’s “forms” technology to index the content of dynamic Web sites is flawed.
Mr. Lederman said:
Deep Web goes out and in real-time sends out search requests to information sources. Each such request is equivalent to a user going to the search form of an information source and filling the form out. Google is attempting to do something different. Using automated tools Google is filling out forms that when executed will retrieve search results which can then be downloaded and indexed by Google. This effort has a number of flaws, including automated tools that fill out forms with search terms and retrieve results will only work on a small subset of forms. Google will not be able to download every document in a database as it is only going to be issuing random or semi-random queries.
In the exclusive interview, Mr. Lederman reveals a new feature. He calls is “smart clustering.” Search results within a cluster are displayed in rank order.
You can read the full text of the interview on the ArnoldIT.com Web site in its Search Wizards Speak series. The interview with Mr. Lederman is the 17th interview with individuals who have had an impact on search and content processing. Search Wizards Speak provides an oral history in transcript form of the origin, functions, and positioning of commercial search and text processing systems.
The interview with Mr. Lederman is here. The index of previous interviews is here.
Stephen Arnold, June 9, 2008
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
