Perfect Search Corporation

An Interview with Ken Ebert

On a flight across country, I found myself in Salt Lake City. Instead of wasting time watching flights queue in the rain, I visited Perfect Search, located in Orem, Utah, about 30 minutes from the Salt Lake airport. Ken Ebert, one of Perfect Search's senior technologists allowed me to explore the system. At lunch in a local eatery, I spoke with Mr. Ebert about Perfect Search. I was particularly curious about the speed with which the system indexed content. The company has integrated semantic technology into its system. Along with key word queries, support for structured data, and other must haves, Perfect Search has developed technology that delivers remarkable performance. Queries on large datasets rendered without latency. The full text of my interview with Ken Ebert appears below. You can get more information about the company on its Web site here.

What was the spark that ignited your work on Perfect Search?

It was a dual spark. One came from search veteran Ron Millett, our company’s Chief Scientist. Back in the early 80s, Ron wrote QuickFinder, which was the world’s first search engine inside a word processor. Later, Ron and his team installed QuickFinder into Groupwise, Novell’s email application. A few years ago, Ron Millett was considering how Moore’s law had applied to hardware, seeing the doubling of processing power every 18 months, but that there had not been any significant speed improvement on the software side of search retrieval technology. As he pondered this, he had a breakthrough about how to approach the query process from a different angle.

During this development period, Ron began collaboration with Dillon Inouye, a professor at Brigham Young University, who had a PhD from Stanford University. Dillon, a gifted visionary, had previous search experience through co-founding Folio Corporation, which was later sold to FAST. While Dillon was contemplating Ron’s ideas, he spent some time assisting his parents who have a farm in central Utah. Dillon later reported that while pondering the issues of search, he had a flood of ideas come to his mind that related to harvesting processes on the farm. The process of using a combine to harvest grain was among the ideas he captured. He wrote and developed a number of ideas based on this experience. After combining Dillon’s visionary perspective and Ron’s unique programming abilities, Ron developed a prototype around these ideas and they knew they had discovered a game-changing and disruptive technology in the field of search.

What is the source of funding?

We have been fortunate to raise $3 million through friends, family, and some smart investors.

Do you plan on additional rounds of financing?

Yes. However, our first desire is to create revenues through product sales and to develop strategic partnerships.

How did you decide to create a search system that makes performance its key differentiator?

The key performance differentiators are our ability to handle massive data sets and our speed. We have a current customer that has over 1 billion records of both structured and unstructured content that we have been able to search on a single low end server. This single server replaced 7 servers that they were previously using for query processing.

We have also made improvements in the indexing process. For this customer, we were able to index their content in 1/100th of the time that it took them with their former solution.

Finally, because our query speed is so fast, we have so much more throughput per server. This significantly reduces the amount of hardware required to handle large query volume. The extra search horsepower could also provide additional search functionality, such as semantic search.

What's the secret sauce for your content processing throughput?

Our secret sauce is a mixture of experience with pipelining and basic search techniques gained over 30 years in implementing and innovating in the search field, mixed with a new approach that utilizes selective application of tuples, elision, and hierarchical accelerators in a patent-protected process.

When did you start the company?

The company was incorporated in January, 2007. Development of the technology started several years before the company was organized.

What platforms do you support?

Currently we have shipping code on Microsoft Windows XP and Microsoft Windows 2003. We also have a version that is ready to deploy on Microsoft Mobile portable devices. We scale from on-device mobile search to enterprise servers to internet-sized server farms, and from 32-bit to 64-bit solutions. Our initial focus has been on Microsoft's platforms, but our roadmap includes much broader support. A port to Linux is currently under development. Other platforms will be supported as opportunities present themselves.

What markets are you focusing on? Can you share a case example? If yes, describe a customer success? How are customers responding to your advanced text processing system?

The first market we are focusing on are those customers who have a Google Search Appliance (GSA). Many of these customers have used their GSA to index their documents, html files, pdf documents, but have avoided using their GSA to index and search their large databases because of document licensing limits. This is because Google counts each record in a database as a document. For example, if a company wanted to use the GSA to index and query its database that had 300 million records, it would cost them over $10 million dollars.

With Perfect Search’s ability to handle large data sets such as in the example above, we can provide a very economical single-server solution. We are finalizing a product that will be an add-on product to the Google Search Appliance, which would enhance the GSA search and extend the ability to search large databases. So the customer still gets the ease of use and functionality of the Google Search Appliance plus the capacity of our add-on product.

As one example, we have a customer that has over 10,000 databases of both structured and unstructured content and over 1.2 billion records. We were able to provide them with an economical solution that allows them to query all of their records on a single server. They are thrilled with the results and it has allowed them to greatly expand their content acquisition strategy.

Some search vendors are rushing to focus on specific niches such as eDiscovery and contact centers. What do you plan for 2009?

In 2009, we are focusing on those customers that have very large data sets and are unsatisfied with their current solutions. As mentioned before, our very first market niche will be GSA customers who need to index and query large databases.

Later in 2009, we will launch a GSA add-on appliance that will enhance email search. This combination of the GSA with our appliance will allow those wanting to do eDiscovery a very competitive solution.

What are three recent technical innovations you have made to Perfect Search that keep your product fresh and relevant for your customers?

Since we are new, all of our innovations are fresh to our customers. We think we approach search from a completely different angle than anyone else out there. By using this unique approach in our indexing and search algorithms, we can achieve excellent results with very high performance. We believe we will bring better performance at a lower cost to our customers.

Microsoft has been inventing and buying search systems. What do you think Microsoft will do with their flock of search systems, including Live.com search, default Vista search, two flavors of SharePoint search, Fast Search, Powerset, and SQL Server search? Will Microsoft's actions have an impact on Perfect Search? If yes, what do you see?

We have watched with interest Microsoft’s efforts to enhance their search capabilities and market competitiveness. You have to include Microsoft’s recent acquisition of Ciao.com, the European e-commerce search engine for $489 million. I think the integration of all these systems is a significant challenge.

We have wondered how Microsoft will be able to scale Powerset, so they can take advantage of the semantic functionality in their other offerings. We actually think that our technology would give Microsoft the computational horsepower they need to be able to offer scalable semantic search.

Microsoft bought Fast Search to be able to establish a larger market share in the enterprise search market and to be able to offer a higher end solution to their customers that needed a more complex search product than SharePoint offers. However, I haven’t seen Microsoft respond to Google’s Search Appliance. According to a certain addled goose, Google may have captured the market lead for revenues in the Enterprise Search market with the GSA. We think our GSA add-on product is well positioned to piggy back on Google’s success in this market and we don’t see Microsoft competing in this arena in the near future.

A number of high profile search vendors are counting on Intel's new multi-core processors to solve their system's sluggish indexing and query processing. How will these next generation CPUs impact Perfect Search? Can you give me a ballpark example of the impact in terms of megabytes processed per second or number of queries supported per minute?

Our current indexing on a single core is getting reasonably good performance without reliance on multiple cores. For example, by using the Perfect Search indexer, our first customer was able to improve indexing by a factor of 100x over the speed of their previous indexer. We are able to index at about 2.5MB/sec. on a single core. By adding two cores and four disks, the rate improves to about 6.5MB/sec. on a 32-bit system.

Our search systems have produced search results of over 25,000 queries per second for exact/near-exact search on structured data on a single core in a fully cached system. Adding additional cores makes it even faster, but we already have much more horsepower than most customers need. They are more interested in a non-cached, disk-based, single-core system with 50 queries per second with hundreds of millions of records. On the whole, more cores are nice, but we perform quite nicely with just one.

As you look at Perfect Search's technical road map, can you characterize the types of enhancements we can expect in the next release?

With a solid indexing and searching in place for Windows systems, our road map includes integration efforts with partners and applications, as well as support for new hardware and OS platforms.

What are the three major trends in enterprise search that you see gaining momentum in 2009?

Steve, you opened our eyes to the most significant trend of 2009 in your keynote speech at the Enterprise Search Summit West. This trend is Google’s continued emergence as the dominant leader in the Enterprise Search market. Besides your comments and analysis, it is surprising to us how little public discussion there is of this fact at the conferences and shows we have attended. I think Google will continue to succeed in 2009 because of the following factors:

• Google brand. We were really struck by one fact discussed at the recent Enterprise Search Summit where a study was conducted among some users. One search result page had the Google brand and the other page didn’t. Both search results were identical. However, users said that the Google branded results were more relevant by a 4-to-1 margin.
• Simplicity. Many buyers of the GSA are attracted to the “plug-n-search” aspect of the technology. You can be up and searching within days of receiving the appliance, compared to the months and months it takes to install a complex Enterprise Search application.
• Economics. With the downturn in the economy, we think that there will be a slowdown in major enterprise software purchases and a push toward cheaper solutions like the Google Search Appliance.

How will you take advantage of these?

Rather than compete against the 800 lb gorilla, we are looking to augment and enhance the Google Search Appliance’s capacity and hopefully catch a ride on their wave.

Mobile search, clustering, concept tagging, and other content processing features are becoming standard features on some systems. Will you be including these features in search?

The focus on Mobile Search by most of the industry has been to launch an internet query from your mobile device and then transmit the query results back to the mobile device.

We can do this as well, but we have also released a Microsoft Windows Mobile device search that we think is unique in the marketplace. Because of our software scalability, we have put our search engine on the mobile device. We index and query all of the content stored locally on the mobile device, including the content of the memory card.

Because of our tremendous horsepower, we provide fast indexing of the device’s content. We also keep new content indexed with an innovative incremental indexing approach. In addition, we provide a sub-second query response. Both indexing and query processing are so fast, that very little computational demand is placed on the device, avoiding power-drain on the limited battery capacity.

Currently no major mobile manufacturer or OS provider offers an indexed search product. At best, they offer a clunky scan search that really slows down when your device is loaded with content. Apple’s iPhone doesn’t offer any sort of search. We currently have our Perfect Search Mobile Device Search available for licensing for the Microsoft Windows Mobile platform. We are in the process of porting it to the iPhone, Symbian and Google Android platforms.

What are the three solutions that you want those looking for a search system to associate with Perfect Search?

We have a distinct competitive advantage to those customers that need to:

• Index and query massive data sets
• Index fast
• Handle significant query volume

The current approach for these customers is to throw significant hardware at their indexing and search problems. However, this is very expensive and doesn’t always scale well. Our Perfect Search technology allows us to be highly competitive in providing a solution to each of these problems.

ArnoldIT Analysis

Perfect Search may be a system that provides a solution to the performance woes that users of IBM DB2, Oracle, and Microsoft SQL Server face. Perfect Search delivers excellent content processing and query processing speed without the hardware imposing the often onerous server requirements needed to make certain data management systems usable. Furthermore, the company's engineers have combined support for structured and unstructured data into one agile system. If you are have large volumes of data to process and must deliver low latency response, you may want to take a close look at the Perfect Search system.

Stephen Arnold, November 25, 2008