ISYS Search Software

An Interview with Ian Davies

Sydney, Australia, is a vibrant city, throbbing with energy and innovation. ISYS Search Software is one of a handful of companies in the search and content processing market capturing customers in the US, Europe, and Pacific Rim. At the recent International Policing: Toward 2020 Conference, ISYS had a high profile due to its strong presence in law enforcement. I wanted to see the company's operation, so I journeyed to the trendy Crow's Nest area of Sydney, a 10-minute taxi ride from the famous Sydney Opera House and Sydney's financial center. In a family-run Chinese restaurant, I put Mr. Davies into my wok and peppered him with questions. Between dim sum delicacies like mango pancakes, he answered my questions in March 2008. My questions are in bold.

What was the genesis of ISYS?

More than 20 years ago, I became aware of “text retrieval” as it was called then, and noticed that all the available products fell into two camps. There were those that had come to PCs from mainframes and were so user-hostile that in practice they were sometimes installed but rarely used. The others had grown up on PCs, and had ample user-friendliness but lacked the rigorous internals that would let them scale properly up to usefully large data volumes. I saw there was an opportunity for a product which had properly sophisticated internals and could handle large volumes, but which was also approachable and easy to use.

We described our early product as an “iceberg”, in that it had a friendly and approachable visage, but the really important bits were never seen. The bit below the water-line was crucial to make the whole thing work when the going got tough. Some of our competitors were “all tip and no berg”.

This principle still serves us well. Today we have all sorts of sophisticated features such as entity recognition, but we make sure the rocket science happens “effortlessly”.

What has changed in search and retrieval in the last 20 years?

It’s a different world. Twenty years ago, our first challenge would be to educate prospective clients in the difference between a search engine and pressing control-F or whatever in their word processor. Now, by virtue of the public internet search engines, people are very much aware of just how useful it is to be able to find things when you need them.

Add to that the incredibly powerful driver that business communication is increasingly electronic. Prior to that, the fact that things existed on paper were a barrier to the usefulness of enterprise search. Now, the base form of interchanged information is electronic, and it tends to only hit paper as an ancillary step.

Of course, the increased capacity of hard disks and bandwidth has been the enabler for this, accompanied by reduction in hardware costs. Everyone has a computer, every computer as an enormous hard disk, and everyone is generating or receiving masses of readily searchable documents and emails.

These factors have certainly helped define the market for search, but they’ve also had dramatic effect on what we, as technologists, have been able to do with search. The machines of today are so much more powerful than ten years ago that we’ve been able to spend available cycles to add really helpful functionality which adds value to search.

That said, we’re pretty zealous about getting fat and lazy on today’s hardware. Our goal, and we’ve always met it, is for every new version of ISYS to run faster than the previous on the same hardware. Performance matters.

ISYS is on a fast track with regards to innovation. What sets our ISYS 8 system apart is that it reflects what our broad base of customers tell us is important. We work with the same engineering diligence to ensure stability and speed, of course, but we are able to provide features and functions because we have a sound engineering platform and incredibly helpful customers. Our customers are our marketing research and product development department.

Your system is extremely well-known in government, legal and intelligence circles. Why is that?

When we launched ISYS commercially, we soon realized only a handful of industries had sufficiently large data volumes to warrant advanced search and retrieval. We concentrated on solving these "big data" problems.

We kept our focus on the public sector, the legal profession, criminal intelligence and the like, because these were the places where search was in most dire need. As a smaller firm, we obviously had to battle some bigger players along the way, but ironically our size has been a big benefit. Whereas the big vendors entered through the front door with broad-sweeping, top-down approaches, we could get in through the back door of a large number of companies with tactical point solutions.

ISYS' speed, stability, and ease of use convinced many in these "big data" sectors to rely on us. Then when the Web "explosion" took place, we were able to take advantage of the growing interest in search.

Now that I think about it, we've kept our original customers, adding more in law enforcement, legal services, and government. And, we've added many other customers from different business sectors, in everything from recruitment and aerospace to financial services and healthcare.

The wonderful news for ISYS is that we all had a need early on, years before Microsoft Windows, the World Wide Web, and the information explosion proliferated. Our customers helped us grow, and our engineering has allowed us to be an aggressive innovator.

Have the customer's needs changed much in 20 years?

Yes and no.

What defined search back then was the significance of the need -- users were after information that truly was mission critical. Now, juxtapose that with today, where search has expanded to address usability and the need to leverage corporate knowledge.

What we have is a keen demand for mission critical search and retrieval, content processing, and analysis. In addition, there are large numbers of organizations that are trying to make the best use of the information in digital form.

Mission-critical search manipulates information to identify a criminal, which may be a matter of public safety, or extract the key fact from information related to a legal matter.

Essential search helps employees find the answer or the information needed to do their work today. Both drive the growth of ISYS. I don't see either need diminishing going forward.

ISYS was one of the first, if not the first, search system vendor to offer a ready-to-run, off-the-shelf search solution, right?

I hesitate to say, "ISYS was the first." There have been so many companies innovating in search. But I think we were one of the first. I was keen to make it possible for a small law firm, a department in an insurance company, or a municipal police force to be able to process its electronic content and immediately improve access to information.

Well before 1990, we recognized and understood these user needs. We have an excellent group of engineers and programmers with a shared vision who were able to translate those needs into a product.

We listened to these customers and continually fed that input back into the product with each successive release. So, to some extent, the seeds for addressing search needs today were sown years ago by the very people who had “can’t fail” requirements for search.

Our first-mover advantage and continued dedication to taking our technology beyond search has enabled us to remain strong in those key markets as well as several others.

I have a list of more than 150 companies offering search solutions and another list of about 125 companies who do content processing. What makes ISYS different from other vendors in the market today?

I’ll once again point to our rich history for starters. There’s no question the enterprise search industry has a lot of smart people driving it, so I think you’ll find plenty of things to get excited about from an innovation standpoint. But what makes ISYS unique is we look to apply this innovation in a practical, real-world manner.

When I attend an industry event and hear the jargon and the buzzwords, I chuckle. ISYS is a search solution that offers features that are useful to a large percentage of users. You don't need a fancy bit of jargon like semantic wiki or enhanced Bayesian algorithms to make a system useful. There needs to be rocket science, without doubt. But as an industry we should be focused on the benefits to the user, not overly obfuscating the process.

At ISYS, we let people use our system, click on the named entities we identify automatically, look at the results grouped by category. The system does the talking, not the buzzwords.

If you look at how the market segments, it breaks out as follows: you have low-end freebie tools for the undifferentiated user; you have the platform vendors that, generally speaking, offer custom-built solutions that offer a high reward but come at the high cost of significant time, money and resources; third, you’ll find specialist vendors who are really pushing the bleeding edge of what’s possible, but for better or worse are appealing to only a finite population; and finally you have vendors like ISYS who are addressing the best of all worlds by delivering advanced, specialized functionality in an affordable, out-of-the-box solution.

ISYS is successful, I am convinced, because it works, and it allows a person who must find information often under great pressure and little tolerance for confusing interfaces, to locate information quickly and easily. And we know it works, because our customers constantly refer new business. I wouldn’t be surprised to learn that our legal and intelligence customers have sold more ISYS installations to their colleagues than we have on our own.

The bottom line is when a life is on the line, you don't want to try and move a little graphic control to increase the sensitivity of the relevance control. You want the right information the first time, every time.

Are there any competitors who have a similar philosophy to that of ISYS?

Of course, we see that the Google Search Appliance and to some extent, the Microsoft SharePoint search (which I think is now called MOSS) are close competitors. We've looked closely at these systems. I think that Google and Microsoft share a common characteristic. Both have a generalized focus and relatively inflexible architectures. An integrator or a very smart licensee can write code, widgets, and adaptors to make these systems perform many search-related functions. But these customizations take time, and time is money.

ISYS, on the other hand, addresses the foundation first by delivering sound relevance, multiple query methods and broad file format and language support. We then expand on that by delivering capabilities like On-The-Fly Categorization, ISYS Entities detection, and ISYS SearchTrends analytics reporting. These “beyond search” capabilities are increasingly in demand by customers, yet are not addressed by other mid-market search tools like the Google Search Appliance or MOSS.

The key point is that an ISYS licensee can install, index, and deploy a search system quickly. If customization is required, our APIs make it possible to adapt or add to our core system. Simply put, the system is up and running without waiting for programmers to make needed adjustments. It is as customizable as anything else, but the difference is that it also does useful things right out of the box.

So are you saying that ISYS does the basics better?

We don't sacrifice the basics for razzle dazzle. We offer some powerful content processing functions. But these functions don't get in the way of delivering results that are on target.

If you talk with most analysts in this industry about best practices for search implementations, they’ll tell you that you need to start with establishing the basics (the 80%); once you have that locked in, you can move into that final 20%, where you can get more sophisticated. We take great pride in the fact that we offer a flexible and extensible solution that takes customers through that entire lifecycle. So, what you have in ISYS is something that can be installed, configured and operational in one afternoon by a single person, and then customized over time as the organization learns more about user search habits, needs, etc. Bottom line is we offer a low barrier for entry in an industry marked by a lot uncertainty, courtesy of the rampant consolidation.

Who is a typical ISYS user today?

Quite honestly, we don’t have a typical user. By design, we deliver broad capabilities that speak to the various requirements of a wide set of users. While we understand that individuals like researches or librarians might require highly specialized tools for their jobs, search has become universal to the average knowledge worker.

We’ve made a conscious effort to not attempt to “understand” your needs or serve as your own personal C-3PO who presumes to know exactly what you want, when you want it. In fact, some of our competitors are integrating technology into their systems that make changes without informing the user or the system administrator.

We’ve designed and engineered our software to account for the different hats a user will wear from day to day, even hour to hour. We offer the breadth of functionality required to address changing user needs without friction. From our perspective, it’s not so much about intelligently interpreting your content for you, but rather giving you intelligent cues to help you quickly and easily connect the dots for yourself.

What's the machine architecture? Are you on premises, hosted, or both?

ISYS offers a standard, on-premise software solution that is available in various personalities. We address the end-user and workgroup need with ISYS:desktop, our client interface. For Intranets, portals, custom web applications and enterprise-wide search, we offer ISYS:web. And finally, for software vendors looking to integrate best-of-breed search into their applications, we ship the ISYS:sdk. Our SDK has been a real source of strength, as evidenced by our list of valued OEM partners.

Who are your partners?

We have a large number. Some of the ones you will recognize include EMC, Autodesk and TOWER Software, to name a few in the US. We have partners here in Australia and elsewhere. In fact, there's been strong interest in our OEM product, particularly with the rise of e-discovery and the need to bolster e-discovery systems with embedded search.

OEMs particularly like that our engine carries a relatively modest footprint, and that the API is quite approachable. We take the same approach with our API as we do with the rest of the product.

What development processes do you use to achieve the high degree of deployability and great out-of-the-box experience you achieve?

We try to combine the best of traditional, rock-solid software engineering principles with some of the better parts of the rapid application development techniques. The pay off from this approach is that ISYS' brilliant, young engineers can try out an idea. Then we can apply our engineering rigor to the innovation. In this way, we can react quickly to customer needs or new ideas we have. These innovations don't require the type of "middleware" or "wrappers" that some of our competitors use to implement a new feature. ISYS is a solid, reliable system without the performance-degrading work-arounds that one finds in certain very well-known search systems.

Our engineers obsess about deployability. Having a great idea and figuring out how it will work under the covers is less than half the job. The rest of the challenge lies in figuring out how we’re going to best present that feature to the user, make it work out-of-the-box, have its function be obvious and then provide configuration control for those who need it.

ISYS is striking with its speed. How does that sort of result come about?

That’s the third half of the job, and again obsession is the key. Performance has got to be endemic in the development team. If you’ve got a great idea but can’t figure out how to make it go fast, then you haven’t got a great idea yet. And you’ve got to maintain that speed across volume. Everything in search would be easy if the data volumes were always low, but the greatest value comes when the data volumes are large. So if your performance goes to water when the volumes grow, you’re just not there.

Performance comes in two ways. The first is algorithmic performance, where the algorithms you come up with scale well with volume. It’s all about curves, and it’s key. If your curves are wrong, you’re never going to scale.

The second is implementation performance, where there’s craftsmanship in how well you implement your algorithm. We spend a lot of time analyzing our code and profiling to ensure the bottlenecks are eliminated.

We like to make every new version more efficient than the last. The ever-improving hardware, including 64-bit, certainly helps but we always achieve better efficiency on a like-with-like basis.

What's the code base?

We code in industry-standard languages. Our focus is on striking a balance between the language an engineer knows best and tried-and-true development, configuration management, and modularity. So far, so good in the technical department, but I think some of the younger engineers want to be given total freedom. We allow some freedom. There's no point in creating components that no one else can figure out. We also won’t sacrifice performance just for the sake of making an engineer's day more pleasant. We care a lot about functionality, deployability and performance. If the engineers have to go an extra mile to get ticks in all three boxes, that’s fine.

The most important thing is to understand the performance implications of your code. We make use of assembler in places. Some say assembler is a dying art and that’s a good thing. We disagree. We think that if your performance analysis shows that you can get another 10% throughput by converting six lines of high-level code into two dozen lines of assembler, then you owe it to your customers to do so. We don’t have much assembler in our code. Very little, in fact. But the bits we do have make a huge difference to performance.

Can customers interact with the system via an API?

Yes, we place a lot of emphasis on our ISYS:sdk offering. Customers have few options when it comes to embedding search into their applications, and we’re happy to say we’ve become a favorite among businesses operating in everything from content management to e-discovery and information lifecycle management. Additionally, ISYS:web ships with a toolkit for customers wanting to implement custom web applications.

What's your newest feature? What's coming in 2008?

We anticipate launching our next major release in the second quarter of 2008. As with our last two releases, the focus of our latest version is on capabilities that take users beyond search, helping answer the question, “You found it, now what can you help me do?” While we’re holding off publicizing specific details about these capabilities, we can tell you that we’re adding a visualization piece; enhancing and unifying our refinement options; incorporating additional functionality that’s specific to the legal market; and solidifying our ability to implement and manage enterprise-wide deployments, which will be key in addressing compliance and e-discovery needs.

Were these customer-driven enhancements?

Yes, we had several customers make suggestions to us. One of the big challenges in working with unstructured content is wrapping structure around it, so that users can quickly and easily sift, sort and navigate content in a meaningful and purposeful way. When we released Version 7 in 2005, we introduced our On-The-Fly Categorization for that very purpose. We expanded on that in 2006 with Version 8 and our ISYS Entities detection capabilities. With our new version in 2008, we’re expanding on that even more to highlight additional intelligence found in the document result set, again with the aim of giving users the insight and context needed to get the most from their searches.

Beyond that, we see significant opportunity in both the legal and corporate compliance markets. These are areas we’ve always addressed successfully, but we wanted to take it to the next level by delivering technology that enables these users to accomplish even more with search.

What are the hot trends in search for the next 12 to 24 months? How will you take advantage of them; for example, go public, partner, sell to a larger firm, etc.

From our perspective, the hottest trend is the demand for embedded search. Various enterprise software systems and their adoption have matured to the point where they can no longer get by with the built-in search they created as one of many non-critical value adds. Search, in many ways, has become one of the more important requirements for these systems, so we think you’ll continue to see these vendors seek out best-of-breed search to help bolster their solutions. This will be especially true with content management systems and e-discovery tools, where the vendor landscape has grown exponentially in recent years, thanks to various corporate compliance laws.

What's next for ISYS? More bundles, vertical builds of the system, other?

ISYS will continue to push the envelope for what’s capable in an affordable, out-of-the-box enterprise search solution. We’ve never believed that effective search has to be expensive and difficult to deploy, and our continued growth and success shows that our customers agree. We’re certainly not looking to give Autonomy or Endeca a run for their money in services, but we’re quite happy to address the areas of the mid-market that the infrastructure vendors don’t touch. There’s tremendous growth opportunity in this segment of the market, so we’ll remain focused on that core.

ArnoldIT Comment

ISYS Search Software is one of the vendors growing at double-digit pace. The company has an outstanding reputation among law enforcement, legal discovery, and intelligence professionals. In my Beyond Search study, ISYS has been identified as one of a small number of content processing innovators named as a "company to watch." You can get more information about at the ISYS website.