Perfect Search: New High-Speed Engine
November 25, 2008
Perfect Search (http://www.perfectsearchcorp.com) has developed a high speed search and content processing system. The demo I saw sizzled with speed. There was speedy content processing and near zero latency query processing and results display. I was impressed and wrangled a free meal with a couple of the company’s seasoned engineers. After lunch, I ordered a fresh bottle of Perrier and grilled one of the start up’s senior technologist. You can read the full text of my interview with Ken Ebert here. Let me highlight two interesting items from the interview:
With regard to speed, I wanted more detail. Mr. Ebert told me:
The key performance differentiators are our ability to handle massive data sets and our speed. We have a current customer that has over one billion records of both structured and unstructured content that we have been able to search on a single low end server. This single server replaced seven servers that they were previously using for query processing.
I probed about the company’s plans for 2009. Perfect Search, unlike some other vendors with whom I have spoken, has embraced the idea of “surfing on Google”, which is one of my standard catchphrases. Mr. Ebert said:
In 2009, we are focusing on those customers that have very large data sets and are unsatisfied with their current solutions. As mentioned before, our very first market niche will be GSA customers who need to index and query large databases. Later in 2009, we will launch a GSA add-on appliance that will enhance email search. This combination of the GSA with our appliance will allow those wanting to do eDiscovery a very competitive solution.
If you want to get more insight into the Perfect Search approach to findability and content processing, read the entire interview. I tested the firm’s system, and I am a believer. Speed thrills or at least speed thrills me when it comes to search.
Stephen Arnold, November 25, 2008
Autonomy: An Interview with Andrew Kanter
November 18, 2008
Andrew Kanter is the chief operating officer of Autonomy, a vendor of enterprise infrastructure and content processing systems, participated in the Search Wizards Speak series. Mr. Kanter addressed a number of topics in this wide ranging discussing, including the firm’s “meaning based computing” technology and the company’s move into cloud computing and hosted services. Autonomy was founded in 1996, but the company has been at the forefront of advanced content processing for more than a decade. Some analysts peg Autonomy as the world’s largest vendor of search systems in the world. The firm’s system is described as a “pan enterprise search platform” and incorporates technology that handles content automatically via the firm’s intelligent data operating layer or IDOL system.
Mr. Kanter said about the tough financial climate that is burdening some information system vendors:
Autonomy has closed some of the biggest deals in our history in recent months, reaching to $20 million, $70 million and more. These companies themselves often don’t know what’s happening within their own walls until it’s too late. In some ways it’s a sad truth but there is always going to be a demand for this type of [Autonomy] solution, whatever happens to the macroeconomic climate.
Mr. Kanter also talked about Autonomy’s ability to index audio and video content which is proliferating at some organizations. He said:
These days it’s very rare that we have a purely text-centric installation because we naturally communicate as human beings through rich forms of information such as voice and video. Autonomy’s IDOL is at the heart of our Autonomy Virage technology, applying the same conceptual analysis to audio and video content as to text. Further, even if demand today is only for text, most enterprises want to have the capability of including audio and video at a later date. They are bringing into their systems conference calls, webinars and other rich media content. In fact, we are seeing an increase in customers using our Autonomy Virage rich media solutions across all verticals. The strength of the platform approach is enable the architecture to evolve to accommodate the enterprise’s changing needs, without having to rip everything out.
You can read the complete interview with Mr. Kanter on the ArnoldIT.com Web site. Navigate to http://www.arnoldit.com/search-wizards-speak/ or click here to access the interview directly.
Stephen Arnold, November 18, 2008
Lemur FLAX: Clever Search Beastie Interview
July 8, 2008
No, I did not interview a real lemur. I tracked down Charlie Hull, one of the wizards driving Lemur Consulting forward. The company makes the open source Xapian information retrieval available as the open source FLAX search engine. Lemur, like Tesuji and dozens of other companies, has tapped the power of open source search and content processing software and crafted a successful business.
FLAX, according to Mr. Hull scales. In an exclusive interview for ArnoldIT.com’s Search Wizards Speak series, he said:
The core technology was originally built to search a collection of 500 million Web pages, and scales easily to over four billion items. We’ve implemented indexes of 30-100 million items on a single standard server. It’s also extremely fast to search a Flax database. We routinely see sub-second retrieval times.
You can see the search system in action at MyDeco.com, a UK-based ecommerce site here.
What I found interesting is the by making FLAX available as open source, the company has generated new customers for the firm’s technical consulting and engineering services. Mr. Hull said:
Our view is that any enterprise search system will necessitate some degree of installation, integration or customization – so a customer will always pay for services. However, with open-source you don’t have to pay any license fees on top. In today’s economic climate this cost saving is more and more important. We’ve seen year-on-year growth of the business, as well as a dawning realization that our open-source approach puts the control back in the hands of the customer – you don’t have to take our word for it that the ‘black box’ of enterprise search is working, you have complete visibility and control over the search system.
Mr. Hull’s secret sauce is technical expertise. The company adds a special ingredient that keeps the company on the fast-track–customer service. The firm prides itself on servicing its customers needs.
In an era when “customer support” means “Don’t bother us,” Lemur is an animal with a clever way to snare clients. You can read the full interview here.
Stephen Arnold, July 8, 2008
Mark Logic: Content Applications Fuel Company’s Growth
June 17, 2008
Mark Logic provides information access and delivery solutions that accelerate the creation of content applications. Customers across a range of industries rely on Mark Logic to repurpose content and deliver that information through channels. Some vendors describe this suite of functions as an enterprise publishing system.
The company has been growing at a furious pace. Dave Kellogg, former Business Objects’ executive, said:
Mark Logic… is a database management system built to natively manage XML documents and optimized for handling vast numbers of them (I mean hundreds of terabytes) with high performance. It’s a read/write system. It has a query language (XQuery). It has transactions and logging. You can use it, by itself–without the need to bolt it on to either a relational database or an application server–as the basis for content applications.
The company’s customers include Oxford University Press, O’Reilly Media, and the Congressional Quarterly. The company builds relationships with its customers. Mr. Kellogg says, “Our philosophy is to sell sell solutions to problems and avoid the stereotypical “drive-by” technology sale, where companies dump the software in the parking lot and leave.”
The full interview appears as part of the Search Wizards Speak series published by ArnoldIT.com. You can read the transcript of the interview with Mr. Kellogg here. The index to the full series of interviews is here.
Stephen Arnold, June 17, 2008
Silobreaker: Sophisticated Intelligence
June 12, 2008
An Interview with Mats Bjore, Silobreaker
I met Mats Bjore at a conference seven, maybe eight years ago. The majority of the attendees were involved in information analysis. Most had worked from government entities or commercial organizations with a need for high-quality information and analysis.
Mr. Bjore and two of his dogs near his home in Sweden.
I caught up with Mats Bjore, one of the wizards behind the Silobreaker service which I profiled in my Web log, in the old town’s Café Gråmunken. Since I had visited Stockholm on previous occasions, I asked the waiter for a cheese plate and tea. No herring for me.
Over the years, I learned that Mr. Bjore shared two of my passions: high-value intelligence and canines. One of my contacts told me that his prized possession is shirt emblazoned with “Dog Father”, crafted by his military client.
Before the waitress brought our order, I asked Mr. Bjore about his interest in dogs and Sweden’s pre-occupation with herring in a mind-boggling number of guises. He laughed, and I turned the subject to Silobreaker.
Silobreaker is one of a relatively small number firms offering a combination intelligence-centric solution to clients and organizations worldwide. One facet of the firm’s capabilities stems from its content processing system. The word “search” does not adequately describe the system. Silobreaker generates reports. The other facet of the company is its deep expertise in information itself.
The full text of my conversation with Mats Bjore appears below:
Where did the idea for Silobreaker originate?
Silobreaker actually has a long history in the sense of the word Silobreaker. When I was working in the intelligence agency and later at McKinsey & Co I was amazed of the knowledge silos that existed totally isolated from each other. I saw the promise of technology to assist in unlocking those Silos, however the big names at that time, Autonomy, Verity, Convera etc failed to deliver, big time… Disappointed and waiting for the technology of the future I registered the name Silobreaker.com , more like a wish for the perfect system. A couple of years later in 2003-2004 I was approached by a team of amazing people–Per Lindh, Björn Löndahl, Jimmy Mardell, Joakim Mårlöv and Kristofer Mansson. These professionals wanted to further develop their software into an intelligence platform. In 2005 my company Infosphere and the software company Elucidon joined forces and we created Silobreaker Ltd as a joint venture. One year later we consolidated software, service and consulting to one brand–Silobreaker.
Today, Silobreaker enables the breaking down of silos built from informational, knowledge, or mental bricks and mortar.
What’s your background?
I am a former lieutenant colonel in the Swedish Army. I was detailed to the Swedish Military intelligence Agency where I founded the Open Source Intelligence function in 1993.
After leaving the government, I became the Scandinavian Knowledge Manager for McKinsey & Company. After several years at McKinsey, I started my own company. Infosphere and the service Able2Act.com.
I am also a former musician in a group called Camping with Penguins. You know that I am a lover of dogs. Too bad you like boxers. You need to get a couple of my friends so you have a real dog. I’m just joking.
I know. I know. What are the needs that traditional search engines like Autonomy, Endeca, and Fast Search (now Microsoft) are not meeting?
Meaning and context and I would also say that traditional engines requires that you always know what to search for. You need to be an expert in your field to make to fully take advantage of the information in databases and unstructured text. With the Silobreaker technology the novice becomes an expert and the expert becomes a discoverer, It might sound like a sales pitch, but its true. Every day in my daily work I have need to jump into new areas, new industries and topics. There is no way that I can formulate keyword search nor have the time to digest 100 or a 1,000 articles that works in the mode of click and read, click and read. With Silobreaker and its technology I start very broadly and the system directly helps me to understand the context of large set of articles in different formats, from different repositories, from different topics. We call this a View 360 with an In Focus summary. Note: here’s an InFocus example provided to me after the interview.
When I search in traditional systems based on the search/ read philosophy, I spend to much time searching and reading and too little time of sense making and analysis. With Silobreaker, I directly start with that process and I create new value, for me and for my clients.
In a conversation with one of the Big Brands in enterprise search, the senior VP told me that services producing answers are “just interfaces”. Do you agree?
“Just interfaces” might be a bit harsh on the companies that actually try to provide direct answers to searches – they actually have some impressive algorithms, but to a certain extent we agree. We simply don’t think that an “answer engine” solves any real information overload problem.
If you want to know “What’s the population of Nigeria” – fine, but Wikipedia solves that problem as well. But how do you “answer” the question “What’s up with iPhone”? There are many opinions, facts, news items, and blogs “out there”. Trying to provide an “answer” to any “question” is very hard to do, maybe futile.
We always emphasize the importance of using our technology for decision-support, not to expect the system to perform the decision-making for you. The problem today is that analysts and decision-makers spend most of their time searching and far too little time learning from and analyzing the information at hand. Our technology moves the user closer to the many possible “answers” by doing much of the searching and groundwork for them and freeing up time for analysis and qualified decision-making. Note: This is a 360 degree view of news from Silobreaker provided after the interview.
There’s significant dissatisfaction among users of traditional key word search systems. What’s at the root of this annoyance?
The more information that is generated, duplicated, recycled, edited and abstracted and in combination with the rapid proliferation of “ I never use a spell checker and I write in your language with my own set of grammar”—– the need for smarter system to actually find what you are looking for will increase. In a couple of years from now, we also see the demise of the mouse and keyboard and the emergence of other means of input, the keyword approach is not just it.
Keyword based search works reasonably well for some purposes (like finding your nearest Swedish herring restaurant), but as soon as you take a slightly more analytical approach it becomes very blunt as a tool.
There is no real discovery or monitoring aspect to keyword based search. The paradox is that you’ll need to know what you’re looking for in order to discover.
Matching keywords to documents doesn’t bring any meaning to the content nor does it put the content in context for the user.
Keyword based search is a bottom-ups approach to relevance. The burden is put entirely on the user to dissect large results in order to find relevant articles, who the key players are, how they relate to each other, and other factors.
This burden creates the annoyance and “research fatigue” and as a result users rarely go beyond the first page of results – hence the desperate hunt amongst providers for PageRank, but which may have no or little bearing on the users real needs.
The intelligence agencies in many ways are the true professionals in content analysis. Why have the systems funded by IN-Q-TEL, Interpol, and MI5/MI6 not caught on in the enterprise world?
These systems are often complex and their “end solutions” are often mix of different software that is not well integrated. We already see a change with our technology. Some government customers look at our free service at Silobreaker.com and have a chance to explore how Silobreaker works without sales people hovering over them.
We want our clients to see one technology with its pieces smoothly integrated. We want the clients to experience information access that, we believe, is far beyond our competitors’ capabilities.
Intelligence agencies have often acquired systems that are too complex and too expensive for commercial enterprises. Some of these systems have been digital Potemkins. These systems provide the user with no proof about why a certain result was generated.
Now, this “black box” approach might be okay when you have a controlled content set, like on the classified side within the intelligence community. But the “real world” needs to makes sense of unstructured information here and now.
You have plus 100,000 major companies in the world, and you have 200 or so countries. Basically the need for technology solutions is the same. For me it’s totally absurd that the government complicates their systems instead of looking at what is working here and now.
Furthermore I think one the reasons that government can do complex and sometimes fruitless projects is that some agencies don’t have to make money to survive. The taxpayers will solve that.
In the commercial sector–profit and time are essential. Another factor that corporations take into account when investing in a system such factors as ease of use.
With the usually high turnover in any industry, a system must be easy to use in order to reduce training time and training costs. In some government sectors, turnover is much lower. People can spend a great deal of time learning how to use the systems. Does this match your experience?
Yes, and I agree with your analysis. I had an email exchange with the chief technical officer of a major enterprise search vendor. He asserted that social search was the next big thing. When I pointed out that social search worked when the system ingested a large amount of information, much available covertly, he argued that general social information was “good enough”? Do you agree?
No I don’t. Now we are talking about quality of the information. If you would index and cross reference XING, Facebook, Linkedin you could display fantastic displays of the connections….. However, how many of this links between people are actually true (in the sense that they actually have met or even have some common ground)?
There is a very large set of people that try to get as many connections as possible, thus diluting the value of true connections. I agree that you need a significant amount of information in order to get a baseline. You also need to validate this kind of data with reality checks in other kind of information sources – offline and online.
My main company, Infosphere, did some research into the financial networks in the Middle East, the fact-based search (ownership, shareholdings, etc) provided one picture, then you have to add the family and social connections, the view from media, then look at resident clusters and other factors. We had more than 8,000 dots ( people ) that we connected. But we were just scratching on the surface.
The graphic displays in Silobreaker are quite useful. In a general description, what are you doing to create on the fly different types of information displays?
The whole philosophy behind Silobreaker is to move away from the traditional keyword based search query which generates just page after page of headline results and forces the user into a loop of continually adjusting the query to find relevance and context.
We see the keyword-based query as a possible entry point, but the graphical search results enable the user to discover, navigate and drill down further without having to type in new keywords. No-one can imagine managing numerical data without the use of descriptive graphical representations, so why do we believe that we can handle vast quantities of textual data in any other way. Well we don’t think we can, and traditional search is proving the point emphatically. Today’s Silobreaker is just giving you a first glimpse of how we (and I’m sure others) will use graphics to bring meaning to search results.
Is Silobreaker available for on premises and for SaaS (software as a service”? What do you see as the future access and use case for Silobreaker?
That’s a good question. Let me say that Silobreaker’s business model is divided into three parts.
First, we have a free news search service that eventually will be add-supported but whose equally important role is to show-case the Silobreaker technology and function as a lead generator for the enterprise offerings.
Second, Our Enterprise Service which is due to be released in September or October 2008 is an online, real-time “clipping service” aimed at companies, banks, consultants as well as government agencies and that will offer a one-stop shop for news and media monitoring from defining what you are monitoring to in-depth content aggregation, analysis and report generation. This service will come with a SaaS facility that enables the enterprise to upload its own content and use the Silobreaker technology to view and analyze it.
Third, we offer a Technology Licensing option. This could range from a license to embed Silobreaker widgets in your own site to a fully operational local Silobreaker installation behind your firewall and customized for your purposes and for your content.
Furthermore, parts of the Silobreaker technology are available as SaaS on request.
Let’s talk about content. Most search systems assume the licensee has content. Is this your approach?
Yes and no, we can facilitate access to some content and also integrate crawling with third-party suppliers or if its very specific assist with specialty crawling.
On top of that we can, of course, integrate the fact sheets, profiles, and other content from my other venture, Able2Act.com which gives any system and any content set some contextual stability.
What are the content options that your team offers? Is it possible to merge proprietary content and the public content from the sources you have mentioned?
Yes, the ideal blend is internal and external content. And that really sets our team apart. Most of the Silobreaker group works with information as the key focus on a daily basis, sometimes 24×7 on certain projects. In other words, we are end users that keeps our ear to ground for information. Most companies out there are either tech people or content aggregators that just sell. We are both.
When you look forward, what is the importance of mobile search? Does Silobreaker have a mobile interface?
Mobile “search” is an extremely important field where traditional keyword-based search just doesn’t cut it. The small screen size of mobile devices, and limited (and sometimes cumbersome) input capabilities is just not suitable for sifting through pages of search results just to find that you need another Boolean operator and have to start all over again. We believe that users must be given a much broader 360 view of what they’re searching for in order to get to the “nugget” information faster. Silobreaker does not currently offer a mobile interface, but needless to say we’re working on it.
What are the major trends that you see emerging in the next nine to 12 months in content processing?
That’s a difficult question. I can identify several areas that seem important to my clients: Contextual processing, cross media integration, side-by-side translations, and smart visualization. Note: I have inserted a Silobreaker link view screen shot Mr. Bjore provided me after our conversation.
Observations
Silobreaker caught my attention when I saw a demonstration of the system before it was publicly available. The system has become more useful to intelligence professionals with each enhancement to the system. Compared to laundry-lists of results, the Silobreaker approach allows a person working in a time-compressed environment to size up, identify, and obtain the information needed. The system’s “smart software” shows that Silobreaker’s learning and unlearning function is part of the next generation of information tools. After accessing information with Silobreaker, I am reminded that key word search is a medieval approach to 21st century problems. Silobreaker’s ability to assist a decision maker makes it clear that technology, properly applied, becomes a force multiplier without pushing human judgment to the sidelines. In one of our conversations, Mr. Bjore drew a parallel between Silobreaker and the canines for which he and I share respect and affection. He said, “Silobreaker works like one of our dogs. Their eyes see what is in front of you, the ears hears the tone of voice, the nose smells what has happened, what is now and what’s around the corner.” I agree. Silobreaker is more than search; it’s an extension of the information envelope. Take a close look at this extraordinarily good system here.
Stephen E. Arnold, June 12, 2008
Deep Web Tech’s Abe Lederman Interviewed
June 9, 2008
Abe Lederman, one of the founders of Verity, created Deep Web Technologies to provide “one-stop access to multiple research resources.” By 1999, Deep Web Technologies offered a system that performed “federated search.” Mr. Lederman defines “federated search” as a system that “allows users to search multiple information sources in parallel.” He added in his interview with ArnoldIT.com:
Results are retrieved, aggregated, ranked and deduped. This doesn’t seem too difficult, but trust me it’s much harder than one might think. Deep Web started out building federated search solutions for the Federal government. We run some highly visible public sites such as Science.gov, WorldWideScience.org and Scitopia.org. We have expanded our market in the last few years and sell to corporate libraries as well as academic libraries.
believes that Google’s “forms” technology to index the content of dynamic Web sites is flawed.
Mr. Lederman said:
Deep Web goes out and in real-time sends out search requests to information sources. Each such request is equivalent to a user going to the search form of an information source and filling the form out. Google is attempting to do something different. Using automated tools Google is filling out forms that when executed will retrieve search results which can then be downloaded and indexed by Google. This effort has a number of flaws, including automated tools that fill out forms with search terms and retrieve results will only work on a small subset of forms. Google will not be able to download every document in a database as it is only going to be issuing random or semi-random queries.
In the exclusive interview, Mr. Lederman reveals a new feature. He calls is “smart clustering.” Search results within a cluster are displayed in rank order.
You can read the full text of the interview on the ArnoldIT.com Web site in its Search Wizards Speak series. The interview with Mr. Lederman is the 17th interview with individuals who have had an impact on search and content processing. Search Wizards Speak provides an oral history in transcript form of the origin, functions, and positioning of commercial search and text processing systems.
The interview with Mr. Lederman is here. The index of previous interviews is here.
Stephen Arnold, June 9, 2008
PolySpot: Usability Fuels Growth
May 19, 2008
Olivier Lefassy, an investment professional turned business intelligence executive, is on the fast track. His firm–PolySpot–is growing at a double-digit pace. The company packages a suite of content processing technologies that “snap in” to licensees’ existing infrastructure. You can read this exclusive interview at http://www.arnoldit.com/search-wizards-speak/polyspot.html.
The idea is to provide powerful information access methods without the costly hand coding and months of tedious work that many vendors impose on their customers.
Instead of displaying a laundry list of results, the company delivers answers to system users. One of the system’s most interesting features, he told ArnoldIT.com for its Search Wizards Speak series, is:
…a Document Collaboration module. You are in a research team for a large financial organization. You locate a useful analyst report about a company. You can open the document and add a comment to it, appended to the original document. You can then put this into a public folder and forward it on to a colleague for his or her comments. We think this is like “document blogging” or annotating. These comments or additional information payloads are indexed “on-the-fly”.
He said, “Usability is key today. For too long the ‘large’ vendors ignored user needs at this level and tried to brainwash the market with talk of algorithms.”
To see the sharp contrast between PolySpot and a long-time player in search, take a look at “Up to Speed on Search” by Phil Muncaster. In that article Mr. Muncaster summarizes Autonomy’s view that some systems are “planes” and others “mere bikes”. The comparison underscores PolySpot’s approach to the market: power without undue complexity. PolySpot’s approach stands on one side of the usability argument and Mr. Muncaster’s essay makes clear that their is another, more complicated side to the argument that appeals to some vendors. In his essay, Mr. Muncaster uses the delightful phrase “particularly keen”, which struck me as quite telling. Could some of the established vendors feel pressured, not just by PolySpot, but the dozens of up-and-comers in information access who offer options to some organizations?
PolySpot’s managing director states clearly that there is a need for a different way to approach information access. The firm’s strong growth in the first three months of 2008 underscores that some European organizations are eager to put euros to work addressing content challenges. You can read the complete interview at ArnoldIT.com here.
Stephen Arnold, May 19, 2008
Former Clandestine Operative Says Automated Systems Not Good Enough
May 13, 2008
Editor’s Note: Robert Steele, former Marine Corp. officer and intelligence operative, was one of the first, if not the first, intelligence professional since World War II to question the relative value of secret sources and technologies in relation to open sources and technologies. Mr. Steele agreed to meet me near his office in suburban Washington, D.C. The full text of the interview appears below. After we spoke, Mr. Steele provided me with illustrations he referenced in our conversation. I have included these in the transcript at the point where Mr. Steele references them. You can read more about Mr. Steele at his Web site, OSS.Net.
How did you get interested in using information that’s readily available to anyone in a library, in newspapers, and online as a source of useful intelligence?
I went into the international spy program at CIA with a Master’s in International Relations, and knew quite a bit about citation analysis and primary research. What I was not expecting over the course of my clandestine career was the obsession with stealing secrets to the exclusion of all that could be known from open sources.
Robert D. Steele
The clandestine officers also refused to interact with the analysts—before leaving for my first overseas assignment, the Chief of Station took me to the analysis side of the house, and on my way there he said something along the lines of “these folks know nothing useful, and we tell them nothing.”
When the Marine Corps asked me to leave CIA to create the Marine Corps Intelligence Center in 1988, I promptly did what I thought the government wanted; that is, I spent $20 million on a codeword analysis center, including a Special Intelligence Communications (SPINTCOM) work station. I thought it would do everything except kill the terrorist.
Was I in for a shock. I had put a PC with Internet access in an isolated room, not connected to any government network. The PC had a modem. I was curious about online and bulletin board systems. In a short time, analysts were leaving their super charged workstations to stand in line to use the PC. These professionals were looking for information that was not in the government system and not known to our officers in the field (including diplomats and commercial or defense attaches).
What a wake up call.
That is when I learned that expensive systems are as good as their sources—narrow casting into the secret world made much of our multi-billion dollar technology virtually worthless. Analysts using the PC showed me that 80 to 90 percent of the information we needed could be obtained using the PC and public information to include direct calls to overt human experts. I also learned that useful information was available in 183 other languages no one in the US Government can speak or understand. Even today, a large number of Washington officials don’t understand the intelligence value of open sources of information including commercial imagery, foreign-language broadcasts that must be accessed locally, and gray literature, such as university yearbooks for a photo of a terrorist. Washington is completely out of touch with human experts that are not US citizens eligible for a secret clearance—the spies don’t want them unless they agree to commit treason, and the analysts are not allowed to talk to them by paranoid ignorant security officials.
Almost every vendor asserts that their systems can “do” business or competitive intelligence. In your experience is this accurate?
Look. BI and CI are not really intelligence.
BI or business intelligence is commonly used as a descriptor for what is nothing more than internal knowledge management, spiced up with a point-and-click graphics dashboard. Not only are most of these system non-interoperable with everything else, they are as smart or as stupid as the digital data they can access.
The reality of information in most organizations is that most of what is really valuable is not digital. And, most CEOs have zero idea what intelligence (decision support) actually means.
CI or competitive intelligence focuses on competitors. What I practice, Commercial Intelligence, focuses on
- External information
- Collaborative work
- Knowledge management
- Organizational intelligence.
Commercial intelligence leverages what can be drawn from the human social networks interacting with an organization and the other sources of information. External information is not information about competitors. It includes such factors as “true cost” of goods and next-generation “cradle to cradle” opportunities. You have to factor in the art and science of retaining Organizational Intelligence. I will send you a diagram that shows my view of this commercial intelligence space.
In my experience, today’s systems are edging toward failure. The systems aren’t very good, useful, or usable. As the Gartner Group recently said about Windows, it is untenable. I like Microsoft for its cash flow—they need to dump the legacy and launch an open source network with shared call centers and Blue Cube power processing.
Intelligenx Discloses Referrals Fuel Rapid Growth
May 12, 2008
In an exclusive interview, Iqbal and Zubair Talib, senior managers of Intelligenx, reveal that referrals have fueled the company’s rapid growth. Intelligenx has a leadership position in directory and “yellow page” search in South Africa, South America, and elsewhere. The company’s profile, despite its US headquarters in suburban Washington, DC, is modest.
The father-son team said:
It seems that our international clients are actively talking about our technology at international conferences. We can always do a better job of marketing, but we put our customers first. Sales occur because people come to us and say, “We want to license your system”… we maintained certain relationships among an elite group of scientists and engineers. We never signed up to give marketing talks at the marketing-oriented venues. Our success comes because certain people understand our technology and recognize that it delivers scale, speed, performance, data management today. Our technology is our marketing.
Unlike search and content processing firms who issue news releases when a Web site signs on to use a well-known search engine or when a vendor announces for the second or third time a reseller deal, Intelligenx keeps innovating and selling.
The company’s system offers almost all of the features associated with the best-known vendors in the search market sector. The Talibs said:
Intelligenx was first to market with technology that offered a true full-text search with what many people call faceted or assisted search results. To achieve this functionality, performance under heavy loads is the prevailing challenge and simply put, our Discovery Engine® solves the problem in what we think is a most elegant fashion “Facets” or “guided navigation” are not just a “checkbox” on a feature matrix but an underlying central philosophy in our technology, the company, and in the development of our system.
You can read about the company’s new stream processing of information, what the Talibs call “cluster flow”. In addition to near real time index updating, additional metadata are generated without adding latency to the system. Another interesting feature of the Intelligenx system is that a licensee can provide its sales people with a real time view of what advertisements are germane to a popular query. The sales person is able to show a prospective advertiser a live report of traffic and the payoff from an advertisement in a specific context.
The company’s technology offers an alternative to the better-known MarkLogic system and the specialist firm, Dieselpoint.
You can read the entire interview on the ArnoldIT.com Web site. The full text of the interview is part of the Search Wizards Speak feature. The exclusive interview is the 13th in this series of first-person accounts of the origin and functionality of important search and content processing systems. Click here to read the interview.
ZyLAB’s Dr. Johannes Scholtes Interviewed
May 5, 2008
ZyLAB’s chief executive officer, Dr. Johannes Scholtes, said in an exclusive interview for the “Search Wizards Speak” series that the company has more than 7,500 licensees world wide. This customer base puts the company on a par with search sector leaders Autonomy, Fast Search & Transfer (Microsoft), and Google.
He told ArnoldIT.com, sponsor of the Search Wizards Speak series:
Our approach has been to say to our customer, “Here’s our list of components. Just select the ones you need. You pay only for these, so we don’t ask our customers to pay huge fees for functions that will never be used.” Our modular approach is now mature, and I see more vendors in Europe and the US emulating what we’ve been doing for a long time. Our customers tell us our “couple-of-day” deployments are very unusual. For us, fast deployment is business as usual for us. These three and six month installation efforts are problems for many organizations, and these become great sales leads for us.
The failure of key word search to meet the needs of today’s organizations is becoming more well-known. ZyLAB, according, to Dr. Scholtes has pushed beyond the search box. He said:
In the basic search, a user can see the number of hits for a query, hit-density ranking, file date and time for creation, modification, and access. There are many other features in basic mode. For advanced search, you can rank on automatically extracted entities, including names, companies, countries, measurements, dates, monetary amounts, and named-phrases. You can rank by semantic relevance using an automatically derived taxonomy or your own taxonomy. Results can be personalized. You can organize result lists in a variety of ways. You can run a query on a linguistic pattern like “a person got a job” and then rank results in these patterns higher than hits in the full text. Through all this additional meta information, we can support clustering, full text similarity inside documents where precision and recall can be set.
He made the point that ZyLAB’s relevance ranking algorithms are not locked up like those from other well-known vendors.
You can read the full interview on the ArnoldIT.com Web site in the Search Wizards Speak section of the ArnoldIT Web site. This is the 12th interview in the series. An index of the previous interviews is here.
Stephen Arnold, May 5, 2008
-
- Subscribe to Beyond Search
Feature archive
News archive
-
