The Bestest Enterprise Search Diagram Ever. Really.
July 19, 2015
I like the word “bestest.” It is so right for diagrams that summarize the complex nature of search. The write up “The Best Enterprise Search Diagram You’ve Ever Seen” is a fetching smile to attract me the person who has never seen a better diagram ever to order a special report. What does the diagram look like? Here’s a not too legible version, but it is close enough for horseshoes:
The pink boxes are contributing activities. The red boxes in the middle of the diagram are direct search management tasks, and the red box in the gray rectangle is the total experience of search. Have at it. Let me know how you fair with your strategy tasks. Also, fill me in on how “search strategy” meshes with the location of the search box. Just askin? I am eager to hear how the search log insight is going to work. I like insight.
For those following this diagram, may I offer a suggestion: Look for a lateral arabesque within your organization or get a Subway sandwich franchise.
Stephen E Arnold, July 19, 2015
Short Honk: Eleasticsearch Information
July 10, 2015
Short honk: For information about Elastic’s Elasticsearch, the open source search system which has proprietary vendors of search systems cowering in fear, navigate to Elasticsearch: The Definitive Guide. Elasticsearch is not perfect, but what software is? Ask United Airlines, the New York Stock Exchange, and the Wall Street Journal about their systems. The book includes useful information about geolocation functions plus some meaty stuff about administering the system once you are up and running. Worth a look.
Stephen E Arnold, July 10, 2015
Enterprise Search and the Mythical Five Year Replacement Cycle
July 9, 2015
I have been around enterprise search for a number of years. In the research we did in 2002 and 2003 for the Enterprise Search Report, my subsequent analyses of enterprise search both proprietary and open source, and the ad hoc work we have done related to enterprise search, we obviously missed something.
Ah, the addled goose and my hapless goslings. The degrees, the experience, the books, and the knowledge had a giant lacuna, a goose egg, a zero, a void. You get the idea.
We did not know that an enterprise licensing an open source or proprietary enterprise search system replaced that system every 60 months. We did document the following enterprise search behaviors:
- Users express dissatisfaction about any installed enterprise search system. Regardless of vendor, anywhere from 50 to 75 percent of users find the system a source of dissatisfaction. That suggests that enterprise search is not pulling the hay wagon for quite a few users.
- Organizations, particularly the Fortune 500 firms we polled in 2003, had more than five enterprise search systems installed and in use. The reason for the grandfathering is that each system had its ardent supporters. Companies just grandfathered the system and looked for another system in the hopes of finding one that improved information access. No one replaced anything was our conclusion.
- Enterprise search systems did not change much from year to year. In fact, the fancy buzzwords used today to describe open source and proprietary systems were in use since the early 1980s. Dig out some of Fulcrum’s marketing collateral or the explanation of ISYS Search Software from 1986 and look for words like clustering, automatic indexing, semantics, etc. A short cut is to read some of the free profiles of enterprise search vendors on my Xenky.com Web site.
I learned about a white paper, which is 21st century jargon for a marketing essay, titled “Best Practices for Enterprise Search: Breaking the Five-Year Replacement Cycle.” The write up comes from a company called Knowledgent. The company describes itself this way on its Who We Are Web page:
Knowledgent [is] a precision-focused data and analytics firm with consistent, field-proven results across industries.
The essay begins with a reference to Lexis, which along with Don Wilson (may he rest in peace) and a couple of colleagues founded. The problem with the reference is that the Lexis search engine was not an enterprise search and retrieval system. The Lexis OBAR system (Ohio State Bar Association) was tailored to the needs of legal researchers, not general employees. Note that Lexis’ marketing in 1973 suggested that anyone could use the command line interface. The OBAR system required content in quite specific formats for the OBAR system to index it. The mainframe roots of OBAR influenced the subsequent iterations of the LexisNexis text retrieval system: Think mainframes, folks. The point is that OBAR was not a system that was replaced in five years. The dog was in the kennel for many years. (For more about the history of Lexis search, see Bourne and Hahn, A History of Online information Services, 1963-1976. By 2010, LexisNexis had migrated to XML and moved from mainframes to lower cost architectures. But the OBAR system’s methods can still be seen in today’s system. Five years. What are the supporting data?
The white paper leaps from the five year “assertion” to an explanation of the “cycle.” In my experience, what organizations do is react to an information access problem and then begin a procurement cycle. Increasingly, as the research for our CyberOSINT study shows, savvy organizations are looking for systems that deliver more than keyword and taxonomy-centric access. Words just won’t work for many organizations today. More content is available in videos, images, and real time almost ephemeral “documents” which can difficult to capture, parse, and make findable. Organizations need systems which provide usable information, not more work for already overextended employees.
The white paper addresses the subject of the value of search. In our research, search is a commodity. The high value information access systems go “beyond search.” One can get okay search in an open source solution or whatever is baked in to a must have enterprise application. Search vendors have a problem because after decades of selling search as a high value system, the licensees know that search is a cost sinkhole and not what is needed to deal with real world information challenges.
What “wisdom” does the white paper impart about the “value” of search. Here’s a representative passage:
There are also important qualitative measures you can use to determine the value and ROI of search in your organization. Surveys can quickly help identify fundamental gaps in content or capability. (Be sure to collect enterprise demographics, too. It is important to understand the needs of specific teams.) An even better approach is to ask users to rate the results produced by the search engine. Simply capturing a basic “thumbs up” or “thumbs down” rating can quickly identify weak spots. Ultimately, some combination of qualitative and quantitative methods will yield an estimate of search, and the value it has to the company.
I have zero clue how this set of comments can be used to justify the direct and indirect costs of implementing a keyword enterprise search system. The advice is essentially irrelevant to the acquisition of a more advanced system from an leading edge next generation information access vendor like BAE Systems (NetReveal), IBM (not the Watson stuff, however), or Palantir. The fact underscored by our research over the last decade is tough to dispute: Connecting an enterprise search system to demonstrable value is a darned difficult thing to accomplish.
It is far easier to focus on a niche like legal search and eDiscovery or the retrieval of scientific and research data for the firm’s engineering units than to boil the ocean. The idea of “boil the ocean” is that a vendor presents a text centric system (essentially a one trick pony) as an animal with the best of stallions, dogs, tigers, and grubs. The spam about enterprise search value is less satisfying than the steak of showing that an eDiscovery system helped the legal eagles win a case. That, gentle reader, is value. No court judgment. No fine. No PR hit. A grumpy marketer who cannot find a Web article is not value no matter how one spins the story.
Attivio ReachesTop 100 Status
June 29, 2015
The Data Dexterity Company announced the brand new Database Trends and Applications (DBTA) 100 and according to Yahoo Finance, Attivio is now on the list: “Attivio Named By Database Trends Applications To Its Prestigious Top 100 List.”
“We are pleased to be recognized by Database Trends and Applications as one of the most important firms in the data space; it further validates the type of feedback that our customers provide on a daily basis,” said Stephen Baker, CEO of Attivio. “As firms continue to be more reliant on maximizing their data to drive business-critical insights, we expect to play a critical role in driving this type of business innovation.”
Attivio joins the ranks of other companies that have made huge innovations in the data industry; they include EMC, Amazon, IBM, and more. Attivio is an industry leader in enterprise systems with its intelligence search platform. Attivio’s search platform enables users to make immediate insights with data visibility. Attivio has a well-known client use that encompasses such names as National Instruments, Nexen, GE, UBS, and Qualcomm. The company believes that there are many innovations to be made from all types, not just the type that is easily found in a database. Attivio uses its search platform to uncover insights in unstructured data that would otherwise be missed by other enterprise search platforms.
We have been following Attivio for many years and by having its name added to DBTA 100 proves it can perform well and deliver useful results. Enterprise search continues to be an important factor for enterprise systems, though people are often forgetting that today. Attivio’s addition to the DBTA 100 stresses that not everyone has forgotten.
Whitney Grace, June 29, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Major SharePoint Features Disclosed
June 23, 2015
SharePoint Server 2016 has caused quite a stir, with users wondering what features will come through in the final version. At Microsoft Ignite last month, rumors turned to legitimate features. Read more about separating fact from fiction in the newest SharePoint release in the CIO article, “Top 4 Revelations about SharePoint.”
The article begins:
“Some of the biggest news to come out of Microsoft Ignite last month was the introduction and the first public demonstration of SharePoint Server 2016 – a demo that quelled a lot of speculation and uneasiness in the SharePoint administrator community. Here are the biggest takeaways from the conference, with an emphasis on the on-premises product.”
The article goes on to say that users can look forward to a full on-premises version, bolstered administrative features, four roles to divide the workload, and an emphasis on hybrid functions. For users that need to stay in the loop with SharePoint updates and changes, stay tuned to ArnoldIT.com. Stephen E. Arnold is a longtime leader in search, and his Web site offers a unique SharePoint feed to keep all the latest tips, tricks, and news in one convenient location.
Emily Rae Aldridge, June 23, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Cloud Search: Are Data Secure?
June 19, 2015
I have seen a flurry of news announcements about Coveo’s cloud based enterprise search. You can review a representative example by reading “Coveo Lassos the Cloud for Enterprise Search.” Coveo is also aware of the questions about security. See “How Does Coveo Secure Your Data and Services.”
With Coveo’s me-too cloud service, I thought about other vendors which offer cloud-based solutions. The most robust based on our tests is Blossom Search. The company was founded by Dr. Alan Feuer, a former Bell Labs’ wizard. When my team was active in government work, we used the Blossom system to index a Federal law enforcement agency’s content shortly after Blossom opened for business in 1999. As government procurements unfold, Blossom was nosed out by an established government contractor, but the experience made clear:
- Blossom’s indexing method delivered near real time updates
- Creating and building an initial index was four times faster than the reference system against which we test Dr. Feuer’s solution. (The two reference systems were Fast Search & Transfer and Verity.)
- The Blossom security method conformed to the US government guidelines in effect at the time we did the work.
I read “Billions of Records at Risk from Mobile App Data Flow.” With search shifting from the desktop to other types of computing devices, I formulated several questions:
- Are vendors deploying search on clouds similar to Amazon’s system and method ensuring the security of their customers’ data? Open source vendors like resellers of Elastic and proprietary vendors like MarkLogic are likely to be giving some additional thought to the security of their customers’ data.
- Are licensees of cloud based search systems performing security reviews as we did when we implemented the Blossom search system? I am not sure if the responsibility for this security review rests with the vendor, the licensee, or a third party contracted to perform the work.
- How secure are hybrid systems; that is, an enterprise search or content processing system which pulls, processes, and stores customer data across disparate systems? Google, based on my experience, does a good job of handling search security for the Google Search Appliance and for Site Search. Other vendors may be taking similar steps, but the information is not presented with basic marketing information.
My view is that certain types of enterprise search may benefit from a cloud based solution. There will be other situations in which the licensee has a contractual or regulatory obligation to maintain indexes and content in systems which minimize the likelihood that alarmist headlines like “Billions of Records at Risk from Mobile App Data Flow.”
Security is the search industry’s industry of a topic which is moving up to number one with a “bullet.”
Stephen E Arnold, June 19, 2015
Enterprise Search: The Last Half of 2015
June 16, 2015
I saw a link this morning to an 11 month old report from an azure chip consulting firm. You know, azure chip. Not a Bain, BCG, Booz Allen, or McKinsey which are blue chip firms. A mid tier outfit. Business at the Boozer is booming is the word from O’Hare Airport, but who knows if airport gossip is valid.
Which enterprise search vendor will come up a winner in December 2015?
What is possibly semi valid are analyses of enterprise search vendors. The “Magic Quadrant for Enterprise Search” triggered some fond memories of the good old days in 2003 when the leaders in enterprise search were brands or almost brands. You probably recall the thrilling days of these information retrieval leaders:
- Autonomy, the math oriented outfit with components names like neuro linguistic programming and integrated data operating layer and some really big name customers like BAE
- Convera, formerly Excalibur with juice from ConQuest (developer by a former Booz, Allen person no less)
- Endeca, the all time champ for computationally intensive indexing
- Fast Search & Transfer, the outfit that dumped Web search in order to take over the enterprise search sector
- Verity, ah, truth be told, this puppy’s architecture ensured plenty of time to dash off and grab a can of Mountain Dew.
In 2014, if the azure chip firm’s analysis is on the money, the landscape was very different. If I understand the non analytic version of Boston Consulting Group’s matrix from 1970, the big players are:
- Attivio, another business intelligence solution using open source technology and polymorphic positioning for the folks who have pumped more than $35 million into the company. One executive told me via LinkedIn, that the SEC investigation of an Attivio board member had zero impact on the company. I like the attitude. Bold.
- BA Insight, a business software vendor focused on making SharePoint somewhat useful and some investors with deepening worry lines
- Coveo, a start up which is nudging close to a decade in age, and more than $30 million in venture backing. I wonder if those stakeholders are getting nervous.
- Dassault Systèmes, the owner of Exalead, who said in the most recent quarterly report that the company was happy, happy, happy with Exalead but provided no numbers and no detail about the once promising technology
- Expert System, an interesting company with a name that makes online research pretty darned challenging
- Google, ah, yes, the proud marketer of the ever thrilling Google Search Appliance, a product with customer support to make American Airlines jealous
- Hewlett Packard Autonomy, now a leader in the acrimonious litigation field
- IBM, ah, yes, the cognitive computing bunch from Armonk. IBM search is definitely a product that is on everyone’s lips because the major output of the Watson group is a book of recipes
- IHS, an outfit which is banking on its patent analysis technology to generate big bucks in the Goldmine cellophane
- LucidWorks (Really?), a repackager of open source search and a distant second to Elastic (formerly Elasticsearch, which did not make the list. Darned amazing to me.)
- MarkLogic, a data management system trying to grow with a proprietary XML technology that is presented as search, business intelligence, and a tool for running a restaurant menu generation system. Will MarkLogic buy Smartlogic? Do two logics make a rational decision?
- Mindbreeze, a side project at Fabasoft which is the darling of the Austrian government and frustrated European SharePoint managers
- Perceptive Software, which is Lexmark’s packaging of ISYS Search Software. ISYS incorporates technology from – what did the founder tell me in 2009? – oh, right, code from the 1980s. Might it not be tough to make big bucks on this code base? I have 70 or 80 million ideas about the business challenge such a deal poses
- PolySpot, like Sinequa, a French company which does infrastructure, information access, and, of course, customer support
- Recommind, a legal search system which has delivered a down market variation of the Autonomy-type approach to indexing. The company is spreading its wings and tackling enterprise search.
- Sinequa, another one of those quirky French companies which are more flexible than a leotard for an out of work acrobat
But this line up from the azure chip consulting omits some companies which may be important to those looking for search solutions but not so much for azure chip consultants angling for retainer engagements. Let me highlight some vendors the azure chip crowd elected to ignore:
Paper.Li Enterprise Search Punts
June 15, 2015
Short honk: I monitor the automated “newsletter” called The Enterprise Search Daily. I am not sure how one receives this publication, but I use this url. In the last few days, there has been minimal—maybe zero—enterprise search news. The publication appears to recycle information about Big Data and text analytics. We will continue to report on the search flounders, oops, I mean, search vendors who offer enterprise search solutions. The problem is that venture backed enterprise search start ups will have to do some fancy dancing to explain why a search for enterprise search brings up items like this:
The Beyond Search team will soldier on with one comment: Enterprise search does not do Big
Data without some careful wordsmithing.
Stephen E Arnold, June 15, 2015
Is Time Running Out for Non Performing, Venture Funded Enterprise Search Vendors?
June 12, 2015
Gee, impatient venture capital firms, grousing partners hungry for a payday, and agitated stakeholders, are these usually cheerful folks worrying about getting their money back with a hefty profit? My hunch is that some who wrote checks might be thinking about a vacation at WalMart instead of a couple of weeks bouncing around Europe or looking at animals in Africa from a Land Rover.
Navigate to “Something Is Rotting under Silicon Valley.” The point of the write up is that the sunshine and unicorn crowd may be getting nervous. The write up points out:
Only seven VC-backed tech companies have gone public so far this year, with just one more (Fitbit) currently on the pricing calendar for June. At this rate, 2015 could go down as the slowest year for VC-backed tech IPOs since the throes of the financial crisis. Moreover, there have been only two strategic sales of VC-backed tech companies valued at over $1 billion (Lynda.com to LinkedIn and Virtustream to EMC).
Forget training (online and walking around the parking lot). The worry may be that some outfits which have sucked in tens of millions of dollars may have — gasp! — liquidity issues and a downward valuation.
The article states:
A less charitable rationale is that too few of these companies [VC bets] have imposed the tough internal discipline — particularly in terms of burn rate — that public equity investors demand. Either way, limited partners in VC funds aren’t getting paid.
What about search? With the implosion of the proprietary search sector and the vaporization of substantive news reports about bubbling sales and profits, the enterprise search sector looks like a The Man with No Man desert scene. The LinkedIn enterprise search groups are somewhat low key. Hello, is anyone there? The automated Paper.Li enterprise search paper is stuffed with information about Big Data. More importantly, Attivio has pivoted… again. Coveo whips the customer support thing. Lucidworks is promising to have a mission. The European enterprise search vendors are making little noise. When did you hear about Exalead, Intrafind, or the stub of Fast deep in the folds of Microsoftland?
My hunch is that this Fortune Magazine article has identified what may be a “Houston, we have a problem” moment for venture funded search vendors. Is there a fix? Nah, just use Elastic or another open source solution. Good news for those who need utility search. Bad news for the bankish MBAs who bet that certain investments would just spin cash.
Here’s another passage I noted in the article:
If I’m a venture capitalist, it might be time to stop staring at the sun and take a peek at the darkening clouds.
Yep, might be time to check out the actual weather, not the pretend environment in those PowerPoints.
Stephen E Arnold, June 12, 2015
An Enterprise Search Mini Case: LucidWorks and Its Accelerating Mission
June 11, 2015
I read “Lucidworks Accelerates Product Focused Mission with Major Fusion Upgrades.” LucidWorks (Really?)—né Lucid Imagination—appears to be working on products. (Note that the company names appears in different ways: “Lucidworks” with variants “LucidWorks”, “Lucid Works,” and “lucidworks”.)
Lucidworks wants to accelerate its mission. Will this be a quick and easy task?
Flashback in time. Lucid Imagination was founded in 2007. You can read about the vision of the company in interviews with these Lucid (no pun intended) executives:
- Marc Krellenstein, formerly Northern Light and one of the founders of Lucid Imagination, March 17, 2009
- Brian Pinkerton, formerly, December 21, 2010, possibly Amazon?
- Paul Doscher, formerly with Exalead, April 16, 2012
- Miles Kehoe, formerly New Idea Engineering, January 29, 2013, now a consultant
- Mark Bennett, formerly New Idea Engineering, March 4, 2014
These interviews make clear the difficult journey that Lucid Imagination took. (What is interesting is that Lucid’s principal competitor was Elasticsearch, now Elastic. That company came from obscurity to the go-to provider of open source search. To be fair, Shay Bannon, founder of Elastic, had compiled considerable experience with the Compass open source search system.)
Why did I cover Lucid in five interviews?
The reason is that open source search appeared to be the salve to soothe the wounds inflicted by proprietary search system vendors. Satisfaction with search was declining. Users were disaffected with high profile proprietary brands. The community approach addressed, in part, the brutal research, development, and customer support costs which search drags to each meeting with stakeholders.
Lucid had a lead; Elastic benefited. Lucid seeks a focus; Elastic is serving customers. Lucid would be an excellent business school case study, ranking at the top along with the Hewlett Packard Autonomy search situation and the Fast Search criminal charges matter. That is rarified case study company.
In the interviews cited above, it is clear that Lucid embraced Solr and made an attempt to emulate the full featured approach to content processing exemplified by Autonomy and Fast Search & Transfer. Elastic, on the other hand, took a more direct approach, relying on Lucene for the heavy lifting, and narrowing its focus to tools which were almost utilitarian. Want to search a log file? Go with Elastic.
The other key difference is the lack of managerial drama at Elastic. Elastic’s management team appears, at least to this observer in Kentucky, as stable. Lucid, on the other hand, has seen the departure of founders early in the company’s history. Presidents arrived and departed. Marketers appeared and disappeared. Major committers joined and then jumped ship; for example, Brian Pinkerton ended up at Amazon, working on its search product. Yonik Seeley also left to start his own search company Heliosearch. Dr. Krellenstein went from strong supported of Lucid to a disaffected founded. He quit.
As recently as September 2014, Lucid Works featured in “Trouble at LucidWorks: Lawsuits, Lost Deals, and Layoffs Plague the Search Startup Despite Funding.” The headline makes several points. First, LucidWorks has ingested more than $40 million, which puts it on a par with Attivio and Coveo in the money department. But Elastic garnered about $70 million at about the same time. The headline also reveals the disjunctions among managers, regardless of which president was on watch. And, the headline focuses on the point that it is a search vendor, which is not in my opinion a particularly magnetic positioning for software.
According to the “Trouble at LucidWorks” article The Guardian and Nordstrom’s abandoned Lucid’s software. The less than flattering Venture Beat story added:
The situation seems to have worsened following shakeups in the sales team, leaving young salespeople inexperienced in the enterprise-software game trying to win deals. “I don’t think any of the sales team hits (their) number except one guy,” said a former employee. And that one guy has resorted to “dropping his pants,” as the sales expression goes, promising to significantly chop the price of a service if his lead commits to buying right away, a different former employee said. The sales goals aren’t increasing. The revenue target for the year is $12 million, right in line with last year, that former employee said. And it doesn’t help that LucidWorks has fumbled with partnerships it was trying to get in place. It was working on alliances with Amazon Web Services, Intel, and Splunk, one former employee told VentureBeat. “Will [Hayes] imploded that with comments he made in the final agreement,” that former employee said of one partnership. And after Hayes stepped up as chief executive in June, he’s laid off people in marketing, sales, and business development. On the technology side of the company, meanwhile, employees have missed deadlines for shipping software to customers, month after month, another former employee said. Outside the office, the company has other distractions — in court, to be exact. Mike Moody, a former senior vice president of engineering at LucidWorks who was terminated in December, sued LucidWorks and certain executives in February for unlawful termination, according to documents submitted to the U.S. District Court for the Northern District of California. LucidWorks is also ensnared in a case it filed against Seeley, one of its founders, in the Superior Court of California, San Mateo County. “This is a case about double-dealing on an employer, which arises from the secretive founding and launching of the company Heliosearch by Yonik Seeley before his resignation from his former employer LucidWorks in October 2013,” the complaint begins. “Unknown to LucidWorks, while Seeley was still employed by LucidWorks, he simultaneously was working directly against LucidWorks’ interests by developing and promoting his new venture Heliosearch as a competing alternative to LucidWorks.”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
