Enterprise Search Is a Growth Industry: No, Really
October 16, 2015
I noticed two things when we were working through the Overflight news about proprietary vendors of enterprise search systems on October 14, 2015.
First, a number of enterprise search vendors which the Overflight system monitors, are not producing substantive news. Aerotext, Dieselpoint, and even Polyspot are just three firms with no buzz in social media or in traditional public relations channels. Either these outfits are so busy that the marketers have no time to disseminate information or there is not too much to report.
Second, no proprietary enterprise search vendor is marketing search and retrieval in the way Autonomy and the now defunct Convera used to market. There were ads, news releases, and conference presentations. Now specialist vendors talk about webinars, business intelligence, Big Data, and customer support solutions. These outfits are mostly selling consulting firms. Enterprise search as a concept is not generating much buzz based on the Overflight data.
Imagine my surprise when I read “Enterprise Search Market Expanding at a 12.2% CAGR by 2019.” What a delicious counterpoint to the effective squishing of the market sector which husbanded the Autonomy and Fast Search & Transfer brouhahas. These high profile enterprise search vendors found themselves mired in legal hassles. In fact, the attention given to these once high profile search vendors has made it difficult for today’s vendors to enjoy the apparent success that Autonomy and Fast Search enjoyed prior to their highly publicized challenges.
Open source search solutions have become the popular and rational solution to information access. Companies offering Lucene, Solr, and other non proprietary information access systems have made it difficult for vendors of proprietary solutions to generate Autonomy-scale revenue. The money seems to be in consulting and add ons. The Microsoft SharePoint system supports a hot house of third party components which improve the SharePoint experience. The problem is that none of the add in and component vendors are likely to reach Endeca-scale revenues.
Even IBM with its Watson play seems to be struggling to craft a sustainable, big money revenue stream. Scratch the surface of Watson and you have an open source system complemented with home brew code and technology from acquired companies.
The write up reporting the double digit comp9ound growth rate states:
According to a recent market study published by Transparency Market Research (TMR), titled “Enterprise Search Market – Global Industry Analysis, Size, Share, Growth, Trends and Forecast 2013 – 2019”, the global enterprise search market is expected to reach US$3,993.7 million by 2019, increasing from US$1,777.5 million in 2012 and expanding at a 12.2% CAGR from 2013 to 2019. Enterprise search system makes content from databases, intranets, data management systems, email, and other sources searchable. Such systems enhance the productivity and efficiency of business processes and can save as much as 30% of the time spent by employees searching information.The need to obtain relevant information quickly and the availability of technological applications to obtain it are the main factors set to drive the global enterprise search market.
TMR, like other mid tier consulting firms, will sell some reports to enterprise search vendors who need some good news about the future of the market for their products.
The write up also contains a passage which I found quite remarkable:
To capitalize on opportunities present in the European regional markets, major market players in the U.S. are tying up with European vendors to provide enterprise search solutions.
Interesting. I do not agree. I don’t see to many US outfits tying up with Antidot, Intrafind, or Sinequa and their compatriots. Folks are using Elasticsearch, but I don’t categorize these relationships as tie ups like the no cash merger between Lexalytics and its European partner.
Furthermore, we have the Overflight data and evidence that enterprise search is a utility function increasingly dominated by open source options and niche players. Where are the big brands of a decade ago: Acquired, out of business, discredited, and adorned with jargon.
The problems include sustainable revenue, the on going costs of customer support, and the appeal of open source solutions.
Transparency Market Research seems to know more than I do about enterprise search and its growth rate. That’s good. Positive. Happy.
Stephen E Arnold, October 16, 2015
Yammer Improvements and Changes on the Horizon
August 27, 2015
A few years ago, Yammer was an integral part of SharePoint’s marketing campaign as they sought to persuade users that they were moving toward a focus on social. With the upcoming release of SharePoint 2016, social is still important, although it feels less forced and more natural this time around. There will be changes to Yammer and Redmond Magazine covers it in their article, “Microsoft Announces Yammer Improvements To Come While Deprecating Some Yammer SharePoint Apps.”
The article says:
“Microsoft announced this week that it is working on a more team-oriented Yammer, and it will be bringing along some mobile app improvements, too. Yammer is Microsoft’s enterprise-grade social networking application that’s part of some Office 365 subscription plans. Yammer can be used as a standalone service, but it’s also used with SharePoint Server products and SharePoint Online implementations.”
To stay current on what else may change with the release of SharePoint Server 2016, stay tuned to ArnoldIT.com. Stephen E. Arnold is an expert on search and the enterprise. His dedicated SharePoint feed is a great way to stay up to date on the latest new surrounding SharePoint.
Emily Rae Aldridge, August 27, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
More Enterprise Search Revisionism: Omitted Companies Are the Major News
August 24, 2015
A flurry of news items hit my Overflight system in the last couple of days. Gartner, one of the expert for hire mid tier consulting firms, issued a “Gartner’s Magic Quadrant for Enterprise Search.” I am not sure if you can access the report. I had to log in to LinkedIn and work through various screens until this gem presented itself to me.
I followed the link and learned that the “Magic Quadrant for 2015” includes these firms:
The Challengers. To me a challenger means a person or thing that engages in any contest, as of skill, strength, etc.
- LucidWorks, founded in 2007
- Mindbreeze, a unit of Microsoft centric Fabasoft in Austria. The search unit fired up a decade ago
- Google, ah, dear old Google and its pricey Google Search Appliances. You can find the license fees for some devices via the GSAAdvantage service. Google has been sort of selling GSAs for a decade.
- Dassault Systems, yep, the French engineering outfit working to convert Exalead’s ageing technology into a product component solution. Exalead dates from 2000. Yikes, that makes the technology 15 years old, an aeon in technology time.
The good news is that LucidWorks has its roots in open source. The other three outfits are proprietary technology.
The second group is Niche Players. The companies in this sector are:
- Expert System. An outfit which opened its doors in 1992 and whose stock is publicly traded. The share price on August 23, 2015 was $2.13 a share
- Recommind, founded in year 2000, is a legal system whose technical approach often reminds me of Autonomy’s systems and methods. The firm was founded in 2000 and now, according to this story, has $70 million in revenue
- Squiz, which is, by golly, not an open source solution despite its origins in the 2001 P@noptic academic/research setting in Australia. Just try searching for that spelling “P@noptic.”
The third group is Visionaries which to me means “given to or characterized by fanciful, not presently workable, or unpractical ideas, views, or schemes.” The dictionary entry here also points out these clarifications: unreal, imaginary, idealistic, impractical, and unrealizable. Here are the search outfits in this category:
- BA Insight. This is an company founded in 2004. The founder raised some venture money and then found himself looking for his future elsewhere. In the presentations I have heard, BA Insight is [a] an enterprise search system replacement for whatever you have running, [b] a business intelligence system, [c] a metatagging machine, [d] some combination of these functions.,
- IBM. Ah, dear, old IBM. The company does the home grown thing with scripts and algorithms from its research labs. IBM was founded in It does the open source thing by building in 1911. The company has had a long time to figure out what to do since the STAIRS III and Web Fountain days. Now IBM search means use of open source, community supported, free Lucene. Plus, It does the acquisition thing with SPSS Clementine (remember than, gentle reader), Vivisimo, i2, and Cybertap, among other information access companies IBM has purchased. At the end of the day, I am not sure what search means because IBM has been promoting the heck out of Watson. You remember Watson. It was a TV game show winner. Watson wrote a cook book. Watson is curing cancer. Watson is doing all sorts of wonderful things. I suppose that’s why it is a visionary with 13 consecutive quarters of revenue decline.
- IHS (Information Handling Service. IHS leverages technology from The Invention Machine (founded in 1992) an acquisition built to locate systems and methods from patent documents. The IHS search system is called Goldfire and positioned as an enterprise search system. IHS, according to Attivio, licenses the Fast Search & Transfer influenced UIA technology platform. IHS for me is a publishing company, but I suppose that doesn’t matter in today’s fluid world.
The final group of search vendors is labeled leaders. So what’s a leader? According to my online dictionary, a leader is a person or thing that leads. And “lead” means to go before or with to show the way; conduct or escort. No, I will not refer to Ashley Madison, gentle reader. I will play this straight. The leaders are:
- Attivio, founded in 2007. It must be a leader because a “visionary” uses the Attivio technology to be a visionary. Is that self referential like articles about Google’s right to be forgotten which must be forgotten?
- Coveo, founded in 2004. This company has been, like Attivio, successful in attracting venture capital.The company once focused on Microsoft Windows as did BA Insight. Now the firm is into customer support but the mid tier consultants remember the good old days of enterprise search.
- Hewlett Packard. Ah, HP, the company wrote a check for $11 billion in 2011, promptly wrote off billions, and embarked on a much loved legal challenge to Dr. Michael Lynch and some other favored individuals. HP, like IBM, has been racking up declining revenues for five consecutive quarters and is in the process of dividing itself into two separate companies. Does this suggest that HP some challenges? Keep in mind Autonomy was founded in the mid 1990s.
- Lexmark. This is a relative newcomer to enterprise search. The company bought Brainware of trigram fame. Lexmark bought the 1980s search darling ISYS Search Software, which was founded in 1988. The company also snagged Kofax, which got into the content processing game with its acquisition of Kapow. I did hear that Lexmark is looking at some shortfalls related to search and content processing. I reported on the chopping of 500 jobs a couple of months ago. But leaders must expect some setbacks like Hewlett Packard. Perhaps Lexmark will reveal the shortfall from its “search related” endeavors. I would peg the number somewhere in the $75 to $80 million range in the last 18 months.
- Sinequa. This marketing centric, social media maven was founded in 2002. The company has some big European clients, but I am not certain that the push into the US has met with the “name in lights” success some French stakeholders expected. Sinequa is obviously a leader in search. I classify the company as a business process outfit, but the mid tier consultants are more informed than an old guy in rural Kentucky.
My view of the enterprise search sector is different.The companies in this list are oldies, a couple dating from the late 1980s and early 1990s. Let’s see. In Internet time, that pegs some technology as prehistoric.,
There is a notable omission too. The list of companies identified by the mid tier outfit has missed the company which has been driving a bulldozer through deals.
What company is that?
Elastic, gentle reader. This outfit is in the process of providing the folks at Goldman Sachs with some information access love. The company has shoved aside the Lucid Works outfit which is scrambling to reposition itself as a Big Data spark something. There are cloud versions of Elastic available for a darned reasonable price. Check out SearchBlox, for example. Keep in mind that Elasticsearch was a second act to Compass, another search system.
A question which I asked myself is, “Why has a mid tier outfit which is so darned expert in enterprise search overlooked the big dog?” Frankly I have no evidence other than the odd little grid in the Linked In post. I assume that the experts at the mid tier firms don’t know much about what’s happening in search. Another thought is that the Elastic folks don’t buy much third party expert input about search. Whatever the reason, I suggest you, gentle reader, become familiar with Elasticsearch in the free or for fee variant.
Another gap I noticed is the omission of the appliance folks. Right off the bat, I think Index Engines, Maxxcat, and Thunderstone deserve a tiny footnote. Maxxcat, for example, is pretty good in the enterprise content indexing arena. Buy a box and plug it in. Index Engines does a great job making some specialized content instantly accessible. And Thunderstone? Well, the company has some darned good technology.
A third lacuna is the omission of the wild and crazy, Fast Search & Transfer tinged SharePoint search. There are upwards of 150 million SharePoint installations. Like it or not, Microsoft also shoves search down my throat each time I use Windows 10. Yikes. The system may have a legacy of considerable interest, but the darned thing is out there. Maybe a teeny tiny footnote? I would suggest that the mid tier outfit identify the vendor which sells more search into Microsoft installations than any other vendor. Nope. I won’t identify this outfit. The president agreed to a Search Wizards Speak interview and then backed out. Too bad for him. No life preserver from me again.
What’s the value of this league table or grid thing from the mid tier consulting firm.
First, it allows the companies in the list to issue a news release. I have already seen references to some of the companies. This post was inspired by the junk mail Linked In shoots at me on a regular basis. There’s nothing like PR which gets a company’s name in front of a bunch of red hot prospects.
Second, the mid tier consulting firm can visit with each company. I can imagine that on those visits, the mid tier consulting firm might just mention the firm’s strategic and tactical for fee services. Hey, if I worked for a mid tier consulting firm, I would be sure to explain why retaining me was the best darned thing since sliced bread. Oh, wait. I worked at Booz, Allen & Hamilton before it drifted into Snowden drifts. I responded to requests; I don’t recall making sales calls. Life is different now I suppose.
Third, the mid tier reports practically force me to write blog posts. I am delighted to be spurred into action.
Fourth, how much does it cost to use these systems? Why not make a table which presents the name of the company, the search system name so that I know what IBM asserts actually performs enterprise search and what HP calls its cloud stuff with Autonomy made ever so easy? Why not states that such and such a search system begins at $X for the license fee and $Y for the on going support, upgrades, and maintenance? Why not present average hourly engineering and technical service fees? Hey, even the best of this animal shelter of disparate systems fail. Did I say crash? Did I say flame out? Did I say deliver irrelevant results? Well, often in my experience.
To wrap up, the Visionaries, the Challengers, the Leaders, and the Niche Players can output news releases. Some my try to dismiss my observations, which is just peachy keen with me. I assume that failed webmasters, thwarted academicians, and unemployed home economics majors will explain that the best of the best appear in the league table.
Present reality any way one wants. I don’t have to make this stuff work anymore. I don’t have to explain to the CFO why the costs associated with enterprise search will continue to go up until the system is removed from the company. I will no longer have to attend a conference filled with cheerleaders for a utilitarian technology which most companies have learned is pretty much the same as it has been since the days of Fulcrum and Verity.
Remember. This is 2015. Most of the technology presented in the mid tier report is getting old. The world wants mobile. The world wants predictive outputs. The world wants search which actually delivers relevant results.
Maybe that is secondary today?
Will I read the complete report if a copy becomes available to me?
Nah. Marketing stuff bores me.
Stephen E Arnold, August 24, 2015
Organizations Should Consider Office 365 Utilization
July 30, 2015
Office 365 has been a bit contentious within the community. While Microsoft touts it as a solution that gives users more of the social and mobile components they were wishing for, it has not been widely adopted. IT Web gives some reasons to consider the upgrade in its article, “Why You Should Migrate SharePoint to Office 365.”
The article says:
“Although SharePoint as a technology has matured a great deal over the years, I still see many businesses struggling with issues related to on-premises SharePoint, says Simon Hepburn, director of bSOLVe . . . You may be thinking: ‘Are things really that different using SharePoint on Office 365?’ Office 365 is constantly evolving and as I will explain, this evolution brings with it opportunities that your business should seriously consider exploring.’”
Of course the irony is that with the new SharePoint 2016 upgrade, Microsoft is giving users a promise to stand behind on-premise installations, but they are continuing to integrate and promote the Office 365 components. Only time and feedback will dictate the continued direction of the enterprise solution. In the meantime, stay tuned to Stephen E. Arnold and his Web service, ArnoldIT.com. Arnold is a longtime leader in search and his dedicated SharePoint feed is a one-stop-shop for all the latest news, tips, and tricks.
Emily Rae Aldridge, July 30, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Enterprise Search: The Last Half of 2015
June 16, 2015
I saw a link this morning to an 11 month old report from an azure chip consulting firm. You know, azure chip. Not a Bain, BCG, Booz Allen, or McKinsey which are blue chip firms. A mid tier outfit. Business at the Boozer is booming is the word from O’Hare Airport, but who knows if airport gossip is valid.
Which enterprise search vendor will come up a winner in December 2015?
What is possibly semi valid are analyses of enterprise search vendors. The “Magic Quadrant for Enterprise Search” triggered some fond memories of the good old days in 2003 when the leaders in enterprise search were brands or almost brands. You probably recall the thrilling days of these information retrieval leaders:
- Autonomy, the math oriented outfit with components names like neuro linguistic programming and integrated data operating layer and some really big name customers like BAE
- Convera, formerly Excalibur with juice from ConQuest (developer by a former Booz, Allen person no less)
- Endeca, the all time champ for computationally intensive indexing
- Fast Search & Transfer, the outfit that dumped Web search in order to take over the enterprise search sector
- Verity, ah, truth be told, this puppy’s architecture ensured plenty of time to dash off and grab a can of Mountain Dew.
In 2014, if the azure chip firm’s analysis is on the money, the landscape was very different. If I understand the non analytic version of Boston Consulting Group’s matrix from 1970, the big players are:
- Attivio, another business intelligence solution using open source technology and polymorphic positioning for the folks who have pumped more than $35 million into the company. One executive told me via LinkedIn, that the SEC investigation of an Attivio board member had zero impact on the company. I like the attitude. Bold.
- BA Insight, a business software vendor focused on making SharePoint somewhat useful and some investors with deepening worry lines
- Coveo, a start up which is nudging close to a decade in age, and more than $30 million in venture backing. I wonder if those stakeholders are getting nervous.
- Dassault Systèmes, the owner of Exalead, who said in the most recent quarterly report that the company was happy, happy, happy with Exalead but provided no numbers and no detail about the once promising technology
- Expert System, an interesting company with a name that makes online research pretty darned challenging
- Google, ah, yes, the proud marketer of the ever thrilling Google Search Appliance, a product with customer support to make American Airlines jealous
- Hewlett Packard Autonomy, now a leader in the acrimonious litigation field
- IBM, ah, yes, the cognitive computing bunch from Armonk. IBM search is definitely a product that is on everyone’s lips because the major output of the Watson group is a book of recipes
- IHS, an outfit which is banking on its patent analysis technology to generate big bucks in the Goldmine cellophane
- LucidWorks (Really?), a repackager of open source search and a distant second to Elastic (formerly Elasticsearch, which did not make the list. Darned amazing to me.)
- MarkLogic, a data management system trying to grow with a proprietary XML technology that is presented as search, business intelligence, and a tool for running a restaurant menu generation system. Will MarkLogic buy Smartlogic? Do two logics make a rational decision?
- Mindbreeze, a side project at Fabasoft which is the darling of the Austrian government and frustrated European SharePoint managers
- Perceptive Software, which is Lexmark’s packaging of ISYS Search Software. ISYS incorporates technology from – what did the founder tell me in 2009? – oh, right, code from the 1980s. Might it not be tough to make big bucks on this code base? I have 70 or 80 million ideas about the business challenge such a deal poses
- PolySpot, like Sinequa, a French company which does infrastructure, information access, and, of course, customer support
- Recommind, a legal search system which has delivered a down market variation of the Autonomy-type approach to indexing. The company is spreading its wings and tackling enterprise search.
- Sinequa, another one of those quirky French companies which are more flexible than a leotard for an out of work acrobat
But this line up from the azure chip consulting omits some companies which may be important to those looking for search solutions but not so much for azure chip consultants angling for retainer engagements. Let me highlight some vendors the azure chip crowd elected to ignore:
Jury Is Still Out on Microsoft Delve
June 11, 2015
Sometimes hailed as Pinterest for the enterprise, Microsoft Delve is a combination of search, social, and machine learning, which produces an information hub of sorts. Delve is also becoming a test subject, as enterprise experts decide whether such offerings intrude into users’ workflow, or enhance productivity. Read more in the Search Content Management article, “Microsoft Delve May Drive Demand for Office365.”
The article summarizes the issue:
“As Microsoft advances further in its mobile-first, cloud-first strategy, new offerings such as Microsoft Delve are piquing companies’ curiosity but also raising eyebrows. Many companies will have to gauge whether services like Delve can enhance worker productivity or run the risk of being overly intrusive.”
As SharePoint unveils more about its SharePoint Server 2016, more will become known about how it functions along with all of its parts, including Delve. It will be up to the users to determine how efficient the new offerings will be, and whether they help or hinder a regular workflow. Until the latest versions become available for public release, stay tuned to ArnoldIT.com for the latest news regarding SharePoint and how it may affect your organization. Stephen E. Arnold is a longtime leader in search and his work on SharePoint is a great go-to resource for users and managers alike.
Emily Rae Aldridge, June 11, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Microsoft Goes Mobile with Delve
April 30, 2015
Microsoft has made enhancements to the core functionality of Delve, as well as rolling out native mobile app versions for iOS and Android. ZDNet breaks the news in their article, “Microsoft Delivers iOS, Android Versions of Delve.”
The article begins:
“Microsoft has made native mobile versions of its Delve search and presentation app available for Android phones, Android wear devices and iPhones. Delve presents in card-like form information from Exchange, OneDrive for Business, SharePoint Online and Yammer enterprise-social networking components. Over the coming months Delve will be adding more content sources, including email attachments, OneNote and Skype for Business.”
This seems like a Microsoft component that has great potential for mobile use, since its focus is “at a glance” information retrieval. Keep an eye on ArnoldIT.com to see what Stephen E. Arnold has to say about it in coming months. Arnold has made a career out of following all things search and enterprise, and he reports his findings at ArnoldIT.com. His dedicated SharePoint feed collects a lot of interesting reporting regarding SharePoint and the rest of Microsoft productivity offerings.
Emily Rae Aldridge, April 30, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Intranet Connections and Super Search Version 13
April 2, 2015
I read “Maximize Productivity with Super Search from Intranet Connections (Version 13.0 Release).”
For decades I have been gathering information about enterprise search and content processing. The name of the company was not familiar to me. The assertions in the news article were, however.
Puzzled, I went through my archive of search vendor information and did not find content about Intranet Connections. I noted the date on this article and wondered if the company were an April Fool’s spoof. I know I am getting on in years, but when I wake up and plop in front of my primitive, coal-fired computer, my memory works reasonably well. I know my name and the day of the week.
The write touts an enterprise search system that includes:
- New Search Engine
- Preview Display Cards
- Intuitive Search Filters
- Advanced Search Options
- Controlled Search Security.
I know from the years of experience I have logged examining, testing, and creating search and content processing systems like the one we sold to the long ago Lycos, that “new” is a slippery concept. For some folks, learning about Google’s site operator is a new thing. For others it is a reminder of the many useful search functions that Google no longer exposes to the ad consumers looking for objective information via Google.com.
I am not sure how often a search system innovates across 13 versions. Intranet Connections seems to have been founded in 1999, which makes the company 16 years young. Most of the long lived search engines don’t change too much from the original core; for example, Autonomy IDOL. On the other hand, other companies just discard a search system and graft in open source Lucene and slap on the “new” label. Others take inspiration from Fast Search and call it new.
The company states on its LinkedIn page here:
Intranet Connections is a business intranet software solution that enables organizations to connect, collaborate and create more efficiently yielding significant time-cost savings and stronger employee engagement. We combine key business tools to automate workflows and processes, while delivering improved communication and collaboration among employees to engage and promote culture within the digital workplace.
How new is Version 13 of a search system. I learned from the write up:
“Enterprise intranet search functionality is proving to be more critical than ever as today’s mature intranets face thousands of data entries, pages, forms, and uploaded corporate documents and policies. Employees’ expectations are higher than ever to deliver on an intranet search utility that is fast, focused, intelligent and super simple. We wanted to introduce intranet search that is not just functional but an entire experience.” Douglas also reports that Super Search was a result of close collaboration with Intranet Connections’ customers who were active in feedback for the design and feature set capabilities, ensuring the product release would enhance their needs for enterprise intranet search.
And adds that it can deliver software capable of “triggering emotions on the Intranet using Intranet design such as theme, videos, and photos.” Furthermore, the system seems to be able to marry an “Intranet and an enterprise social network.”
These are significant assertions.
Okay, that sounds great but we are in Version 13, not Version 1.1. The new version was announced in August 2014. In “Intranet Search Designed for Maximum Productivity” I learned:
The first thing customers will notice is the completely redesigned user interface. It is really geared towards making search simple and fast as possible for the average user. We also introduced “one-click filtering”. If a user knows they are looking for a document, a form, or a person, they have the option to filter their results with a single click. This automatically removes search results that aren’t in the specified category. More advanced search options are available for power users, but are hidden by default. These users can choose to filter their results by specific sites, application, modified date, author, or tags. We also introduced the content of feature cards. Most of the time, users can determine if a search result is what they are looking for by the title, category, or short description. However, if there are multiple documents that are similar, a little bit more information may be necessary. Instead of requiring the user to click into the item to view more details, and navigate away from search, we introduced the concept of a feature card. Additional summary information for a search result can be displayed within the search screen, preventing the need to jump back and forth from search and content.
Intrigued by my Overflight systems lack of information about Super Search, I visited the company’s Web site. I learned that the system begins at $15,000, which strikes me as a bargain. Low cost search systems often face significant financial demands as the company struggles to keep pace with the needs of customers, support demands, and the inevitable tweaks that are needed to deal with the wild and crazy nature of behind-the-firewall content.
At www.IntranetConnections.com I learned that the company makes “Intranet software made for you.” I assume that means me. I do have a 2.5 million test corpus which has been known to take days of indexing. One German company promised speedy performance, and I had to leave the system on for five days before I could run a test query. The initial crawl failed because this particularly German, Lucene based system choked on Microsoft’s file locks. Yep, every Microsoft system has these types of files. I wondered, “Yo, why not provide some tools to deal with this like a !readme.txt file.”
Back to Intranet Connections.
The company delivers what I think of as a one-stop, 7-11 solution. The Web site highlights a people directory, forms, document management, Intranet Web sites, but not search. After I scrolled through information about the corporate Intranet, the finance Intranet, and the healthcare Intranet. But no direct link to Super Search.
I used my tools to examine the site and located a blog post about Super Search in an article labeled “Super Search Launch [sic] Scavenger Hunt.” There was a phrase about Super Search, “And much more”. But there was no link. I did locate a link to a story with the title “Super Search (V13.0) dated January 27, 2015. That page did provide links to a feature guide, a support page, a webinar recording (Does anyone have webinar fatigue as I do?) and a recursive link to the blog. There is also a link to the installation guide. The guide is 300 words long and helpful provides me with a user name and password. The guide also makes clear that I need to be deep into the Microsoft world. Mac and Linux users do not seem to be encouraged. Unlike the German outfit, Intranet Connections provides a link to information necessary to get the search engine working.
It appears that the company offers an alternative to Microsoft SharePoint. The firm, based in Vancouver, has 1,600 customers. Some have a high profile like NASA and the Mayo Clinics. My hunch is that the company has assembled / developed a suite of software. Search is included.
Other observations:
- I struggled with the “new” concept. I mean after 13 years, how “new” is “new.”
- I had to do some poking around to get access to the fact sheet and basic information
- The pricing of $15,000 seems to apply to the full collection of software available from the company
- I have yet to figure out how I managed to know zero about a company with a search system named “super.”
I need to improve my enterprise search information collection. Some help from vendors with more comprehensive and easy-to-find information would be helpful.
Stephen E Arnold, April 2, 2015
Semantic Search Becomes Search Engine Optimization: That Is Going to Improve Relevance
March 27, 2015
I read “The Rapid Evolution of Semantic Search.” It must be my age or the fact that it is cold in Harrod’s Creek, Kentucky, this morning. The write up purports to deliver “an overview of the history of semantic search and what this means for marketers moving forward.” I like that moving forward stuff. It reminds me of Project Runway’s “fashion forward.”
The write up includes a wonky graphic that equates via an arrow Big Data and metadata, volume, smart content, petabytes, data analysis, vast, structured, and framework. Big Data is a cloud with five little arrows pointing down. Does this mean Big Data is pouring from the sky like yesterday’s chilling rain?
The history of the Semantic Web begins in 1998. Let’s see that is 17 years ago. The milestone is in the context of the article, the report “Semantic Web road Map.” I learned that Google was less than a month old. I thought that Google was Backrub and the work on what was named Google begin a couple, maybe three years, earlier. Who cares?
The Big Idea is that the Web is an information space. That sounds good.
Well in 2012, something Big happened. According to the write up Google figured out that 20 percent of its searches were “new.” Aren’t those pesky humans annoying. The article reports:
long tail keywords made up approximately 70 percent of all searches. What this told Google was that users were becoming interested in using their search engine as a tool for answering questions and solving problems, not just looking up facts and finding individual websites. Instead of typing “Los Angeles weather,” people started searching “Los Angeles hourly weather for March 1.” While that’s an extremely simplified explanation, the fact is that Google, Bing, Facebook, and other internet leaders have been working on what Colin Jeavons calls “the silent semantic revolution” for years now. Bing launched Satori, a knowledge storehouse that’s capable of understanding complex relationships between people, things, and entities. Facebook built Knowledge Graph, which reveals additional information about things you search, based on Google’s complex semantic algorithm called Hummingbird.
Yep, a new age dawned. The message in the article is that marketers have a great new opportunity to push their message in front of users. In my book, this is one reason why running a query on any of the ad supported Web search engines returns so much irrelevant information. In my just submitted Information Today column, I report how a query for the phrase “concept searching” returned results littered with a vendor’s marketing hoo-hah.
I did not want information about a vendor. I wanted information about a concept. But, alas, Google knows what I want. I don’t know what I want in the brave new world of search. The article ignores the lack of relevance in results, the dust binning of precision and recall, and the bogus information many search queries generate. Try to find current information about Dark Web onion sites and let me know how helpful the search systems are. In fact, name the top TOR search engines. See how far you get with Bing, Google, and Yandex. (DuckDuckGo and Ixquick seem to be aware of TOS content by the way.)
So semantic in the context of this article boils down to four points:
- Think like an end user. I suppose one should not try to locate an explanation of “concept searching.” I guess Google knows I care about a company with a quite narrow set of technology focused on SharePoint.
- Invest in semantic markup. Okay, that will make sense to the content marketers. What if the system used to generate the content does not support the nifty features of the Semantic Web. OWL, who? RDF what?
- Do social. Okay, that’s useful. Facebook and Twitter are the go to systems for marketing products I assume. Who on Facebook cares about cyber OSINT or GE’s cratering petrochemical business?
- And the keeper, “Don’t forget about standard techniques.” This means search engine optimization. That SEO stuff is designed to make relevance irrelevant. Great idea.
Net net: The write up underscores some of the issues associated with generating buzz for a small business like the ones INC Magazine tries to serve. With write ups like this one about Semantic Search, INC may be confusing their core constituency. Can confused executives close deals and make sense of INC articles? I assume so. I know I cannot.
Stephen E Arnold, March 27, 2015
Enterprise Search Is Important: But Vendor Survey Fails to Make Its Case
March 20, 2015
I read “Concept Searching Survey Shows Enterprise Search Rises in the Ranks of Strategic Applications.” Over the years, I have watched enterprise search vendors impale themselves on their swords. In a few instances, licensees of search technology loosed legal eagles to beat the vendors to the ground. Let me highlight a few of the milestones in enterprise search before commenting on this “survey says, it must be true” news release.
A Simple Question?
What do these companies have in common?
- Autonomy
- Convera
- Fast Search & Transfer?
I know from my decades of work in the information retrieval sector that financial doubts plagued these firms. Autonomy, as you know, is the focal point of on-going litigation over accounting methods, revenue, and its purchase price. Like many high-tech companies, Autonomy achieved significant revenues and caused some financial firms to wonder how Autonomy achieved its hundreds of millions in revenue. There was a report from Cazenove Capital I saw years ago, and it contained analyses that suggested search was not the money machine for the company.
And Convera? After morphing from Excalibur with its acquisition of the manual-indexing ConQuest Technologies, a document scanning with some brute force searching technology morphed into Convera. Convera suggested that it could perform indexing magic on text and video. Intel dived in and so did the NBA. These two deals did not work out and the company fell on hard times. With an investment from Allen & Company, Conquest tried its hand at Web indexing. Finally, stakeholders lost faith and Convera sold off its government sales and folded its tent. (Some of the principals cooked up another search company. This time the former Convera wizards got into the consulting engineering business.) Convera lives on in a sense as part of the Ntent system. Convera lost some money along the way. Lots of money as I recall.
And Fast Search? Microsoft paid $1.2 billion for Fast Search. Now the 1998 technology lives on within Microsoft SharePoint. But Fast Search has the unique distinction of facing both a financial investigation for fancy dancing with its profit and loss statement and the distinction of having its founder facing a jail term. Fast Search ran into trouble when its marketers promised magic from the ESP system. When the pixie dust caused licensees to develop an allergic reaction, Fast ran into trouble. The scrambling caused some managers to flee the floundering Norwegian search ship and found another search company. For those who struggle with Fast Search in its present guise, you understand the issues created by Fast Search’s “sell it today and program it tomorrow” approach.
Is There a Lesson in These Vendors’ Trajectories?
What do these three examples tell us? High flying enterprise search vendors seem to have run into some difficulties. Not surprisingly, the customers of these companies are often wary of enterprise search. Perhaps that is the reason so many enterprise search vendors do not use the words “enterprise search”, preferring euphemisms like customer support, business intelligence, and knowledge management?
The Rush to Sell Out before Drowning in Red Ink
Now a sidelight. Before open source search effectively became the go to keyword search system, there were vendors who had products that for the most part worked when installed to do basic information retrieval. These companies’ executives worked overtime to find buyers. The founders cashed out and left the new owners to figure out how to make sales, pay for research, and generate sufficient revenue to get the purchase price back. Which companies are these? Here’s a short list and incomplete list to help jog your memory:
- Artificial Linguistics (sold to Oracle)
- BRS Search (sold to OpenText)
- EasyAsk (first to Progress Software and then to an individual investor)
- Endeca to Oracle
- Enginium (sold to Kroll and now out of business)
- Exalead to Dassault
- Fulcrum Technology to IBM (quite a story. See the Fulcrum profile at www.xenky.com/vendor-profiles)
- InQuira to Oracle
- Information Dimensions (sold to OpenText)
- Innerprise (Microsoft centric, sold to GoDaddy)
- iPhrase to IBM (iPhrase was a variant of Teratext’s approach)
- ISYS Search Software to Lexmark (yes, a printer company)
- RightNow to Oracle (RightNow acquired Dutch technology for its search function)
- Schemalogic to Smartlogic
- Stratify/Purple Yogi (sold to Iron Mountain and then to Autonomy)
- Teratext to SAIC, now Leidos
- TripleHop to Oracle
- Verity to Autonomy and then HP bought Autonomy
- Vivisimo to IBM (how clustering and metasearch magically became a Big Data system from the company that “invented” Watson) .
The brand impact of these acquired search vendors is dwindling. The only “name” on the list which seems to have some market traction is Endeca.
Some outfits just did not make it or who are in a very quiet, almost dormant, mode. Consider these search vendors:
- Delphes (academic thinkers with linguistic leanings)
- Edgee
- Dieselpoint (structured data search)
- DR LINK (Syracuse University and an investment bank)
- Executive Search (not a headhunting outfit, an enterprise search outfit)
- Grokker
- Intrafind
- Kartoo
- Lextek International
- Maxxcat
- Mondosoft
- Pertimm (reincarnated with Axel Springer (Macmillan) money as Qwant, which according to Eric Schmidt, is a threat to Google. Yeah, right.)
- Siderean Software (semantic search)
- Speed of Mind
- Suggest (Weitkämper Technology)?
- Thunderstone
These are not a comprehensive list. I just wanted to layout some facts about vendors who tilted at the enterprise search windmill. I think that a reasonable person might conclude that enterprise search has been a tough sell. Of the companies that developed a brand, none was able to achieve sustainable revenues. The information highway is littered with the remains of vendors who pitched enterprise search as the killer app for anything to do with information.
Now the survey purports to reveal insights to which I have been insensitive in my decades of work in digital information access.
Here’s what the company sponsoring the survey offers:
Concept Searching [the survey promulgator], the global leader in semantic metadata generation, auto-classification, and taxonomy management software, and developer of the Smart Content Framework™, is compiling the statistics from its 2015 SharePoint and Office 365 Metadata survey, currently unpublished. One of the findings, gathered from over 360 responses, indicates a renewed focus on improving enterprise search.
The focus seems to be on SharePoint. I thought SharePoint was a mishmash of content management, collaboration, and contacts along with documents created by the fortunate SharePoint users. Question: Is enterprise search conflated with SharePoint?
I would not make this connection.
If I understand this, the survey makes clear that some of the companies in the “sample” (method of selection not revealed) want better search. I want better information access, not search per se.
Each day I have dozens of software applications which require information access activity. I also have a number of “enterprise” search systems available to me. Nevertheless, the finding suggests to me that enterprise search is and has not been particularly good. If I put on my SharePoint sunglasses, I see a glint of the notion that SharePoint search is not very good. The dying sparks of Fast Search technology smoldering in fire at Camp DontWorkGud.
Images, videos, and audio content present me with a challenge. Enterprise search and metatagging systems struggle to deal with these content types. I also get odd ball file formats; for example, Framemaker, Quark, and AS/400 DB2 UDB files.
The survey points out that the problem with enterprise search is that indexing is not very good. That may be an understatement. But the remedy is not just indexing, is it?
After reading the news release, I formed the opinion that the fix is to use the type of system available from the survey sponsor Concept Searching. Is that a coincidence?
Frankly, I think the problems with search are more severe than bad indexing, whether performed by humans or traditional “smart” software.
According the news release, my view is not congruent with the survey or the implications of the survey data:
A new focus on enterprise search can be viewed as a step forward in the management and use of unstructured content. Organizations are realizing that the issue isn’t going to go away and is now impacting applications such as records management, security, and litigation support. This translates into real business currency and increases the risk of non-compliance and security breaches. You can’t find, protect, or use what you don’t know exists. For those organizations that are using, or intend to deploy, a hybrid environment, the challenges of leveraging metadata across the entire enterprise can be daunting, without the appropriate technology to automate tagging.
Real business currency. Is that money?
Are system administrators still indexing human resource personnel records, in process legal documents related to litigation, data from research tests and trials in an enterprise search system? I thought a more fine-grained approach to indexing was appropriate. If an organization has a certain type of government work, knowledge of that work can only be made available to those with a need to know. Is indiscriminate and uncontrolled indexing in line with a “need to know” approach?
Information access has a bright future. Open source technology such as Lucene/Solar/Searchdaimon/SphinxSearch, et al is a reasonable approach to keyword functionality.
Value-added content processing is also important but not as an add on. I think that the type of functionality available from BAE, Haystax, Leidos, and Raytheon is more along the lines of the type of indexing, metatagging, and coding I need. The metatagging is integrated into a more modern system and architecture.
For instance, I want to map geo-coordinates in the manner of Geofeedia to each item of data. I also want context. I need an entity (Barrerra) mapped to an image integrated with social media. And, for me, predictive analytics are essential. If I have the name of an individual, I want that name and its variants. I want the content to be multi-language.
I want what next generation information access systems deliver. I don’t want indexing and basic metatagging. There is a reason for Google’s investing in Recorded Future, isn’t there?
The future of buggy whip enterprise search is probably less of a “strategic application” and more of a utility. Microsoft may make money from SharePoint. But for certain types of work, SharePoint is a bit like Windows 3.11. I want a system that solves problems, not one that spawns new challenges on a daily basis.
Enterprise search vendors have been delivering so-so, flawed, and problematic functionality for 40 years. After decades of vendor effort to make information findable in an organization, has significant progress been made. DARPA doesn’t think search is very good. The agency is seeking better methods of information access.
What I see when I review the landscape of enterprise search is that today’s “leaders” (Attivio, BA Insight, Coveo, dtSearch, Exorbyte, among others) remind me of the buggy whip makers driving a Model T to lecture farmers that their future depends on the horse as the motive power for their tractor.
Enterprise search is a digital horse, an one that is approaching break down.
Enterprise search is a utility within more feature rich, mission critical systems. For a list of 20 companies delivering NGIA with integrated content processing, check out www.xenky.com/cyberosint.
Stephen E Arnold, March 20, 2015
-
- Subscribe to Beyond Search
Feature archive
News archive
-
