October 30, 2014
I know zippo about online advertising. What have I got to advertise? After a talk with the owner of a high-traffic blog with videos, I flipped through my archive. With Google creeping toward a for fee test for YouTube lovers, I found “YouTube Shoots Google In Foot” interesting. Google is dependent on advertising. The dinosaur, the jets, and the synthetic biology initiative–everything depends on ads. Search has been marginalized. This write up explains that Google is annoying its users. Er, what’s new?
The nifty part was:
A rather emotionally charged Google ad has been making its way around the internet lately, heavily advocating the power of Google search. Ironically, all of the searches displayed in the ad fail to show Google ads that would typically come up in a normal, adblock-free search, suggesting that Google prefers to show a cleaner page in the commercial. They’ve also allegedly been asking users to disable adblock for some of their sites, while paying AdBlock Plus to whitelist their other Adwords ads, saving an estimated $887 million last year.
I am not sure I understand this, but it seems as if Google may be looking beyond ads to fund the money YouTube consumes each quarter.
Stephen E Arnold, October 30, 2014
October 27, 2014
One of my two or three readers sent me a link to “The 10 Stuff Ups We All Make When Interpreting Research.” The article walks through some common weaknesses individuals make when “interpreting research.” I don’t agree with the “all” in the title.
This article arrived as I was reading a recent study about search. As an exercise on a surprisingly balmy Sunday afternoon in Kentucky, I jotted down the 10 “stuff ups” presented in the Interpreting Research article. Here they are in my words, paraphrased to sidestep plagiarism, copyright, and Google duplication finder issues:
- One study, not a series of studies. In short, an anomaly report.
- One person’s notion of what is significant may be irrelevant.
- Mixing up risk and the Statistics 101 notion of “number needed to treat” gets the cart before the horse.
- Trends may not be linear.
- Humans find what they want to find; that is, pre existing bias or cooking the study.
- Ignore the basics and layer cake the jargon.
- Numbers often require context. Context in the form of quotes in one on one interviews require numbers.
- Models and frameworks do not match reality; that is, a construct is not what is.
- Specific situations do matter.
- Inputs from colleagues may not identify certain study flaws.
To test the article’s premises, I I turned to a study sent to me by a persona named Alisa Lipzen. Its title is “The State of Knowledge Management: 2014. Growing role & Value of Unified Search in Customer Service.” (If the link does not work for you, you will have to contact either of the sponsors, the Technology Services Industry Association or Coveo, an enterprise search vendor based in Canada.) You may have to pay for the report. My copy was free. Let’s do a quick pass through the document to see if it avoids the “stuff ups.”
First, the scope of the report is broad:
1. Knowledge management. Although I write a regular column for KMWorld, I must admit that I am not able to define exactly what this concept means. Like many information access buzzwords, the shotgun marriage of “knowledge” and “management” glues together two abstractions. In most usages, knowledge management refers to figuring out what a person “knows” and making that information available to others in an organization. After all, when a person quits, having access to that person’s “knowledge” has a value. But “knowledge” is as difficult to nail down as “management.” I suppose one knows it when one encounters it.
2. Unified search. The second subject is “unified search.” This is the idea that a person can use a single system to locate information germane to a query from a single search box. Unified suggests that widely disparate types of information are presented in a useful manner. For me, the fact that Google, arguably the best resourced information access company, has been unable to deliver unified search. Note that Google calls its goal “universal search.” In the 1980s, Fulcrum Technologies (Ottawa, Canada) search offered a version of federated search. In 2014, Google requires that a user run a query across different silos of information; for example, if I require informatio0n about NGFW I have to run the query across Google’s Web index, Google scholarly articles, Google videos, Google books, Google blogs, and Google news. This is not very universal. Most “unified” search solutions are marketing razzle dazzle for financial, legal, technical, and other reasons. Therefore, organizations have to have different search systems.
3. Customer service. This is a popular bit of jargon. The meaning of customer service, for me, boils down to cost savings. Few companies have the appetite to pay for expensive humans to deal with the problems paying customers experience. Last week, I spent one hour on hold with an outfit called Wellcare. The insurance company’s automated system reassured me that my call was important. The call was never answered. What did I learn. Neither my call nor my status as a customer was important. Most information access systems applied to “customer service” are designed to drive the cost of support and service as low as possible.
“Get rid of these expensive humans,” says the MBA. “I want my annual bonus.”
I was not familiar with the TSIA. What is its mission? According the the group’s Web site:
TSIA is organized around six major service disciplines that address the major service businesses found in a typical technology company.
Each service discipline has its own membership community led by a seasoned research executive. Additionally, each service discipline has the following:
- Focused research agenda
- Dedicated research team
- Benchmark study
- Dedicated track at Technology Services World conferences
- Member Advisory Board
In addition, we have a research practice on Service Technology that spans across all service discipline focus areas.
My take is that TSIA is a marketing-oriented organization for its paying members.
Now let’s look at some of the the report’s key findings:
The people, process, and technology components of technology service knowledge management (KM) programs. This year’s survey examined core metrics and practices related to knowledge capture, sharing, and maintenance, as well as forward-looking elements such as video, crowd sourcing, and expertise management. KM is no longer just of interest to technical support and call centers. The survey was open to all TSIA disciplines, and 50% of the 400-plus responses were from groups other than support services, including 24% of responses from professional services organizations.
October 25, 2014
I read “Google’s Head of Android to Oversee Its Most Important Products.” Interesting news. Larry Page is moving on up. For me, search is moving on down. Here’s the passage I highlighted:
The promotion punctuates Pichai’s quick rise inside the company as well as CEO Larry Page’s desire to focus on off-loading some of his management duties to better focus on overall business strategy. While Google’s search and advertising business still generates $50 billion a year in revenue, some financial analysts fear its business is slowing. The company last week reported that paid clicks for the third quarter rose 17 percent from the same period last year. That compares with 26 percent growth the year before.
My view is that search is the loser in this deal. Mr. Brin is chasing balloons and solving death. Mr. Page wants to think about making money from a higher level.
The Google has change in neon lights in front of the dinosaur bones. Metaphor? You bet. You can research this with reasonable precision and recall on either Yandex.com or iseek.com. Google? Well, you get lots of ads on your desktop computer. On your mobile phone? Well, not too many ads fit on the tiny screen. Aye, that’s the problem, Captain.
Stephen E Arnold, October 25, 2014
October 25, 2014
Interesting idea: Amazon is not a business. The story “Ex-Microsoft CEO Steve Ballmer Says Amazon Isn’t a Real Business” floats this idea. The argument is that Amazon does not make money. Let’s assume this notion is correct. Can an outfit that is not a real business deliver real search? The answer is, “Yep.” Dozens of search vendors have been living from hand to mouth for many years. Each of these companies would assert that they have state-of-the-art, supercalifragilisticexpialidocious technology.” But there’s the money thing, isn’t there. Hakia seems to be the most recent search vendor to go into slow down mode. Others are taking their foot of the gas pedal as well. There is some truth to the real businesses make money premise. Will Amazon deliver “real” search. Probably a long shot.
Stephen E Arnold, October 25, 2014
October 22, 2014
I read “HP Will Unveil new Computing Product Called Sprout Next Week.” I am not sure I understand a multi function device that combines a touch screen, an overhead projector, and a 3D scanner. Why? An important function is missing.’
Where is Autonomy Idol? It’s a multi billion dollar technology wonder. A “Powered by Idol” would be a nice marketing touch.
Shortly after the acquisition, I heard that HP wanted to embed Autonomy Idol in a range of devices. Well, after three years, why isn’t Idol included in Sprout?
I can envision the late night TV ad now. I want to see a touch screen, overhead projector, and 3D scanner promoted as the next American Idol.
Stephen E Arnold, October 22, 2014
October 19, 2014
I read “Panopto Recognized as a “Leader” in Gartner’s Magic Quadrant for Enterprise Video Content Management.” I learned that Panopto is a “leader” in the mid tier consulting firm’s league table of enterprise video content management vendors.
According to the write up,
In use at Fortune 500 companies and leading academic institutions worldwide, Panopto is the only video platform that provides organizations with an integrated, off-the-shelf solution for video recording, live streaming, video management, and inside-video search. Within the enterprise, Panopto is being used to improve sales enablement, streamline employee onboarding, scale corporate training, and simplify the delivery of executive communications. Within universities, Panopto is the fastest-growing solution for flipping the classroom, capturing lectures, and streaming campus events.
For many years, I have pointed out that locating information in a large collection of videos remains a dicey proposition. Years ago I explored Exalead’s video search system. It worked, but it imposed hefty computational burdens on system resources. Perhaps that has changed under Dassault’s stewardship? I wonder if the Exalead system could be used to manage an organization’s video? I don’t know, and the company does not appear in the Gartner version of the Boston Consulting Group’s matrix.
Another possible omission is Autonomy’s line up of video management and findability tools. I recall seeing a demonstration of a system with the “powered by IDOL” label. Has Autonomy withdrawn from the field of video play?
In our ongoing investigation of Google technology, we learned that it is easy to use YouTube as a convenient, low cost video management system. But no Google in the BCG inspired matrix.
I don’t know too much about video management, but when I scanned the graphic available from this link, I noticed these companies’ absence?
The inclusion of some products/companies with which I was not familiar was interesting; for example, Perceptive Software (part of Lexmark and owner of ISYS test search). The separate entries and Kulu Valley and Qumu struck me as an anomaly. Qumu, I believe, owns Kulu. Does 1 + 1 = a super leader or is this an oversight or just a failure to update?
Congratulations to Panopto. But in the back of my mind there is a reason why some consultancies lag behind McKinsey, BCG, and the pre-break up Booz, Allen & Hamilton. Mentioning “mid tier” reminds me of IDC’s sale of my content on Amazon without my permission under the “expert” Dave Schubmehl’s name.
Stephen E Arnold, October 18, 2014
October 15, 2014
There is a presentation “Kicking the Bukkit: Anatomy of an Open Source Meltdown” by Ryan Michela, a developer with experience in open source. Over several years, a game open source project rose and fell. I am not too interested in open source games. At the end of the Slideshare document, there are five reasons an open source game project failed.
Let me summarize these and encourage you to work through he full 55 slide deck. How many of these issues may have an impact on open source search systems. Keep in mind that commercial enterprises like Attivio and IBM make use of open source technology.
- Inclusion of decompiled code in an open source project
- License issues
- Ties ups within the community before a project gains momentum
- No contributor license agreement
- Disgruntled developers in the community.
The presentation includes a quote that I noted:
It only takes one unhappy developer to kill an unprotected project.
Is there an open source search company vulnerable to one or more of these issues? I can name a couple. I wonder if the firm’s funding sources are concerned about their investment “kicking the bucket”?
Stephen E Arnold, October 15, 2014
October 10, 2014
I worked through the 34 page report “Industry Watch. Search and Discovery. Exploiting Knowledge, Minimizing Risk.” The report is based on a sampling of 80,000 AIIM community members. The explanation of the process states:
Graphs throughout the report exclude responses from organizations with less than 10 employees, and suppliers of ECM products and services, taking the number of respondents to 353.
The demographics of the sample were tweaked to discard responses from organizations with fewer than 10 employees. The sample included respondents from North America (67 percent), Europe (18 percent) and “rest of world” (15 percent).
Some History for the Young Reader of Beyond Search
AIIM has roots in imaging (photographic and digital imaging). Years ago I spent an afternoon with Betty Steiger, a then well known executive with a high profile in Washington, DC’s technology community. She explained that the association wanted to reach into the then somewhat new technology for creating digital content. Instead of manually indexing microfilm images, AIIM members would use personal computers. I think we connected in 1982 at her request. My work included commercial online indexing, experiments in full text content online, a CD ROM produced in concert with Predicasts’ and Lotus, and automated indexing processes invented by Howard Flank, a sidekick of mine for a very long time. (Mr. Flank received the first technology achievement award from the old Information Industry Association, now the SIIA).
AIIM had its roots in the world of microfilm. And the roots of microfilm reached back to University Microfilms at the close of World War II. After the war, innovators wanted to take advantage of the marvels of microimaging and silver-based film. The idea was to put lots of content on a new medium so users could “find” answers to questions.
The problem for AIIM (originally the National Micrographics Association) was indexing. As an officer at a company considered in the 1980 as one of the leaders in online and semi automated indexing methods, Ms. Steiger and I had a great deal to discuss.
But AIIM evokes for me:
Microfilm —> Finding issues —> Digital versions of microfilm —> CD ROMs —> On premises online access —> Finding issues.
I find the trajectory of a microfilm leading to pronouncements about enterprise search, content processing, and eDiscovery fascinating. The story of AIIM is a parallel for the challenges the traditional publishing industry (what I call the “dead tree method”) has, like Don Quixote, galloped, galloped into battle with ones and zeros.
Asking a trade association’s membership for insights about electronic information is a convenient idea. What’s wrong with sampling the membership and others in the AIIM database, discarding those who belong to organizations with fewer than 10 employees, and tallying up the survey “votes.” For most of those interested in search, absolutely nothing. And that may be part of the challenge for those who want to get smart about search, findability, and content processing.
Let’s look at three findings from the 30 plus page study. (I have had to trim because the number of comments and notes I wrote when reading the report is too massive for Beyond Search.)
Finding: 25 percent have no advanced or dedicated search tools. 13 percent have five or more [advanced or dedicated search tools].
Talk about good news for vendors of findability solutions. If one thinks about the tens of millions of organizations in the US, one just discards the 10 percent with 10 or fewer employees, and there are apparently quite a large percentage with simplistic tools. (Keep in mind that there are more small businesses than large businesses by a very wide margin. But that untapped market is too expensive for most companies to penetrate with marketing messages.) The study encourages the reader to conclude that a bonanza awaits the marketer who can identify these organizations and convince them to acquire an advanced or dedicated search tool. There is a different view. The research Arnold IT (owner of Beyond Search) has conducted over the last couple of decades suggests that this finding conveys some false optimism. For example, in the organizations and samples with which we have worked, we found almost 90 percent saturation of search. The one on one interviews reveal that many employees were unaware of the search functions available for the organization’s database system or specialized tools like those used for inventory, the engineering department with AutoCAD, or customer support. So, the search systems with advanced features are in fact in most organizations. A survey of a general population reveals a market that is quite different from what the chief financial officer perceives when he or she tallies up the money spent for software that includes a search solution. But the problems of providing one system to handle the engineering department’s drawings and specifications, the legal departments confidential documents, the HR unit’s employee health data, and the Board of Director’s documents revealing certain financial and management topics have to remain in silos. There is, we have found, neither an appetite to gather these data nor the money to figure out how to make images and other types of data searchable from a single system. Far better to use a text oriented metasearch system and dismiss data from proprietary systems, images, videos, mobile messages, etc. We know that most organizations have search systems about which most employees know nothing. When an organization learns about these systems and then gets an estimate to creating one big federated system, the motivation drains from those who write the checks. In our research, senior management perceives aggregation of content as increasing risk and putting an information time bomb under the president’s leather chair.
Finding: 47% feel that universal search and compliant e-discovery is becoming near impossible given the proliferation of cloud share and collaboration apps, personal note systems and mobile devices. 60% are firmly of the view that automated analytics tools are the only way to improve classification and tagging to make their content more findable.
The thrill of an untapped market fades when one considers the use of the word “impossible.” AIIM is correct in identifying the Sisyphean tasks vendors face when pitching “all” information available via a third party system. Not only are the technical problems stretching the wizards at Google, the cost of generating meaningful “unified” search results are a tough nut to crack for intelligence and law enforcement entities. In general, some of these groups have motivation, money, and expertise. Even with these advantages, the hoo hah that many search and eDiscovery vendors pitch is increasing potential customers’ skepticism. The credibility of over-hyped findability solutions is squandered. Therefore, for some vendors, their marketing efforts are making it more difficult for them to close deals and causing a broader push back against solutions that are known by the prospects to be a waste of money. Yikes. How does a trade association help its members with this problem? Well, I have some ideas. But as I recall, Ms. Steiger was not too thrilled to learn about the nitty gritty of shifting from micrographics to digital. Does the same characteristic exist within AIIM today? I don’t know.
October 7, 2014
Google, believe it or not, is responsible in part for the problems with enterprise search. The idea is advanced in “Why the “Google Paradigm” Has Damaged Enterprise Search.” The core of the argument is that people use Google for Web search. The resulting perception is that “enterprise search is as easy as Google web search, and that a central index of an enterprise is the right way to do enterprise search. ”
Google’s entrance into enterprise search was one of the companies earliest attempts to enter a market in which revenue came from a subscription or license, not a fee for advertising. The Google Search Appliance was a server loaded with a version of Google’s Web search system. Based on our work with the first GSA, it was clear that like many other Google products and services from the 2001 to 2004 period, Google was operating on some Googley assumptions; for example:
- Google assumed that a version of its Web search system stripped of its ad matching was good enough for finding textual content in an organization
- The company assumed that Autonomy, Endeca, and Fast Search & Transfer, the dominant enterprise search vendors at this time were too complex for most technical staff in an organization. The time and complexity of these systems contributed to the high user dissatisfaction with these systems. The high cost of these industry leaders’ systems contributed to management grousing about search.
- Google assumed that it could disintermediate traditional information technology departments and deal directly with end users.
Google crafted a server that was positioned as a “search toaster.” The low price of the basic unit was less than $2,000 and sported an interface that required the licensee to plug in basic information and click a button to start the indexing process.
The Google Search Appliance by 2007 had an estimated 50,000 licensees. At that time, the product line had expanded but the locked down nature of the Google Search Appliance and the key word approach of the system was creating sales opportunities for other search appliance vendors; namely, Thunderstone, Maxxcat, and Index Engines.
Google added features and fiddled with the license fees, hardening the GSA product line with hot backups, connectors, and extensibility via licensed vendors. Few analysts paid much attention to the product licensing fees for the various “GB” or Google boxes. If you want to get a sense of the costs for building out a GSA system that can process 100 million documents, navigate to www.GSAadvantage.gov and search for the Google’s search appliances. The costs work out to be comparable or slightly higher than a similar installation from Autonomy, Endeca, or Fast. The high prices remain today.
Google learned from the GSA experience. Instead of offering an enterprise cloud solution, the company has left a limited and pricey GSA product line in the market and provided a modest commitment to this enterprise search solution. Google’s cloud solution manifests itself in Google’s site search features. I am waiting for Google either to kill the product line or amp up its commitment. In my opinion, the GSA is in no man’s land at this time. It appears that not even Google can respond to the needs of the enterprise findability users. If any company could crack the code, would it not be Google or a Xoogler’s start up?
As the GSA emerged as placeholder product, professionals became more and more dependent on Google’s Web search. In Europe, for example, Google’s Web search commands an market share in excess of 80 percent. In Denmark, Google’s share of Web search is north of 90 percent. In the US, Google has a 65 to 75 percent share of Web search, depending on which consultancies’ numbers one uses.
The word “search” became synonymous with Google. Enterprise search vendors began to use jargon other than search. This step was a natural reaction to hearing from prospects, “We want a search that works just like Google.” What the prospects meant was a system that was easy to use and seemed to deliver useful results in the hits displayed at the top of a results list, a page of images, or a map showing a location.
Google Web search, not the Google Search Appliance, reflected a broader shift in the information access market. Users of Web search and enterprise systems wanted and still want:
- Systems that do not require the user to invest much time and effort in getting an answer
- Systems that can produce useful outputs whether text, images, or maps with data displayed on them
- Systems that delivered “answers” without the delays (latency) many enterprise systems force on users.
Google’s ability to respond to this enterprise demand has been ineffectual. Like other Web search vendors, key word retrieval does not solve the problems basic search systems spawn. The GSA is evidence that Google does not have the key to unlock the revenue vault for enterprise search.
What Google search has done (inadvertently I might add) has been to make crystal clear that users do not want to work hard for information users perceive as useful. Precision and recall are irrelevant because voting and advertisers influence Google Web search results. Users love Google’s outputs.
In the organization, procurement teams, individual users, and senior management boil their needs down to one simple statement: “We want search that is just like Google.”
That’s a big, big problem for search and content processing vendors. Google Web search is not about relevance, objective information, or accuracy. Google is easy and “good enough.” In an organization, people want easy. But in an organization the results have to be timely, comprehensive in terms of what information is available to an organization, and accurate.
On the Web Google can skip content that is malformed or stored on a server that does not respond to a Google spider quickly enough. In an organization, the content has to be available. On the Web, the advertisers and the uses’ own behavioral data pays the bills. In an organization, the organization has to pay the bills. Google has more money from a different business model than most organizations. Google pumps money into plumbing to deliver the service that makes money. Organizations want to fix the amount spent on search and the funds are not infinite.
For search vendors, the problem of Google’s dominance in Web search makes product differentiation difficult. Google’s business model creates challenges for vendors who have to justify the “value” and hence the “cost” of their search systems. For traditional search vendors, ease of use is very, very difficult because of the nature of the questions enterprise system users have.
Google is a mirror in which societal, cultural, and intellectual changes in information access are reflected. For many years, I have called attention to the verbal push ups search vendors use to try and make sales. The struggled Hewlett Packard have had with Autonomy provide a glimpse of how “value” can be difficult to change into hard cash Microsoft’s Delve illustrates that search for Office 365 is a combination of contacts, alerts, and personalization, not key word search. The dependence on enterprise search companies for cash from venture capital sources illustrates that traditional search is a very, very tough business to make into something sustainable and profitable without financial life support. The expectations that Watson will become a $10 billion business in 60 months is disconnected from the experience of other smart companies. In the history of enterprise search, only Autonomy reported revenues of more than $800 million from enterprise licenses. IBM projects more than 10 X this revenue in 60 months. It took Autonomy more than a decade to hit $500 million.
The reality is that Google is not the problem. Google is a metaphor for what users want when it comes to information access.
The write up asserts:
The Google paradigm also ignores the challenge of scalability. Indexing the enterprise for a centralized enterprise search capability requires major investment. In addition, centralization runs counter to the realities of the working world where information must be distributed globally across a variety of devices and applications. The amount of information we create is overwhelming and the velocity with which that information moves increases daily.
Interesting statement. For me, the problem of the Google paradigm is that it another bit of jargon that sidesteps a what information retrieval must deliver in today’s business environment. Whoever cracks the code can make money. My hunch is that Google probed the enterprise search market and is trying to figure out how to make it pay off in a significant way. Google may be trapped in the same problem space through which other enterprise search and content processing vendors slog. The question may be, “Is there a way out of the swamp and into a land of milk, honey, sustainable revenue, and healthy margins?”
Stephen E Arnold, October 7, 2014
October 5, 2014
Who owns Business Insider? Oh, maybe Amazon, a company with arguably the most ineffective search system available to me. Just try to winnow a list of books by those one can buy right now versus those one can buy at some time in the future? Oh, right. I just read “Here’s the Evidence That Google’s Search Results Are Horribly Biased.”
Now Google is an ad outfit. Pay money and one gets traffic. Seems simple. Google also flogs its own products like the really, really popular Google Search Appliance. Google is battling Amazon in the world of WalMart like cloud cost cutting. Sigh. Is it a surprise that Google has a small amount of self interest in what it does, delivers, and presents? Who argues for precision and recall at Google?
Here’s what I learned:
An internal document recently leaked from Yelp showed how the company believed Google was siphoning off up to 20% of its clicks, directing them to Google’s results. Yelp even persuaded the European Union to reopen an antitrust investigation into the way it said Google abused its 90% share of the European search market to manipulate results. (Business Insider heard Yelp’s intervention came after CEO Jeremy Stoppelman found himself at a dinner in San Francisco this spring with EC president José Manuel Barroso, where he was able to bend Barroso’s ear about Google’s market share.) So Yelp has launched a coalition, including Trip advisor and Consumer Watchdog, under the banner “Focus On The User” to draw attention to the way Google results are displayed.
Yep, revelation. Is it possible for a merchant to influence a Yelp review? Hmmm. Is any “free” search system bias free? Any countries filtering search results or system access to control information flow? Hmmmm.
Stephen E Arnold, October 5, 2014