Exclusive Interview: Hadley Reynolds, IDC
August 18, 2010
One of the big guns in search and content processing consulting is IDC, a firm to which many senior managers turn when trying to figure out the fast-changing world of digital information. IDC describes itself as:
the premier global provider of market intelligence, advisory services, and events for the information technology, telecommunications, and consumer technology markets. IDC helps IT professionals, business executives, and the investment community make fact-based decisions on technology purchases and business strategy. More than 1000 IDC analysts provide global, regional, and local expertise on technology and industry opportunities and trends in over 110 countries worldwide. For more than 46 years, IDC has provided strategic insights to help our clients achieve their key business objectives. IDC is a subsidiary of IDG, the world’s leading technology media, research, and events company.
I go along with the description because every once in a while I do a project for IDC, and I do like to get paid for my research.
Hadley Reynolds is a featured speaker at the Lucene Revolution Conference, Boston, Mass., October 7 and 8, 2010. Information about the conference is available at http://www.lucenerevolution.com.
Mr. Reynolds focuses on understanding business transformation opportunities through the application of search technologies to traditional business models and the role search innovation is playing in creating new business opportunities. Prior to joining IDC, he guided the Center for Search Innovation at Microsoft Fast. A former Delphi Group consultant, he brings business and technical expertise to bear on his work in search.
I tracked down Hadley Reynolds, one of IDC’s senior consultants in the search practice. We conversed in Framingham, Massachusetts. We talked at the Starbuck’s in Framingham. I have transcribed our talk in the dialogue below:
Hadley, thanks for taking the time to talk with me?
Glad to. I read your blog and I must say that I don’t agree with some of your points.
No problem. The blog is designed to spark discussion, not compete with the work you and your colleagues do at IDC.
We do not consider your comments on the world of search competitive in any way. Go for it.
Okay, let me get right to the point. I noticed that you are on the program for the Lucene Revolution Conference. Why the interest in open source software?
Anyone who has been watching the enterprise search market in recent years has to be impressed with the growth record of the Lucene: in technical enhancements, community participation, and acceptance in the commercial market.
Enterprise search is usually a proprietary solution. Open source relies on a community. Is the community angle real or a myth?
When the Lucene project was in its early days, I was highly skeptical about the likelihood of a community of sufficient horsepower coalescing around it and carrying forward Doug Cutting’s foundational work. However, with IBM and other smaller commercial players like Attivio, Palantir, i2, Lucid Imagination, and others both using and contributing to the codebase, plus continuing contributions from search “independents”, I think the skepticism about critical mass and future support can be laid to rest.
I see some companies playing what I call the “open source card”. A couple of examples are IBM and Oracle. IBM uses Lucene/Solr in OmniFind 9.1, and Oracle bought Sun Microsystems and then MySQL. Won’t that confuse enterprise customers?
The trend in IT is toward more open source software, not less. First it was accepted that common resources like the Linux operating system and the Apache Web Server and the Mozilla browser were fine alternatives for the enterprise. Then we saw open source databases like MySQL and others, and multiple content management systems like Alfresco, Drupal and many more. So it’s not surprising that we now have successful open source search in Lucene/Solr and that commercial companies who utilize search as a component, and even those that sell search-based applications, would want to take advantage of the strength of these products both for themselves and for their customers.
I am interested in Lucene/Solr but I also track Drupal, Hadoop, and other open source projects. What are benefits you see in open source software?
The benefits of using Lucene/Solr are not hard to enumerate: rapid time-to-market, high-level functionality, flexibility and customization, low entry level cost, positive future outlook for technology evolving with the state of the art.
I agree. But the reason there is proprietary software is to deliver a solution with “one throat to choke.” What are the drawbacks of open source software where no one may be responsible for a code fix, a new function, or a code widget?
The drawbacks include: enhancements on a community timetable only, potentially expensive customization, requirement for advanced skills in-house or near-at-hand, delivered functionality will trail the truly cutting edge search software specialist firms, and system life costs can become significant.
When I think of IDC I think of commercial software. With this open source “revolution” on display in the promotion for the Lucene Revolution Conference and the stepped up marketing of the conference sponsor, Lucid Imagination, something is happening and it may not be the traditional enterprise play. When someone asks you why you don’t use a commercial search solution, what do you tell them?
I believe that for many search problems in the enterprise, the smart approach is to select the tool that gets the job done with the least delay and at the most reasonable investment level. More and more frequently, open source search software is becoming the tool of choice.
Wow it looks like your phone is going to jump off the table.
Right, I need to get back.
How do people contact you?
Your readers can get me by sending an email to hreynolds at idc dot com.
Stephen E Arnold, August 18, 2010
Sponsored post. Okay?
Open Source May Be Disruptive
August 16, 2010
I do not know much about software, information, or the big-time doings of corporate giants. I am not an azurini, a self-appointed poobah, or the cat’s paw of a group of 20-something MBAs from schools that require a great family and a high IQ. I am an addled goose. I float around in a pond filled with mine drainage water and offer some humble observations which the great and powerful dismiss as silly, irrelevant, or just plain incorrect.
No problemo.
However, even the intellectual black hole of the addled goose’s analytic muck pond can figure out from two articles that open source is scaring the heck out of a really tough, superstar executive.
You make your own decision about the accuracy and significance of these two news stories:
First, point your browser thingy with monitoring functions activated so those watching know you really did navigate to a “real” news source and read “Oracle Kills Open Solaris, Moves Development behind Closed Doors.” The idea is pretty easy to understand. Those super-marketers at Sun Microsystems gave away an operating system as open source. Nope. Oracle’s Larry Ellison and his sharp-pencil crowd shut that door. The reason? Open source equals bad business. “Bad”, I presume, means a threat to Oracle’s pricing tactics. Free and Oracle are not words that I associate when I hear the word “Oracle.”
A happy quack to http://jordanhall.co.uk/general-articles/dont-be-evil-licensing-1301401/ for this great illustration.
Second, Oracle is suing Google over its use of Java. Now Java is sort of a piggy, but, hey, lots of universities teach Java, and it can be quite useful when running in today’s nifty hardware environments. Overlook those flaws which have been documented in some detail in the Software Engineering podcasts at www.se-radio.net. Notice: SE-Radio is not exactly an Adam Carolla Leo LaPorte type podcast. You can get some information about the this tussle between two former bosom buddies by tapping to “Initial thoughts on Oracle vs Google Patent Lawsuit.” Yep, those are links to patent documents, so I don’t think the skimmers among my readers will invest much dwell time on the Tirania post.
Nevertheless, the headlines may be enough for a “real” azure chip consultant. The details are murky and former English majors and sociology minors won’t spend too much time doing the analysis a “real” poobah does.
Let me Cliff Notes it: Larry Ellison is a smart, rich person. He understands that open source is a problem for a company like Oracle that charges really big fees for its software. Open source with its unruly developers and hard-to-make-do-push-ups work ethic are the enemy. The fix. Kill open source. If total annihilation is not possible, make open source expensive in terms of legal fees. The way law works for rich people is that a rich person’s lawyers can make a less rich person spend lots of time fighting the rich person’s legal actions. Eventually money wins, particularly when there are fuzzy wuzzy ideas like open source, intellectual property, and rich people arguing as the main action.
There is just one snag. Even rich people have trouble keeping those peasants under control. For my readers who stayed awake during world history, you know that lots of peasants with sticks and rocks can be a real problem. Honk off enough peasants, and the excitement can end in a revolution.
At this moment in the capitalistic, free market sun:
- Lots of companies are strapped for cash. Free is pretty darned appealing when you have to decide how to pay the light bill, the actress assisting the company at a trade show, and paying the lease on a new Bimmer.
- Open source is pretty good, and there are some robust solutions available with the click of a mouse. Examples include Drupal, Hadoop, Lucene/Solr and * lots * more.
- The open source stuff is fun. Training and certification for Oracle or other “carrier class” enterprise solutions are not as much fun as blasting around the lake on a jet ski at 30 knots.
If I focus on relational databases, I am in a Roman ruin. You can see or at least imagine the splendor of the structure. But rebuilding it after a crash and getting it back to the “way it was” is just too much work, too expensive, and too labor intensive. Why not build a new structure, using the tips and methods that HGTV puts on display each night on my local cable channel. Need a granite counter top and have neither money, stone cutting tools, nor expertise. Hey, just shoot over to Home Depot and get an epoxy alternative. That’s the open source approach: New materials, new methods, and new benefits.
Roman ruin. What’s the cost of a rebuild and then upkeep? How do you modify a limestone flying balcony? You don’t. Get some slaves.
Is Q-Go a Yugo?
August 9, 2010
Last week, I received a call from a fancy pants MBA about NLP or natural language processing. NLP seems to be a new opportunity. NLP has been around a while, and like the formerly hot notion “taxonomy” and “semantics”, NLP is in vogue. The question concerned a company I knew about, Q-Go. I dipped into my Overflight service and realized that the company had gone quiet. In some cases, “going quiet” is a prelude to either a massive investment like Palantir’s $90 million or closing up shop like Convera did earlier this year.
Q-Go provides an application aimed at redefining customers’ web searching experiences. Research indicates that a growing number of customers are sick of turning to search engines for answers because they get responses with millions of unrelated websites.
According to their website, “31 percent of users are unhappy with their online interaction with web sites and 70% are unable to find what they are looking for.” And Q-Go asserts that it has the answer for the airline, financial services, and telecommunication industries. Q-Go reduces customer service issues by providing a search application that can more successfully interpret the meaning behind user questions—in all major Western languages. The approach sounds like InQuira’s.
The result? Fewer customer service calls, lowered costs, and higher conversion rates. It almost sounds too good to be true. With a guaranteed six-month return on investment, the only downside I see is that there are still some languages Q-Go can’t work with. But I’m guessing that will eventually change if the company avoids the “quiet” state.
Stephen E Arnold, August 12, 2010
Summer Search Rumor Round Up
July 26, 2010
The addled goose has been preoccupied with some new projects. In the course of running around and honking, he has heard some rumors. The goose wants to be clear. He is not sure if these rumors are 100 percent rock solid. He does want to capture them before the mushy information slips away:
Source: http://oneyearbibleimages.com/rumors.gif
First, the goose heard that there will be some turnover at Microsoft Fast. The author of some of the posts in the Microsoft Enterprise Search Blog may be leaving for greener pastures. You can check out the blog at this link. What does this tell the goose? More flip flopping at Microsoft? Not sure. Any outfit that pays $1.2 billion for software that comes with its own police investigation is probably an outfit that would scare the addled goose to death. The blog is updated irregularly with such write ups as “Crawling Case Sensitive Repositories Using SharePoint Server 2010” and “SharePoint 2010 Search ‘Dogfood’ Part 3 – Query Performance Optimization.” Ah, the new problem of upper and lower case and the ever present dog food regarding performance. I thought Windows most recent software ran as fast as a jack rabbit. Guess not.
Second, a number of traditional search vendors are poking around for semantic technology. The notion that key words don’t work particularly well seems to be gaining traction. The problem is that some of the high profile outfits have been snapped up. For example, Powerset fell into the Microsoft maw and Radar Networks was gobbled by Paul Allen’s love child, Evri. Now the stampede is on. The problem is that the pickings seem to be slim, a bit like the t shirts after a sale at the Wal-Mart up the road from the goose pond here in Harrods’s Creek. For some lucky semantic startups, Christmas could come early this year. Anyone hear, a sound like “hack, hack”. Oh, that must be short for Hakia. You never know.
Third, performance may have forced a change at HMV.co.uk in merrie olde England. Dieselpoint was the incumbent. I heard that Dieselpoint is on the look out for partners and investors. The addled goose tried to interview the founder of the company but a clever PR person sidelined the goose and shunted him to the drainage ditch that runs through Blue Island, Illinois. Will Dieselpoint land the big bucks as Palantir did.
Fourth, the goose heard that a trio of Microsoft certified partners with snap in SharePoint search components were looking for greener pastures. What seems to be happening is that the easy sales have dried up since Microsoft started its current round of partner cheerleading. The words are there, but the sales are not. Microsoft seems to want the money to flow to itself and not its partners. Who is affected? The goose cannot name names without invoking the wrath of Redmond and a pride of PR people who insist that their clients are knocking the socks off the competition. However, does the enterprise need a half dozen companies pitching metatagging to SharePoint licensees? I think not. If sales don’t pick up, the search engine death watch list will pick up a few new entries before the leaves fall. Vendors in the US, Denmark, Germany, Austria, and Canada are likely to watching Beyond Search’s death watch list. Remember Convera? It spawned Search Technologies. Remember the pre Microsoft Fast? It spawned Comperio? When a search engine goes away, the azurini flower.
Fifth, what’s happened to the Oracle killers? I lost track of Speed of Mind years ago. There was a start up with a whiz bang method of indexing databases. I haven’t heard much about killing Oracle lately. In fact, stodgy old Oracle is once again poking around for search and content processing technology according to one highly unreliable source. With SES11g now available to Oracle database administrators, perhaps the time is right to put some wood behind a 21st century search solution.
If you want to complain about one of these rumors, use the comments section of this blog. Alternatively, contact one of the azurini outfits and get “real” verification. Some of their consultants use this blog as training material for the consultants whom you compensate. No rumor this. Fact.
Stephen E Arnold, July 26, 2010
Freebie
Endeca and Agile Business Intelligence
July 26, 2010
If you have not read the interview about Fetch Technologies, you might want to take a look. Fetch is a company that sucks in content and makes it available for analysis. Among its features is an innovative programming method. The idea is that the old style business intelligence approach is too slow for today’s business and operational environment. Fetch is an information platform, and it has a number of advocates. Also, in the same sector are equally accomplished outfits such as Kapow Tech and JackBe, among others. Vendors like Exalead have made significant headway in business intelligence, challenging some of the old line outfits to up their game. IBM bought SPSS but I am waiting for significant innovation. SAS acquired Teragram and Memex, so I expect big things from these firms. Autonomy has a Hummer filled with business intelligence clients, and that firm continues to chew into the old line firms cut off from the fast moving client herd. In short, business intelligence is a big deal.
Endeca has been in the business intelligence business for many years. I did a report that pegged the date in the 2002 to 2003 period, maybe even earlier. I was, therefore, not surprised with the information revealed in “New Study Details Top Questions Effective CIOs Must Ask to Determine Agile BI Readiness.” With the stampede to business intelligence, it is obvious even to a first year business school student at an academic backwater like the one I attended that something is causing the corporate antelope to take off.
The cause of the shift, in my experience, boils down to four factors:
First, traditional business intelligence is complicated and requires dedicated headcount to get up and running and create the reports managers require.
Second, the managers are usually clueless about what constitutes “good data.” In fact, with lousy data, the business intelligence systems produce outputs that may mislead the clueless MBA from my alma mater. The reports often baffle me, but I am an addled goose and don’t really have corporate grade bloodlines.
Third, the time windows in which decisions must be taken continue to get smaller. Whether real or induced by iPhone attention deficit disorder is irrelevant. Crunching data from the dinosaur systems takes too long. Most of the azure chip consultants seem okay with the idea that systems their firms recommend make guessing a standard business practice.
Fourth, disparate data are very expensive to normalize. That’s why outfits like Fetch and Palantir are doing pretty well. Palantir, as you may know, is now valued at $1.0 billion and sucks in disparate data, outputs reports, and pretty much leapfrogs the more traditional outfits.
What did the Endeca study reveal?
Here are the points that jumped out at me:
- Analysts have to create reports for more than 70 percent of those in the survey sample
- Time is short, deadlines vary.
- About half of those in the sample found business intelligence systems too hard.
No surprises for me.
What interested me is that a company with a strong foundation in eCommerce is pushing business intelligence. My view is that Endeca, like other vendors in search and content processing, want to generate more revenues from their technology, content connectors, and partnerships with value added resellers.
The challenge for Endeca will be to deal with the inertia within many large companies. Endeca is not alone in chasing business intelligence. In fact, Endeca has been pushing business intelligence to some degree for a number of years. So far the high performers have been companies with a combination of technology, content processing capabilities, and the ability to solve specific business problems.
My hunch is that general purpose business intelligence systems are going to face a long slog uphill. The newer players have to contend with one another, price cuts, the economy, and the marketing challenge.
Perhaps a bespoke survey will do the trick? My view is that Endeca like other search vendors is looking for a way to generate revenue. Like eDiscovery, the realities of the marketplace are going to make it tough for the many business intelligence vendors to find a pot of gold at the end of the BI rainbow.
Furthermore, I think that the business intelligence push is one more indication that “pure search” has been disrupted significantly. Furthermore, can some search vendors deliver actionable business intelligence or just results list with lipstick and mascara?
Stephen E Arnold, July 26, 2010
Freebie
ArnoldIT July 2010 For-Fee Columns
July 6, 2010
Stephen E Arnold’s July 2010 columns have been filed. These will appear between August and October 2010 due to the lag time in print publication. Information World Review has shifted from paper to an online-only publication, so that column should be available in the month of July. Here’s a run down of what I covered for each of these publishing firms:
Information Today, published by Information Today in Medford, New Jersey. The column is “The Palantir Play: A Blend of Open and Closed.” Palantir received about $90 million in venture funding. The column considers the implications of the firm’s open source and proprietary technology blend. www.infotoday.com
Information World Review, published by Bizmedia Ltd., runs my column in its online journal. This month’s column is “Will Open Source Boost SAP?” My view is that I hope so. SAP has a long hill to climb with its aging locomotives R/3 and NetWeaver. www.iwr.co.uk
KMWorld, published by Information Today in Medford, New Jersey. The column is “Google Communications: Regular, Blended, or Transformational?”. I consider the differences between Google’s approach to enterprise unified communications and what such companies as Cisco, Microsoft, and Verizon offer. www.kmworld.com
Smart Business Network publishes about 20 regional business magazines. My column appears in each region’s publication. This month’s column is “Why a Web Site Is So Yesterday.” The idea is that other types of electronic presence is required. A Web page won’t pull the marketing cart in today’s world. www.sbnonline.com
In addition, the ArnoldIT.com team is generating original content for Access Innovations and IGear. If you want to add original content to your blog, let me know. I can provide you with options and costs for this service. The Beyond Search blog and my columns have created a spontaneous demand for substantive content on technical and business issues. Quite a surprise for this aging goose. One plus is that traffic to a Web site increases with the ArnoldIT.com “triple tap” method. Forget traditional public relations and consider the original content methods. Write seaky2000@yahoo.com for information.
Stephen E Arnold, July 6, 2010
This is a commercial message bought by Stephen E Arnold in order to generate vast sums of money and help companies wanting a marketing method that really works.
-
- Subscribe to Beyond Search
Feature archive
News archive
-
