Exclusive Interview: Hadley Reynolds, IDC
August 18, 2010
One of the big guns in search and content processing consulting is IDC, a firm to which many senior managers turn when trying to figure out the fast-changing world of digital information. IDC describes itself as:
the premier global provider of market intelligence, advisory services, and events for the information technology, telecommunications, and consumer technology markets. IDC helps IT professionals, business executives, and the investment community make fact-based decisions on technology purchases and business strategy. More than 1000 IDC analysts provide global, regional, and local expertise on technology and industry opportunities and trends in over 110 countries worldwide. For more than 46 years, IDC has provided strategic insights to help our clients achieve their key business objectives. IDC is a subsidiary of IDG, the world’s leading technology media, research, and events company.
I go along with the description because every once in a while I do a project for IDC, and I do like to get paid for my research.
Hadley Reynolds is a featured speaker at the Lucene Revolution Conference, Boston, Mass., October 7 and 8, 2010. Information about the conference is available at http://www.lucenerevolution.com.
Mr. Reynolds focuses on understanding business transformation opportunities through the application of search technologies to traditional business models and the role search innovation is playing in creating new business opportunities. Prior to joining IDC, he guided the Center for Search Innovation at Microsoft Fast. A former Delphi Group consultant, he brings business and technical expertise to bear on his work in search.
I tracked down Hadley Reynolds, one of IDC’s senior consultants in the search practice. We conversed in Framingham, Massachusetts. We talked at the Starbuck’s in Framingham. I have transcribed our talk in the dialogue below:
Hadley, thanks for taking the time to talk with me?
Glad to. I read your blog and I must say that I don’t agree with some of your points.
No problem. The blog is designed to spark discussion, not compete with the work you and your colleagues do at IDC.
We do not consider your comments on the world of search competitive in any way. Go for it.
Okay, let me get right to the point. I noticed that you are on the program for the Lucene Revolution Conference. Why the interest in open source software?
Anyone who has been watching the enterprise search market in recent years has to be impressed with the growth record of the Lucene: in technical enhancements, community participation, and acceptance in the commercial market.
Enterprise search is usually a proprietary solution. Open source relies on a community. Is the community angle real or a myth?
When the Lucene project was in its early days, I was highly skeptical about the likelihood of a community of sufficient horsepower coalescing around it and carrying forward Doug Cutting’s foundational work. However, with IBM and other smaller commercial players like Attivio, Palantir, i2, Lucid Imagination, and others both using and contributing to the codebase, plus continuing contributions from search “independents”, I think the skepticism about critical mass and future support can be laid to rest.
I see some companies playing what I call the “open source card”. A couple of examples are IBM and Oracle. IBM uses Lucene/Solr in OmniFind 9.1, and Oracle bought Sun Microsystems and then MySQL. Won’t that confuse enterprise customers?
The trend in IT is toward more open source software, not less. First it was accepted that common resources like the Linux operating system and the Apache Web Server and the Mozilla browser were fine alternatives for the enterprise. Then we saw open source databases like MySQL and others, and multiple content management systems like Alfresco, Drupal and many more. So it’s not surprising that we now have successful open source search in Lucene/Solr and that commercial companies who utilize search as a component, and even those that sell search-based applications, would want to take advantage of the strength of these products both for themselves and for their customers.
I am interested in Lucene/Solr but I also track Drupal, Hadoop, and other open source projects. What are benefits you see in open source software?
The benefits of using Lucene/Solr are not hard to enumerate: rapid time-to-market, high-level functionality, flexibility and customization, low entry level cost, positive future outlook for technology evolving with the state of the art.
I agree. But the reason there is proprietary software is to deliver a solution with “one throat to choke.” What are the drawbacks of open source software where no one may be responsible for a code fix, a new function, or a code widget?
The drawbacks include: enhancements on a community timetable only, potentially expensive customization, requirement for advanced skills in-house or near-at-hand, delivered functionality will trail the truly cutting edge search software specialist firms, and system life costs can become significant.
When I think of IDC I think of commercial software. With this open source “revolution” on display in the promotion for the Lucene Revolution Conference and the stepped up marketing of the conference sponsor, Lucid Imagination, something is happening and it may not be the traditional enterprise play. When someone asks you why you don’t use a commercial search solution, what do you tell them?
I believe that for many search problems in the enterprise, the smart approach is to select the tool that gets the job done with the least delay and at the most reasonable investment level. More and more frequently, open source search software is becoming the tool of choice.
Wow it looks like your phone is going to jump off the table.
Right, I need to get back.
How do people contact you?
Your readers can get me by sending an email to hreynolds at idc dot com.
Stephen E Arnold, August 18, 2010
Sponsored post. Okay?
New Blog with ArnoldIT.com Content
August 18, 2010
A new Web log and information services débuts today (August 18, 2010). Redefining Monitoring, owned by IGear Corporation, provides news, information, and commentary about cloud-centric monitoring. IGear’s technology embraces numerically-controlled machines and production systems as well as other business processes. IGear’s cloud technology makes it possible to “take the pulse” of smart machines, systems, and complete production operations 24×7. The IGear dashboard provides an intuitive, graphical display. Access from mobile devices, netbook computers, or desktop systems is supported. You can access the Web log at www.redefiningmonitoring.com.
In a statement released by IGear today, the company said:
IGear, a leader in the monitoring industry, has launched Redefining Monitoring at www.redefiningmonitoring.com, a blog that covers news and information about monitoring and the latest in cloud computing technologies.
“We decided to launch the blog after much discussion with clients, colleagues and staff. There are many exciting issues and options which come across my desk every day, and this is a great way to share them as well as our thoughts about their impact with a broader group of people. We can add some ‘color and shading’ to the disciplines involved in monitoring a range of production equipment and manufacturing systems, among others,” explained Don Korfhage, president of IGear.
One of IGear’s principal backers said, “we are experiencing a new era in monitoring driven by advances in cloud and wireless technologies along with the desire of people to have information at their fingertips, 24×7”.
Redefining Monitoring has several standard sections, including news, features and recent posts. Anyone interested in updates can subscribe to the blog through Feedburner and comments are welcome.
IGear serves as the foundation of numerous OEM equipment builder private label offerings. Since 1986, IGear software has been the foundation of reliably collecting critical data from thousands of machines globally.
IGear provides valuable information and alerts to OEMs and their customers enabling them to make better decisions and operate in a lights-out capacity. With IGear, OEMs more proactively service equipment, isolate problems, and optimize their service technicians – the result – a quicker resolution and more satisfied end-user customer.
“I/Gear – Always On” – ushering in a new era in monitoring.
For more information, navigate to the IGear Web site at www.igearonline.com.
ArnoldIT.com provides content for this news and information service. If you are interested in focused, professional content with high impact, write seaky2000 at yahoo dot com. The ArnoldIT.com Overflight system generates content for www.taxodiary.com, www.theseed2020.com, and the Beyond Search blog. Beyond Search’s content pushes beyond SEO.
Kenneth Toth, August 18, 2010
Sponsored post
Google Street View Faces German Skepticism
August 17, 2010
Google has caught Germany by surprise with its sudden and quiet announcement to carry on with its controversial Street View service in 20 German cities. As reported in “Google Itself Is Responsible for the Massive Skepticism”, more shocking for the Germans including many of the politicians who are away on vacation, is the limited one month period given by the company to request for their homes to be blurred out from the images provided by the service.
Google has drawn skepticism by this move, which is seen by many as a deliberate attempt to reduce the number of objections to be registered against Street View. However, the public and commentators on the editorial pages are divided over the issue of digital mapping of Germany. Some call Google as setting its own rules, and a non-cooperator, compromising only under public pressure, while some hail Street View as the map of the future, and do not want to be left out!
Leena Singh, August 17, 2010
LinkedIn’s Data Infrastructure
August 17, 2010
LinkedIn is one of the premier networking sights for business professionals. It is a great way for professionals to meet other industry professionals or stay connected to their colleagues. LinkedIn has millions of members located in various countries so this adds up to massive amounts of data being process daily. In “LinkedIn’s Data Infrastructure” LinkedIn’s Principal Engineer and Engineering Manager Jay Kreps provides attendees at the Hadoop Summit an insight look at how LinkedIn processes data.
LinkedIn keeps the majority of its vital data offline. The offline data process is relatively slow so LinkedIn processes batches of data daily. They use the open source program Hadoop in their daily calculations. The term open source describes any type of program where the owner provides the source code along with the software license that allows users to modify the software if necessary. Hadoop is a popular framework because it is designed to help users work with massive amounts of data.
Kreps made sure to mention two of LinkedIn’s most vital open source projects, Azkaban and Voldemort. The article describes Azkaban as “an open source workflow system for Hadoop, providing cron-like scheduling and make-like dependency analysis, including restart. Its main purpose in the system is to control ETL jobs which are responsible for pushing the database as well as any event logs to Voldemort.
Voldemort can simply be described as a storage facility for LinkedIn’s NoSQL key/value. LinkedIn produces a daily relationship graph that is used for querying in web page results. The data must go through an extensive process which was once done by a database. However, this process was counterproductive because the database had to modify first and then the data had to be manually moved. Voldemort makes partitioning along with the entire data movement process easier and more productive.
Readers can go to Data Applications and Infrastructure at LinkedIn Hadoop Summit 2010 to view the data path and additional information. LinkedIn also has a handy index structure implemented in the Hadoop pipeline for extensive searches. The open source Lucene/Solr is used in the search features to make sure users can conduct advanced searches and obtain accurate information quickly. Open source was instrumental in LinkedIn being able to build a productive database able to specifically handle their massive data load which was exactly what the doctor ordered.
April Holmes, August 17, 2010
Facebook Pages Become Customer Support Centers
August 17, 2010
Consumers are the driving force behind any successful business. Many companies are behind when it comes to their CRM (customer relationship management) and though they may have excellent products customers are unable to get the quality support they need and deserve. Issues with customer service can lead to customers jumping ship and taking their money elsewhere. Many businesses have Facebook pages aimed at consumers. Facebook is improving by “brining customer service software to businesses living inside the world’s biggest social network.” “Facebook Pages Become Customer Support Centers” provides a little insight on the new support system designed by Parature. Customers will be able to choose from several different options and find the answers they need quickly. Users will no longer be a victim of the dreaded phone tree Hades. A language processing vendor will become one of the first to use the new service. Time will tell if this customer service software is effective but sometimes talking to a real person is the best fix. We think this repurposing of Facebook has significant implications for the hapless customer support search sector.
April Holmes, August 17, 2010
2010 Trends in Open Source Systems Management
August 17, 2010
Zenoss wanted to examine the system management trend among IT professionals in attendance at the USENIX Large Installation System Administration (LISA) conference so they conducted a system management survey in 2006, 2007, 2008 and 2009. They were especially interested in the utilization of open source software to handle the IT management needs of large groups. Survey results were compared to data obtained from the Zenoss user community. Out of 974 survey participants over 98% of them acknowledged they use open source in their enterprises. More than 66% of the Zenoss community survey participants preferred to use open source if available. Quality of support was the main reason enterprises liked open source. In addition 50% of those surveyed were using cloud technology. Read the Zenoss blog “2010 Trends in open Source Systems Management” to find a few of the results obtained as well as view the complete results of the 2010 Open Source Systems Management Survey.
Our view is that open source is going to put search and content processing into a martini shaker and deliver James Bond’s potent cocktail to some unwitting tipplers.
April Holmes, August 17, 2010
Semantic Enterprise: The StartUps
August 17, 2010
The enterprise software market is full of possibilities and there are many opportunities available for new semantic enterprise start ups. In “Semantic Enterprise: The StartUps” readers are provided with a list of semantic enterprise start ups. The company, Ontoprise oversees a number of products. Ontoprise has an impressive client base but with so many different products it can become difficult to keep up with emerging changes which can negatively impact their longevity in the market. Revelytix is another start up but it has the potential to thrive as a semantic enterprise vendor because its emergent analytics is gaining momentum and interest. The start up Franz has gained its reputation on the sale of development tools and database management products. There is a big market for development tools and database management and if Franz can continuously evolve with technology they could be a lasting vendor. There is a huge profit at stake and each start up vendor has to build a thriving business in order to get a piece of the pie.
April Holmes, August 17, 2010
Google, Acquisitions, and Like What?
August 16, 2010
When I was finishing a client’s research report in 2005, I learned that Google was poking around Riya. Nothing came of that rumor, and I got the idea to take a look at some of Google’s patent applications as a possible pool of information for what eventually became Google Version 2.0: The Calculating Predator. I read “Google to Acquire Like.com after Leaving Them at the Altar in 2005” and a thought struck me. Google’s acquisitions in the last six or seven months seem to be duplicative; that is, the features and functions of the acquired companies seem to be similar to systems and methods mentioned or described in Google patent applications. Applications are interesting to me because these documents provide a small window through which to glimpse some of the thinking going on inside a company’s research groups. Most applications just sit there and gather dust. Some become real patents. Patent applications are one bobbing cork in an organization’s business stream. Not definitive, patent applications are interesting.
The questions in my mind this morning are:
- Google’s spending spree seems to me to be similar to the approach employed by Yahoo in the pre-Semel years. That strategy helped contribute to Yahoo’s technical fragmentation. Does Google’s management team have a method to minimize fragmentation and prevent the construction of technical silos?
- Has it become sufficiently difficult to get Google engineers to work on certain projects that Google’s management finds it more expedient to buy a company with a dedicated team? If the answer is, “Yes, buying a company produces a ready-to-code team”, how will this shift affect Goggle’s “Googliness”? Being Googley is the culture, so perhaps the culture at Google is shifting?
- When a company gets sucked up by another entity, time is required to deploy the landing gear, get on an approach, and then bring the aircraft down for a safe landing on the aircraft carrier. More time is required to move the aircraft from the flight deck to its designated location and get the pilot and crew settled into their quarters. The problem is that Google’s competitors are moving too. Google, particularly in the social space, may have to spend more time moving aircraft than deploying them to harm the companies’ enemies. Amazon, Apple, Facebook, and now Oracle are attacking the Google. How much time does Google have to align its parts and get its assets where they are urgently needed?
What I find interesting is that so much at Google seems to be going in a surprising direction. There is the hassle in South Korea, the Oracle Java dust up, the Facebook targeted ad thrust which is a potentially big problem, and the testiness of the Cupertino crowd reflecting its leaders luminescene into the eyes of Googzilla, and the pressure cooker of the Verizon probe to tier wireless fees. If advertising revenue softens, the Google will have a chance to demonstrate the wisdom of its current approach to innovation and monetization.
Perhaps the logic of Math Club management will break new ground in organizational development?
Stephen E Arnold, August 16, 2010
Definitely a freebie
Open Source May Be Disruptive
August 16, 2010
I do not know much about software, information, or the big-time doings of corporate giants. I am not an azurini, a self-appointed poobah, or the cat’s paw of a group of 20-something MBAs from schools that require a great family and a high IQ. I am an addled goose. I float around in a pond filled with mine drainage water and offer some humble observations which the great and powerful dismiss as silly, irrelevant, or just plain incorrect.
No problemo.
However, even the intellectual black hole of the addled goose’s analytic muck pond can figure out from two articles that open source is scaring the heck out of a really tough, superstar executive.
You make your own decision about the accuracy and significance of these two news stories:
First, point your browser thingy with monitoring functions activated so those watching know you really did navigate to a “real” news source and read “Oracle Kills Open Solaris, Moves Development behind Closed Doors.” The idea is pretty easy to understand. Those super-marketers at Sun Microsystems gave away an operating system as open source. Nope. Oracle’s Larry Ellison and his sharp-pencil crowd shut that door. The reason? Open source equals bad business. “Bad”, I presume, means a threat to Oracle’s pricing tactics. Free and Oracle are not words that I associate when I hear the word “Oracle.”
A happy quack to http://jordanhall.co.uk/general-articles/dont-be-evil-licensing-1301401/ for this great illustration.
Second, Oracle is suing Google over its use of Java. Now Java is sort of a piggy, but, hey, lots of universities teach Java, and it can be quite useful when running in today’s nifty hardware environments. Overlook those flaws which have been documented in some detail in the Software Engineering podcasts at www.se-radio.net. Notice: SE-Radio is not exactly an Adam Carolla Leo LaPorte type podcast. You can get some information about the this tussle between two former bosom buddies by tapping to “Initial thoughts on Oracle vs Google Patent Lawsuit.” Yep, those are links to patent documents, so I don’t think the skimmers among my readers will invest much dwell time on the Tirania post.
Nevertheless, the headlines may be enough for a “real” azure chip consultant. The details are murky and former English majors and sociology minors won’t spend too much time doing the analysis a “real” poobah does.
Let me Cliff Notes it: Larry Ellison is a smart, rich person. He understands that open source is a problem for a company like Oracle that charges really big fees for its software. Open source with its unruly developers and hard-to-make-do-push-ups work ethic are the enemy. The fix. Kill open source. If total annihilation is not possible, make open source expensive in terms of legal fees. The way law works for rich people is that a rich person’s lawyers can make a less rich person spend lots of time fighting the rich person’s legal actions. Eventually money wins, particularly when there are fuzzy wuzzy ideas like open source, intellectual property, and rich people arguing as the main action.
There is just one snag. Even rich people have trouble keeping those peasants under control. For my readers who stayed awake during world history, you know that lots of peasants with sticks and rocks can be a real problem. Honk off enough peasants, and the excitement can end in a revolution.
At this moment in the capitalistic, free market sun:
- Lots of companies are strapped for cash. Free is pretty darned appealing when you have to decide how to pay the light bill, the actress assisting the company at a trade show, and paying the lease on a new Bimmer.
- Open source is pretty good, and there are some robust solutions available with the click of a mouse. Examples include Drupal, Hadoop, Lucene/Solr and * lots * more.
- The open source stuff is fun. Training and certification for Oracle or other “carrier class” enterprise solutions are not as much fun as blasting around the lake on a jet ski at 30 knots.
If I focus on relational databases, I am in a Roman ruin. You can see or at least imagine the splendor of the structure. But rebuilding it after a crash and getting it back to the “way it was” is just too much work, too expensive, and too labor intensive. Why not build a new structure, using the tips and methods that HGTV puts on display each night on my local cable channel. Need a granite counter top and have neither money, stone cutting tools, nor expertise. Hey, just shoot over to Home Depot and get an epoxy alternative. That’s the open source approach: New materials, new methods, and new benefits.
Roman ruin. What’s the cost of a rebuild and then upkeep? How do you modify a limestone flying balcony? You don’t. Get some slaves.
Oracle and Google: Interesting Enemies
August 16, 2010
“How Oracle Might Kill Google’s Android and Software Patents All at Once” is an interesting write up. You should read it and make your own decision about its premise. For me, I was struck by the notion that Android itself could be jeopardized. Here’s the passage that got my wheels turning:
That fact not only highlights that Oracle is just as “open source friendly” as Google, but that it’s also more responsible in developing open source software in such a way that it doesn’t recklessly expose itself to being sued the way Sun did, or the way Google did.
I am not sure I see Oracle as quite the champion of open source that Roughly Drafted suggests. The article continued:
Once the iPhone and other platforms reach Verizon over the next six months, Android’s sales will scale back down domestically, and all the platform will have to recommend itself is a lot of adware, malware, copyright violations and fraudware pushed underhanded developers looking to bilk an audience in a market with no curator. On top of all this, it will also have a top software maker seeking to eviscerate its core development platform, necessitating a significant reworking of what Android even is. Who wants to invest in development for that? Especially if all your work is just going to be pirated by all the Android freetards.
If this assertion is accurate, Google will have to scramble to find another potential money stream. My hunch is that Google may have difficulty monetizing at the scale of AdWords. After 11 years, Google still sells text ads and variations thereof for the lion’s share of its revenues. What happens if the AdWords’ money-machine throws a gear? I recall hearing or reading that Google’s management expects to make money – lots of money – from Android. No Android. Bummer. No Android and declining AdWords revenue. Will the Math Club may be working at Burger King?
The write up also includes a patent “hook”. No matter how lousy the patent system is, I think that change will be slow and litigation will be only marginally faster. Nevertheless, the write up presents an interesting argument. I suppose Oracle folks will no longer be encouraged to sell Google Search Appliances or use Google Apps to address certain issues with Oracle’s own “glue” code like Fusion.
Stephen E Arnold, August 16, 2010
Freebie
-
- Subscribe to Beyond Search
Feature archive
News archive
-
