RedMonk and Open Source
May 18, 2011
If you have worked with traditional consulting firms, you know that “open” is not part of the standard method. At RedMonk, open is a pivot point. The company provides a range of services to organizations world wide which have a need for intelligence about open source software. RedMonk has emerged as one of the leaders in the open source community, providing traditional advisory services as well as specialized capabilities tailored to the fast growing open source sector.
You can learn read an exclusive interview with Stephen O’Grady, one of the founders of RedMonk. In a wide ranging discussion with Stephen E Arnold, publisher of Beyond Search, Mr. O’Grady talks about open source technology and its impact on traditional commercial, proprietary software.
In response to a question about the business implications of open source software, he said:
As with every other market with credible open source alternatives, the commercial landscape of search has unquestionably been impacted. Contrary to some of the more aggressive or doom crying assertions, open source does not preclude success for closed source products. It does, however, force vendors of proprietary solutions to compete more effectively. We talk about open source being like a personal trainer for commercial vendors in that respect; they can’t get lazy or complacent with open source alternatives readily available.
He continued:
Besides pushing commercial vendors to improve their technology, open source generally makes pricing more competitive, and search is no exception here. Closed source alternatives remain successful, but even if an organization does not want to use open source, search customers would be foolish not to use the proverbial Amdahl mug as leverage in negotiations.
You can read the complete interview with Mr. O’Grady at http://wp.me/pf6p2-4A2. He will be participating in the Lucene Revolution Conference as well.
Don C. Anderson, May 18, 2011
Freebie
Knowledge XChanger Version 7.0 from Comintelli
May 18, 2011
ThomasNet News showcases “Comintelli Launches New Knowledge XChanger7.0, Promising Reduced Information Overload.” This press release from Comintelli heralds the latest version of their lauded information portal. The Swedish company prides itself on its responsiveness to client feedback:
Several Comintelli customers and partners have helped benchmark the performance and scalability of the new architecture and provided their knowledge, search, and user interface (UI) resources during the development process. “’One of Comintelli’s long-standing operating principles is to build our solutions based on customer feedback and enhancing usability,’ said Jesper Martell, CEO.
Knowledge XChanger integrates with systems such as SharePoint and automatically aggregates, structures, analyzes, and delivers data. Emphasizing the product’s ability to address the needs of sizable enterprises, the write-up describes five areas of emphasis: large-scale content distribution in a variety of formats; fast and flexible data import handlers; improvement in search performance and speed; customizable interfaces; and easier content creation and sharing.
Cynthia Murrell, May 18, 2011
Freebie
Exclusive Interview: Stephen O’Grady, RedMonk
May 18, 2011
Introduction
The open source movement is expanding, and it is increasingly difficult for commercial software vendors to ignore. Some large firms have embraced open source. If you license, IBM OmniFind with Content Analytics, you get open source plus proprietary software. Oracle has opted for a different path, electing to acquire high profile open source solutions such as MySQL and buying companies with a heritage of open source. Sun Microsystems is now part of Oracle, and Oracle became an organization of influence with regard to Java. Google is open source, or at least Google asserts that it is open source. Other firms have built engineering and consulting services around open source. A good example is Lucid Imagination, a firm that provides one click downloads of Lucene/Solr and value-add software and consulting for open source search. The company also operates a successful conference series and has developed specialized systems and methods to handle scaling, big data, and other common search challenges.
I wanted to get a different view of the open source movement in general and probe about the more narrow business applications of open source technology. Fortunately I was able to talk with Stephen O’Grady, the co-founder and Principal Analyst of RedMonk, a boutique industry analyst firm focused on developers. Founded in 2002, RedMonk provides strategic advisory services to some of the most successful technology firms in the world. Stephen’s focus is on infrastructure software such as programming languages, operating systems and databases, with a special focus on open source and big data. Before setting up RedMonk, Stephen worked as an analyst at Illuminata. Prior to joining Illuminata, Stephen served in various senior capacities with large systems integration firms like Keane and consultancies like Blue Hammock. Regularly cited in publications such as the New York Times, NPR, the Boston Globe, and the Wall Street Journal, and a popular speaker and moderator on the conference circuit, Stephen’s advice and opinion is well respected throughout the industry.
The full text of my interview with him on May 16, 2011 appears below.
The Interview
Thanks for making time to speak with me.
No problem.
Let me ask a basic question. What’s a RedMonk?
That’s my favorite question. We are a different type of consultancy. We like to say we are “not your parents’ industry analyst firm.” We set up RedMonk in 2002.
Right. You take a similar view of industry analysts and mid tier consulting firms that I do as I recall.
Yes, pretty similar. We suggest that the industry analysis business has become a “protection racket… undoubtedly a profitable business arrangement, but ultimately neither sustainable nor ethical. In fact, we make our content open and accessible in most cases. We work under yearly retained subscriptions with clients.
Over the last nine years we have been able to serve big household names to a large number of startups. We deliver consulting hours, press services, and a variety of other value adds.
Quite a few firms say that. What’s your key difference?
We are practical.
First, RedMonk is focused on developers, whom we consider to be the new “kingmakers” in technology. If you think about it, most of the adoption we’ve seen in the last ten years has been bottom up.
We’re “practitioner-focused” rather than “buyer-focused”. RedMonk is focused on developers, whom we consider to be the new “kingmakers” in technology. If you think about it, most of the adoption we’ve seen in the last ten years has been bottom up. Our core thesis is that technology adoption is increasingly a bottom up proposition, as demonstrated by Linux, Apache, MySQL, PHP, Firefox, or Eclipse. Each is successful because these solutions have been built from the ground floor, often in grassroots fashion.
Third, we are squarely in the big data space. The database market was considered saturated, but it exploded with new tools and projects. A majority of these are open source, and thus developer friendly. We are right in the epicenter of that shift.
Do you do commissioned research?
No, we don’t do commissioned research of any kind. We just don’t see it as high value, even if the research is valid.
How has the commercial landscape of search specifically, and data infrastructure generally, been impacted – for better or for worse – by open source?
As with every other market with credible open source alternatives, the commercial landscape of search has unquestionably been impacted. Contrary to some of the more aggressive or doom crying assertions, open source does not preclude success for closed source products. It does, however, force vendors of proprietary solutions to compete more effectively. We talk about open source being like a personal trainer for commercial vendors in that respect; they can’t get lazy or complacent with open source alternatives readily available.
Isn’t there an impact on pricing?
Great point.
Besides pushing commercial vendors to improve their technology, open source generally makes pricing more competitive, and search is no exception here. Closed source alternatives remain successful, but even if an organization does not want to use open source, search customers would be foolish not to use the proverbial Amdahl mug as leverage in negotiations.
When the software is available for free, what are customers paying for?
Revenue models around open source businesses vary, but the most common is service and support. The software, in other words, is free, and what customers pay for is help with installation and integration, or the ability to pick up the phone when something breaks.
A customer may also be paying for updates, whereby vendors backport fixes or patches to older software versions. Broadly then, the majority of commercial open source users are paying for peace of mind. Customers want the same assurances they get from traditional commercial software vendors. Customers want to know that there will be there someone to help when bugs inevitably appear: open source vendors provide that level of support and assurance.
What’s the payoff to the open source user?
That’s my second favorite question.
The advantages to this model from the customer perspective are multiple, but perhaps the most important is what Simon Phipps once observed: users can pay at the point of value, rather than acquisition. Just a few years ago, if you had a project to complete, you’d invite vendors in to do a bake off. They would try to prove to you in an hour or two demo that their software could do the job well enough for you to pay to get it.
This is like an end run, right?
In general, but we believe open source software inverts the typical commercial software process. You download the software for free, employ it as you see fit and determine whether it works or not. If it does, you can engage a commercial vendor for support. If it doesn’t, you’re not out the cost of a license. This shift has been transformative in how vendors interact with their customers, whether they’re selling open source software or not.
The general complexion of software infrastructure appears to be changing. Relational databases, once the only choice, are becoming rather one of many. Where does search fit in, and how do customers determine which pieces fit which needs?
The data infrastructure space is indeed exploding. In the space of eighteen months we’ve gone from relational databases are the solution to every data problem to, seemingly, a different persistence mechanism per workload.
As for how customers put the pieces together, the important thing is to work backwards from need. For example, customers that have search needs should, unsurprisingly, look at search tools like Solr. But the versatility of search makes it useful in a variety of other contexts; AT&T for example uses it for Web page composition.
What’s driving the adoption of search? Is it simply a function of data growth, as the headlines seem to imply, or is there more going on?
Certainly data growth is a major factor. Every year there’s a new chart asserting things like we’re going to produce more information in the next year than in all of recorded history, but the important part is that it’s true. We are all–every one of us–generating massive amounts of information. How do you extract, then, the proverbial needle from the haystack? Search is one of the most effective mechanisms for this.
Just as important, however, has been the recognition amongst even conservative IT shops that the database does not need to be the solution to every problem. Search, like a variety of other non-relational tools, is far more of a first class citizen today than it was just a few short years ago.
What is the most important impact effective search can have on an organization?
That’s a very tough question. I would say that one of the most important impacts search can have is that a good answer to one question will generate the next question. Whether it’s a customer searching your Web site for the latest Android handset or your internal analyst looking for last quarter’s sales figures, it’s crucial to get the right answer quickly if you ever want them to ask a second.
If your search fails they don’t ask a second question, you’ll either have lost a potential customer or your analyst is making decisions without last quarter’s sales figures. Neither is a good outcome.
Looking at the market ahead, what trends do you see impacting the market in the next year or two? What should customers be aware of with respect to their data infrastructure?
There are a great many trends that will affect search, but two of the most interesting from my view will be the increasing contextual intelligence of search and the accelerating integration of search into other applications. Far from being just a dumb search engine, Solr increasingly has an awareness of what specifically it is searching, and in some cases, how to leverage and manipulate that content whether it’s JSON or numeric fields. This broadens the role that search can play, because it’s no longer strictly about retrieval.
And integration?
Okay, as for integration, data centers are increasingly heterogeneous, with databases deployed alongside MapReduce implementations, key-value stores and document databases.
Search fills an important role, which is why we’re increasingly seeing it not simply pointed at a repository to index, but leveraged in conjunction with tools like Hadoop.
What kind of threat does Oracle’s lawsuit over Google plus Java pose to open source?How does it compare to the SCO controversy with Linux some years back?
In my view, Oracle’s ongoing litigation of Google over Java related intellectual property has profound implications for both participants, but also for the open source community as a whole.
The real concern is that the litigation, particularly if it is successful, could have chilling effects on Java usage and adoption. As far as SCO is concerned, this is somewhat different in that it targets a reimplementation of the platform in Android rather than the Java platform itself. SCO was threatening Linux rather than a less adopted derivative.
While users of both Java and MySQL should be aware of the litigation, however, realistically the implications for them, if any are, are very long term. No one is going to abandon Java based open source projects, for example, based on the outcome of Oracle’s suit.
It seems like everyone who is anyone in the software world has an open source strategy, even through to Microsoft’s embrace of php. Should information technology executives and decision makers, who were once suspicious of open source, be suspicious of software vendors without a solid open source strategy?
With the possible exception of packaged applications, open source is a fact of life in most infrastructure software markets. Adoption is accelerating, the volume of options is growing, and – frequently – the commercial open source products are lower cost. So it is no surprise that vendors might feel threatened by open source.
But even if they choose not to sell open source software, as many do not, those without a solid open source interoperability and partnership story will be disadvantaged in a marketplace that sees open source playing crucial roles at every layer of the data center. Like it or not, that is the context in which commercial vendors are competing. Put more simply, if you’re building for a market of all closed source products, that’s not that large a market. In such cases, then, I would certainly have some hard questions for vendors who lack an open source strategy.
Where can a reader get more information about RedMonk?
Please, visit our Web site at www.redmonk.com.
ArnoldIT Comment
RedMonk’s approach to professional services is refreshing and a harbinger of change in the consulting sector. But more importantly, the information in this interview makes clear that open source solutions and open source search technology are part of the disruption that is shaking the foundation of traditional computing. Vendors without an open source strategy are likely to face both customer and price pressure. Open source is no longer a marginalized option. Companies from Twitter to Cisco Systems to Skype, now a unit of Microsoft, rely on open source technology. RedMonk is the voice of this new wave of technical opportunity.
Stephen E Arnold, May 18, 2011
Protected: Fixing Development Projects in SharePoint 2010
May 18, 2011
Topix and Blekko Search
May 17, 2011
I noticed a change in the Topix.com search system over the weekend. Results come from Blekko.com. As you may know, Blekko.com is a good option to have in hand when the results from Google or Bing miss the mark. For some of my queries, I use Blekko.com before I wade through the Google result lists.
In my tests of the new search system, I did notice one somewhat annoying characteristic. To see what created extra work for me, navigate to www.topix.com. You will see a page of “Recent US News News & Discussions.” Ignore the “News News” weirdness and look at the search box. Enter a query for a popular topic. I tried “sharepoint” and “dancing with the starts.”.
Notice that the search results appear with a site limiter command; specifically, site:topix.com shown in the screenshot below:
The implementation is a hosted service much like the Blossom.com search solution I use for the Beyond Search blog. The problem is that the index is stale. In the “sharepoint” query, I got a hit to Houston Newswire but the story was no longer available: http://www.topix.com/wire/houston/p5.
What’s happening is that the new content in Topix is not being indexed quickly by Blekko. Now what I mean by quickly is that I need to be able to locate news stories with my query terms over the last two or three days. I also need to locate today’s stories. Topix is a “news news” site, and I need the search system to refresh its index quickly.
What’s the fix? Easy, copy the query string from the Blekko Topix results page, in my case, site:topix.com sharepoint and navigate to Google.com. Then paste the query string into the Google search box and you get more current hits on the topic in which you are interested.
Can Blekko speed up its indexing cycle? Sure, and I think the company will. Google and other search systems create priority lists for content acquisition. Blekko needs a similar high priority operation to make Topix search useful to those who look for “news news”, to use Topix’ own weird lingo.
How much faster is the Google cycle? On my spot tests, Google was lagging by hours. Blekko was lagging by days but even more disturbing was the Blekko Topix results did not present content. I was getting for my sharepoint query a list of jobs shown below.
This is a really bad relevance issue, which when combined with the index staleness, is causing me to use Google.com to search Topix.
Hopefully this issue will be remediated soon.
Stephen E Arnold, May 17, 2011
Freebie
Google News: Entertainment Tonight Redux
May 17, 2011
I found the posts about the revamp to Google News fascinating. Some history. Google News has not been monetized. Google News has chugged along for years. Google News has not mesmerized me. I prefer the approach of the lesser known Videotwitter.net or Newsnow. The point of view in “Eventually, One Of These Updates Will Make Google News Not Suck At Tech News, Right?” is in line with mine. However, I think that Google is trying rather hard to become what I call “consumery”. With 40 percent of ad budgets flowing to television, Google has to find a way to “TV-ize” its news experience. Here’s the passage I noted in the TechCrunch article:
it’s so obvious what Google’s algorithms are looking for, there’s a not-so-secret trick on how to game them. Instead of rushing to get a post out about a hot story, just wait a few hours. By then, the story will bubble towards the top of Google News’ tech section and if you time it correctly and you’re a site that Google News watches (some good, some bad), you can easily be the top headline for the entire section (which also means placement on the main site as well). This works because Google News favors recent “takes” on a story instead of actual sources of information. You could argue that’s okay in some situations, but often these “takes” are just like the Today’s THV bullshit: republishing AP content late. In other words, if you’re in the business of breaking news, you’ll almost never find your stories on Google News. Instead, you’ll find someone else re-purposing your story there hours later and reaping tens of thousands of pageviews as a result of the sloth and/or jackassery.
The language is a bit more frisky that we use in Beyond Search, but the idea is on the money. However, I don’t think TechCrunch goes far enough. The challenge Google faces in tech news exists in other high value, niche sectors too.
Google is in a rush to the middle in an effort to pump up page views, eyeballs, and, hence, ad revenues. The problem for me is that other outfits do a better job of delivering links to stories important to me. Even Videotwitter does a better job of covering the really hot click generators like pop culture and sports. As for technology, I find Techmeme useful along with Daily Rotation, and the tech section of Newsnow. Each of these does an arguably better job of flagging what’s hot and making it dead easy for me to get to the source.
My question is, “Will Google’s consumery approach pay off?” The company has many initiatives, and I find them somewhat distracting. I remember what happened when the long lost CompuServe lost its way. Thrashing does not a business grow in my experience. Google is starting to move toward the Entertainment Tonight approach to content. Interesting.
Stephen E Arnold, May 17, 2011
Freebie unlike the late, great CompuServe
Blekko Gets Discovered
May 17, 2011
We have known about Blekko for quite a while. The New York Times, more recently, figured out that the search engine’s 30 percent growth in the last few months is no accident.
Now that the New York Times writes about Blekko, it is official. Move over Google and Bing, there’s a new search engine on the block. In the New York Times’ “An Engine’s Tall Order: Streamline the Search,” writer Damon Darlin explains the problems with Google search results and Blekko.com’s solution. We learned:
“While you may get them (Google search results) very rapidly, they may not be all that useful and dependable.” Various search engine optimization efforts help to artificially move a Web page or article to the top of Google’s search results page. “Web pages are created specifically to fool Google’s search algorithm in order to get a higher ranking.“
Industry veteran Rich Skrenta is behind Blekko.com, which “uses a search algorithm like Google’s or Bing’s but also gets humans, mostly volunteers, to identify the sites they know, trust and visit most often and to put those at the top of the search results.” Coverage in the New York Times helps put Blekko on the mainstream radar.
About time.
Rita Safranek, May 17, 2011
Freebie unlike the hard copy of the New York Times
Google, Skyhook and Information
May 17, 2011
In the more-legal-hassles-for-Google department, The Register reports “Court Rejects Google Call to End ‘Android Not Open’ Suit.” Google tried to get this case dismissed, but Massachusetts Superior court justice Judith Fabricant rejected the motion, saying that she’d like to see Skyhook’s evidence.
Long a provider of mobile device location services, Skyhook accuses Google of strong-arming mobile handset makers into giving them an unfair advantage:
“The suit specifically claims that Andy Rubin, who oversees Google’s Android project, told Motorola co-CEO Sanjay Jha that if Motorola didn’t drop Skyhook from its phones, Google would remove official Android support from the devices. This would mean the devices could not use proprietary Google apps or the Android name. The suit says that whereas Google paints Android as open source, Google still maintains exclusive oversight of the OS.”
As Google continues to push boundaries, the law keeps pushing back. The company is still growing by leaps and bounds, though, so these struggles must be worth it to them. So far, they seem to be coming out ahead. The documents related to the case may provide some insight into how Google uses Android to help ensure that Google achieves its revenue goals and business objectives. Money, we think, is going to become a key factor in Google’s business activities. Our view is that the online advertising money machine is going to get some competition. And certain fast growing markets like China are going to force Google to pull the type of pricing moves associated with professional publishers and marketers of commercial data.
What insights will the Skyhook matter provide into Google’s open source methods? Interesting.
Cynthia Murrell, May 17, 2011
Freebie
Maps on Steroids
May 17, 2011
Here is an interesting link from the people behind Mapsys.info: “Public Data Visualization with Google Maps and Fusion Tables”.
“Visualizing” public data basically means mapping information that is relevant to a community. A good working example mentioned in the posting is San Francisco’s Bay Area bike accident tracker. The map’s legend decodes the various colored dots as the type of accident and how it came to be recorded.
Source: http://mapsys.info/
A screenshot of the coding needing to display a map with personalized details is offered in the posting. The star of the show is the integration with a fusion table, a tool offered by Google to house data sets to be presented on a map. Added functionality is included by using “SQL-like query syntax” and leveraging “the Python libraries Google provides for query generation and API calls”. This allows you to pick smaller data sets out of the fusion table.
So behind the scenes, this looks like another example of search moving beyond the token keyword. You won’t hear any complaints out of us. I remember creating maps using old fashioned methods when I was working on my engineering degree. This method delivers accuracy and time savings.
Sarah Rogers, May 17, 2011
Freebie
Google and the $64 Billion Question
May 17, 2011
Short honk: No big ideas about this. I found the statement in “Google’s 2010 Economic Impact” fascinating:
Today we’re announcing that Google provided $64 billion of economic activity to businesses, website publishers and non-profits in 2010. This is an 18% increase from the economic impact total in 2009. Here’s how it works: for every $1 a business spends on Google AdWords, they receive an average of $8 in profit through Google Search and AdWords.
I like the $64 billion number. Big and resonant. However, I had one question, “Why is Google pitching its economic impact?” Google makes its economic impact with each quarterly report. Is something putting pressure on Google to squeeze such congratulatory utterances from the “official Google blog”?
My hunch. Something is up or down. Maybe online ad performance? Maybe hype to pump up Google bonds? Interesting timing.
Stephen E Arnold, May 17, 2011
Freebie unlike Google Adwords with their 8 to 1 return. Wow. How about some big data to support the assertion?
-
- Subscribe to Beyond Search
Feature archive
News archive
-
