New Open Source Search Information Service Available
April 16, 2012
Open source search was not a viable option for the enterprise in 2003 when ArnoldIT started work on the first Enterprise Search Report. Stephen E. Arnold wrote two more editions before he decided that proprietary search solutions were becoming “look alikes.” In the ArnoldIT 2011 study, The New Landscape of Enterprise Search, Stephen E Arnold and his editorial team decided not to cover open source search solutions because the sector was moving rapidly and no large players had emerged. Now almost a year after the New Landscape of Enterprise Search, the pace of innovation has increased significantly and there are some significant commercial open source search ventures in the US and elsewhere.
The ArnoldIT editorial team, which consists of librarians and technologists, recommended that we begin the task of identifying important articles to determine if there were sufficient mass to warrant a Beyond Search type of publication focused on open source search. We concluded that there was an increasing flow of information about open source search. Therefore, we want to share this information with others who have an interest in what is shaping up to be a disruptive force in information retrieval.
We want to help document that there is a new approach to enterprise search. The solutions involve the cloud, toolkits, and ready-to-run services available with a mouse click. The vendors pushing forward range from companies which have an established profile in the business community; for instance, IBM and Lucid Imagination. There are some open source search solutions which are not widely known in certain organizations; Xapian and Summa Summix come to mind. In between there are dozens of open source search, content processing, and hybrid services.
ArnoldIT recently completed a study of open source search option. After finishing our research for a client, we decided to move forward on a new information service. OpenSearchNews.com will discuss big data search solutions, including Amazon’s CloudSearch service, Basho Riak, and Constellio. If you are not familiar with these solutions and have an interest in search, you will want to check out OpenSearchNews.com.
The new microsite, now publicly available, publishes Monday through Friday and provides critical commentary, information about products, and highlights additional sources about open source search. The information service will report about the companies, trends, and products which offer an alternative to the seven figure solutions from proprietary enterprise search solutions. The approach of the service will be similar to that taken by researchers who want information that provides essential facts and links to high-value sources of information. The service will provide up-to-date news and analysis about the dynamic market for open source search and will publish Monday to Friday at www.opensearchnews.com. Additional information about the new information service is available on the site’s About page. Keep in mind that we don’t do “real” news. We have more in common with researchers and analysts than those who work for organizations embracing the tenets of Mr. Murdoch.
Recent stories include:
- Enterprise Adoption of Solr Lucene Rises
- In the Future, Enterprise Search Will Be a Service
- Lucid Works 2.0 Attracts Enterprise Suitors
Emily Aldridge, the editor of the publication, is an MLS and expert searcher who demonstrated exceptional capabilities in tracking down information about products and projects with names like Hounder, Oxyus, and Piscator.
Emily Aldridge, editor of the new information service, said:
“Open source search has become a fast-growing segment of the enterprise search and big data markets. The number of companies competing in this segment is growing. Large commercial enterprises are embracing open source and providing useful software to anyone who wants to use it. Two good examples are the contributions of Lucid Imagination and LinkedIn. The Danish government has supported an open source search initiative which provides search features for libraries looking to provide a patron with a single search box for a range of content in different collections.”
The information service will cover cloud solutions, open source search appliances, and mention commercial services which have open source software under the glossy exteriors of products and services from Amazon and IBM. We will also cover related subjects such as proprietary cloud search services. Comments will be accepted, and like other ArnoldIT information services we hope to combine useful information with some pointed observations.
Like Beyond Search, we will roll out new features and functions over time. We plan to use Google’s AdSense to help offset the cost of producing the service. If you want to learn more about the publication, contact us at seaky2000 at yahoo dot com.
Don C. Anderson, Senior Engineer, ArnoldIT, April 16, 2012
Sponsored by Pandia.com
Amazon and the Poobahs
April 15, 2012
I read “What Amazon’s Ebook Strategy Means.” Interesting but a few degrees off center. The main point seems to be that one cannot believe corporate executives. A second point is that corporate executives in publishing should abandon efforts to protect content with digital rights management schemes.
Good start. I find that public statements by anyone require some untangling. “Meaningful use” comments from Administration officials is an example of the difference between what may be meant, what one understands, and what is actually going on.
I also like the definition of three terms. I understand disintermediation. That’s what has happened to corporate libraries when “point and click” interfaces made MBAs and English majors into expert online searchers. There was a cost savings angle too. Even better. I am semi-okay with the definition of monopoly. However, in the online world, monopolies emerge because of economics, human motivation, and the nature of systems to enjoy nodes. Big nodes are often good. Little nodes are okay, but it is tough to make them pay off unless there is a business angle. The point is important because traditional publishing represents nodes from a different time. Online nodes are here, and they squeeze, subsume, and erode nodes from a different era. I don’t think the Vanderbilts and JP Morgans of the past were into this nuance, but if these gentlemen were alive today, the business nuances would be acted upon.
I am not so sure about eh word “monopsomy.” Most buyers follow habitual behavior. This is an aspect of online which is little appreciated. The idea is that once a person online has fallen into a groove, getting out of that groove is tough. No matter how much criticism is aimed at Apple iTunes, try to get most users to change. For that “one click away” baloney. So monopsomy is a five dollar word which simply does not apply to online and user habit.
Armed with these terms, the article asserts:
If the major publishers switch to selling ebooks without DRM, then they can enable customers to buy books from a variety of outlets and move away from the walled garden of the Kindle store. They see DRM as a defense against piracy, but piracy is a much less immediate threat than a gigantic multinational with revenue of $48 Billion in 2011 (more than the entire global publishing industry) that has expressed its intention to “disrupt” them, and whose chief executive said recently “even well-meaning gatekeepers slow innovation” (where “innovation” is code-speak for “opportunities for me to turn a profit”). And so they will deep-six their existing commitment to DRM and use the terms of the DoJ-imposed settlement to wiggle out of the most-favored-nation terms imposed by Amazon, in order to sell their wares as widely as possible.
Several observations:
First, I sure wouldn’t want to try and figure out how to make traditional publishing work in today’s world. I left that sector in the 1990s because the writing was on the wall. Shrinking margins, a shift in media channels, and the quest for blockbusters spoke to me. I am not sure traditional publishing is much more than a chase to find the one book that sells. James A Twitchell documented this a long time ago in Carnival Culture. The blend of online and the carnival are a potent combination.
Second, Amazon itself is vulnerable. The shrinking margins and the increasingly aggressive and somewhat clever behavior tells me that Jeff Bezos and his merry band know that extraordinary measures are required. The race is to capture users and leverage habitual behavior before another company does. The shortest distance between a book reader and habit is low ball prices. The WalMart approach is part of the Google Android play. Buying customers is a time honored retail method. YouTube is buying an audience with free content. Amazon is buying an audience with cheap Kindles and maybe once again cheap books. But Amazon has to pump up the revenue, control costs, and lock in its customers. Tough job.
Third, the content landscape has already shifted. Literacy in the US is a goner for large segments of the populace. Whether it is the tiny sound bite articles in Men’s Journal or the emergence of books which are collections of items, books are becoming something a relatively modest percentage of the 320 million people in the US consume. The same pressure which newspapers and magazines experience applies to book publishers. Look at the emergence of videos instead of white papers. Scary, cheap, easy, and very non-book.
What’s habit got to do with this? Traditional publishers have their habits. Buy low, sell high, and hope for a blockbuster. Online habits are different. Amazon, Apple, Facebook, and Google are big nodes and each node fosters habitual behavior among its users. Traditional publishing, therefore, is a subset of an online node. Subsets, in may cases, are expendable or have to become luxury items. The online nodes become the arbiters of taste, fashion, and what’s hot, good, or visible. Publishers have this role but for a rapidly decreasing segment of the online universe.
Chasing blockbusters is a tough business. Cutting costs and avoiding financial catastrophe is also a tough job. I am not sure either the publishers or Amazon is up to the task. And DRM? Changing habits is difficult. Isn’t it better to form the habits and use the systems and methods of capitalism than offer advice from the sidelines? The context of online has changed the rules. The old business models are interesting but less useful in the world of online.
Stephen E Arnold, April 15, 2012
Sponsored by Pandia.com
The Wall Street Journal Calls Motorola a Toy
April 12, 2012
I think the Wall Street Journal is owned by the News Corp. I have a tough time keeping track of real journalists these day. Here in Kentucky, the local newspaper is nuking folks who cost too much and place a burden on certain benefit plans.
I did find the headline “Google’s $12 Billion Toy” interesting. You can locate a copy of the story in the hard copy and environmentally unfriendly version of the Wall Street Journal in Section B1, pages 1-2. There is an electronic version online at this link. Don’t hassle me if it goes dark like so many traditional publishers’ content.
The main point is not about wire tapping, bribes, or nepotism. Nah, the subject of a real journalistic foray is Google’s purchase of Motorola Mobility Holdings. The angle is that Google has a “toy.” Now I used to like toys. Today I am more into gadgets. In fact, calling Motorola Mobility a toy is out of step with what those interested in technology find enthralling. Toy is an object with which children play, an amusement, a trifle, a diminutive thing, etc. My hunch is that by calling Google’s owning Motorola Mobility a toy, the Wall Street Journal was suggesting that Google’s management is childish, immature, and in need of a distraction. The juxtaposition of $12 billion and toy is a stylistic touch that may land the real journalist a gig writing for Jimmy Kimmel or the zany crowd at Saturday Night Live. Comedy writing might be a good back up for real journalists if more layoffs arrive at the big time, real newspapers. With the story’s arrival from a unit of the News Corp., I am just not sure whether the viewpoints about Google’s business acumen are serious or some of that “Google is not our pal” attitude which appears to surface from time to time.
Here’s the passage I noted:
Google’s competitors weren’t as charitable, with one rival executive privately dubbing the undertaking a “hairball.”
Hmm. A hairball. But which competitor?
But I particularly enjoyed this statement:
And that may be the scariest part of all for Google’s investors. The company really believes it can be all things to all people.
My view is that a company which is taking action may not be perfect, but it sure as heck is a more satisfying approach to business than the public removal of senior managers, allegations of bribery and unauthorized listening to voice mail, and the disclosure of alleged secret lunches with government figures.
I feel more comfortable with Google than with some of the antics of real journalists and their owners. Just my opinion from rural Kentucky.
Stephen E Arnold, April 12, 2012
Sponsored by Pandia.com
Beavers and Real Journalists Do What They Do
April 6, 2012
I find this amusing and a good example of the “real” journalists’ approach to information. Navigate to “UK Broadcaster Sky News Admits E-mail Hacking.” Here’s the passage I noted:
John Ryley, head of Sky News, said the instances involved suspected criminal activity. “We stand by these actions as editorially justified and in the public interest,” he said in a statement. “We do not take such decisions lightly or frequently.” An external review of e-mail records is under way at the broadcaster’s instigation, he said, but no grounds for concern have yet been found. “Sky News is committed to the highest editorial standards. Like other news organizations, we are acutely aware of the tensions that can arise between the law and responsible investigative journalism.”
CEOs who “revise” history may be on to something. As an addled goose, I am delighted to say, “I am neither a revisionist of history nor a “real” journalist.” Beavers do what beavers do. I wonder who owns the Wall Street Journal. A “real” journalistic outfit no doubt.
Stephen E Arnold, April 6, 2012
Sponsored by Pandia.com
Newspapers Losing Revenue: Time for a Change
March 30, 2012
Newspaper acquisition time? I was surprised by a headline I landed on while browsing Business Week; an article titled, “Newspapers Lose $10 in Print for Every Digital $1” grabbed my attention.
According to the article, newspapers in the United States lost $10 in print advertising revenue in 2011 for every dollar gained online. The article cites a study by Pew Research and blames the 7.9 percent ad revenue loss on competition from tech intermediaries. Newspapers are hurting tremendously in the online arena. Paid news sites and print copies are declining in revenue because consumers want their news fast and free, usually via mobile apps and free news blogs.
Newspaper groups have failed to capitalize on the volume of personalized data available online in the face of increased competition from companies including Google (GOOG) and Facebook, which are selling advertising targeted to consumers based on their interests and demographics, typically at higher ad rates, Rosenstiel said. Newspapers have slowly shifted their businesses online, led in part by the recent success of New York Times Co. (NYT)’s plan to charge readers for access to its newspapers’ websites. Pew’s study estimates as many as 100 newspapers are expected to offer a digital subscription model in the coming months.
No matter how one exercises ingenuity, the newspapers have a broken business model and a customer base indifferent to old information in print or online. Users are not likely to pay subscription fees, even for traditional and trusted organizations, if the material is available elsewhere for free. News groups should reconsider new business models or becoming partners with data-driven companies, or else it could be sell off and go fishing time.
Andrea Hayden, March 30, 2012
Sponsored by Pandia.com
Publishers Pose Threats to Text Mining Expansion
March 26, 2012
Text mining software is all the rage these days due to its ability to make significant connections by quickly scanning through thousands of documents. This software can recognize, extract and index scientific information from vast amounts of plain text, allowing computers to read and organize a body of knowledge that is expanding too fast for any human to keep up with. However, Nature.com recently reported on a some issues that have developed in this growing industry in the article “Trouble at the Text Mine.”
According to the article, text mining programmers Max Haeussler and Casey Bergman have run into trouble trying to get science publishers to agree to let them mine their content.
The article asserts:
Many publishers say that they will allow their subscribers to text-mine, subject to contract and the text-miners’ intentions, and point to a number of successful agreements. But like many early advocates of the technology, Haeussler and Bergman complain that publishers are failing to cope with requests, and so are holding up the progress of research. What is more, they point out, as text-mining expands, it will be impractical for individual academic teams to spend years each working out bilateral agreements with every publisher.
While some publishers are getting on board the text mining train, many are still trying to work out how to take advantage of the commercial value before signing on. Too bad it takes more than a degree in English to make text mining deliver useful results. Bummer.
Jasmine Ashton, March 26, 2012
Sponsored by Pandia.com
Reed Elsevier and Broken Hips
March 25, 2012
Break a hip when you are old and life gets really tough. Publishing companies like Thomson Reuters, Wolters Kluwer, and Reed Elsevier have taken their share of knocks and bumps. Now Reed Elsevier has a broken hip. The woes of scientific, technical, and professional publishers are like a bum ticker. Now a broken hip threatens the life expectancy and possibly the survival of the entity limping like Chester in Gunsmoke.
I am floating in the pond filled with mine drainage in rural Kentucky. It’s a good day as far as Kentucky goes. Even the hunters have taken a day off, so no sounds of gun fire roils the sylvan beauty of an abandoned horse farm. Yep, lots of horse farms for sale. The rich folks can no longer afford the sport of kings. Even kings and princes have had to cut back. What could be more enticing than a non working horse farm?
Oh, I have an idea. A traditional publishing company offering really high value properties like Variety, the must read newspaper of the Hollywood set. Wait. I am incorrect. I get Hollywood news from free Web sites; for example, E!Online, Celebrity Gossip, Entertainment Weekly, Hollywood Reporter, and TMZ.com. Granted a headline like “Ex Kony Soldier Warlord May Have Cursed Jason Russell” is not up to Variety’s headline writing, but it is free, has some exotic varietal images, and, of course, video and ads. Did I mention ads?
I learned about the new attempt to sell Variety in “Daily Variety Up for Sale.” Here’s the passage I noted:
It’s not the first time Reed has sought a buyer for its Hollywood newspaper. In 2008, the company put all of the U.S publications owned by its Reed Business Information unit up for sale. But amid that year’s financial crash, it was unable to find a buyer willing to purchase them all. Since then, the company has sold all of its other publications, including Publisher’s Weekly and Broadcasting & Cable. The move comes as Variety — famous for a show-biz lexicon of industry-specific terms like “boffo” and tongue-twisting headlines like “Sticks Nix Hick Pix” — has lost its standing as Hollywood’s dominant trade newspaper amid heated competition from online-only publications Deadline and the Wrap, as well as its traditional competitor, the Hollywood Reporter.
So now the economy is better and the market for print trade publications is apparently ripe for a sale of the property which “has lost its standing as Hollywood’s dominant trade newspaper” is on the block.
Several questions:
Amazon and Distribution Changes
March 23, 2012
Major online retailer, Amazon, is in the process of relaxing its policies regarding its ebook distribution. In the article “Amazon Publishing to Sell Series of eBooks Outside the Kindle Store,” we learn a little more about the decision to spread the wealth.
The recent retaliation of Barnes and Noble and other wholesale booksellers against Amazon has sparked a change in how Amazon will continue to do business. Barnes and Noble, as well as other booksellers, pulled the print editions of books published by Amazon Publishing from their shelves this month in retaliation to Amazon’s “Kindle store only” policy. It is a policy that Barnes and Noble management believes undermines the industry and creates monopoly.
In response to their action, Amazon announced that its latest series of short biographies will be sold outside of the Amazon marketplace. No word on whether the other stores will pick up the books but Amazon has assured the public that they want their books to be distributed as widely as possible.
Amazon has confirmed that its latest addition to the Amazon Publishing roster, a series of short biographies edited by James Atlas, will indeed be sold outside of the Amazon ecosystem in both print and ebook form.
Why is it that Amazon is ridiculed for its apparent lack of availability when Apple has long had a foothold in the marketplace as a “buy here, pay here only” business? Apple products are available at your nearest electronics provider (as are Amazon Kindles). Yet when it comes to content, you must go through iTunes in order to utilize all aspects of your product.
We think that a distribution shift is an important part of access. Are consumers and search lost in the mulcher?
Stephen E Arnold, March 20, 2012
Sponsored by Pandia.com
Quote to Note: Publisher Strips the Internet Bare
March 15, 2012
Quote to Note. Interesting write up by a poobah fearful of losing his elephants. Point your vile browser thing at “John R. MacArthur: Internet Con Men Ravage Publishing.” Here is the quote I noted:
As far as I know, there isn’t a single profitable online-only magazine or newspaper in the United States and there isn’t a single profitable newspaper or magazine with an online edition that is seriously considering dropping its print edition.
The write up was free when I located the essay. I don’t have much of an opinion on the arguments in the poobah’s write up. Too late.
Stephen E Arnold, March 15, 2012
Sponsored by Pandia.com
Library Association Flexes Its Muscles
March 15, 2012
The American Library Association had plenty to say to publishers about the recent inflation of ebook prices. In “Libraries Protest Random house Price Hike” we get a more in depth look at the problem.
The Random House publishing company recently informed libraries that the wholesale price for ebooks would be on the rise by more than 20 percent. That’s a huge hike increase for something with no tangible, physical, product attached to it. The increase would affect new adult releases and children’s ebooks would double.
This new protocol stems from the publishers believe that ebooks can be “repeatedly circulated” without wearing out. Barring file destruction, they can be circulated indefinitely.
This comes as a shock to most libraries around the country. Many local branches are going through a tough year as finances that were already spread thin are stretched to the limit due to stiff budget cuts across the board. The publishers actions have caused the American Library Association to stand up and defend its right to fairly priced goods and the patrons rights to indefinite circulation.
While I appreciate Random House’s engagement with libraries and its commitment to perpetual access,” ALA president Molly Raphael said in the statement, “I am deeply disappointed in the severe escalation in e-book pricing reported today. Calling on our history together and our hope to satisfy mutual goals moving forward, the American Library Association strongly urges Random House to reconsider its decision. In a time of extreme financial constraint, a major price increase effectively curtails access for many libraries, and especially our communities that are hardest hit economically.
But while the move comes at a bad time for libraries in a financial bind, and is certainly not in good taste, it’s hard to blame a company that is worried about its own future both physically and financially for trying to cover all of its bases and make a buck or two where it can. In an age where technology is taking over, books and other forms of leisure have been relegated to the back burner as the latest and greatest craze takes over. We’ve seen companies like Sony and Kodak go bankrupt or nearly so and discontinue the production of their most trusted moneymakers. Is it too farfetched to think that novels have become a novelty?
Stephen E Arnold, March 15, 2012
Sponsored by Pandia.com
-
- Subscribe to Beyond Search
Feature archive
News archive
-
