Advances in Search Drive Big Data Future
June 4, 2013
Big Data is getting the lion’s share of attention in the enterprise search market. As the cost of data storage continues to plummet, even smaller organizations are interested in capturing the meaning that may exist in massive amounts of routine data, or Big Data. So the Big Data market is exploding. Smart Data Collective is tracking the trends in their article, “7 Big Data Trends That Will Impact Your Business.”
The article highlights trends that will impact the average business. The third trend noted is advances in search and LucidWorks is a standout contender in this category:
“Sifting through massive amounts of data to find that preverbal needle in the haystack is no simple task. Over time we will likely see more big data solutions injecting search support into their solutions. Leading the way in this endeavor are LucidWorks, IBM, Oracle through the acquisition of Endeca (full disclosure, I’m a former Endeca employee), Autonomy and MarkLogic. LucidWorks combines an open source stack of Lucene/Solr, Hadoop, Mahout and NLP.”
So for those who are interested in capturing and using Big Data, but want an interface they can negotiate, LucidWorks is a great choice. Their LucidWorks Big Data is an application development platform that incorporates the power of Lucene, Solr, Hadoop, Hive, and other major open source powerhouses into a usable, efficient platform. Try it and see.
Emily Rae Aldridge, June 4, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Elasticsearch in Under a Minute
May 30, 2013
Elasticsearch is generating a lot of buzz of late. For those who are curious, JavaWorld is offering a brief step-by-step tutorial entitled, “ElasticSearch on EC2 in less than 60 seconds.” It begins:
“Curious to see what all the ElasticSearch hubbub is about? Wanna see it in action without a lot of elbow grease? Then look no further, friend – in less than 60 seconds, I’ll show you how to install ElasticSearch on an AWS AMI. You’ll first need an AWS account along with an SSH key pair. If you don’t already have those two steps done, go ahead and do that. The steps that follow suggest a particular AMI; however, you are free to select the instance type. Micro instance types are free to use; consequently, you can get up and running with ElasticSearch in less than a minute for free.”
The instructions continue, but only a developer or programmer would be able to follow along. That’s okay, because they are the audience. However, for those who are interested in an out-of-the-box solution that satisfies both developers and end-users, LucidWorks would be worth investigating. On the market longer than Elasticsearch, LucidWorks is a more trusted solution that works for enterprise search and Big Data. Most importantly, LucidWorks’ security track record is so strong, that other companies, including MapR, seek out LucidWorks as a partner in order to provide security and peace of mind to customers.
Emily Rae Aldridge, May 30, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
StackSearch Launches Search as a Service
May 29, 2013
Search as a service subscription models are becoming more popular with the increasing need for scalable, stackable enterprise solutions. A new option is now on the market as StackSearch released their Qbox search as a service. Read more in the press release at Wall Street Journal entitled, “Stacksearch Launches Qbox Search-as-a-Service.”
The release begins:
“StackSearch, Inc. today announced the availability of Qbox.io search-as-a-service. Qbox, available via a tiered monthly subscription model, was built ‘by developers, for developers’ and empowers developers to incorporate supported and fully managed ElasticSearch indexes into their apps without having to worry about the hassle of installing, maintaining, and scaling on-premise search infrastructure.”
While the concepts StackSearch built on seem relevant, some organizations may hesitate to adopt. First, security and other issues often arise with new companies. StackSearch was founded in 2012 and has yet to be vetted across the industry. Also, the new solution depends on ElasticSearch, a new company itself, and one that has already encountered security and reliability concerns. While agility and scalability are features that enterprises find enticing, security reigns supreme. So organizations are still likely to turn to a tried-and-true company like LucidWorks for their valuable enterprise search and Big Data needs, without sacrificing innovation and creativity.
Emily Rae Aldridge, May 29, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Enterprise Search: Can Word Choice Rescue a Dogpaddling Business?
May 23, 2013
I read “Ontology Slays Data Integration and Ignites Semantic Search Revolution.” I found several things interesting about the write up.
First, there is the word choice: “slays,” “ignites,” and “revolution.” In case you have forgotten, an ontology is, according to the Catholic Encyclopedia:
Though the term is used in this literal meaning by Clauberg (1625-1665) (Opp., p. 281), its special application to the first department of metaphysics was made by Christian von Wolff (1679-1754) (Philos. nat., sec. 73). Prior to this time “the science of being” had retained the titles given it by its founder Aristotle: “first philosophy”, “theology”, “wisdom”. The term “metaphysics” (q.v.) was given a wider extension by Wolff, who divided “real philosophy” into general metaphysics, which he called ontology, and special, under which he included cosmology, psychology, and theodicy. This programme has been adopted with little variation by most Catholicphilosophers. The subject-matter of ontology is usually arranged thus:
- The objective concept of being in its widest range, as embracing the actual and potential, is first analyzed, the problems concerned with essence (nature) and existence, “act” and “potency” are discussed, and the primary principles — contradiction, identity, etc. — are shown to emerge from the concept of entity.
- The properties coextensive with being — unity, truth, and goodness, and their immediately associated concepts, order and beauty — are next explained.
- The fundamental divisions of being into the finite and the infinite, the contingent and the necessary, etc., and the subdivisions of the finite into the categories (q.v.) substance and its accidents (quantity, quality, etc.) follow in turn — the objective — reality of substance, the meaning of personality, the relation of accidents to substance being the most prominent topics.
- The concluding portion of ontology is usually devoted to the concept of cause and its primary divisions — efficient and final, material and formal –the objectivity and analytical character of the principle of causality receiving most attention.
My reaction? The use of the term ontology in the context of “slays,” “ignites,” and “revolution” seems a little frisky.
Second, the product referenced in the news release offers some relief. I find the explanation of the product in terms of what it is not quite interesting; to wit:
Ontology 4 is built to five key principles that separate it from traditional data integration technologies:
- No schema – Ontology uses a searchable, semantic model built on proven graph-based technology.
- No Integration – Ontology uses a semantic model to find and combine data relating to business entities fragmented across the enterprise.
- No Big Bang – Ontology’s semantic model embraces on-going changes while delivering value early and iteratively over the duration of a project.
- No Search Restriction – Ontology’s semantic search find’s information across application data, documents and emails.
- No Upfront Risk. – No integration to data sources, No unnecessary tying up of team resources, No feasibility surprises, and No problem changing project requirements.
“The Internet is the world’s largest source of data, yet no one integrates it. They search it,” concluded Enweani. “So, when it comes to enterprise data, we say ‘Search, don’t Integrate.”
Third, enterprise search and the vendors engaged in the discipline demonstrated at two enterprise search summits in the last two weeks a strong shift away from the use of the word “search.” Synonyms included customer relationship management, discovery, search based applications, and similar distancing terms.
Perhaps more colorful word choice and the use of old style rhetorical flourishes will breathe life into a dogpaddling business sector. As one vendor which recently experienced a CEO shuffle because the firm once again missed its numbers, “We are now a platform.”
Will word choice deliver revenue? Investors hope so.
Stephen E Arnold, May 23, 2013
Sponsored by Augmentext
Demographics and an Another Daunting Challenge for Search
May 22, 2013
I read “Pew: 94% Of Teenagers Use Facebook, Have 425 Facebook Friends, But Twitter & Instagram Adoption Way Up.” The main point is that Facebook has what I would call a monopolistic position when it comes to teens and their friends. I am not sure Facebook is the home run play in places like rural Chile, but where there is money, infrastructure, and gizmos, Facebook is on top.
The point which struck me is, “What happens when an outfit is on top?” Revenue accrues and so does attention.
The research which the write up summarizes contains an interesting factoid or two. For example, teens are, if the data are correct, are shifting from online services which use words to online services which use pictures. (Will video be far behind?) Here’s the passage I noted:
Twitter and Instagram are far behind Facebook, but both have made impressive gains. Twitter was used by only 12% of teens in 2011 but more than doubled that to 26% in 2012. with usage of 26% and 11%. Instagram doesn’t appear to have been measured in 2011, so surveyed growth can’t be determined. But it comes in with an impressive third place at 11%.
Several observations are warranted.
First, search is somewhat of a disappointment when one tries to locate specific information in text form. Last night at dinner, a prominent New York attorney said, “It may just be me but I am having more difficulty finding exactly what I am looking for.” The comment bedevils quite a few people. I suggested that the prominent attorney hire a legal researcher. The prominent attorney replied, “I suppose I will have to.” Lesson: Finding information is getting more difficult, not easier. Keep in mind that the problem exists for words. Search is a challenge for some folks, and vendors have been trying to crack the code for 40, maybe 50 years.
Second, what information is embedded in digital images? What “metamessages” are teens sending when a snapshot is launched into the Twitter or Instagram world? More important, what search system is needed to locate and figure out the information in an image? My view is that geocoding and personal information may offer some important clues. But do we have a search system for these content repositories which works for the hapless attorney, a marketer, or a person looking for information about a runaway teen? In my view, not yet, and not by a long shot.
Third, is the shift from text to images by the teen demographic in the study sample a signal that text is losing its usefulness or relevance? The notion that those entering the workforce in a few years wedded to Tweets and snapshots may be an important cultural shift in some parts of the developed world.
The big question remains, “How will one find information to answer a question?” Text search is a problem. The brave new world hinted at in the Pew study poses more findability challenges. I am not sure the current crop of search and content processing challenges can resolve the problem to my satisfaction. The marketers will assert the opposite. The reality is that findability will remain a central problem for the foreseeable future.
Search is most easily resolved by ignoring its problems or reducing the problem to predictive algorithms in a “mother knows best” approach to information. That may work for some, but not everyone.
Stephen E Arnold, May 21, 2013
Sponsored by Augmentext
Forget SEO Say Hello to AdWords
May 20, 2013
SEO is a hot topic as it is necessary for any marketing and PR plan to take shape. Unfortunately, Search Engine Watch reports that many are taken advantage of by SEO companies. Their recent post, “Moving Forward With a Broken Compass: A Plea to SEOs,” goes as far to say that what these companies deliver is borderline criminal.
The writer of this particular post establishes his ethos at the other end of the spectrum of quality of work delivered. The author describes a time where he went to attend a regular meeting at his client’s conference room but mentions that he never saw past that front conference room.
However, one day was different:
“I was surprised when the client offered to take us for a tour of their entire facility to have us meet the people we had been actually been working for. The client took my co-workers and I around their office complex and warehouses. They introduced us to people we had never before met, stating things like ‘This is Bob from Company X. They didn’t have a job before the work you’ve done for us. We built Bob’s office and the warehouse for his company off the back of what you’ve been doing.’”
Whether SEO delivers what it promises or not, this is beside the point. If you want traffic, buy AdWords.
Megan Feil, May 20, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
The Growth of Global Search Spending
May 17, 2013
What is going on with search? According to Search Engine Watch in “Kenshoo Global Search Spending Grows 15%, CPCs Drop In Q1” the trends point to global search spending going up, but the costs-per-click are in the doldrums. Kensoo’s Global Search Advertising Trends tracked the search trends and found many positive factors: click-through-rates are up 62%, search spending is 15% up year-over-year, and the total spending is up 32% from 2011. It all looks great, but then the global average CPC dropped $0.39.
It seems the search market is an inevitable seesaw, but what is making it rise and descend so much? Possibly the lack of marketers who have not caught onto mobile. The immediate culprit would be social media, but the just might be a red herring:
“Social media is certainly sucking up all the buzz, but is this what’s depressing CPCs? Not according to Aaron Goldman, CMO of Kenshoo. ‘Social is certainly hot, but we’re not seeing it cannibalize search budgets,’ Goldman said. ‘Rather social is being funded from other channels, such as display and offline.’ He adds that social ad platforms including Facebook Exchange are taking budget from other ad exchanges and networks.”
We’ll just pin it on a bunch of economic factors that mesh together and form near indecipherable tracings. Bet your dollar that Google will be affected; the Google Glasses won’t be able to protect them from this burn if it continues.
Whitney Grace, May 17, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Enterprise Search Europe Spring 2013
May 16, 2013
The enterprise search show ended today, May 16. The presentations, except the one by Stephen E Arnold, were scintillating, thought provoking, and solid evidence that enterprise search is the crown jewel of enterprise software systems. Forget the grousing about Fast Search & Transfer, Autonomy, and the millions upon millions poured into outfits trying to generate a profit by licensing software which makes it easy to locate a needed document using a traditional personal computer, a laptop, or a notebook computer. Mobile phones and tablets are, alas, not yet the camels inside the enterprise search tent.
I learned about the importance of knowing what users want. I learned about providing users with systems which auto suggest, display relevant links, and eliminate the annoying task of reading a document to determine if it has useful information for the user.
Progress never stops. I would point out that Stephen E Arnold’s slide showing that precision and recall were making incremental progress over a decade. The flat line was in sharp contrast to his utterly fantastic suggestion that the complexity of modern search systems and their costs were increasing. One Scandinavian business development professional said to Mr. Arnold, “So you think the costs of search are going up like that, like the take off of the jet plane.”
Mr. Arnold, I overheard, said, “Yep, especially when the systems don’t work as advertised, require expensive unbudgeted investments, and produce more complaints than changing the health care dental deductible.” The Scandinavian shook his head in disbelief and wandered off in search of more comforting conversation.
A screen capture from Stephen E Arnold’s anomalous presentation. The cost and complexity curves rise more aggressively than the precision and recall curve. Who needs relevance when modern systems can deliver search without the user’s performing any intellectual effort prior to accepting what a system delivers.
I did come away with three broad thoughts once I cleared my mind of the fog of confusion that Stephen E Arnold’s obfuscation machine delivered.
First, Apple’s and Google’s conferences sell out in a very short time. Perhaps some of those turned away from the Apple and Google events could pick up a few IQ points and simultaneously get the inside dope on the hottest enterprise application — enterprise search? Two enterprise search vendors generated more than $100 million in revenues in the 45 year history of the enterprise search sector’s lifetime. Definitely enterprise search is the go to market. Measured in terms of academics, advisors, and unemployed home economics majors, search is where the action is.
Second, the technology on display was a great refresher for me. I learned about users’ dissatisfaction with search a decade ago. If I understood the presenter, user dissatisfaction is unchanged. About half of those who use an enterprise findability system are not thrilled with the experience. Progress is, it seems, modest. On the other hand, consistency in user opinion helps size the magnitude of the opportunity. I have not attended an enterprise search event for several years. I must admit I don’t think I missed any important developments. The content was, in my opinion, familiar.
Third, the technical bits had to bite and claw to get podium time. The outlier Stephen E Arnold actually used some equations. No other presenter made that mistake. The majority of the presentations focused on management issues. There were variously described as “governance,” “content management,” and planning. For those with an MBA and a love of enterprise search, there are, I concluded, many opportunities for consultants. Several of the folks who sell their expertise pointed out “I am not a technical expert,” “I can’t code,” and my favorite “Enterprise search is just one of the specialties I have.” Ah, billable time for uninformed advice. A career tip.
What’s the future of enterprise search?
One speaker said, “Search is not a good word to use.”
No kidding.
Edward Stephens, Stephen E Arnold’s more intelligent cousin, May 17, 2013
Sponsored by Augmentext
LucidWorks Stories the Evolution of Big Data Search
May 16, 2013
Dan Kuznetsky is a trusted authority in enterprise search. He brings his expertise to the topic of search in Big Data in his latest article for ZDNet, “Evolution of Search in Big Data as Told by LucidWorks.”
After a discussion of how LucidWorks is contributing to the open source community through its participation in the Apache Software Foundation, Kuznetsky goes on to explore this interesting development model:
“LucidWorks is one of a growing number of technology companies that are building products based upon open-source software that was created in products hosted by the Apache Software Foundation. It is fascinating how they are cooperating to build the basic technology and then focusing on different competitive market niches. Each time I mentioned what I thought was a competitor, the folks from LucidWorks pointed out that those companies are partners that are trying to use their individual strengths together to serve the market. This is an area that is worth watching.”
Kuznetsky hit on the strength of LucidWorks and the rest of the value-added open source market. Innovation is encouraged, many benefit, but each company finds a niche that makes it profitable, but also useful. In this way, innovation is encouraged, open source development is encouraged, and users benefit from continuous improvement and support for the solutions in which they invest. Sounds like a win-win.
Emily Rae Aldridge, May 16, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Holy Grail of Search and Big Data?
May 14, 2013
A Business Wire press release caught our eyes recently as it announced the distribution of LucidWorks Search with the MapR Platform for Apache Hadoop. “MapR Technologies Distributes Enterprise-Grade Search with Hadoop Platform” shares that now customers will have predictive analytics, search, discovery and advanced database operations at their fingertips on a single platform.
Integrating LucidWorks technology with MapR beefs up the added value that LucidWorks Search offers as far as security, connectivity and user management. Additionally, MapR announced that the M7 Edition is available; this combines unprecedented Hadoop and NoSQL capabilities together in one platform.
According to Ben Woo, managing director, Neuralytix:
“Integrating search capabilities into Hadoop is an important milestone for the industry and represents tremendous opportunity for customers to find new insight and derive value from Big Data. This is an enormous step forward especially in time-sensitive processes such as fraud detection where Big Data must be searched as it streams into the enterprise.”
MapR’s chief application architect tells us that using search and big data is not just about analyzing social media content and Web traffic. We wonder…big data and search: has the holy grail (or one of them) been found?
Megan Feil, May 14, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
-
- Subscribe to Beyond Search
Feature archive
News archive
-
