• Home
• About
• Writers
• Landscape
• Wizards Index
• Fraud
• Wanna Be Like Larry?

Search Silver Bullets, Elixirs, and Magic Potions: Thinking about Findability in 2012

I feel expansive today (November 9, 2011), generous even. My left eye seems to be working at 70 percent capacity. No babies are screaming in the airport waiting area. In fact, I am sitting in a not too sticky seat, enjoying the announcements about keeping pets in their cage and reporting suspicious packages to law enforcement by dialing 250.

I wonder if the mother who left a pink and white plastic bag with a small bunny and box of animal crackers is evil. Much in today’s society is crazy marketing hype and fear mongering.

Whilst thinking about pets in cages and animal crackers which may be laced with rat poison, and plump, fabric bunnies, my thoughts turned to the notion of instant fixes for horribly broken search and content processing systems.

I think it was the association of the failure of societal systems that determined passengers at the gate would allow a pet to run wild or that a stuffed bunny was a threat. My thoughts jumped to the world of search, its crazy marketing pitches, and the satraps who have promoted themselves to “expert in search.” I wanted to capture these ideas, conforming to the precepts of the About section of this free blog. Did I say, “Free.”

A happy quack to http://www.alchemywebsite.com/amcl_astronomical_material02.html for this image of the 21st century azure chip consultant, a self appointed expert in search with a degree in English and a minor in home economics with an emphasis on finger sandwiches.

The Silver Bullets, Garlic Balls, and Eyes of Newts

First, let me list the instant fixes, the silver bullets,  the magic potions, the faerie dust, and the alchemy which makes “enterprise search” work today. Fasten your alchemist’s robe, lift your chin, and grab your paper cone. I may rain on your magic potion. Here are 14 magic fixes for a lousy search system. Oh, one more caveat. I am not picking on any one company or approach. The key to this essay is the collection of pixie dust, not a single firm’s blend of baloney, owl feathers, and goat horn.

1. Analytics (The kind equations some of us wrangled and struggled with in Statistics 101 or the more complex predictive methods which, if you know how to make the numerical recipes work, will get you a job at Palantir, Recorded Future,  SAS, or one of the other purveyors of wisdom based on big data number crunching)
2. Cloud (Most companies in the magic elixir business invoke the cloud. Not even Macbeth’s witches do as good  a job with the incantation of Hadoop the Loop as Cloudera,but there are many contenders in this pixie concoction. Amazon comes to mind but A9 gives me a headache when I use A9 to locate a book for my trusty e Reeder.)
3. Clustering (Which I associate with Clustify and Vivisimo, but Vivisimo has morphed clustering in “information optimization” and gets a happy quack for this leap)
4. Connectors (One can search unless one can acquire content. I like the Palantir approach which triggered some push back but I find the morphing of ISYS Search Software a useful touchstone in this potion category)
5. Discovery systems (My associative thought process offers up Clearwell Systems and Recommind. I like Recommind, however, because it is so similar to Autonomy’s method and it has been the pivot for the company’s flip flow from law firms to enterprise search and back to eDiscovery in the last 12 or 18 months)
6. Federation (I like the approach of Deep Web Technologies and for the record, the company does not position its method as a magical solution, but some federating vendors do so I will mention this concept. Yhink mash up and data fusion too)
7. Natural language processing (My candidate for NLP wonder worker is Oracle which acquired InQuira. InQuira is  a success story because it was formed from the components of two antecedent search companies, pitched NLP for customer support,and got acquired by Oracle. Happy stakeholders all.)
8. Metatagging (Many candidates here. I nominate the Microsoft SharePoint technology as the silver bullet candidate. SharePoint search offers almost flawless implementation of finding a document by virtue of  knowing who wrote it, when, and what file type it is. Amazing. A first of sorts because the method has spawned third party solutions from Austria to t he United States.)
9. Open source (Hands down I think about IBM. From Content Analytics to the wild and crazy Watson, IBM has open source tattooed over large expanses of its corporate hide. Free? Did I mention free? Think again. IBM did not hit $100 billion in revenue by giving software away.)
10. Relationship maps (I have to go with the Inxight Software solution. Not only was the live map an inspiration to every business intelligence and social network analysis vendor it was cool to drag objects around. Now Inxight is part of Business Objects which is part of SAP, which is an interesting company occupied with reinventing itself and ignored TREX, a search engine)
11. Semantics (I have to mention Google as the poster child for making software know what content is about. I stand by my praise of Ramanathan Guha’s programmable search engine and the somewhat complementary work of Dr. Alon Halevy, both happy Googlers as far as I know. Did I mention that Google has oodles of semantic methods, but the focus is on selling ads and Pandas, which are somewhat related.)
12. Sentiment analysis (the winner in the sentiment analysis sector is up for grabs. In terms of reinventing and repositioning, I want to acknowledge Attensity. But when it comes to making lemonade from lemons, check out Lexalytics (now a unit of Infonics). I like the Newssift case, but that is not included in my free blog posts and information about this modest multi-vehicle accident on the UK information highway is harder and harder to find. Alas.)
13. Taxonomies (I am a traditionalist, so I quite like the pioneering work of Access Innovations. But firms run by individuals who are not experts in controlled vocabularies, machine assisted indexing, and ANSI compliance have captured the attention of the azure chip, home economics, and self appointed expert crowd. Access innovations knows its stuff. Some of the boot camp crowd, maybe somewhat less? I read a blog post recently that said librarians are not necessary when one creates an enterprise taxonomy. My how interesting. When we did the ABI/INFORM and Business Dateline controlled vocabularies we used “real” experts and quite a few librarians with experience conceptualizing, developing, refining, and ensuring logical consistency of our word lists. It worked because even the shadow of the original ABI/INFORM still uses some of our term 30 plus years later. There are so many taxonomy vendors, I will not attempt to highlight others. Even Microsoft signed on with Cognition Technologies to beef up its methods.)
14. XML (there are Google and MarkLogic again. XML is now a genuine silver bullet. I thought it was a markup language. Well, not any more, pal.)

Please, keep in mind that I have some other candidates like frameworks, governance (a particularly distastefully notion to me  because it reeks of English Leather and hair gel slapped on a problem related to editorial policy), and hybrid infrastructures, but in this free blog post, let’s roll with these 14 chunks of Aladdin’s magic carpet, I Dream of Jeannie, and Yoda-inspired methodologies.

Common Themes among the 14 Elements of Faerie Magic

What hooks these magical components is the idea that snake oil salesmen cure a variety of ills with one of these multi-purpose information wand waves or elixir chuggings.

I am too young to remember a horse drawn cart with a family selling snake oil coming to town. But one of my relatives  pitched home made cure alls, what the pragmatist would call “snake oil”,  in her vaudeville days. The Arnold family’s bottled elixir would, according to the pitch, “cure baldness, settle the stomach, remove warts, and heal skin eruptions.” Several years ago, we came across the “secret” formula for my family’s snake oil.

What was the secret? Whiskey. Cheap stuff, probably rot gut or white lightning. We are Arnolds after all.

Alcohol made up 90 percent of our cure all. For some folks a snort of the Arnold “medicine” would indeed provide temporary relief and with five or six bottles of the stuff, blissful oblivion and a headache the next morning. Doesn’t this sound like the aftermath of buying a “too good to be true” search solution?

The three themes for these content magic “fairy tales” are:

1. Avoidance of the phrase “enterprise search.” Most information technology processionals have figured out what two thirds of an enterprise search systems users have known for years. Search doesn’t work particularly well. If it did, there would not be so many search vendors trying to do an about face and embrace almost any other buzzword except “enterprise search.”I would not be surprised to see this bound phrase added to Google’s list of banned words, concepts, and phrases.
2. Confusing a part with a whole solution. I have two or three readers, and I have a hunch that one person may have actually coded, installed, debugged, tuned, and customized a search system which processed content on more than 50,000 servers and struggled to get 20 million or more content objects “findable.” If you have, you understand that an information retrieval system consists of numerous subsystems, features, functions, and even third-party components that are only loosely integrated with the mother ship code base. What is immediately evident in this list is that when you buy one of these magic wands, piles of pixie dust, or pipe dreams,you are getting a component. Yes, a part. A chunk. Think in terms of buying an automobile. The dealer is selling you the oil filter and one bucket seat. Try driving that to Trader Joe’s for Two Buck Chuck, chump.
3. Pretending the components are really simple, a “no brainer”. Now I understand what happens. An unemployed journalist or a terminated middle school teacher needs a job. Signing on the work on a “search” project or a “researcher” at a D plus or C minus consulting firm is a necessary step in paying rent, getting the child orthopedic shoes, or mom and dad enjoying a Panama Canal cruise in the winter. The problem is that “reading about” information retrieval or “using Google” is superficial experience. Experts have taken a wrench to the content processing subsystem’s engine room to adjust a relevancy threshold. I know it comes as a shock to the folks who hire happy, friendly people to help with complex systems problems, but look on the bright side. When you are terminated or your employer goes out of business, you can convert that political science degree into a credential for search and content processing consulting. You can run an indexing boot camp or write a blog for other “experts” as I did. I even include Latin in some of my posts. Yep, simplify NLP or metatagging. No problemo for the knowledge management and the content management crowd.

Do these “magic solutions” like the Star Wars’ “force”, work in isolation? Each of the silver bullets requires an arm manufacturing infrastructure, training, and maintenance, lots and lots of “stuff” to work beyond “demo scale”. These solutions are mostly marketing smoke and mirrors. In my opinion, what is going on is that the problem of “enterprise search” has been marginalized. The big problem has been restated as a silver bullet solution. Presto. No problem with enterprise search when you buy one of the 14 magic carpet rides.

Some of the original enterprise search giants fouled the pond like sheep making water unfit for cows. The remediating step which is now fueling the market for the pixie dust vendors is a How to Win Friends and Influence People maneuver. That’s right. The vendors are not selling total solutions. The pixie dust crowd is in a desperate, often frantic hunt for revenues and buzz word magic is generating some revenue. Hooray. Search does not sell. Magic wand waving does. What works today is finding some marketing bait to put on the sales hook.

That’s the state of the market with regard to pixie dust, silver bullets, and Peter Pan at this time.

Implications for 2012

I know that the end-of-year prophecies by the Nostradamus emulators will be coming along for the next two or three months. I am going to skip the predictions and make several opinionated observations about the environment for enterprise search and content processing. I am 67, an addled goose, and I live in rural Kentucky in semi retirement. This is a fact that some AtomicPR, azure chip consultants, and nearly bankrupt conference organizers choose to overlook. If you have a tendency to experience sudden anger or high blood pressure, you will want to stop reading now.

First, we are going to have more  outright failures of search and content processing vendors. The first sign of trouble is crazy marketing. The next sign is executive churn. Now the AtomicPR types and the cat’s paw private equity firms will find a way to do a Dancing with the Stars’ two step around “financial collapse.” The fact is that it is tough to make sales, and many organizations are looking for “good enough” solutions. We define “good enough” as low cost and without the add ons one gets when the team from Pimp My Ride works on 1985 Chevrolet Suburban. The implication behind this statement is that CEOs, stakeholders, and commission sales professionals will shovel more pixie dust over a procurement team or contract officer than ever before. Most vendors’ goal is to move money from the buyer to the magic wand seller. When one joins a health club, what does on buy? Pain? Nope. Those who join a health club buy a mental feel good jolt because now that 20 pounds of fat is really going to go away. Let’s have a Dairy Queen mini blizzard to celebrate joining the gym.

Second, the 14 silver bullets will be increasingly subsumed into non-search applications. The reason is that quite a few Fortune 1000 organizations pay big bucks for big fixes to big problems. I know that it makes no logical sense for a clueless company to ignore a functional cloud based search solution like Blossom Software’s technology for a couple thousand dollars a year only to sign on for a multi million dollar deal with an IBM, Microsoft, Oracle, or SAP.But that’s what happens. One side effect is that the vendors of pixie dust chase smaller deals with even more alacrity and zest. These folks ignore that the marketing costs are likely to chew up the profit, but history majors may not know about cost accounting. When the smaller buyers lock the door, the vendors just amp up the effort. In 2012, marketing about magic is going to ratchet up a couple of notches. When the money to pay the light and water bills runs out, the vendors will go away when the funding source pulls the plug or does a non cash merger or some other Olympus-style financial sleight of hand.  I learned about content processing deal involving  a four year license discount if the customer ponied up most of the four year fee in an upfront, lump sum. The deal was a government wide license in a country in which a war is raging and political instability is documented on the nightly news. What do you think the likelihood of a renewal in four years is?

Third, the funding sources will actively help close deals. Here in Harrod’s Creek, we sense that waves of influence wielding are evident on the horizon west of Half Moon Bay. By 2012, the surfers will be out and the locals will be complaining about parking and selling board wax, granola, and bottled tap water to the wave catchers. Examples range from the hyperbole about “social networks as a solution to findability” to “mobile search as the new revolution in information retrieval.” I am not sure how social a pharmaceutical drug researcher will be, but I think the system used to “find” chemical information will not be available on Bing or Google in 2012. I don’t think even the multi-faceted Endeca system (now Oracle) knows much about chemical structures and combinatorial chemistry. MBAs can manage chemical plants. MBAs cannot do chemistry in most cases.  I don’t think mobile search will make it easy for a person working on intercept analyses for a law enforcement agency to do much “real” work on an iPhone or an Android device. But, hey, I am often wrong. In 2012, I expect the repositioning of search, information retrieval, and content processing to become absolutely poetical. There is a need for English majors in technology consulting after all.

Wrap Up

I assume I will get the pleading letters from “friends” who want me to write about search and content processing like the “real” consultants. I will see missives in the Beyond Search comments section which begin, “Steve, you know that ….”. I fully expect those conference organizers who pay me to show up and give a talk about my current research projects will say, “We need to make sure the exhibitors are happy.”

I hear this type of smarmy quasi-adult “advice” frequently. Know what? I find it amusing. Think about it. I am 67, live in an environmental hell hole, reside in a state where two thirds of the population does not know where Canada and Mexico are, and is awash in red ink. In Kentucky, public schools close for the Derby. Those who disagree with my opinions offer gentle, condescending smiles. I delete the too familiar emails asking me to remove or revise a blog post because my view is different from someone else’s. I flatly reject offers to “educate” me via a webinar with a luminary about the bold features of a product. When these inputs arrive, I ask, “Am I drilling into a marketer’s or stakeholder’s front tooth?”

I have been in the consulting and “getting paid to analyze”  business for a long time. I got my training at the money machine you know as Halliburton. Yep, Dick Cheney’s stomping grounds. Then I received a mind meld and thought transfer from Dr. William P Sommers, president of the “old and respected” Booz, Allen & Hamilton with a PhD and an MBA  at age 21. Dr Sommers was so smart, seasoned Booz, Allen consultants avoided his dimly lit office, his raised dais, his Louis IV escritoire, and the side chairs covered in suede that looked suspiciously like a freshly butchered calves.

I have learned over the last 40 years that it is sometimes preferable to provide thought provoking context for blips that look to the uninformed observer  like discrete, unrelated actions, and random events.

Here goes.

Most of the writing, talking, lecturing and training about enterprise search is about generating money for the software vendors, consultants, and newly minted experts who may be more productive working in a pizza joint on Lexington Avenue.

Repeating what a vendor says is the easy part of the challenge. The hard part is making stuff work for users. Now ask yourself this question, “What certified Oracle data base administrator or certified platinum SharePoint consultant wants to put himself or herself out of a job?”

Search is important. Many working in search are not objective and not qualified to advise on enterprise information systems and  methods. The global financial depression has ushered in what I call “the pay to play” model. Objective information is tough to identify and often very difficult to discern. Which mobile operating systems is more successful? Android, Apple’s, other. The answer is, “It depends on one’s point of view.”

Information retrieval is hard to implement, do, and sustain. Search mostly disappoints. Search is useless when one has to pinpoint a “right answer” without additional, knowledge work. Perhaps there is  a silver bullet solution and, who knows, maybe it is here right now. I am blinded by pixie dust and cannot see reality like the “real” experts. What do you expect from someone my age with impaired vision. On the bright side, the information in this blog is free. Thank our sponsors, please.

Stephen E Arnold, November 10, 2011

Sponsored by no one. This is a freebie. Enjoy your weekend distractions. The search problem will keep until Monday morning.

Comments

Copyright © 2008 Beyond Search · ArnoldIT.com · Disclaimer · Revolution News theme by Brian Gardner · Log in