New Idea’s Founder Speaks, New Search Tools Service in Beta
July 21, 2008
New Idea Engineering is one of those specialized engineering firms that keep a low profile because the company is swamped with work. Miles Kehoe and Mark Bennett, the two founders of New Idea have deep experience with search and related technologies. Messrs. Kehoe and Bennett, , revealed in an interview for the Search Wizards Speak series the premise of their firm:
New Idea has from Day One tried to make our products and tools cross-vendor, but none of the major vendors has any incentive to do so until customers start objecting.
This is a clear statement of one reason why search vendors may not rush to resolve some issues for their customers. Now The problems with enterprise search are now becoming more widely known. New Idea’s founders explain why:
…The biggest problem we see in failed implementations is that the technology the customer picked is just not the right one for their environment. Corporate IT managers have to remember that a great demo is indistinguishable from product, but sometimes they seem willing to accept the vendor’s demo as a suitable substitute for their environment. There is also a mind set in many IT departments that search is either not critical - it’s still often a “check-box item” - or that it must be terribly easy…
You can read the full text of the interview here. Additional information about New Idea is here. The company has a useful Web log, and a new addition to the New Idea arsenal of useful resources is a listing of software tools that can help untangle some of the Gordian knots in an enterprise search deployment. An alpha version of the new service called Search Components Online is available here.
Disclaimer: I have provided some information about open source and shareware content transformation tools. Kudos to the New Idea Engineering team for creating a much-needed listing that can help those struggling with flawed enterprise search systems or consultants trying to help their customers get their system back online. I have linked to the company’s enterprise search Web log and cheerfully nabbed nuggets from the company’s informed postings.
Stephen Arnold, July 21, 2008
Federated Search: List of Presentations
July 21, 2008
Deep Web Technologies Web log showcases almost two dozen presentations about federated search. The list was compiled by Sol Lederman, one of the key figures at Deep Web Technologies. An interview with Abe Lederman appeared in the Search Wizards Speak series in June 2008 here. The company’s Web log is FederatedSearchBlog.com, and you will want to navigate to this useful list of presentations. Note a couple of the referenced presentations are on a third party service called Slideshare, which some authors referenced by Deep Web Tech rely upon, is troublesome. Kudos to the Deep Web Tech team for their work.
Stephen Arnold, July 21, 2008
Adea Introduces Connector for GSA for SAP
July 21, 2008
Did the acronyms baffle you? GSA is the Google Search Appliance. SAP is the ubiquitous vendor of expensive enterprise software. Adea, a consulting firm, has created an adaptor to “hook” the GSA into an SAP environment. The idea is that instead of the expensive SAP search solutions (TREX and Endeca, for instance), you can buy Google boxes.
The product is Ocelli. The software makes it possible to search SAP data in a simpler way than SAP provides in its default environment. Among the Ocelli functions is support for role-based searches and results display.
For more information about Adea, click here. The Ocelli product information page is here. I don’t have current pricing information at this time.
As Google’s enterprise team gets pulled into more opportunities, companies such as Adea will find it worth their while to create custom software. Google itself seems willing at this time to let its partners carry the ball for integration and customer support in some accounts.
Microsoft started its partner program in a benign way as well.
Stephen Arnold, July 21, 2008
iPhone BI
July 20, 2008
Business intelligence is coming to the iPhone, the mobile device that delights those with tiny fingers and a youthful love of multi-function gizmos. Cliff Saran’s “Oracle and Salesforce Develop iPhone Business Apps”, published on July 11, 2008, pulled two announcements together that I had overlooked. One of my engineers has an iPhone, and I watch him surf the Web, make the occasional call, and send misspelled SMS messages to me about projects’ status. You can read Mr. Saran’s interesting article here.
Mr. Saran identifies two companies pushing into what is for me a territory near the BlackBerry frontier. First, he describes Oracle’s Business Indictors. The id4ea is that a hip CFO will use her iPhone to “to view the latest company financial trends and enables sales managers to receive alerts on sales performance and customer satisfaction issues.” You can learn more about Business Indicators here but the Oracle Web site is a sometimes snail-like machine. Be patient.
He also describes Salesforce Mobile for the iPhone. This application “allows Salesforce users to view and edit records (accounts, opportunities), log sales or service activities such as e-mails, phone calls, and in-person meetings, and assign tasks and events to colleagues.” More about SMiP is here.
From my vantage point in rural Kentucky, it is too soon for me to make a call about the iPhone’s ability to gain a foothold in the enterprise. But my engineer is, as he says, “likin’ it”. Also, I want to see how the Apple and Google mobile initiatives interact. Could a showdown be coming between Apple and Google with Research in Motion relegated to the scrub team?
Stephen Arnold, July 20, 2008
More Burton Group Wisdom about Enterprise Search
July 20, 2008
On July 17, I commented on the first part of an interview conducted by Margie Semilof. Her interview subjects were two consultants attached to the Burton Group. You can read my views and opinions about Part I here.
The second part arrived today with a publication date of July 14, 2008. Like the US snail mail service, express delivery can vary. You can read the full text of the interview with Guy Creese, a content management expert at Burton Group, a Midvale, Utah-based research company here.
I don’t get too excited about content management systems, and I am quick to point out that CMS has created more challenges than CMS has resolved. Software experts in CMS who want to make the leap into enterprise search or a closely allied field remind me of some of the MBAs at Booz, Allen & Hamilton when I worked there 35 years ago. These confident folks figured that with a half hour and some journal articles, no discipline was beyond their ken.
Now to the second part of the interview, which is titled “ABCs of Enterprise Search.” So, my first point is that the poetic convention is abandoned and the interview does not cover the sweep of enterprise search. The interview hits a handful of points, scattering generalizations to dazzle the interviewer.
Several points warrant comment.
First, Mr. Creese, CMS expert, narrows enterprise search to “finding content on your website as well as searching behind the firewall.” Hmmm. In a recent search installation, one of the key requirements was sending queries to a number of public Web sites, concatenating the results, and generating answers. The focus was not finding, but answering questions. Since finding systems have a dismal track record in user satisfaction, I am baffled by this definition.
Second, the interviewer asks a question about tools to normalize content so that a query goes across collections, data types, and sources. The answer given my Mr. Creese is about what’s happening in the market and what Mr. Creese calls segmenting the search market in “a Goldilocks fashion.” I have no idea how transformation and normalization tools relate to the Goldilocks reference. I ask myself, “What the heck is this CMS expert referencing.” Maybe the editor fouled up. Wacky stuff. In his answer, Mr. Creese suggested Microsoft could not develop Fast Search technology itself. I think Microsoft has tried to develop search technology and found that it was not competitive with products from Microsoft Gold partners and tried to buy a Cadillac. Fast is more of a Chevrolet Vega, and I think Microsoft has yet to confront the challenges the acquisition puts on Microsoft’s doorstep.
Third, the interviewer asks, “Who is in charge of enterprise search?” The CMS expert names people who could be in charge. The answer in most organizations is, “No one.” Search is a hot potato and responsibility moves around like a miniature poodle with fleas. I find that more organizations are using procurement teams because the single point responsibility does not work very well.
Fourth, the interviewer wants to know about Microsoft’s acquisition of Fast Search & Transfer. Mr. Creese, if I understand his answer, is that Microsoft wanted a migration strategy and a way to get around the document limit in SharePoint search. My understanding is that Microsoft itself may be confused about why it bought Fast Search & Transfer. The price tag was astounding. The Fast ESP installations require quite a bit of baby sitting to work. Integration so far consists of a Web part. One source in Norway told me that Microsoft is cleaning up the rumors swirling around Fast Search’s financial dealings. In short, the logic is clear in the Microsoft news releases. The reality is a trifle muddy.
Fifth, the interviewer wants to know what one gets with Fast Search & Transfer search technology. In my experience, you get a box of parts. Licensees have to have these assembled to deliver a system that meets specific requirements. As long as the resources are available, Fast ESP–as well as a number of other vendors’ systems–can perform many marvelous functions. When resources aren’t sufficient, well, there’s some trouble in paradise.
My thought is that the answers as published reveal a CMS expert who has not been deeply involved in multiple enterprise system projects. Just like the confident MBAs at Booz, Allen so long ago. It’s easy to talk; harder to be deeply informed.
Agree? Disagree? Help me learn.
Stephen Arnold, July 20, 2008
Really Useful Microsoft Jargon Demystified
July 19, 2008
Please, navigate to the Enterprise Search Web log here. The July 18, 2008, post “Microsoft Terms for SQL Server Search Components” is worth its weight in Windows XP OEM installation disks. Dr. Search, the author, defines indexing, collection, and tokenization, which I found quite useful. Even better, the mysterious yet wizardly Dr. Search includes a link to an article by Mark Bennett. “Contrasting Relational and Full-Text Engines” makes it clear Mr. Bennett knows more about search technology than I do. I suppose I should insult him, but this 2004 essay is too useful. I will probably recycle large chunks of his knowledge in a future writing, so I have to tread lightly. Download the article here. Note that it is the property of New Idea Engineering, an outfit with a solid track record.
An exclusive interview with the founders of New Idea Engineering will appear in the ArnoldIT.com Search Wizards Speak series here. I will post a news item in Beyond Search so you can read the full interview with the New Idea technology Jedi knights. The interview is as useful as “Contrasting Relational and Full Text Engines.” Highly recommended. New Idea Engineering’s Web site is here.
Stephen Arnold, July 19, 2008
Corporate Social Networks: Might Be Losers
July 19, 2008
On July 17, 2008, ReadWriteWeb.com let the cat out of the bag. Marshall Kirkpatrick’s essay “Corporate Social Networks Are a Waste of Money, Study Finds”. Please, read the full post plus the useful linked stories embedded in the essay.
The source of this insight is Edward Moran, a brain-for-hire at Deloitte Touche Tohmatsu, one of the remaining Big Eight largely untarnished by service scandals in the last few years. As a veteran of Booz, Allen & Hamilton, I am reluctant to embrace these types of studies without reading and studying them first hand. I am, therefore, not agreeing or disagreeing with the news media’s summary of the story or Mr. Kirkpatrick’s analysis.
I wanted to capture the most important point for me; to wit:
They [corporate social networks] are stupid.
Brilliantly summarized, Mr. Kirkpatrick.
I continue to be skeptical of social networks as a user. As a person who has worked in law enforcement and intelligence for short periods of time over the years, I think social networks are extremely useful. I don’t want to participate. I want to peruse the information about the topics, who reads what, who comments, on what, and other tasty info nuggets.
My view of social networks is probably not what the cheerleaders for social networks and collaborative systems had in mind for their products. That’s okay with me. Keep ‘em coming. Pump up that usage. Every little clickstream helps.
Stephen Arnold, July 19, 2008
Search All Information: The Categorical Affirmative Is Alive and Well
July 18, 2008
I think I took a logic class in 1964 from Dr. William Brown, a Ph.D. with an encyclopedic knowledge of Will Rogers, the American humorist. Dr. Brown could whip out one liners to make a point. I watched as my classmate when he said, “Will Rogers said, ‘A man only learns in two ways, one by reading, and the other by association with smarter people.’” My classmate avoided categorical affirmatives and negatives for the remainder of the semester. I’m not sure if my classmate understood the quotation or if the attention Dr. Brown directed at my classmate drove the lesson home.
This morning my news reader presented me with this headline: “We Offer a Single Place to Search All Information”. I clicked the link and read an interview conducted by Ruth Samson for DQ Channels, a publication with which I was not familiar. Her interview subject was Sanjay Manchanda, Director, Business Division, Microsoft India. You must read the interview here.
The subject of the interview is Microsoft, Fast Search & Transfer, and assorted closely-aligned issues. Here’s my list of what caught my attention:
- Demand for SharePoint Search comes from “companies that have huge amounts of data and need to search quickly and constantly.” My notes to myself: What’s huge? SharePoint native search dies at 50 million documents, sooner if the documents are multi-megabyte jobs with SmartTags and other goodies stuffed inside the wrapper.
- Unique selling proposition: “we have built our search capabilities so deeply that it is seamless. We offer a single place to search information of different types. Our recent acquisition of FAST, a provider of enterprise search solutions, also gives us an edge over other players in the segment.” My notes to myself: No way. Search is a crazy quilt. Fast Search is not one thing. Powerset is older technology given a new coat of paint. Baloney. Seamless! Not true. The stitches are evident, far apart, and made with a nail, not a needle.
- Microsoft has 3,600 partners. My notes to myself: Hmm. Are these SharePoint partners or total Microsoft partners? 50 to 60 partners in India. Why the spread?
- Cost of SharePoint is variable. Starts low, then increases as the number of users goes up. My notes to myself: Ah, exactly like the IBM and Oracle models. Business model may not work in the present economic environment. Search server in India costs R20,000 or $470. Now that’s a deal with Fast Search targeting $150,000 in early 2007.
- Search is “the next killer app”. My notes to myself: Nope, search is disappointing, a utility function, and not what users want. If this Microsoft expert Sanjay Manchanda is right, Microsoft is chasing the wrong ambulance.
The interview struck me as a marketing effort. Not only was the “all” spurious, the idea that Microsoft search is “seamless” is not congruent with my experience. What’s seamless is loading a third party search system such as Isys Search Software and getting something that doesn’t take a platoon of Microsoft engineers scrambling around for days.
The interview has some one-liners that would make Will Rogers smile. Dr. Brown would still be fussing over the “all”. I am hung up on “seamless”. Quite an interview.
Stephen Arnold, July 18, 2008
Google: Stepping on the Enterprise Search Gas Pedal
July 18, 2008
Richard Martin’s “Pressed by Rivals, Google Accelerates Enterprise Search Efforts” is a revelation. The story appeared on July 17, 2008, and you can read the full text here. (You may see an annoying fly over ad and then be redirected. The wacky url worked for me.)
In my opinion, The most interesting point in the scoop was:
Unifying a single search interface across those varied systems, which often use different protocols and programming languages, has become something of a holy grail.
The religious metaphor is lost on me, but I think Mr. Martin’s point is that search is supposed to make it possible to access data, regardless of its location or format, from one interface.
I agree.
My view is that this is not a search problem and, therefore, the GOOG, Microsoft, and the companies mentioned in the article will not solve this problem with search technology.
The challenge is transformation of data in such a way that new operations are possible. Who wants to search? I want to know where a particular item comes from and how certain I can be that the item is accurate or good enough to use in a decision.
In my opinion, talking about enterprise search is a waste of time. Google is definitely going to deliver, but the solution will not be a search technology. The problem is better resolved in terms of data management. Google has some interesting technology in this practice area, and I discuss it in my new study Beyond Search, published by the Gilbane Group.
Stephen Arnold
What Is SharePoint?
July 18, 2008
A few publishers print tabloids and magazines. A hard copy of System Management News, July 15, 2008, arrived today. My lunch appointment was running late so I flipped through the newsprint tabloid and saw this headline, “Microsoft’s SharePoint Hits Sweet Spot as the Next Killer App.” The author of this apologia is Patrick Hynds, president of Critical Sites and a Microsoft Regional Director. Mr. Hynds is a good writer, and he does let his enthusiasm for SharePoint sparkle at every opportunity. The hard copy has a Web corollary at www.sysmannews. A digital version of Mr. Hynds’s analysis is here. I urge you to read the original.
The key point in the write up is that SharePoint is a “killer app”. For me, the most interesting point in the article was:
Let’s hope Microsoft doesn’t get too visionary for its own good and and expand SharePoint beyond the sweet spot it now occupies so well. It is all about collaboration and acceleration of information sharing.
The notion of a server as a killer application befuddles me. Actually, quite a bit about SiteServer, oh, I mean SharePoint makes me think about the fees consultants can assess. Without a doubt, SharePoint packs more buzz word goodness per byte than almost any other Microsoft application.
First, you have to figure out which SharePoint to license or use. The two versions are:
- WSS or Windows SharePoint Services 3.0.WSS includes Windows 2003 Server and Windows 2003 Small Business Server.
- MOSS or Microsoft Office SharePoint Server 2007.
There are numerous differences between the two platforms but both offer search, wikis, Web logs, and calendaring.
To get SharePoint to work, you will need other Microsoft server products. The two must haves are SQL Server and Exchange. Exchange is also described as a collaboration application, presumably because some executives still send email with attachments.
Rube Goldberg: Software Architect
Whenever I read cheerleaders’ scripts, I am impressed with how each each is to remember. The reality of being a cheerleader is different from the surface appearance and the facile nature of the chants. SharePoint is for me like a cheerleader and a catchy chant. “Push em back, push em back, way back.”
SharePoint slices, dices, chops, shreds, and cuts julienne potatoes, which I would not recognize if you showed me one now. To my simplistic mind, SharePoint is a layer of spackle that one uses to fill in the gaps among islands of Microsoft functions. A properly deployed and resourced SharePoint makes it possible to create a document, make it available to authorized users, and publish the document as a Web page. The search, the collaboration, the federating of email, local, and remote information, and the rest of the bells and whistles is a huge, sprawling favela (Brazilian slum).
Source: http://www.travelblog.org/Photos/171232.html
Underneath the layers of code is a content management system. But to make the CMS work, you have to buy other Microsoft servers, use Microsoft programming tools, and drink Microsoft KoolAid.
The problem is that SharePoint has great marketing and lousy plumbing.

