Enterprise Information’s Missing Pieces
June 5, 2008
In 2001, I found myself on a panel talking about electronic information and enterprise search. The venue was Internet World. That’s right the once dominant trade show for the brave new world of online.
I’m not sure how I ended up on the program, but I recall I was there, facing an audience of 250 people. Put the word “Internet” on a hand lettered sign in a diner’s window and a crowd would gather. The Internet has evolved but the missing pieces in the information puzzle are still with us.
Here’s an image from my PowerPoint deck.
Web log graphics are “crunched” and the result is difficult for me to read. Let me highlight each of these nine pieces of the enterprise information puzzle.
- Graphical editor
- Database engine
- Version controls
- Site manipulation tools (that is, publishing tools)
- Personalization tools
- Search engine
- Administrative interface
- Usage tracking
- Security services
Nothing is missing. The nine elements are identified in the graphic, and in your own organization you have each of these functions up and running. Some puzzle pieces work better than others. These are complex sub systems and functions. Variability and unevenness are to be expected.
My point in 2001 was that each of these pieces was not fitted to the others. The parts are there, but until integration across different sub systems and functions, the puzzle is incomplete. In fact, you don’t even have a decent picture of what the integrated results will look like.
Back to the Present
In 2001, I talked about a “digital Vegematic”. As you may know, a Vegematic was a plastic and metal device that could perform a number of chopping, slicing, and dicing operations. A digital Vegematic ingested information, processed it, and delivered to the users usable outputs.
In my talk, I highlighted a demonstration I had see of the NCompass Labs system. This company had developed a software system that would perform the many different operations needed to put the puzzle pieces together. I created content; I saved it; I repurposed it; and I output it as a Web page and as a printed one-page document.
nCompass Labs was acquired by Microsoft. The company’s technology influenced the product we know as SharePoint today. Here’s schematic of the system shortly after Microsoft acquired the company. Scan the diagram, then look at the questions below.
Here are the questions. This is not a test; it’s a thought experiment about technology in 2001 and technology in 2008:
- Does this system scale?
- Is the system stable?
- Does the system deliver reasonable performance without bottlenecks?
- Can you maintain this system with your present technical expertise?
- Do you know a human being who can make these moving parts work together in less than 48 hours?
My answers to these questions summarize the conclusion to my Internet World talk seven years ago.
First, yes, the system does scale if I have enough money, hardware, and time to push data through the many layers of the system.
Second, yes, the system is stable if I gate the amount of information fed into the system. Surges could cause bottlenecks. Bottlenecks choked or crashed one or more of the sub systems.
Third, is the performance reasonable. With processor load in the 20 percent range, no problem. When processor loads rose about 80 percent, there was a degradation in performance. The system was serialized, so when one process slowed, the next process sat idle until the first process completed its work and made the output available to the dependent process.
Four, I cannot maintain the 2001 system alone. It involves too many moving parts. The skills needed to handle SQLServer are not particularly useful in working out the problems with the content connectors.
Fifth, I do not know a single person with the ability to get this system up and running in 24 hours.
These points are unchanged in June 2008. After seven years, we have not yet put the puzzle pieces together. There are very few organizations with a cohesive, integrated information environment. Think of your own employer or the companies with which you work. How screwed up are these organizations information systems.
The complexity of the systems, the need for trained human hands to fiddle, and the dearth of resources means that the problem I identified in 2001 still exist. In many organizations, the information situation has gone backwards, deteriorated. With staff reductions and turnover, the bright minds who could “find” the needed information are gone.
Even successful outfits like Amazon suffer from this complexity and “puzzle piece” problem. Several months ago, I was asked to attend a meeting at Amazon. At a break, I asked two Amazonians (male variety) if there was a single code base and specific programming requirements. The answer surprised me. I recall that one of the Amazonians said, “Man, we do two-pizza teams. When there’s a problem, a small team has to fix the glitch. It’s exciting because we never know what language was used. It’s a giant puzzle.”
In closing, when systems and functions operate in an atomic fashion there is no overall information picture. Until the pieces are assembled, it is hard to know if something is missing. Before the pieces are assembled, there is not homogeneity in the information fabric.
In this type of “virtual silo” set up, it is remarkable that systems function at all. The two-thirds dissatisfaction with enterprise search, one could assert, should be closer to 90 percent. I’ll will let you know what my assessment of the missing pieces in 2016. My hunch is that progress will be made, just slowly.
Stephen Arnold, June 7, 200
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
