Successful Enterprise Search Management

by Stephen E. Arnold and Martin White

Successful Enterprise Search Management

Martin White and Stephen E. Arnold have co-written "Successful Enterprise Search Management." The study distills the authors' more than 50 years of search and information access experience into 130 information-packed pages. The project consumed eight months. Instead of filling the study with profiles of vendor technologies, Messrs. White and Arnold narrowed their focus to the most important issues in the management of enterprise search procurement, deployment, and enhancement. The study mentions dozens of vendors, includes numerous case studies, and contains illustrations of business procedures. When the manuscript was in near-final form, the authors asked other experts in enterprise search to review the text, make suggestions, and identify oversights. Messrs. White and Arnold took the reviewers' observations to heart and made adjustments. The result is a useful study that presents information not readily available in other books about search, content processing, and information access in organizations. A complete table of contents appears below. The electronic version of the book will be available for download in December 2008, on or close to December 15, 2008. Hard copies of the study will be in January 2009. You can order the publication directly from the publisher, Galatea in England. Click here to order.

Foreword

Acknowledgements

Part I: Setting the scene

1. Ensuring Success, Reducing Risk
   1. Introduction
   2. In the beginning
   3. Exponential growth
   4. The rise of enterprise search
   5. The start of the journey
2. Why Enterprise Search is Difficult
   1. Expectations
   2. What is enterprise search?
   3. What makes search a challenge?
   4. Management challenges
   5. Technical challenges
      1. Scaling
      2. Upgrading
      3. Tuning
      4. Customising
      5. Transforming
   6. Informational issues
      1. Versions
      2. Exceptions
      3. Volatility
   7. Cracking the tough nut
3. The Enterprise Search Industry
   1. Market overview
   2. Google and the enterprise
   3. Pressure and compression
      1. Business intelligence
      2. Superplatforms
      3. Competitive implications
   4. eDiscovery and brand names
      1. eDiscovery
      2. Brand name vendors
   5. Fast-growing sectors
      1. Specialist systems
      2. Up-and-coming vendors
      3. Challenges for eDiscovery and brand name vendors
   6. New players in enterprise search
      1. Low-cost and open source systems
      2. Search utilities
      3. Start-ups
   7. Characteristics of the enterprise search market
      1. Implications

Part II: How search works

1. The Technology of Search
   1. Introduction
   2. Components of a search system
      1. Crawler sub-system
      2. Document processing sub-system
      3. Indexing sub-system
      4. Query processing sub-system
      5. Administrative sub-system
   3. Infrastructure
      1. Technology to handle peaks and valleys
   4. Assisted navigation
      1. Adding advanced text processes
   5. Content processing
      1. Semantic processing
      2. Entity extraction
      3. Classification
2. Emerging Search Technologies
   1. New technologies
      1. Discovery
      2. Federated search
      3. Social search
      4. Semantic search
   2. What about ontologies, taxonomies and classification systems?
      1. Classification system
      2. Knowledge base
      3. Ontology
      4. Taxonomy
      5. Keeping perspective
   3. Practical suggestions regarding search technology
3. Managing Security
   1. Collection and document level security
      1. Organisation-wide policy
   2. Access control
   3. Approaches to secure search
      1. Vendor approaches to security
   4. Managing data privacy
   5. Secure search administration
      1. Programming policies for security
   6. A search security checklist
      1. Define the system scope
      2. Plan, modify, follow the plan, adapt, and then repeat
      3. Players and constituencies
      4. Security metrics
      5. Specify thresholds and parameters
      6. Perform security audits
4. Managing Language
   1. Introduction
      1. The challenges posed by the real world of language
      2. Handling multiple languages
      3. Same words – different meanings
      4. Specific challenges for search technology
   2. Linguistic analysis
      1. Latent semantic indexing (LSI)
   3. Entity extraction
   4. Searching for personal names
   5. De-duplication
   6. Managing collections in multiple languages

Part III: How to select a vendor

1. Defining Your Enterprise Search Requirements
   1. Introduction
   2. Total cost of search implementation
      1. License fee
      2. Software maintenance
      3. Upgrades
      4. Additional modules
      5. Database licenses
      6. Professional services
      7. Hardware and infrastructure
      8. Training
      9. In-house support team
   3. Making a business case
      1. Defining user requirements
      2. Current search system audit
      3. Enterprise application road map
      4. Search screens
      5. Collection analysis
   4. Federated search requirements
   5. Repository method
2. Writing the Tender Specification
   1. Introduction
   2. Defining objectives
   3. Information systems strategy
   4. Compiling the RFP
      1. Accessibility support
      2. APIs
      3. Crawls and indexing
      4. File types
      5. Proof of concept
      6. Risk assessment
      7. Security management
      8. Upgrade release schedule
   5. Supporting a global implementation
      1. Using the incumbent systems integrator
      2. Using a new systems integrator
      3. Using a specialist search integrator
3. Selecting a Vendor
   1. Introduction
   2. The selection team
   3. Developing a short list
   4. The presentation
   5. The difficult questions
      1. Additional software licenses
      2. APIs
      3. Customisation
      4. Hardware and software
      5. License terms
      6. Preferred architecture
      7. Search reporting
      8. Security
      9. Taxonomy management
      10. Training and support
   6. The debrief
4. Proof of Concept
   1. Methods exist, not followed
   2. A definition
   3. Steps in the proof of concept
      1. Overall work flow
      2. Management track
      3. Test corpus track
      4. Infrastructure track
   4. Case example: demos are not systems
   5. A plan
   6. Test corpus
   7. Document distribution and collection
      1. Special cases
      2. Legacy data
      3. Confidential data
      4. Copyrighted information
      5. Practical considerations
   8. A General Mills bake off
      1. What to test
      2. Test queries
   9. Avoiding pitfalls
   10. Reducing risk

Part IV: How to implement search

1. Managing the Search Team
   1. Introduction
   2. At the very beginning
   3. Roles and responsibilities
      1. Search (Information Discovery) Manager
      2. Search Technology Manager
      3. Information Specialist
      4. Search Analytics Manager
      5. Search Support Manager
      6. The core team
   4. Supporting global enterprise search
   5. Creating a global Center of Search Excellence
   6. Reporting lines
2. Deploying the Search Solution
   1. Scope
   2. What to do
   3. Why search is different
      1. Information is fluid
      2. Search is not one system
      3. Content changes
   4. Dependencies
   5. Two dependency hot spots
      1. The diagram
      2. The coding loop
      3. The security loop
   6. Avoiding delays and cost overruns
      1. Lack of management support
      2. Too much customisation
      3. Content woes
   7. Tips and tricks
      1. Plan early and adapt to changes
      2. Obtain management support
      3. Limit features
      4. Minimise customization
      5. Figure out the content scope
      6. Security
      7. Use consultants
3. Search Usability
   1. Introduction
      1. Ranking and relevance
      2. Other search metrics
   2. The search dialogue
   3. Federated search
   4. Search page design
      1. Limiting the scope of the search
      2. Numeric values
      3. Other common search problems
      4. Best bets and URL management
      5. Auto categorization
      6. User-centric design
4. Search Analytics
   1. Introduction
   2. A multiplicity of parameters
   3. Search logs
      1. Number of searches per hour/day
      2. Top 50 searches by search terms/query
      3. Top 50 searches leading to no results being presented
      4. Top 50 searches leading to no document being selected
      5. Top 50 most requested documents
      6. Top 50 searches where more than three pages of hits were
      7. presented
      8. Top 50 searches by duration of search
      9. Searches by number of results presented
      10. Saved searches
      11. Searches where new documents are alerted to the user
   4. It takes more than search logs
   5. Taking action
5. Search System Upgrades
   1. Backstory
   2. Case example: major vendor
   3. Case example: Microsoft SharePoint
   4. The dilemma
   5. Trade-off analysis needed
      1. Data you need
      2. Past history
      3. What other licensees tell you
      4. Vendor’s sales or support engineers
      5. The web
      6. Third-party search experts
      7. Interpreting the data
   6. Reasons to upgrade
   7. Reasons not to upgrade
   8. A road map
      1. Spell out what’s to be done
      2. Plan for a restore or fail over
      3. Get commitment form your team
      4. Cut over day
      5. Understanding the storage method
      6. Key points to note
   9. Net net

Part V: Wrap-up

1. 17 Action This Day
   1. Introduction
   2. A growing appetite
   3. A structured approach
      1. Planning
      2. Preparation
      3. Implementation
      4. Adaptation
   4. A strategic perspective

Appendix 1 – Summary of Key Questions for Successful Enterprise Search

Glossary