LBS II: Query Processing Performance Degrading
May 27, 2009
Question: When we brought our search system online a month ago, query processing was snappy. Now it is noticeably slower. What’s going on? How do we speed up performance?
Answer to this Little Baffler: The addled goose needs much more information about your specific implementation of search. You will want to take a close look at some hot spots that crop up, regardless of the system. Here are some places to start your investigation:
- Check the log files for spikes. Increased usage and / or users sending more complex queries can choke a system. If you have experienced spikes, you may have to throw hardware at the problem, add bandwidth, or take more drastic steps such as refusing queries under certain conditions.
- Look at the hardware, RAM, and storage for the query processing server or cluster. Ask your service provider or your colleagues what’s under the hood. Search vendor specifications are often understated. One immediate fix is to add RAM. Other hardware architectural changes may be necessary if the initial set up in not up to snuff.
- Verify that indexing and crawling are taking place on dedicated machines. If you are sharing servers or running a sub optimized virtual environment, you may see the symptom as slow query processing and not realize that the cause is the implementation of the indexing and crawling subsystems.
How common are performance problems in enterprise search? Very common.
Stephen Arnold, May 27, 2009