Technology from Harrod’s Creek
Stephen E. Arnold
October 6, 2003

Search: The New World Order of Subjective Relevance

It’s official. USA Today, the New York Times, and the Washington Post say that search means pay for clicks.

Says one former Monster.com professional riffed (Amerispeak for made redundant), “Traffic on sites is one of the keys to success today. Without the money to buy key words, online commerce companies will find the rules changing.”

How have the rules changed?

According to a former NBC Web professional, “Web sites are shifting from brochureware and static information to interactive media. In addition to the need for compelling content, Web sites will have to advertise using a wide range of tools. As traffic becomes more concentrated at a handful of key hubs, the cost for buying traffic can only go one way... up.”

What can an organisation with an informational Web site do to generate traffic?

As it turns out, there are a number of steps one can take to increase the likelihood of getting traffic from a search engine such as Google, Yahoo, or any of the other of search hubs that exist. A checklist of ideas includes these relatively easy actions that do not require the marketer to put pounds or euros in an advertiser”s pocket.

First, each page should carry a clear statement of the page’s content. This title appears sometimes in the title bar of most browsers when a page loads. For those whose memory of HTML is hazy, the syntax is <title>ABC Ltd’s Directory of Marketing Services</title>.

Second, insert metatags. These are the Web marketers’ equivalent of index terms, preferably selected to be meaningful to an informed user and directly related to the content of the page. The syntax illustrated extends the ABC Ltd’s example above: <META name="keywords" content="Web design, Photoshop, MySQL">.

Third, insert a meta Description tag. A quick check of the Web pages listed in Internet Resources Newsletter, a good barometer of Web sites of interest to information professionals (www.hw.ac.uk/libwww/irn/irn108/irn108.html) reveals that fewer than 50 percent of the sites include a meta Description tag. The meta Description tag is a short sentence that provides specific information about a the specific Web page. To illustrate with the ABC Ltd. example, insert <META name="description" content="ABC Ltd. is the leading Web promotion and marketing consulting firm in Tetbury, Glou.">.

Several caveats to these three tags. The big three search superpowers — Google, Yahoo (Inktomi), and Microsoft Search — have scripts that ignore, delete from the crawl list, or demote in relevance ranking Web pages that use these three tags to spoof the spider. A cornucopia of tricks can be ferreted from the bright and lively description of Web search at Jupiter Media’s Search Engine Watch newsletter. (www.searchenginewatch.com). Without digging into the use of metatag spam and similarly clever tactics, too many words, forbidden words, and misleading descriptions may result in a Web page or a site being treated as a second-class citizen.

Web sites that offer certain types of content can find themselves indexed as well. Search traffic experts agree that creating Web pages with useful information is a plus. Scholars and researchers have known about the value of content for a number of years, but the hip hop Web marketing firms have claimed this insight as their own.

Another plus is frequently updating of content in order to keep it “fresh.” Indexing scripts—described as spiders or crawlers—note the date and time stamp of Web pages. The more frequently a Web page’s content changes, presumably the fresher the content. The careful reader will note that it is possible to fool the spider, but in general the idea is that “fresh” content has more value than “stale content.” Once a spider realizes that a Web site hasn’t changed for a while, the spider will scale back the frequency with which it crawls that site.

Web sites looking for traffic, therefore, will want to provide useful information to the intended reader. Indexing spiders can be tuned to ignore certain types of content, so the Web marketer will want to provide valid HTML, XML, or content that can be indexed by most search engines. Although Google and Inktomi can make sense out of many file types, indexing of these documents is less than perfect, so the canny Web marketer will stick to the easily parsable HTML and XML approach.

The downside to using these metatags is that it costs money to add useful meta information to Web pages. Another often overlooked expense is the cost of updating metadata. The benefit of metadata usually outweighs its cost even though many search marketing gurus pooh-pooh the usefulness of metadata when buying words delivers results that are more reliable.

The concept of “useful information” is too arcane for consideration in this column. The Web marketer will want to fiddle with white papers, listings, news headlines, and product descriptions to see which combination of factoids results in more frequent visits by the indexing services. Web server logs contain records of spider visits in most cases, so it is a relatively easy task to compare frequency of visits by a particular spider before and after content changes are implemented.

Another aspect of content that is getting quite of bit of interest is the use of backlinks to a site (favored by Google) and pointers to other sites (favored by Ixquick, for example). Each indexing company has a different way of handling links, but in general links are often considered an indicator of a site’s importance. Web marketers, therefore, often include useful links for their intended visitors.

What happens if a Web mogul implements these tags, creates and updates useful content, and offers meaningful links to other sites and Google, Yahoo, and the hundreds of other Web indexing spiders still ignore one’s site?

You buy advertising.

The good news is that buying advertising works in most cases. Another bit of cheerful information is that most Web advertising uses the checkbook principle of accounting. An advertiser commits to a specific sum of money—say, £5,000 for 12 weeks of exposure, page views, words, or other metric in play. When the money is gone, the advertiser has the option of adding money to the account or stopping the program.

There is some bad news in Web advertising, and it is often hidden under layers of wooly verbiage in the advertisement contract.

First, there are different types of advertisements. Among the most popular are “words” or “phrases.” The price for these varies over time. The advertiser buys a key word such as “Web marketing.” This phrase may commend as much as £1 per click on overture and about half this on FindWhat.com. Our friends at ABC Ltd. allocate £1,000 to this phrase and garner 1,000 clicks in the first day of exposure.

Here’s the clever part. If ABC Ltd. does not allocate more money to the phrase “Web marketing”, another firm can buy the term at a higher price. Thus, the newcomer to key word marketing will want to be sensitive to the auction pricing model that operates.

LookSmart (originally Australian but not thoroughly Americanized) was one of the leaders in “paid inclusions.” Inktomi also offers a similar program for Yahoo and the other sites licensing search from it. Inktomi has provided search and paid inclusion to Microsoft. When Microsoft’s new search service becomes available, one wonders if Inktomi’s relationship with Microsoft will continue.

Unlike key words, paid inclusions appear whether the person entering a query uses a word, phrase, or concept related to the subject of the inclusion. In short, a paid inclusion puts a link in a user’s list of search results whether the subject of the advertisement is related to the query. Although some purists may find irrelevant hits offensive, most users seem oblivious to their presence. Paid inclusions are priced on a block of clicks. Prices are in flux, so the rate schedule can vary over time.

Web marketers can purchase advertisements on specific Web sites’ pages. For Web site operators wanting traffic from a particular geographic region, a combination of print and online advertising in specific media and on a few high-traffic sites may be a prudent approach. Costs for traditional advertising work on the “rate card” basis calculated on a projected number of exposures. Buying traditional advertisements can be pricey when the art work, management time, and follow up activities are included.

What’s the optimal approach for an organization that wants more traffic? The pragmatic approach is to develop the necessary tags for the spiders. Google, Yahoo, and many other search services permit a free Web page submission. The free is attractive until one realizes that the indexing service may take four weeks or more to add a site to the crawl list. Even then, there is no guarantee.

The pragmatic Web marketer will turn to a submission service. Submission services, submission consultants, and submission software are abundant. One useful service, according to the experts in traffic, is Submit Express (www.submit-express.com). The service offers a free service and a for-fee service. For about £18, Submit Express will hose 75,000 search engines with submission details for a Web site. For those skeptical of a 75,000 site submission, the Add Pro service provides a form-based service to help out the more conservative Web marketer.

For indexing professionals and those with degrees in information science, the new world order cares precious little about precision and recall. One wonders if objective indexing will undergo a renascence. One can only hope.

 

Contact Mr. Arnold at:

Postal Box 320
Harrod’s Creek, Kentucky 40027, USA
Voice: (502) 228-1966
Facsimile: (502) 228-0548
Electronic mail: sa@arnoldit.com