• Home
• About
• Writers
• Landscape
• Wizards Index
• Fraud
• Wanna Be Like Larry?

Mysteries of Online 6: Revenue Sharing

This is a short article. I was finishing the revisions to my monetization chapter in Google: The Digital Gutenberg and ran across notes I made in 1996, the year in which I wrote several articles about online for Online Magazine. One of the articles won the best paper award, so if you are familiar with commercial databases, you can track down this loosely coupled series in the LITA reference file or other Dialog databases.

Terms Used in this Write Up

My “mysteries” series has evoked some comments, mostly uninformed. The number of people who started working in search when IBM STAIRS was the core tool are dwindling in number. The people who cut their teeth in the granite choked world of commercial online comprise an even smaller group. Commercial online began with US government funding in the early 1960s, so Ruby loving script kiddies are blissfully ignorant of how online files were built and then indexed. No matter. The lessons form foundation stones in today’s online world.

Indexing and Abstracting: A Backwater

Aggregators collect content from many different sources. In the early days of online, this meant peer reviewed articles. Then the net gathered magazines and non-peer reviewed publications like trade association magazines. Indexing and abstracting in the mid 1960s was a backwater because few publishers knew much about online. Permission to index and abstract was often not required and when a publisher wanted to know why an outfit was indexing and abstracting a publication, the answer was easy. “We are creating a library reference book.” Most publishers cooperated, often providing some of the indexing and abstracting outfits with multiple copies of their publications.

Some of the indexing and abstracting was very difficult; for example, legal, engineering, and medical information posed special problems. The vocabulary used in the documents was specialized, and word lists with Use For and See Also references were essential to indexing and abstracting. The abstract might define a term or an acronym when it referenced certain concepts. When abstracts were included with a journal article, the outfit doing the indexing and abstracting would often ask the publisher if it was okay to include that abstract in the bibliographic record. For decades publishers cooperated.

The reason was that publishers and indexing and abstracting outfits were mutually reinforcing operations. The published collected money from subscribers, members, and in some cases advertisers. The abstracting and indexing shops earned money by creating print and electronic reference materials. In order to “read the full text”, the researcher had to have access to a hard copy of the source document or, in some cases, a microfilm instance of the document.

No money was exchanged in most cases. I think there was trust among publishers and indexing and abstracting outfits. Some of the people engaged in indexing and abstracting crated products so important to certain disciplines that courses were taught in universities worldwide to teach budding scientists and researchers how to “find” and “use” indexes, abstracts, and source documents. Examples include the Chemical Abstracts database, Beilstein, and ABI/INFORM, the database with which I was associated for many years.

Pay to Process Content

By 1982, some publishers were aware that abstracting and indexing outfits were becoming important revenue generators in their own right. Libraries were interested in online, first in catalogs for their patrons, and then in licensing certain content directly from the abstracting and indexing shops. The reason for this interest from libraries (medical, technical, university, public, etc.) was that the technology to ingest a digital file (originally on tape) was becoming available. Second, the cost of using commercial online services which would make hundreds of individual abstract and index databases available was variable. The library (academic or corporate) would obtain a password and a license. Each database incurred a charge, usually billed either by the minute or per query. Then there was online connect charges imposed by outfits like Tymnet or other services. And there were even charges for line returns on the original Lexis system. Libraries had limited budgets, so it made sense for some libraries to cut the variable costs by loading databases on a local system.

By 1985, full text became more attractive to users. The reason was that A&I (abstracting and indexing) services provided pointers. The user then had to go find and read the source document. The convenience of having the bibliographic information and the full text online was obvious to anyone who performed research in anything other than a casual, indifferent manner. The notion of disintermediation expanded first in the A&I field because with full text, why pay to crate a formal bibliographic record and manually assign index terms. The future was full text because systems could provide pointers to documents. Then the document of interest to the researcher could be saved to a file, displayed on screen, or printed for later reference.

The shift from the once innovative A&I business to the full text approach threw a wrench into the traditional reference business. Publishers were suspicious and then fearful that if the full text of their articles were in online systems, subscription revenues would fall. The publishers did not know how much risk these systems poses, but some publishers like Crain’s Chicago Business wanted an upfront payment to permit my organization to crate full text versions of certain articles in the Crain publications. The fees were often in the five figure range and had additional contractual obligations attached. Some of these original constraints may still be in operation.

Negotiating an online deal is similar to haggling to buy a sheep in an open market. The authors were often included among the sheep in the traditional marketplace for information. Source: http://upload.wikimedia.org/wikipedia/commons/thumb/0/0e/Haggling_for_sheep.jpg/800px-Haggling_for_sheep.jpg

Revenue Sharing

Online vendors like Dialog Information Services knew that change was in the air. Some vendors like Dialog and LexisNexis moved to disintermediate the A&I companies. Publishers jockeyed to secure premium deals for their full text material. One deal which still resonates at LexixNexis today was the New York Times’s arrangement with LexisNexis for the New York Times’s content. At its height, the rumor was that LexisNexis paid more than $1 million for the exclusive that put the New York Times’s content in the LexisNexis services. The New York Times decided that it could do better by starting its own online system. Because publishers saw only part of the online puzzle, the New York Times’s decision was a fateful one which has hobbled the company to the present day. The New York Times did not understand the cost of the infrastructure and the importance of habituated users who respond to the magnetism of an aggregate service. Pull out a chunk of content, even the New York Times’s content, and what you get is a very expensive service with insufficient traffic to pay the overall cost of the online operation. Publishers making this same mistake include Dow Jones, the Financial Times, and others. The publishers will bristle at my assertion that their online businesses are doomed to be second string players, but look at where the money is today. I rest my case.

To stay in business, online players cooked up the notion of revenue sharing. There were a number of variations of this business model. The deal was rarely 50 – 50 for the simple reason that as contention and distrust grew among the vendors, the database companies, and the publishers, knowledge of costs was very difficult to get. Without an understanding of costs in online, most organizations are doomed to paddling upstream in a creek that runs red ink. The LexisNexis service may never be able to work off the debt that hangs over the company from its money sucking operations that date from the day the New York Times broke off to go on its own. Dow Jones may never be able to pay off the costs of the original Dow Jones online service which ran on the mainframe BRS search system and then the expensive joint venture with Reuters that is now a unit in Dow Jones called Factiva. Ziff Communications made online pay with its private label CompuServer service and its savvy investments in high margin database and operations that did business as Information Access. Characteristic of Ziff’s acumen, the Ziff organization exited the online database business in the early 1990s and sold off the magazine properties, leaving the Ziff group with another fortune in the midst of the tragedy of Mr. Ziff’s health problems. Other publishers weren’t so prescient.

With knowledge in short supply, here were the principal models used for revenue sharing:

Tactic A: Pool and Payout Based on Percentage of Content from Individual Publishers

This was a simple way to compensate publishers. The aggregator would collect revenues. The aggregator would scrape off an amount to cover various costs. The remainder would then be divided among the content providers based on the amount of content each provider contributed. To keep the model simple (it wasn’t) think of a gross online revenue of $110. Take off $10 for overhead (the actual figure was variable and much larger). The remainder is $100. One publisher provided 60 percent of the content in the pay period. Another publisher provided 40 percent  of the content in the pay period. One publisher got a check for $60 and the other a check for $40. The pool approach guarantees that most publishers get some money. It also makes it difficult to explain to a publisher how a particular dollar amount was calculated. Publishers who turned an MBA loose on these deals would usually feel that their “valuable” content was getting short changed. It wasn’t. The fact is that disconnected articles are worth less in a large online file than a collection of articles in a branded traditional magazine. But most publishers and authors today don’t understand this simple fact of the value of an individual item within a very large collection.

I was fascinated when smart publishers would pull out of online services and then  try to create their own stand alone online services without understanding the economic forces of online. These forces operate today and few understand them after more than  40 years of use cases.

Tactic B: Minimums

The second model is that the publisher demands a minimum. Some publishers then want to participate in the pool as well. Different tactics are possible. The idea is that the publisher would get some money and then a piece of the action. This worked for special cases, but most publishers were not able to negotiate these types of deals. The reason goes back to what happens when a large amount of information on a specific topic is aggregated. Publishers and authors think their work is of monumental importance., A tiny fraction of what’s published is important. An even smaller portion is monumental. Some monumental articles like Albert Einstein’s essays in the early 20th century were not monumental until several decades drifted past. The online savvy company would focus on getting as many sources as possible, confident that the information would appear in multiple sources. Even if the information was “secret” at one time, with multiple sources, the “secret” would be in the online file. This is the core of open source intelligence, and it works. Most people don’t understand this principle thoroughly today.

Tactic C: A Quid Pro Quo

A third approach to revenue was to shift from hard cash payouts to other benefits. These ranged from managing fulfillment of photocopies (a major hassle for a traditional publisher) and sharing the revenue derived from hard copy sales to providing access to the online file containing the publisher’s information without a charge. In the commercial online world, the publisher who understood online could use an expensive online service without paying. The money that would have be paid to use the service was, therefore, preserved. The payment to the publisher was this cash conservation. Other publishers wanted information. The database vendor would provide briefings, often coincident with a contract cycle. The idea was that the publisher was part of a grand experiment and getting information straight from those who knew how online worked. Other variations were possible, but the goal was to compensate the content provided without distributing cash.

Tactic D: Mixing and Matching

Most online services and database producers would mix and match these compensation schemes. The advent of the Internet and the ad model has marginalized these tactics, however. But if you poke around, you can find vestiges of these three approaches.

What about the Service Bureaus?

Keep in mind that in the early days of online, the systems were hard to use. The main vendors were Dialog Information Services, SDC (Systems Development Corp.), the European Space Agency, LexisNexis, and a handful of others. These outfits operating the plumbing, provided online connectivity, and handled maint3enance of the search systems. The database producers were contractors to the service bureaus.

The relationship between the online vendors and the database producers was similar to that between the database producers and the publishers. There was one important difference. The database producers knew how online worked and understood how little control over the customers and the revenue from the online usage of their files.

Distrust became the handmaiden of database producers. Publishers distrusted the database producers and the online vendors. Database producers distrusted publishers and the online vendors. Online vendors distrusted database producers and publishers. The result of this phase change meant that by the early 1990s, no one felt comfortable in what was once an exclusive, in-crowd.

Source: http://www.ww1battlefields.co.uk/flanders/images/york/yorks_viewabove.jpg

Little wonder that when a graphical browser, the Internet, and simple electronic publishing became available, the stampede online began. Service bureaus still exist, but we call them managed service providers or data center vendors. The economics are still murky, but most are on some sort of pay as you go basis. The nonlinear pricing surprises still exist, but these are part of the economics of online. The cost spikes are not more tolerable than they were 40 years ago because options exist. Options did not exist 40 years ago. That’s one plus of today’s online world.

Few people even today understand the complexity of running an online service. When people tell me that their systems people can run an online service, I get out of Dodge City. Online services are specialized, complex, and black holes of money if a trophy generation wonder makes a trivial mistake. Don’t believe me. Look at your company’s data center costs and now project that into a public online environment with tens of thousands of users every day. What about millions of users? If you can’t see the cost issues, don’t read this Web log. Stop. Get into gardening or house painting.

Wrap Up

The learnings that I keep on my notecards are easy to summarize:

1. Traditional media business models and the original online business models no longer work reliably in today’s online world. Google’s power comes from its plumbing and its business model. You can’t have one without the other no matter what a trophy generation parvenu consultant asserts.
2. Distrust is part of the landscape. Combined with ignorance of the economics of online, the mixture is volatile and potentially dangerous in terms of legal liability.
3. Misunderstanding of the consequences of trivial decisions about online lead to technical, managerial, customers, and financial problems. To the uninformed, these problems seem random, disconnected, and unpredictable. The problems are not what they seem as long as one has the appropriate knowledge. Without that knowledge, information problems are unfixable by trial and error because the person or company will run out of time and money in most cases before the solution is implemented. Look at the present efforts of America Online. Ask.com, Microsoft, the New York Times, and Yahoo. Lots of effort; minimal or negative progress in terms of cost control, customer uptake, and revenue.

I have skipped over most of my note cards. This seems to be a fertile business topic. I may provide other thoughts from my notecards. Feel free to provide factual anecdotes, verifiable examples, and data on this topic. Telling me I am stupid or an idiot is okay. I admit that I am an addled goose, but I prefer information not the ad hominem approach to my opinions.,

Stephen Arnold, February 16, 2009

Comments

Copyright © 2008 Beyond Search · ArnoldIT.com · Disclaimer · Revolution News theme by Brian Gardner · Log in