CyberOSINT banner

Real Journalism Forks Real Humans

October 9, 2015

AP’s Robot Journalists Are Writing Their Own Stories Now” suggests that wizards who suggest that automation creates jobs may want to outthink their ideas. Remember the good old days. The Associated Press, United Press International and other “we use humans” news gathering organizations hired people. Now some of the anecdotes about real journalists are derogatory. I never met a journalist who was inebriated at 9 30 am. Noon? Maybe?

In the write up, the Associated Press, which has a fascinating approach to its ownership, rolled out Automated Insights. The idea was that software filtered and assembled real news stories.

Well, how is that working out?

IBM’s CEO believes that automation will not decimate the work force. Gannett is making an effort to buy up more newspapers so these too can be tooled to the tolerances of the Louisville Courier Journal. Fine newspaper. Fine operation.

And the AP itself? Well, the accumulated loss continues to go up. I recall reading “Employment Rates Are Improving For Everyone But Journalism Majors.”

I noted this passage in a NASDAQ write up:

The prospect of technology-driven job destruction is a matter of great debate for many scientists, technologists, and economists, some of whom predict massive losses in the labor market. In the past, new technology has destroyed jobs and created new ones, but some experts wonder if the increasing power of information technology will leave relatively less and less for people to do.

Journalism majors, unemployed “real” journalists, and contract journalists once called stringers—life is only going to get better. Lyft will make it easier for some folks to become taxi drivers. There are plenty of jobs as data scientists, a profession eager for those who can write prose. There are also opportunities to become experts in search and content processing. Hey, words are words.

Stephen E Arnold, October 9, 2015

Predictions about Technology: Digital Retreading

October 7, 2015

I like it when a person tells me that software or a human can predict the future. My question is, “If the predictions are spot on, why is the owner of the prediction system talking? Why not play fantasy football, pick stocks, or hang out at Keeneland during an auction and buy horses whose value will skyrocket?

The answer is, “Err, well, hmmm.”

Exactly. Predicting the future is a bit like imagining oneself putting on soccer boots and filling in for the injured Lionel Messi. Easy to thing. Essentially impossible to do.

The fix is to be fuzzy. Instead of getting into a win-lose situation, there are caveats. I find these predictions and their predictors amusing. Not as enjoyable as the antics of something like IBM cognitive computing marketed by Bob Dylan or the silliness of Hewlett Packard management activities. But close, darned close.

I read “Gartner: Top 10 Strategic Technology Trends For 2016.” I noted this statement from the capitalist tool:

…the evolution of digital business is clearly at the heart of what is covered.

Okay, the trends are going to identify trends which will allow an MBA or a savvy marketer to look at business and understand how “business” will evolve. Darwin to the future, not Darwin from the past I assume.

The question in my mind is, “Are these retread ideas?”

Here are three “trends” which caught my attention. To get the full intellectual payload, you will need to read the article or, better yet, seek out a Gartner wizard and get the trend thing straight from the horse’s mouth. Yep, right, mouth.

Trend 2: Ambient user experience.

I remember hearing about ambient computing years ago. The idea was that one could walk around and compute. I also ran across Deloitte’s identification of a similar trend months ago. But it was in the late 1990s or early 2000s when an MIT person talked about the concept. Obviously if one is computing whilst walking around, there is an experience involved. With mobile devices outselling tethered devices, it seems disingenuous to talk about this trend. According to Forbes, the capitalist tool:

Gartner posits that the devices and sensors will become so smart that they will be able to organize our lives without our even noticing that they are doing so.

I like posit. The word means “to dispose or set firmly, assume or affirm the existence of, and propose as an explanation.” Yep, posit something that academics and blue chip consulting firms have been saying for a while.

Trend 4: Information of Everything

Now these universal statements are rhetorical tactics which make my tail feathers stand up. “Everything” is a broad concept. A critical reader may want to ask, “Will you provide me with information about line 24 million in Watson’s 100 million lines of code?” The “everything” is going to provide this answer. Nope. Logical flaw. But here’s how the capitalist tool, a font of logical thought, presents this “information of everything” trend:

According to Gartner, by 2020, 25 billion devices will be generating data about almost every topic imaginable. This is equal parts opportunity and challenge.  There will be a plethora of data, but making sense of it will be the trick. Those companies that harness the power of this tidal wave of information will leapfrog competitors in the process.

I like the plethora. I like the leapfrog. I like the tidal wave. I have a sneaking suspicion that most folks with a computer device have experienced a moment of information confusion. “With every topic imaginable”, confusion is a familiar neighbor. Now how long has this concept of lots of information from lots of devices with communications capability been around? Forbes, the capitalist tool, published in June 2014 “A Very Short History of the Internet of Things.” If the Forbes’ writer had taken the time to look at that article, the concept poked its nose into the world in the early 1930s. Well, that is only 80 years ago. But it is a trend. Hmm. Trend.

Now my favorite.

Trend 9. Mesh App and Service Architecture

The notion that computer systems able to exchange information is a good one. I can’t recall when I learned about this concept. Wait. No, I remember. It was in 1963 when I took my first class in computer programming. The professor, a fine autistic polymath, explained that the mainframe—a 1710—was a collection of components. He said in 1962 that different machines would talk to one another in the future. Well, there you go. A third rate university with dullards like me in class got a prognostication which seems to be true. That was more than half a century ago. Here’s the modern version of this old chestnut:

More apps are being built to be plugged together, and the value of the combination is much greater than the sum of the parts.  As Lyft has integrated with comparable offerings in other countries, its ability to expand its offering for traditional customers traveling abroad and the reverse has meant faster growth with minimal cost implications.

Enough of these walks down memory lane. Three observations:

  1. These trends are recycled concepts
  2. The presentation of the trends is a marketing play, nothing more, nothing less
  3. Mid tier consulting firms are trying really hard to sound very authoritative, important, and substantial.

That would work if footnotes provides pointers to those who offered the ideas before. Whether a blue chip consulting firm like Deloitte or a half wild computer science professor in the Midwest, the trends are not trends.

We are, gentle reader, looking at digital retreaded tires. A recap. A remold. Old stuff made fresh. Just don’t drive too quickly into the future on these babies. Want to bet on this?

Stephen E Arnold, October 7, 2015

Watson Weekly: Chasing Sales via Ads. Forget Thought Leadership

October 6, 2015

I was exploring the topics business intelligence and Big Data. I was intrigued by “Is Thought Leadership a Waste of Money?” My reaction was, “Nope, thought leadership is good.” Who wants to fool around with regular marketing methods.

What’s the write up say?

I highlighted this passage from a person who does not know about the genesis of Strategy & Business and the somewhat addled Booz, Allen executive who wanted a BAH branded Economist to generate revenue:

Once upon a time back in 1994, Joel Kurtzman, the then-editor-in-chief of Strategy & Business, coined the term “thought leader” as a means for identifying people within the business marketplace that merited our attention. Thought leaders were the individuals within their respective industries who offered fresh, creative ideas and commentary on industry problems and trends. Two decades later, much of today’s thought leadership has gone from original to repetitive. It’s not that business leaders, C-level executives, or entrepreneurs don’t have great ideas or valuable insights. The problem is a bit more complex.

But here’s the shocker. Strategy & Business was a reaction by Booz, Allen & Hamilton to publications and marketing campaigns mounted by other blue chip consulting firms.

Advertising, at least for blue chip firms, was somewhat low brow. The notion of pumping drivel into the in boxes of Fortune 1000 executives was also distasteful. Today advertising is the cat’s pajamas.

IBM is proving that nothing beats banging one’s own drum even if no one knows what the band is playing.

I opened my dead tree edition of the New York Times this morning )October 6, 2015), and what did I see? The work of Ogilvy & Mather? Sure looks like it. Big ad buy. Big images. Big assertions.

Cognitive computing via Watson. Yikes, where is the smarter planet? I did some poking around and came across “Tangled Up in Big Blue: IBM Replaces Smarter Planet With … Bob Dylan.”

IBM began to realize that the message of Smarter Planet — basically that computing is and will be integral to everything, as manifested in innovations such as smart power grids and connected cars — is no longer a differentiator for the business, explained Mr. Iwata. The emerging pattern, as harnessed and fostered by its Watson technology, is that these super computing capabilities can be built into anything digital because they live in the cloud.

IBM’s senior vice president of marketing Jon Iwata allegedly said:

“This will resonate strongly with not only our current clients but…companies and decision makers and software developers who aren’t currently IBM clients.”

The result in the dead tree newspapers I saw presented page upon page of IBM Watson marketing. Here are some of the pages from this morning’s print campaign in the New York Times and the Wall Street Journal:

ibm ad

The massive ad campaign reveals that Watson consists of 100 million lines of code. No comment about bugs counts, however.

Obviously, this snapshot is too small to read. Put down your smartphone and buy the dead tree newspapers. Here are the themes I noted:

  1. Buzzwords
  2. Components that you, gentle reader, can assemble like Potassium ferrocyanide in chem lab when the teacher is inattentive
  3. Images of youthful, diverse people who are obviously into Watson
  4. Copy, lots of copy.

The information recycles that which is available on the IBM Watson Web site. The difference is that the multi page ads are the equivalent of a Bunker Buster dropped into the somewhat indifferent world of search and content processing. How will the likes of minnows like Coveo, dtSearch, Elasticsearch (now Elastic), Recommind, Sinequa, legions of business analytics firms, the specialists pitching everything from indexing (Smartlogic) to semantics (SenseBot), and all manner of information access vendors scattered across a somewhat Martian like landscape. Sure, there may be water, but can one survive on the stuff?

IBM is skipping the thought leader thing and going right to big buck advertising. I can imagine this scenario taking place in Joe Coffee’s. The IBM marketing team is meeting with the ad agency’s equivalent of Bindy Irwin. The scene is a hip coffee shop near the Watson office in Manhattan.

IBM Watson Wizard (IWW): We need something big to get this Watson bandwagon rolling?

Mad Ave Ad Exec (MAAE): Yes, big. We need to do big.

IWW: Let’s brainstorm here? Do you want another cappuccino with the neat latte art?

MAAE: Sure, sure. But make mine a macchiato.

[IBM Watson executive returns with more cappuccino and one artisan cafe macchiato.]

IWW: Who wants the macchiato? What have you got for me?

MAAE: Okay, we have been talking while you were standing on line? By the way, do you want one of us to pay for the coffee?

IWW: Nah, we’ve got more than a billion to burn. Let’s get to it.

MAAE: Here’s the idea. Imagine putting the Watson cognitive computing message in front of every, and I mean every, New York Times and Wall Street Journal reader. We warm up with some Monday Night Football buys and then, bang, we hit the buyers with the message, “Cognitive computing.”

IWW: Well, print? What about viral videos? What about social media?

MAAE: We will do that. We can pay some mid tier consulting types to send out Watson tweets?

IWW: But that did not get any traction?

MAAE: Tweets are good. We need to provide a big bang to make the tweet thing happen.

IWW: What’s the message?

MAAE: We were thinking think. But 21st century style. We want to go with outthink thing.

IWW: Out think. I like it.

MAAE: Now picture this. You know how everyone learned about chemical symbols in high school?

IWW: Yes, but I got a D.

MAAE: No problem. Here’s the picture. [Ad person grabs napkin and sketches a hexagon with a happy face.

ibm happy face

We show the components of the Watson system as little chemical symbols with codes in them.

IWW: Symbols? Codes? It looks like a happy face with an F in it.


MAAE: Grab your mental iPhone. Snap this happy icon with the Fd. You see “face detection.” Fd. Crystal clear. Non verbal. Immediate.

IWW: I don’t understand.

MAAE: Work with me on this. We make a list of the APIs and the buzzwords and put them into a graphic. We call the page “IBM Watson is the platform for cognitive business.”

IWW: Oh, like the structures computational chemists use to visualize complex constructs?

MAAE: What’s a computational structure whatever? I know a happy face thing with a hexagon. This gets the message across. Zap. Like an Instagram, right?

IWW: I get it. I get it.

MAAE: You like it, right? Big bang. Big splash. Big message but simple, clear, easy to grasp.

IWW: How many New York Times and Wall Street Journal readers know what API means?

MAAE: We’ve grab the upside. Wait for it. We will hook the Watson cognitive thing with a superstar. We are thinking Bob Dylan.

IWW: Bob Dylan. I remember him. Butwasn’t there some talk about drugs, political activism, maybe something with Croatia in France?

MAAE: Ancient history and myth. He’s an icon. Picture this. Bob Dylan becomes the image of cognitive computing. Can’t miss. Cannot miss. Winner. We become the messaging for API. Watson APIs will be huge. The chatter about text extraction, image tagging, and concept expansion. Deafening.

IWW: Wow, that sounds almost as powerful as the Jeopardy game show promotion. I really liked that game show thing. Watson won too.

MAAE: Right. That’s the value of post production. Now. One final point. Jules here came up with a great idea while you were waiting on line. We take the rock solid facts about Watson. Jules thinks this was your idea, and it is a great one. Watson. Only 100 million lines of code, you know, more than in a Volkswagen-type fuel emission system. We sprinkle these facts under a headline like “A cognitive business is a business that thinks.” Stir in Dylan and you can write your own ticket in this cognitive computing thing.

IWW: But what about outthink thing? You said the new hook was outthink.

MAAE: Yes, yes, outthink is the glue. Cognitive API outthink. Huge. I will send a contract over to you later today.

IWW: Do you think we will make any sales?

MAAE: Sales? Sure, sure. Winner. Be sure to turn around that contract. We need to get rolling like a rolling stone. Winner.

What other boosters did Watson receive on October 6, 2015. Well, the IBM Big Blue Boss is on CNBC. Not as perky as Bindy, but pretty excited about granting CNBC an exclusive.

One question: What about revenues? You know three years of declining revenue.

Stephen E Arnold, October 6, 2015





Stephen E Arnold, October 6, 2015

Alphabet Spells a Digital Ling Temco Vought

October 4, 2015

I read “O Is for Official: Google to Morph into Alphabet Today.” Here’s the MBA (not technology) maneuver:

The Wall Street maneuver doesn’t alter day-to-day operations at any of the companies. Though, as we’ve reported [the we is the real journalistic outfit Recode], the financial engineering will give Alphabet more room to justify its spending to investors, and more leverage to scoop up companies. Inside the corporate campuses the move may, if implemented as designed, give the companies more room to innovate and provide employees with more space to feel entrepreneurial.

Fine with me. I would point out that the shift from DEC Alta Vista engineering to “financial engineering” is, like 2006, an important turning point for the Backrub wizards. Messrs Brin and Page have moved from National Science Foundation thinking to the world of Jimmy Ling and T Boone Pickens.

The parallel for me is Tyco (which did not work out for the art lover at the company) and my favorite roll up Ling Temco Vought. As you may recall, the okay outfit Ling Electric Co. “morphed” into Ling Temco Vought and then, at the end of the trail, LTV Corp. and finally into LTV Steel. From circuits to commodities between 1947 and 2000 when the outfit filed for bankruptcy.

I recall that someone with lots of money told me that Jimmy Ling “invented” the modern conglomerate. I don’t buy that. I think of his innovation as what I call the three Rs: Roll up, Restate, and Rename. As I was finishing graduate school, LTV grew from $35 million in turnover to about $4.0 billion. Nifty.

The notion of diversification complemented strong positions in certain sectors like aerospace.

What happened was that there was pushback and investigations. The company lost touch with some markets, reorganized, and eventually ended up bankrupt. There are a number of books and business school case studies about LTV’s exciting journey.

The point is that the Alphabet Google is embarking on a journey which may allow me to watch AG follow LTV’s footsteps down a path which, one hopes, leads to a brighter, less contentious future.

Now the big question, “Is this shift or morph as the article states a good thing?” Yep, some stakeholders, lots of lawyers, and many pundits have a digital Comstock to mine. The Alphabet Google thing may be a digital variant of Ling Temco Vought.

Stephen E Arnold, October 4, 2015

Massive Analytic: The First Precognitive Analytics Platform

October 1, 2015

Let me reflect a moment. IBM is doing cognitive computing. I am assuming that the on going PR and marketing activities are accurate representation of money making technologies.

Massive goes IBM one better. The Massive Analytic outfit claims on its Web site that it delivers “effortless data driven decisions.” The product or service is Oscar AP, which allows you to “analyze all your data with artificial precognition.”

Interesting. About five weeks ago, I read “SAP, Oracle and HP Don’t Get Big Data, Claims Massive Analytic Chairman.” In the article, I learned:

Large IT vendors such as SAP, Oracle and HP don’t understand how to properly help their customers to make the most of big data, being more concerned about locking them into their ecosystems than providing them with true analytical insight. That’s according to George Frangou, executive chairman and founder of “precognitive data analytics” platform Oscar AP, which Frangou described as “an AI that allows people to foresee the future and the outcome of their decisions” which “makes Minority Report real”.

That reminded me of Recorded Future, an outfit partially funded by the Alphabet Google thing and In-Q-Tel, the US government intelligence community’s investment fund. Recorded Future rolled out in 2008 after a year or so of gestation. Massive Analytic took its first breath in 2010. I assume the wiggle room created by the term “precognitive” allows Massive Analytic to claim the adjective “first.”

The write up about Massive Analytic contained a statement which I found interesting. I circled this in red, gentle reader:

according to Frangou, larger competitors such as SAP, Oracle and HP “don’t get it” when it comes to making the most of big data and analytics. “They don’t get it because the driver for them is to sell kit. … You’re into millions of dollars before you start,” he said, attacking the aggressive sales tactics of the big vendors, which he said are designed solely to sell the product and not to provide support. “And by the way, the actual algorithms don’t scale either, so you’re into lots of people and manual intervention,” he added. Because of this, Frangou said Massive Analytic is “quite unashamedly following a displacement strategy to displace the incumbents because they’re not getting it”.  He added that SAP HANA, Oracle Exalytics and HP Haven are essentially the same product because they’re built on the same base code.

It is true that most analytics vendors recycle what the engineers and mathematicians with MBAs learned in their university courses. I am not sure about the “algorithms don’t scale.” There are issues with algorithms, but as the work by SRCH2 shows, there is a great deal of innovation opportunity in optimization.

But the point which I find slightly jarring is the reference to SAP, Oracle, and HP “built on the same base code.”

Well, maybe. SAP uses home brew code (anyone remember TREX), acquired stuff from Business Objects (Inxight), and open source snippets. Oracle uses the wild and crazy home brew code, acquired code from “analytics” outfits like Endeca, and confections from some of the Oracle partner ecosystem. HP—an example for MBA cases studies for the next couple of decades—uses home brew, acquired technology from outfits like Autonomy, and probably scripts written by the Board of Directors and Meg Whitman in their spare time.

What the three companies share is, therefore:

  1. Code written by employees and contractors
  2. Code from open source and licensed libraries
  3. Code from companies acquired in moments of great wisdom.

The wrappers each of these companies exposes to its customers and partners make it easy to use the popular programming conventions, recycle structured query language, and exploit reasonably stable Web conventions.

I would suggest that once one looks under the hood of one of these companies’ projects, there will be a world of differences. There is a simple reason or two.

First, some familiar bits and lots of unfamiliar or downright extraterrestrial methods translate to job security and on going consulting work. Who wants to lose a night Oracle DBA job? Not anyone I know.

Second, enterprise software is about customization. I know the yap about enterprise apps, but these apps are little more than customized scripts to allow a hapless marketer with a degree in home economics to pull down a standard report.

I will leave it to you to unravel the mysteries of precognitive analytics and the assertion that HP, Oracle, and SAP are peas in a pod.

Stephen E Arnold, October 1, 2015

HP Autonomy: Why Do the Deal?

September 30, 2015

I read “If Ray Lane Hated HP’s Autonomy Move So Much, How Did It Happen?”

Darned good question. The article reviews information which suggests that HP chairman was uncomfortable with the tie up. Also, HP’s CFO gave the deal a thumbs down.

According to the article:

Reached for comment an HP spokesperson reiterated that the Autonomy buy had unanimous support from the board.

I assume this is the HP way.

Stephen E Arnold, September 30, 2015

TweetedTimes: Gone Goose?

September 29, 2015

Short honk. The TweetedTimes allowed “members” to create aggregations of content. I used to look at the semantic and Big Data pages. A few days ago, I noticed that the TweetedTimes, which I think was a Yandex property, went dark on September 23, 2015. I checked to see what was in the Twitter fire hose. Zilch. That says a lot.

Stephen E Arnold, September 29, 2015

The Nikkei Financial Times Deal: Journalism and Quality

September 29, 2015

I read a Harry Quebert type article this morning called with typical big time journalism understatement “The Financial Times and the Future of Journalism.”

Yep, the future. Of journalism.

The set up is an interview which has been converted to a chatty, informed narrative with commentrary from the person asking the questions (a New Yorker “contributor”, which I think means contract worker) and statements from a full-time equivalent at the Financial Times, a salmon colored newspaper consumed by  750,000 quality-centric readers.

The quotes in this blog post come from the CEO of the Financial Times which sold to Nikkei, a Japanese outfit for 40 times the FT’s 2014 revenues. So $37 million netted Pearson, the former owner of the FT, about $1.4 billion. Like the HP purchase of Autonomy, I will be interested to see how the purchase plays out. Obviously Pearson was neither willing nor able to put the FT on a pedestal of cash. The former owner of Madame Tussaud’s wax museum sold the newspaper. Let Nikkei realize the long term benefits of FT ownership I assume.

The write up by the New Yorker magazine, which has pretty good cartoons, is a darned interesting journalistic artifact itself. But I want to focus on some of the statements in the write up, allegedly made by the FT CEO who played a big part in the deal with the Japanese buyer.

I noted this statement:

Nikkei wanted to prepare for the transition to digital, which has been slow in its home market.

My recollection may be fuzzy, but I thought that the Japanese were exploring the digital world, databases, and all sorts of software based activities in Japan’s Fifth Generation Project in the early 1990s. Hey, that was like yesterday in traditional print publishing.

The FT executive allegedly said to the reporter:

I think if you were to summarize the vision that we both share, it would be about growth. We both think there is a very good growth opportunity for the F.T. That requires a long-term perspective. It requires investment. They have committed to that. And for a news organization like the F.T. right now, that’s music to one’s ears, frankly.”

I like the long term growth perspective. Apparently Pearson was not on board with this concept about investment without significant payoff. As a result, Pearson shopped the FT and netted a nine figure payout.

Why did Pearson opt to sell and not pump cash into the FT? Here’s the explanation from the FT executive:

What is lacking is some fuel in the tank and the ability to spread our wings a bit.”

Pearson apparently lacked “fuel”. I wonder if the “fuel” is patience, money, financial resources, or wisdom. The billion dollar deal looked pretty snappy to me. Imagine. More than one billion for a newspaper. That’s the color of money.

Apparently the FT boss perceives those from the Far East—that is, beyond Dover—as adopting Adam Carolla’s “In 50 Years We’ll All Be Chicks” approach. None of this City and Wall Street aggressiveness toward revenues and profits. Here’s a passage I highlighted in green, the color of money:

But Japanese newspapers, including the ones owned by Nikkei, are also known for taking a less aggressive stance toward news than many of their Western counterparts.

Will the FT remain independent as other newspapers and real journalistic endeavors do the inclusion, sponsored content, advertising thing? According to the FT executive:

“Editorial independence is absolutely fundamental to the way we operate,” he replied.

He allegedly added:

But I think the most important thing is they understand our values and editorial independence. I’m not going to tell them how to run Nikkei, and they are not going to tell us how to do editorial independence at the F.T. They are very clear about that.

I like that understanding. The owner will not assert control over something the person owns. Shared values among the quality journalists effervesce from this factoid.

The most important passage concerns the FT business model. Here’s the explanation of the FT’s “vindicated”, super-charged approach to generating at some time in the future oodles of dough from a global market of discerning news consumers:

The replacement was cheap trial subscriptions. If you go to the F.T.’s Web site today and try to read a story, you will be prompted to take out a month-long subscription for a dollar….“We are now able to measure, optimize, and track all of these readers and changes with real insight that we could never do before. It sounds dry, but it’s not. It’s really understanding readers, what matters to them. We are never going to edit by numbers, but we are going to inform all of our decisions around data.”

Not only that, the FT is going back to the model for the newspapers which have become pedestal mounted historical artifacts. Newspapers are back as the “trusted” folks in the information business. I know that I trusted newspapers until I read about US diplomacy and yellow journalist in the period from 1895 to 1898.

“Precisely because of digital disruption, precisely because there is so much information and news and information out there, the value of a trusted guide, the value of a trusted brand” has gone up, he said.

Yep, those families losing sons in the dust up among the US, Spain, and Cuba understand that trust stuff.

Then there is a statement which seems to bring the future payday for the new owner of the FT tantalizingly near:

But we fundamentally believed that if it’s quality journalism, people will pay for it. That’s been vindicated.”

From my point of view, what’s been vindicated is that there was a buyer willing to pony up more than $1 billion for a brand, several hundred thousand readers, and a Web site offering a  $1.00 trial subscription. I assume that is the definition of vindication from Pearson’s point of view. I am not sure about Nikkei’s point of view. If the FT’s senior management had an agreement with Pearson designed to keep the FT’s senior management on board, was some of the money shared with the FT leadership? Good question.

I also highlighted in red ink red, not money green, this statement attributed to the FT executive:

But he also said insisted that things are changing in journalism and that the business climate is improving. “There is a belief in journalism,” he said.

Stepping back I thought about the New Yorker’s analysis of the FT deal. Much of the verbiage could be used to describe how the New Yorker feels about its approach to news and information.

I asked myself, “Is this article about the FT or is it about the New Yorker’s perception of quality journalism?” Another good question.

And what about search. Does anyone recall the Endeca FT Newssift project? I do. Moving on.

Stephen E Arnold, September 29, 2015

The HP Autonomy Enterprise Search Epic Continues

September 26, 2015

I don’t play baseball anymore. I did. I was okay, but one of the fellows who lived in my neighborhood in central Illinois played very well. He played everyday. After a stellar high school career, he became a fielder in the major leagues. The pressure was too much. He made bad decisions. He tried to claw back to the starting rotation. Instead of swinging with the relaxed, fluid motion I recalled from our days of playing together, he tried to hit a home run every time at bat. His confidence dwindled away, and he became a person who did not perform. Last I heard, he had fallen victim to his inner demons and was searching for a panacea. But, in my opinion, he struck out. Bad management.

Definition of panacea:

noun 1. a remedy for all disease or ills; cure-all. 2. an answer or solution for all problems or difficulties:

I thought about this person when I read “Deal Divided H-P Leaders” in the September 26, 2015, Wall Street Journal. You may need to pay to access this article which is available at as “Hewlett Packard’s Then Chairman Ray Lane Tried to Quash Autonomy Acquisition.”

The main point of the write up is that HP wanted a panacea, and the senior management of HP thought Autonomy, a search and content processing company, was the answer to HP’s revenue challenges.

The Wall Street Journal points out that the Chairman of the Board of Directors was supportive of the multi billion dollar deal and then wanted to kill the deal.

Also, the WSJ identifies what I would call a “management” problem; to wit:

HP missed other red flags in assessing the Autonomy deal. In 2013, the Journal reported that outside auditors for Autonomy had noted that an Autonomy executive had alleged improper accounting practices at the company [Autonomy]. However, HP executives briefed on the allegations hadn’t passed them along to HP’s Board or to Mr. Apotheker [president and Autonomy deal supporter].

The Wall Street Journal article includes a point I made in my 2003 analysis of Autonomy, a version of which appeared in the first edition of the Enterprise Search Report.

Revenues from software which allows employees to locate information germane to work activities has for decades faced a major hurdle; namely, making sales and keeping customers. The problem, which persists today, is that enterprise search vendors have a tough time making basic key word search command the type of license fees and corporate commitment which enterprise resource planning, accounting, and compliance-related systems demand.

Enterprise search vendors have, again for decades, explained that search and retrieval was something more than finding a needed document. The buzzwords used for decades invoke “knowledge management,” “business intelligence,” and “customer support.” Each of these is baloney, but enterprise search vendors trapped. Making search work in the fast changing content environments in which organization operate was a tough technical problem. The costs of engineering fixes was uncontrollable, and, not surprisingly, enterprise search vendors layered on additional functions in an effort to make sales, charge more, and stay in business.

Autonomy, along with IBM and OpenText, were firms which grew search via acquisition. Autonomy was perhaps the most successful of the roll up tacticians. The firm acquired Verity, a system which dated from the 1980s and added it to Autonomy’s earlier video management acquisitions, document management acquisitions, and other bits and pieces accumulated since Autonomy opened for business in the late 1990s.

Each acquisition added revenue to Autonomy’s financial reports and the customers of these acquisitions became candidates for other Autonomy products. At the time of the HP purchase decision, Autonomy had about six or seven times the revenue of Endeca, another late 1990s search vendor. (Oracle bought Endeca for $1.1 billion in 2011. Other search vendors sold in the 2008 t0 2014 period traded from much lower purchase prices; for example, IBM bought Vivisimo for $20 million, a figure which was equivalent to one year Vivisimo revenues.)

HP did not, in my opinion, understand that search and retrieval was a business that broke the backs of many bright MBAs and whiz kid engineers. HP assumed that its management team would triumph in generating billions from Autonomy’s core technology. I think some of Autonomy’s innovations are important, but I know that Autonomy was able to generate six or seven times the revenue of the number two search vendor in 2011 because it managed a portfolio of content processing companies and did a pretty good job of generating revenue from lines of business ADJACENT to search and retrieval.

HP wanted the 1990s technology of Autonomy to generate billions. HP quickly learned that its view of Autonomy did not match what Autonomy’s management team built.

I am not sure how bright folks at HP could not look at the failures of Convera, Delphes, Entopia, Siderean, and other search vendors and not ask, “What’s different about search?”

HP wanted a panacea. HP demonstrates the type of problem my friend who became a major league player had and still has. In the big leagues, swinging for the fences, seeking a silver bullet, and looking for a quick fix is easy. Finding a fix for a company with problematic business models and conflicting management views is very difficult.

What does the HP experience suggest? After decades of enterprise search hyperbole, reality is different from the word picture sales professionals create in the minds of those whose desperation clouds their thinking.

My view is that HP has struck out. Bad management in my opinion.

Stephen E Arnold, September 26, 2016

Spark: Another Open Source Game Changer

September 24, 2015

Gentle reader, I know that knowledge about Spark is as widespread as information about the woes of the Philadelphia Eagles. My understanding of Spark is that is is an open source engine for large scale data processing. It is faster than Hadoop. It is easy to use. It is flexible enough to allow the intrepid Spark aficionado the combine structured query language, streaming, and analytics in one software system. Spark runs “everywhere.” For more about Spark, see this Apache project page.

Spark is one of the next big things, poised to ignite innovation, consulting revenues, innovations, and vendor repositionings.

I approached “Game-Changing Real-time Uses for Apache Spark” in order to learn how Spark can change the game for real time data and information work. Game changing means that old school outfits are going to lose because the new game has new rules, new players, and new everything.

The write up identified these ways Spark will change some quite significant markets:

  • Credit card fraud detection
  • Network security
  • Genomic sequencing
  • Real time ad processing
  • Medical

My goodness, Spark will become the number one enabling technology for some very problematic market spaces.

Let’s look at what Spark will do to real time ad processing. The write up reports:

One advertising firm uses Spark, on MapR-DB, to build a real-time ad targeting platform. The system looks at user data and decides which ads to show users on the Internet based on demographic data. Since advertising is so time-sensitive, advertisers have to move fast if they want to capture mindshare. Spark Streaming is one way to help them do that.

What strikes me is that Spark requires programmers, software engineering, and then integration of different components. If an error manifests itself, the Spark solution may require those who embrace it to perform some old fashioned work.

In a sense, the game hasn’t changed at all. Open source software reduces license fees and provides a developer with some freedom from license restrictions. On the other hand, the difficult task of getting a complex system to work as intended remains.

My hunch is that Spark is an interesting open source project. The consultants and start ups see Spark as an opportunity. The game changing nature of Spark is potential energy, not a sure thing.

Stephen E Arnold, September 23, 2015

Next Page »