Now Business Intelligence Is Dead
July 18, 2012
I received a “news item” from Information Enterprise Software, an HTML email distributed by InformationWeek Software. The story was labeled “Commentary.” I did not think that “real” journalists engaged in “commentary.” Isn’t there “real” news out there to “cover” or “make.”
Read the article. Navigate to “If BI Is Dead, What’s Next?” The “commentary” is hooked to an azure chip consultant report called “BI Is Dead! Long Live BI” which costs a modest $250. You can buy this document from Constellation Research here. First, let’s look at the summary of the report and then consider the commentary. I want to wrap up with some blunt talk about analytic baloney which is winging through the air.
Here’s the abstract so get your credit card ready:
We [Constellation Research] suggest a dozen best practices needed to move Business Intelligence (BI) software products into the next decade. While five “elephants” occupy the lion’s share of the market, the real innovation in BI appears to be coming from smaller companies. What is missing from BI today is the ability for business analysts to create their own models in an expressive way. Spreadsheet tools exposed this deficiency in BI a long time ago, but their inherent weakness in data quality, governance and collaboration make them a poor candidate to fill this need. BI is well-positioned to add these features, but must first shed its reliance on fixed-schema data warehouses and read-only reporting modes. Instead, it must provide businesspeople with the tools to quickly and fully develop their models for decision-making.
I like the animal metaphors. I must admit I thought more in terms of baloney, but that’s just an addled goose’s reaction to “real” journalism.
The point is that business intelligence (I really dislike the BI acronym) can do a heck of a lot more. So what’s dead? Excel? Nah. Business intelligence? Nah. A clean break with the past which involved SAS, SPSS, and Cognos type systems? Nah.
Information about point and click business intelligence should be delivered in this type of vehicle. A happy quack to the marketing wizard at Oscar Mayer for the image at http://brentbrown98.hubpages.com/hub/12-of-the-Worst-Sports-Logos-Ever
So what?
Answer: Actually not a darned thing. What this report has going for it is a shocking headline. Sigh.
Now to the “commentary.” Look a pay to play report is okay. The report is a joint work of InformationWeek and the Constellation report. Yep, IDC is one of the outfits involved in the study. The “commentary” is pretty much a commercial. Is this “real” journalism? Nah, it is a reaction to a lousy market for consulting studies and an attempt to breathe controversy into a well known practice area.
Here’s the passage I noted:
We all saw the hand wringing in recent years over BI not living up to its promise, with adoption rates below 20% or even 10% of potential users at many enterprises. But that’s “probably the right level” given the limitations of legacy BI tools, says Raden. I couldn’t agree more, and I’ve previously called for better ease of use, ease of deployment, affordability, and ease of administration. What’s largely missing from the BI landscape, says Raden, is the ability for business users to create their own data models. Modeling is a common practice, used to do what-if simulation and scenario planning. Pricing models, for instance, are used to predict sales and profits if X low-margin product is eliminated in hopes of retaining customers with products A, B, and C.
So what we are learning is that business intelligence systems have to become easier to use. I find this type of dumbing down a little disturbing. Nothing can get a person into more business trouble faster than fiddling around with numbers and not understanding what the implications of a decision are. Whether it is the fancy footwork of a Peregrine or just the crazy US government data about unemployment, a failure to be numerically literature can have big consequences.
LinkedIn and Desperation Marketing: The State Farm Case
July 13, 2012
I am all for making sales. What I found interesting this week (July 8 to July 12, 2012) was a flurry of four spam emails from what I believe to be LinkedIn’s marketing operation. I poked around a little and realized that I had signed up for a Louisville (Kentucky) sales discussion group. When I say “I”, one of the goslings who manages my social presence on LinkedIn joined the group. We are researching the local market for a project, and I assume joining a LinkedIn group of local businesses was a good idea. Wrong.
Is this State Farm’s favorite marketing department lunch meat? Yummy, spam.
The sender was a person who believed that I would be interested in an “Entrepreneurial Career Opportunity” with State Farm Insurance. Now anyone who runs a query for me on Bing, Google, or Yandex will be able to conclude that I probably am a long shot for this type of work:
After reviewing your LinkedIn profile, I was impressed with the experiences you’ve had in your career and would love the chance to chat with you regarding our career opportunities!I am expecting several openings in 2012/2013 in Louisville and surrounding areas. I am looking for qualified candidates to become our next State Farm Agents. We offer a 7 month paid training program at your current salary (subject to a cap of $144K). Following your training you would earn renewable income from an existing book of business, $30,000 in signing bonuses, retirement benefits, worldwide travel incentives, office set-up assistance and more. We are not a franchise, so there is NO franchise fee to start a business with State Farm. We have an 85% success rate on all new agents and the support system that we offer to our agents is the best in the industry. This is a great business opportunity!I’m not necessarily looking for someone who is looking for a job; I’m looking for highly successful individuals…
The job is to earn six figures selling insurance in Louisville. Okay. Now Louisville is in my view wallowing in the economic hog slop. There are quite a few people out of work. I know because we are adding staff to Augmentext, so I have a pretty good sense of the level of desperation in the job market. I don’t understand why State Farm is having such a tough time finding door to door, hammer dialing, bright white teeth and big smile workers. Unemployment is about 15 percent, maybe as high as 20 percent around Harrod’s Creek. What’s up?
I did some poking around and the sender is a State Farm insurance person is based in Nashville and has a colleague named Jerry D. I wrote Ms. Swing, suggesting she do a better job of screening her spam. I also requested that she not spam me with four identical emails in a span of minutes. One works just fine, thank you. She apparently told her boss, “Jerry”, whom I had a tough time understanding on his panting and nerve-tinged voice mail. Jerry wanted me to call him so he could explain the process used by State Farm. He gave me a phone number to call too: 615 692 6149. I did not call. You feel free to call.
Traditional Publisher Spots Web Display Ad Woes
July 9, 2012
I read “Analysis: Microsoft Loss Reflects Web Display Ad World’s Woes.”
Is there a difference between knowing about a problem and fixing the problem? I think there is.
I find it fascinating when a management and growth challenged publishing company spots management and growth woes in a software company. The headline is a trifle cryptic. The “loss” refers to Microsoft’s interesting nuking of its $6.3 billion property aQuantitative. The number is interesting because it is not too far from the money Thomson Corp. paid for Reuters in 2007. More about this Thomson Reuters $17 billion deal in a sentence or three.
A rubber stamp to avoid.
The story makes the point that the notion of charging people to put ads on a Web site is not working out as planned. There is some explanation of why Web ads are not delivering; for example, Web pages are not where the action is. TV advertising is. Dave Morgan, “a veteran and entrepreneur,” is quoted as saying, “That’s where the big money and margins are.” Omitted is the requirement of an engaged audience, but that’s neither here nor there.
The other hot methods for online advertising are, according to Wine.com CEO Rich Bergsund Bergsund, “a lot of display ads, but now mainly uses Google paid-search, affiliate marketing and comparison shopping engines.”
No push back from me. I have three points to make:
First, Thomson Reuters may be fast approaching its own write off opportunity. The Thomson Reuters hook up is, in my opinion, starting to look like an even higher stakes game than the Microsoft aQuantitative marriage.
Second, traditional publishers and professional publishers are able to write about what works to generate money. Raising subscription prices and pay walls work but don’t generate the type of revenue that keeps huge information outfits out of the Sea of Red Ink. What’s fascinating is that traditional publishing and information companies can describe a problem and its fixes, but these same companies do not seem to be able to implement these solutions. I find this fascinating because the stakes are much higher than run-of-the-mill Monday morning quarterbacking.
Third, big does not translate to sustainable. The Microsoft aQuantitative is a recent case example of how spreadsheet fever makes financial services firm drool and quiver with joy. The actual management and fiscal reality demands severe and painful action.
Net net: I think there may be more cutbacks at Microsoft and in companies like Thomson Reuters. Just my viewpoint.
Stephen E Arnold, July 9, 2012
Sponsored by Polyspot
Exalead Two Years after the Sale
July 4, 2012
I was looking at my Overflight page for Exalead, the search system acquired by the French “systems” company Dassault in mid 2010. I have not run across Exalead in my work, but I have moved “beyond search.” A number of companies once dominant in enterprise search have been nudged out from under the microscope by outfits which are triggering more attention here in Harrod’s Creek. To put this comment in context, let me mention several firms: Attivio, Artirix, Digital Reasoning, Elasticsearch, Ikanow, Lucid Imagination, Polyspot, Quid, and Sphinx to pluck a few from the “hot company queue” on one of my infamous 4×6 hot pink note cards.
We maintain some public Web pages which make it easy to see at a glance how much coverage a particular vendor gets. You can find the splash page for Overflight on ArnoldIT.com and the Exalead page here.
A snippet of the Exalead information on my Overflight service at http://arnoldit.com/overflight/search/report.php?name=exalead
The news focuses quite understandably on Dassault Systèmes, the parent company. The content push is in the area of managing manufacturing information. What is interesting is that Dassault now faces some new competition in this space. Check out Inforbix for one example.
The blog posts are even more fascinating. Exalead is following other search vendors with webinars. I have expressed my personal webinar fatigue and my annoyance at the stream of “invitations” to join a forward thinking executive. I will learn new things and the time will be well spent. I don’t buy it. I can read more quickly than I can absorb serial conversation. But that’s just me.
The second thing I noticed was information about the company’s winning an award. I have seen a number of similar announcements. Recognition is important, but my question was, “How many people were in the competition?” and “How were winners selected?” I have not heard of LT Innovative or its award, but that may be my limitation.
Third, I saw a link to a blog post which further links to information which explains how to build an “SBA in 10 minutes.” An “SBA” is a search based application. The idea is that most organizations have had it with proprietary search systems. In order to get out of a fouled nest, vendors have worked up language that suggests “finding” without using the scarlet phrase “enterprise search.” My question, “Is it really possible to build a real application in a real 10 minutes?” Heck, I can’t get MailChimp to output text without weird fonts in less than 15 minutes. I even struggled to get a Microsoft PowerPoint on my iPad 3. Fortunately one of the goslings had set up a Dropbox account and that worked. Total time: 20 minutes from start to finish.
What is missing from the flow of information about Exalead is information about major account wins. We try to highlight some of the important deals which we learn about; for example, “Microsoft Snags a Big Search Project.”
Two Traditional Publishers Still Trying to Make Online Pay Off
July 2, 2012
I found it interesting that the Chicago Tribune is undergoing another online redesign. My father, who is now 91, wanted to use the online version of the Chicago Tribune. The Web site was essentially impossible for him on both his desktop computer and his iPad. Since the hard copy of the paper was no longer available in Peoria, Illinois, he gave up. The Tribune is now trying to do a combination of free and pay wall in an effort to generate revenue, buzz, and engagement. For the story, navigate to “Redesigned Chicago Tribune Web Site to Wall Off Some Content.” Please, keep in mind that the story can be blocked from public access at any time, so you will have to become creative to locate the story. Here’s the passage which caught my eye:
Registration soon will be required to access premium features such as columnists, reviews, in-depth and investigative reporting and new content from outside news sources, the newspaper announced Tuesday. All content, including premium content behind the new registration wall, will be free. Visitors who choose not to register will still have unlimited access to basic content, which includes breaking news, photos and videos.
I can get local news from services such as Topix.com. Although not perfect, my father can click an icon and see information.
To add spice to the new service, the Chicago Tribune will be adding additional content. The new approach seems to be working. “Tribune Digital Signups Exceed Wildest Dreams” asserts:
Chicago Tribune executives are ecstatic over initial reaction to their digital membership program, the first step toward instituting a partial pay wall for online content. As of early Sunday, more than 22,000 readers had registered as digital members since the redesigned website launched at 6pm Thursday. [June 28, 2012]. The news item reported:
At some point in the future, those who’ve registered will have the option to pay for premium content, including material from the Tribune, the Economist and Forbes. Details of the pay wall plan will be determined by feedback from users…
The challenge will be to generate enough money to pay for the digital effort, offset losses in traditional advertising, and make up the shortfall from the fixed and variable costs which are dragging down traditional newspapers. Will the Chicago Tribune hit its financial goals? I hope so. I also hope that the company gets enough money to invest in the interface. My 91 year old father is not getting any younger but to win him back as a subscriber, the Tribune has to deliver a service which is usable.
The Future of Google: Search, Sales, or Showmanship?
June 28, 2012
The news “thing” makes me nervous. Mr. Murdoch wants to sell his “real” journalist news properties it seems or he wants to take some other action. Bloggers are reporting on events in near real time, confusing me with reports of various events which, in most cases, have little significance in the world of the addled goose in rural Kentucky.
I wanted to raise one question, triggered by the flood of information about Google’s demonstration day at Google I/O. “What is Google?”
I am okay with whatever Google does. The company does not have a material impact on my life. For many, however, Google is the Alpha and the Omega of “the future.”
I scanned the Thomson Reuters’ “news” story “Google Goes Up Against Amazon, Apple with Nexus Tablet.” The main idea is that Google is—yawn—in the hardware business, competing with Amazon and Apple. This is new information? Hardly. I also took a look at “Project Glass Demo: Hangouts in Air.” How does skydiving improve the relevance of the queries I run on Google? Ah, future improvements when I jump from an aircraft and want to locate a doc in the box.
Let me step back.
First, Google is dependent on advertising for revenue. I read but do not fully agree with the assertions in “IgnitionOne: Growth Slows For Paid-Search Spend.” For the sake of intellectual joy, let’s assume that:
Marketers continue to allocate budgets to U.S. paid-search advertising campaigns, but growth slowed to 15.5% in the second quarter of 2012, compared with the prior two quarters. In Q4 2011, paid-search ad spend grew 22.4%; and in Q1 2012, 30.3%, according to a quarterly report that IgnitionOne will release Thursday [June 28, 2012].
I keep trying to relate Google’s hardware and technical sky diving to ad revenues. Frankly, I am coming up a few cans short of a six pack. Google can generate ad revenue by participating in large markets. A good example is China. Oh, I almost forgot that Google has created a bit of tension between itself and the Chinese government. There are quite a few Chinese mobile phone manufacturers using Android, but these phones don’t generate direct ad revenue for Google. I wonder if that gap can be closed. In the hardware game, Google has pitted itself against Amazon, Apple, and Microsoft. In general, I wonder if there might be more revenue to gain with positive relationships with these firms. Oh, I almost forgot that a scorched earth policy is being used with these three giants. Well, let’s hope that ad revenues are not softening because selling hardware can be tricky. I wonder if Barnes & Noble has cracked the code. Oh, I almost forgot that the classic middle man bookseller may be losing money on hardware.
The Huber Affair: Demining Now Underway
June 26, 2012
Google is working overtime to keep attention focused on technical issues. You can wallow in the smart software encomium in the New York Times. (See “How Many Computers to Identify a Cat? 16,000” in the June 27, 2012, environmentally unfriendly newspaper or you can give the newspaper’s maybe here, maybe gone link at http://goo.gl/Twl9I.) The Google I/O Conference fast approaches, so there are the concomitant write ups about a Google hardware and news in “Google’s I/O Conference: New Operating System, Tablet”.
But there are two personnel stories which seem to haunt the company at what is the apex of the Google techno-promo machine: Larry Page’s minor voice problem and a person few people outside of Google have heard about. Both of these are potential “information minefields.” Google does not, as far as I know, have an effective demining system in place.
I have avoided commenting directly on the health thing. You can get the story or what passes for a story in “Google CEO Larry Page and the Healthy Way to Answer, ‘What’s Wrong?’” But I do have an opinion about the wizard responsible for Local Search, Maps, Earth, Travel, Payments, Wallet, Offers, and Shopping. I read more about about one Google executive than I expected in “This Exec May Have The Hardest Job At Google, And His Colleagues Are Tired Of Seeing Him Get Trashed In The Press.”
The basic idea, as I understand it, is:
Last week, we [Business Insider] published a story headlined: “Depending On Whom You Ask, This Google Exec Is Either ‘Weak’ Or He Just Drew The Short Straw?
The publication did some digging and learned from “senior sources”:
Their view is that Huber is a top-notch Google executive who asked for the hardest challenge his boss could give him and he got it – in the form of nascent, unproven products and an executive reporting to him that ended up being vastly under-qualified for her job.
The weak link in the Google brain mesh was a person from PayPal. Yikes. A female goofed with some PayPal type projects. The story wraps up:
Sponsored Content: Facebook Hits a Snag
June 22, 2012
The SEO (search engine optimization) crowd thought it had a winner with sponsored content. Pay Facebook money. Crank out some verbiage. Watch those clicks come tumbling along. What seems to be tumbling are Facebook ad opportunities. “Problems for Monetization: Lawsuit Forces Facebook to Let You Opt Out of Sponsored Story Ads” suggests a “stumbling block.” Was Facebook assuming that its “members” would ingest advertising as news without complaint? Personally I enjoy advertising centric editorial content. I used to work at a newspaper and then a big magazine company. The advertorials were often labeled. Sure, tiny type was used, but if you looked, you would see the words “Sponsored by…” or “A message from …” or a similar statement. I flipped through a slick travel magazine in the doctor’s office and it looked to me as if most of the editorial content was sponsored. But I may be overly sensitive.
Here’s the passage I noted:
For those less familiar with Sponsored Stories, when a Facebook user interacts with a business, such as by Liking a Page or Page’s post, using an application, or checking in to a physical business, that business can pay to have the news feed story that could normally appear be shown more prominently or frequently in the web and mobile news feed, or in the ads sidebar to friends. Because they seem like organic content, and feature the faces and names of friends as an automatic trusted referral, they’re clicked more often and are more influential on viewers than traditional ads.
Even the lingo is from the Land of SEO. Example: “Organic” just like beets and carrots from the farmer down the road here in Kentucky.
An alleged advertorial. Source: http://pdfcast.org/pdf/writing-sample-advertorial
Sponsored content is a very big deal. The reason is that consumers tune out ads. Do you remember the commercials which run in motion picture theaters before the show starts. I don’t. I play with one of my electronic distraction devices. On a desktop computer’s big screen, there is enough real estate to stuff a range of ads to lure the surfer. On a mobile device, the ads are really annoying. So how does one pump up the click throughs? Easy. Sponsored content that is shaped information.
Shaped information is tough for some people to identify. To get a sense of the challenge, check out A 50 Year History of Disinformation by Peter Viemeister. As a result, the content is consumed and according to information in the article cited above, performs “much better than traditional ads.”
What’s the fix?
Some are harsh. Facebook users can elect to turn off the ads. Yikes, bummer. Others can be sidestepped such as a provision to have users under 18 “represent they have received parental consent.”
Here at Beyond Search we label sponsored content, which generates questions. People reading Beyond Search wonder why a company like Polyspot would sponsor a story about search. Well, Polyspot is in the search business and we are covering topics germane to Polyspot’s interests. No big surprise.
The Alleged Received Wisdom about Predictive Coding
June 19, 2012
Let’s start off with a recommendation. Snag a copy of the Wall Street Journal and read the hard copy front page story in the Marketplace section, “Computers Carry Water of Pretrial Legal Work.” In theory, you can read the story online if you don’t have Sections A-1, A-10 of the June 18, 2012, newspaper. Check out a variant of the story appears as “Why Hire a Lawyer? Computers Are Cheaper.”
Now let me offer a possibly shocking observation: The costs of litigation are not going down for certain legal matters. Neither bargain basement human attorneys nor Fancy Dan content processing systems make the legal bills smaller. Your mileage may vary, but for those snared in some legal traffic jams, costs are tough to control. In fact, search and content processing can impact costs, just not in the way some of the licensees of next generation systems expect. That is one of the mysteries of online that few can penetrate.
The main idea of the Wall Street Journal story is that “predictive coding” can do work that human lawyers do for a higher cost but sometimes with much less precision. That’s the hint about costs in my opinion. But the article is traditional journalistic gold. Coming from the Murdoch organization, what did I expect? i2 Group has been chugging along with relationship maps for case analyses of important matters since 1990. Big alert: i2 Ltd. was a client of mine. Let’s see that was more than a couple of weeks ago that basic discovery functions were available.
The write up quotes published analyses which indicate that when humans review documents, those humans get tired and do a lousy job. The article cites “experts” who from Thomson Reuters, a firm steeped in legal and digital expertise, who point out that predictive coding is going to be an even bigger business. Here’s the passage I underlined: “Greg McPolin, an executive at the legal outsourcing firm Pangea3 which is owned by Thomson Reuters Corp., says about one third of the company’s clients are considering using predictive coding in their matters.” This factoid is likely to spawn a swarm of azure chip consultants who will explain how big the market for predictive coding will be. Good news for the firms engaged in this content processing activity.
What goes faster? The costs of a legal matter or the costs of a legal matter that requires automation and trained attorneys? Why do companies embrace automation plus human attorneys? Risk certainly is a turbo charger?
The article also explains how predictive coding works, offers some cost estimates for various actions related to a document, and adds some cautionary points about predictive coding proving itself in court. In short, we have a touchstone document about this niche in search and content processing.
My thoughts about predictive coding are related to the broader trends in the use of systems and methods to figure out what is in a corpus and what a document is about.
First, the driver for most content processing is related to two quite human needs. First, the costs of coping with large volumes of information is high and going up fast. Second, the need to reduce risk. Most professionals find quips about orange jump suits, sharing a cell with Mr. Madoff, and the iconic “perp walk” downright depressing. When a legal matter surfaces, the need to know what’s in a collection of content like corporate email is high. The need for speed is driven by executive urgency. The cost factor clicks in when the chief financial officer has to figure out the costs of determining what’s in those documents. Predictive coding to the rescue. One firm used the phrase “rocket docket” to communicate speed. Other firms promise optimized statistical routines. The big idea is that automation is fast and cheaper than having lots of attorneys sifting through documents in printed or digital form. The Wall Street Journal is right. Automated content processing is going to be a big business. I just hit the two key drivers. Why dance around what is fueling this sector?
More Predictive Silliness: Coding, Decisioning, Baloneying
June 18, 2012
It must be the summer vacation warm and fuzzies. I received another wild analytics news release today. This one comes from 5WPR, “a top 25 PR agency.” Wow. I learned from the spam: PeekAnalytics “delivers enterprise class Twitter analytics and help marketers understand their social consumers.”
What?
Then I read:
By identifying where Twitter users exist elsewhere on the Web, PeekAnalytics offers unparalleled audience metrics from consumer data aggregated not just from Twitter, but from over sixty social sites and every major blog platform.
The notion of algorithms explaining anything is interesting. But the problem with numerical recipes is that those who use outputs may not know what’s going on under the hood. Wide spread knowledge of the specific algorithms, the thresholds built into the system, and the assumptions underlying the selection of a particular method is in short supply.
Analytics is the realm of the one percent of the population trained to understand the strengths and weaknesses of specific mathematical systems and methods. The 99 percent are destined to accept analytics system outputs without knowing how the data were selected, shaped, formed, and presented given the constraints of the inputs. Who cares? Well, obviously not some marketers of predictive analytics, automated indexing, and some trigger trading systems. Too bad for me. I do care.
When I read about analytics and understanding, I shudder. As an old goose, each body shake costs me some feathers, and I don’t have many more to lose at age 67. The reality of fancy math is that those selling its benefits do not understand its limitations.
Consider the notion of using a group of analytic methods to figure out the meaning of a document. Then consider the numerical recipes required to identify a particular document as important from thousands or millions of other documents.
When companies describe the benefits of a mathematical system, the details are lost in the dust. In fact, bringing up a detail results in a wrinkled brow. Consider the Kolmogorov-Smirnov Test. Has this non parametric test been applied to the analytics system which marketers have presented to you in the last “death by PowerPoint” session? The response from 99.5 percent of the people in the world is, “Kolmo who?” or “Isn’t Smirnov a vodka?” Bzzzz. Wrong.
Mathematical methods which generate probabilities are essential to many business sectors. When one moves fuel rods at a nuclear reactor, the decision about what rod to put where is informed by a range of mathematical methods. Special training experts, often with degrees in nuclear engineering plus post graduate work handle the fuel rod manipulation. Take it from me. Direct observation is not the optimal way to figure out fuel pool rod distribution. Get the math “wrong” and some pretty exciting events transpire. Monte Carlo anyone? John Gray? Julian Steyn? If these names mean nothing to you, you would not want to sign up for work in a nuclear facility.
Why then would a person with zero knowledge of how numerical recipes, oddball outputs from particular types of algorithms, and little or know experience with probability methods use the outputs of a system as “truth.” The outputs of analytical systems require expertise to interpret. Looking at a nifty graphic generated by Spotfire or Palantir is NOT the same as understand what decisions have been made, what limitations exist within the data display, and what are the blind spots generated by the particular method or suite of methods. (Firms which do focus on explaining and delivering systems which make it clear to users about methods, constraints, and considerations include Digital Reasoning, Ikanow, and Content Analyst. Others? You are on your own, folks.)
Today I have yet another conference call with 30 somethings who are into analytics. Analytics is the “next big thing.” Just as people assume coding up a Web site is easy, people assume that mathematical methods are now the mental equivalent of clicking a mouse to get a document. Wrong.
The likelihood of misinterpreting the outputs of modern analytic systems is higher than it was when I entered the workforce after graduate school. These reasons include:
- A rise in the “something for nothing” approach to information. A few clicks, a phone call, and chit chat with colleagues makes many people expert in quite difficult systems and methods. In the mid 1960s, there was limited access to systems which could do clever stuff with tricks from my relative Vladimir Ivanovich Arnold. Today, the majority of the people with whom I interact assume their ability to generate a graph and interpret a scatter diagram equips them as analytic mavens. Math is and will remain hard. Nothing worthwhile comes easy. That truism is not too popular with the 30 somethings who explain the advantages of analytics products they sell.
- Sizzle over content. Most of the wild and crazy decisions I have learned about come from managers who accept analytic system outputs as a page from old Torah scrolls from Yitzchok Riesman’s collection. High ranking government officials want eye candy, so modern analytic systems generate snazzy graphics. Does the government official know what the methods were and the data’s limitations? Nope. Bring this up and the comment is, “Don’t get into the weeds with me, sir.” No problem. I am an old advisor in rural Kentucky.
- Entrepreneurs, failing search system vendors, and open source repackagers are painting the bandwagon and polishing the tubas and trombones. The analytics parade is on. From automated and predictive indexing to surfacing nuggets in social media—the music is loud and getting louder. With so many firms jumping into the bandwagon or joining the parade, the reality of analytics is essentially irrelevant.
The bottom line for me is that the social boom is at or near its crest. Marketers—particularly those in content processing and search—are desperate for a hook which will generate revenues. Analytics seems to be as good as any other idea which is converted by azure chip consultants and carpetbaggers into a “real business.”
The problem is that analytics is math. Math is easy as 1-2-3; math is as complex as MIT’s advanced courses. With each advance in computing power, more fancy math becomes possible. As math advances, the number of folks who can figure out what a method yields decreases. The result is a growing “cloud of unknowing” with regard to analytics. Putting this into a visualization makes clear the challenge.
Stephen E Arnold, June 18, 2012