Use the Sentiment Analysis Luke
December 22, 2015
The newest Star Wars film is out in theaters and any credible Star Wars geek has probably seen the film at least twice. One theme that continues to be prevalent in the franchise is the use of the mystical, galactic power the Force. The Force gives the Jedi special powers, such as the ability to read a person’s mind. Computer Weekly says that data will be able to do the same thing in: “Sentiment Analysis With Hadoop: 5 Steps To Becoming A Mind Reader.”
While the article title reads more like a kit on how to became a psychic cheat, sentiment analysis has proven to predict a person’s actions, especially their shopping habits. Sentiment analysis is a huge market for companies wanting to learn how to reach their shoppers on a more intimate level, predict trends before they happen, and connect with shoppers in real-time. Apache Hadoop is a tool used to harness the power of data to make anyone with the right knowledge a mind reader and Twitter is one of the tools used.
First-data is collect, second-label data to create a data dictionary with positive or negative annotations, third-run analytics, fourth-run through a beta phase, and fifth-get the insights. While it sounds easy, the fourth step is going to be the biggest hassle:
“Remember that analytic tools that just look for positive or negative words can be entirely misleading if they miss important context. Typos, intentional misspellings, emoticons and jargon are just few additional obstacles in the task.
Computers also don’t understand sarcasm and irony and as a general rule are yet to develop a sense of humor. Too many of these and you will lose accuracy. It is probably best to address this point by fine-tuning your model.”
The purpose of sentiment analysis is teaching software how to “think” like a human and understand all our illogical ways. (Hmm…that was a Star Trek reference, whoops!) Hadoop Apache might not have light sabers or help you find droids, but it does offer to help understand consumers spending habits. So how about, “These are the greenbacks you have been looking for.”
Whitney Grace, December 22, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Business Intelligence Free Trials. Do Not Forget Great Free Trials
December 8, 2015
Want to dive into next generation business intelligence without spending several hundred thousand dollars? If I were younger, I would think about diving. I might even think about business intelligence.
Navigate to “7 Great Business Intelligence Software With Free Trials.” You will learn about software, which the write up describes as “great”, like these:
- BIME, a data warehouse
- Cyfe, an all in one business analytics dashboard
- Decibel Insight, a Web analytics software
- SalesforceIQ, a system to analyze customer relationships
- Sisense, an “insightful Big Data analysis tool”
- Wave Analytics Cloud, visualization to help you understand your customer base
- Zoho Reports, a “simple business intelligence app”
I knew about Cyfe and Zoho, and these are useful tools. I did not know about the other products.
The use of the word “great” might be a bit of an overstatement, but when it comes to business intelligence hyperbole seems to be part of the standard marketing tool kit. Great. The write up may do some annoying. The links to the companies mentioned point to the article itself. Content marketing done with the care of a fast food cook listening to an iPod, watching other workers actually work, and dreaming about the weekend.
Stephen E Arnold, December 8, 2015
EHR Promises Yet to Be Realized
December 1, 2015
Electronic health records (EHRs) were to bring us reductions in cost and, just as importantly, seamless record-sharing between health-care providers. “Epic Fail” at Mother Jones explains why that has yet to happen. The short answer: despite government’s intentions, federation is simply not part of the Epic plan; vendor lock-in is too profitable to relinquish so easily.
Reporter Patrick Caldwell spends a lot of pixels discussing Epic Systems, the leading EHR vendor whose CEO sat on the Obama administration’s 2009 Health IT Policy Committee, where many EHR-related decisions were made. Epic, along with other EHR vendors, has received billions from the federal government to expand EHR systems. Caldwell writes:
“But instead of ushering in a new age of secure and easily accessible medical files, Epic has helped create a fragmented system that leaves doctors unable to trade information across practices or hospitals. That hurts patients who can’t be assured that their records—drug allergies, test results, X-rays—will be available to the doctors who need to see them. This is especially important for patients with lengthy and complicated health histories. But it also means we’re all missing out on the kind of system-wide savings that President Barack Obama predicted nearly seven years ago, when the federal government poured billions of dollars into digitizing the country’s medical records. ‘Within five years, all of America’s medical records are computerized,’ he announced in January 2009, when visiting Virginia’s George Mason University to unveil his stimulus plan. ‘This will cut waste, eliminate red tape, and reduce the need to repeat expensive medical tests.’ Unfortunately, in some ways, our medical records aren’t in any better shape today than they were before.”
Caldwell taps into his own medical saga to effectively illustrate how important interoperability is to patients with complicated medical histories. Epic seems to be experiencing push-back, both from the government and from the EHR industry. Though the company was widely expected to score the massive contract to modernize the Department of Defense’s health records, that contract went instead to competitor Cerner. Meanwhile, some of Epic’s competitors have formed the nonprofit CommonWell Health Alliance Partnership, tasked with setting standards for records exchange. Epic has not joined that partnership, choosing instead to facilitate interoperability between hospitals that use its own software. For a hefty fee, of course.
Perhaps this will all be straightened out down the line, and we will finally receive both our savings and our medical peace of mind. In the meantime, many patients and providers struggle with changes that appear to have only complicated the issue.
Cynthia Murrell, December 1, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Candidates as the Top Business Intelligence Providers
November 30, 2015
I read “Top 10 Business Intelligence Software Services.” I must admit that I had heard of only two of the outfits on this list. I noted that there was zero information about the methodology used, who the “experts” were generating the list, and the particular angle each of the companies takes to “business intelligence.”
Here are the 10 outfits identified in the write up:
- Cyfe
- GoodData
- HasOffers
- Kissmetrics
- Looker
- Moz
- Phocus
- Salesforce Analytics Cloud
- Similar Web
- Tableau
I would suggest there are some other business intelligence vendors you may wish to review. I profile a dozen vendors in CyberOSINT: Next Generation Information Access. None of these outfits in the article made the cut for my study. I do provide information about why certain vendors were selected. I profile another batch of vendors in my forthcoming monograph The Dark Web Dilemma. Again, none of the vendors in the article’s list of 10 “leaders” made it past my researchers’ analyses.
Who cares about Google and In-Q-Tel backed outfits or companies pushing into high value real world applications of intelligence? The answer, gentle reader, is that I do.
Stephen E Arnold, November 30, 2015
IBM Cognos 2015 Pricing
November 27, 2015
IBM offers many products and services. Getting a firm, fixed cost for some of these can be tough. Asking Watson may not result in too many useful IBM cost outputs. A company’s IBM representative may be able to deliver the goods.
Imagine my delight when I read a semi content marketing item called “IBM Cognos business intelligence offers Self-Service BI.”
Here are the data I found interesting:
Cognos BI on Cloud offers three levels of user pricing and four levels of administrator pricing. User pricing is as follows:
- A workgroup license is $75 per user, per month, with a minimum subscription of 50 users and a minimum six-month term. It is renewed semi-annually with monthly billing.
- A standard license is $95 per user, per month, with a minimum subscription of 100 users and a minimum one-year term. It is billed monthly and renewed annually.
- An enterprise license is $125 per user, per month, with a minimum subscription of 150 users and a minimum one-year term. It is billed monthly and renewed annually.
Administrator pricing is as follows:
- Analytics Administrator (authorized user [AU]): List price is $15,100 per AU; typical discount is 30% and annual support percentage is 20%.
- Analytics Explorer (authorized user and processor [PVU]): $2,500 per AU; typical discount is 30% and annual support percentage is 20%.
- Analytics User Authorized (user and processor [PVU]): $1,350 per AU; typical discount is 30% and annual support percentage 20%.
- Information Distribution (processor [PVU]): $500 per PVU; typical discount is 30% and annual support percentage is 20%.
The “menu” includes the variable pricing elements which IBM has used for decades. When we licensed ABI/INFORM document delivery to IBM, I happily implemented the same pricing scheme. Wow, does that approach yield revenue? Yep, it does.
I would point out that the write up does not beat the Watson drum. I find this amusing because Watson is marketed by the Watson as an analytics champion. See, for example, “It’s Come to This for IBM: Watson Is Now a Gimmick App on the iPhone.” But never fear, Big Blue fans, IBM said in October 2015 that it was tweaking Cognos. How? According to eWeek, “IBM Redesigns Cognos Analytics to Resemble Watson Analytics.”
IBM has a bit of a revenue and profits hill to climb. IBM has the analytics tools to track its financial progress. Tools, however, do not equal sustainable, organic revenues.
Storm clouds remain even with the Weather Channel data.
Stephen E Arnold, November 27, 2015
How Semantic Technology Will Revolutionize Education
November 27, 2015
Will advanced semantic technology return us to an age of Socratic education? In a guest post at Forbes, Declara’s Nelson González suggests that’s exactly where we’re heading; the headline declares, “The Revolution Will Be Semantic: Web3.0 and the Emergence of Collaborative Intelligence.” In today’s world, stuffing a lot of facts into each of our heads is much less important than the ability to find and share information effectively. González writes:
“Most importantly, Web3.0 is opening paths to collaborative intelligence. Isolated individual learning is increasingly irrelevant to organizational health, which is measured largely through group metrics. Today, public and private institutions live or die based on the efficiency, innovation, and impact of corporate efforts.”
The post points to content curators like Flipboard and Pinterest as examples of such collective adaptive capacity, then looks at effects this shift is already beginning to have on education. González gives a couple of examples he’s seen around the world, and discusses ways collaboration software like his company’s can facilitate new ways of learning. See the article for details. He writes:
“Web 3.0 is unleashing a kind of ‘back to the future’ innovation, the digital democratization of what élites have always practiced: deep learning through imitative apprenticeship, humanistic personalization via real-time observation, and mastery through crowdsourced validation. Silicon Valley is thus enabling us all to become the sons and daughters of Socrates.”
Launched in 2012, Declara set out to build better bridges between online sources of knowledge. The company is based in Palo Alto, California.
Cynthia Murrell, November 27, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
MarkLogic Does Ecommerce
November 25, 2015
On their blog, MarkLogic announces they are “Eliminating Shopper Fatigue: Making Online Commerce Faster, More Accurate.” Anyone who has tried to shop online for a very particular item understands the frustration. Despite all the incentives to quickly serve up exactly what a customer is looking for, ecommerce sites still struggle with searches that get too specific. Writer (and MarkLogic chief marketing officer) Michaline Todd gives this example: A site that sells 652 different versions of a “screwdriver” returns zero results to the phrase “one-quarter-inch slotted magnetic screwdriver.” You know it must be there somewhere, but you have to comb through the 652 screwdriver entries to find it. That or give up and drive to the local hardware store, where a human will hook you up with exactly what you need. Good for local business, but bad for that ecommerce site.
Todd says the problem lies in traditional relational databases, upon which any eCommerce sites are built. These databases were not meant to handle unstructured data, like supplier-created product descriptions. She describes her company’s solution to the problem, which naturally includes MarkLogic’s NoSQL technology:
“The beauty of NoSQL is that it’s a schema-agnostic data model that ingests data in whatever its current form. Codifyd uses MarkLogic to quickly and reliably merge millions of data points from thousands of suppliers into a product catalogue for each of its clients. By gathering such fine-tuned information instantaneously, Codifyd recommends products matched to specific attributes in real time, increasing customer trust, loyalty and retention. This more precise information also allows retailers to bundle relevant product offers in a set, improving upselling and increasing the average order size. For example, a retailer can serve up the ‘one-quarter-inch slotted magnetic screwdriver’ the customers searched for as well as a toolkit that contains that particular screwdriver.”
Todd notes that Codifyd also dramatically speeds up the process of posting entries for new products, since unstructured data can be reproduced as-is. Launched in 2001, MarkLogic proudly declares that theirs is the only enterprise-level NoSQL platform in existence. The company is headquartered in San Carlos, California, and maintains offices around the world.
Cynthia Murrell, November 25, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Business Intelligence Services Partnership Between Swedish Tech Companies Zinnovate and Yellowfin
November 25, 2015
The article titled Business Intelligence Vendor Yellowfin Signs Global Reseller Agreement with Zinnovate on Sys-Con Media provides an overview of the recent partnership between the two companies. Zinnovate will be able to offer Yellowfin’s Business Intelligence solutions and services, and better fulfill the needs that small and mid-size businesses have involving enterprise quality BI. The article quotes Zinnovate CEO Hakan Nilsson on the exciting capabilities of Yellowfin’s technology,
“Flexible deployment options were also important… As a completely Web-based application, Yellowfin has been designed with SaaS hosting in mind from the beginning, making it simple to deploy on-premise or as a cloud-based solution. Yellowfin’s licensing model is simple. Clients can automatically access Yellowfin’s full range of features, including its intuitive data visualization options, excellent Mobile BI support and collaborative capabilities. Yellowfin provides a robust enterprise BI platform at a very competitive price point.”
As for the perks to Yellowfin, the Managing Director Peter Baxter explained that Zinnovate was positioned to help grow the presence of the brand in Sweden and in the global transport and logistics market. In the last few years, Zinnovate has developed its service portfolio to include customers in banking and finance. Both companies share a dedication to customer-friendly, intuitive solutions.
Chelsea Kerwin, November 25, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Axel Springer Snaps Up Business Insider
November 24, 2015
I often find myself at Business Insider, reading about a recent development. That’s why I was intrigued by the article, “Sold! Axel Springer Bets Big on Digital, Buys Business Insider” at re/code. Though for me the name conjures an image of a sensationalistic talk-show host with a bandana and a wide vocal range, Axel Springer is actually a publisher based in Germany, and has been around since 1946. We note that they also own stake in the Qwant search engine, and their website touts they are now the “leading digital publisher in Europe.” This is one traditional publisher that is taking the world’s shift to the digital realm head on.
Writer Peter Kafka sees a connection between this acquisition and Axel Springer’s failed bid to buy the venerable Financial Times. He writes:
“Axel Springer is a Berlin-based publisher best known as the owner of newspapers Die Welt and Bild. In July, it missed its chance to buy the Financial Times, the august, 127-year-old business news publisher, when it was outbid at the last second by Japan’s Nikkei. Business Insider shares very little in common with the FT, other than they both deal with financial topics: While the FT has built out its own digital operations in recent years, it’s a subscription-based business whose stock-in-trade is sober, restrained reporting. Business Insider is a fast-twitch publisher, pitched at readers who’ve grown up on the Web and based on a free, ad-supported business model. While the site was famous for its you-bet-you’ll-keep-clicking headlines and slideshows, it also did plenty of serious reporting; in the last year it has been on an expansion binge, adding a British outpost, a new tech site and a new gambit that’s supposed to create viral content that lives on platforms like Facebook. Today’s transaction appears to link the FT and BI: Industry executives think Springer’s inability to land the Financial Times made them that much hungrier to get Business Insider.”
Perhaps, but this deal may be a wise choice nevertheless. Digital news and information is here to stay, and Business Insider seems to have figured out the format. We’ll see how Axel Springer leverages that know-how.
Cynthia Murrell, November 24, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
No Mole, Just Data
November 23, 2015
It all comes down to putting together the pieces, we learn from Salon’s article, “How to Explain the KGB’s Aazing Success Identifying CIA Agents in the Field?” For years, the CIA was convinced there was a Soviet mole in their midst; how else to explain the uncanny knack of the 20th Century’s KGB to identify CIA agents? Now we know it was due to the brilliance of one data-savvy KGB agent, Yuri Totrov, who analyzed U.S. government’s personnel data to separate the spies from the rest of our workers overseas. The technique was very effective, and all without the benefit of today’s analytics engines.
Totrov began by searching the KGB’s own data, and that of allies like Cuba, for patterns in known CIA agent postings. He also gleaned a lot if info from publicly available U.S. literature and from local police. Totrov was able to derive 26 “unchanging indicators” that would pinpoint a CIA agent, as well as many other markers less universal but useful. Things like CIA agents driving the same car and renting the same apartment as their immediate predecessors. Apparently, logistics agents back at Langley did not foresee that such consistency, though cost-effective, could be used against us.
Reporter Jonathan Haslam elaborates:
“Thus one productive line of inquiry quickly yielded evidence: the differences in the way agency officers undercover as diplomats were treated from genuine foreign service officers (FSOs). The pay scale at entry was much higher for a CIA officer; after three to four years abroad a genuine FSO could return home, whereas an agency employee could not; real FSOs had to be recruited between the ages of 21 and 31, whereas this did not apply to an agency officer; only real FSOs had to attend the Institute of Foreign Service for three months before entering the service; naturalized Americans could not become FSOs for at least nine years but they could become agency employees; when agency officers returned home, they did not normally appear in State Department listings; should they appear they were classified as research and planning, research and intelligence, consular or chancery for security affairs; unlike FSOs, agency officers could change their place of work for no apparent reason; their published biographies contained obvious gaps; agency officers could be relocated within the country to which they were posted, FSOs were not; agency officers usually had more than one working foreign language; their cover was usually as a ‘political’ or ‘consular’ official (often vice-consul); internal embassy reorganizations usually left agency personnel untouched, whether their rank, their office space or their telephones; their offices were located in restricted zones within the embassy; they would appear on the streets during the working day using public telephone boxes; they would arrange meetings for the evening, out of town, usually around 7.30 p.m. or 8.00 p.m.; and whereas FSOs had to observe strict rules about attending dinner, agency officers could come and go as they pleased.”
In the era of Big Data, it seems like common sense to expect such deviations to be noticed and correlated, but it was not always so obvious. Nevertheless, Totrov’s methods did cause embarrassment for the agency when they were revealed. Surely, the CIA has changed their logistic ways dramatically since then to avoid such discernable patterns. Right?
Cynthia Murrell, November 23, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
-
- Subscribe to Beyond Search
Feature archive
News archive
-
