Artificial Intelligence Spreading to More Industries
May 10, 2016
According to MIT Technology Review, it has finally happened. No longer is artificial intelligence the purview of data wonks alone— “AI Hits the Mainstream,” they declare. Targeted AI software is now being created for fields from insurance to manufacturing to health care. Reporter Nanette Byrnes is curious to see how commercialization will affect artificial intelligence, as well as how this technology will change different industries.
What about the current state of the AI field? Byrnes writes:
“Today the industry selling AI software and services remains a small one. Dave Schubmehl, research director at IDC, calculates that sales for all companies selling cognitive software platforms —excluding companies like Google and Facebook, which do research for their own use—added up to $1 billion last year. He predicts that by 2020 that number will exceed $10 billion. Other than a few large players like IBM and Palantir Technologies, AI remains a market of startups: 2,600 companies, by Bloomberg’s count. That’s because despite rapid progress in the technologies collectively known as artificial intelligence—pattern recognition, natural language processing, image recognition, and hypothesis generation, among others—there still remains a long way to go.”
The article examines ways some companies are already using artificial intelligence. For example, insurance and financial firm USAA is investigating its use to prevent identity theft, while GE is now using it to detect damage to its airplanes’ engine blades. Byrnes also points to MyFitnessPal, Under Armor’s extremely successful diet and exercise tracking app. Through a deal with IBM, Under Armor is blending data from that site with outside research to help better target potential consumers.
The article wraps up by reassuring us that, despite science fiction assertions to the contrary, machine learning will always require human guidance. If you doubt, consider recent events—Google’s self-driving car’s errant lane change and Microsoft’s racist chatbot. It is clear the kids still need us, at least for now.
Cynthia Murrell, April 10, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Update from Lucene
May 10, 2016
It has been awhile since we heard about our old friend Apache Lucene, but the open source search engine has something new, says Open Source Connections in the article, “BM25 The Next Generation Of Lucene Relevance.” Lucene is added BM25 to its search software and it just might improve search results.
“BM25 improves upon TF*IDF. BM25 stands for “Best Match 25”. Released in 1994, it’s the 25th iteration of tweaking the relevance computation. BM25 has its roots in probabilistic information retrieval. Probabilistic information retrieval is a fascinating field unto itself. Basically, it casts relevance as a probability problem. A relevance score, according to probabilistic information retrieval, ought to reflect the probability a user will consider the result relevant.”
Apache Lucene formerly relied on TF*IDF, a way to rank how users value a text match relevance. It relied on two factors: term frequency-how often a term appeared in a document and inverse document frequency aka idf-how many documents the term appears and determines how “special” it is. BM25 improves on the old TF*IDF, because it gives negative scores for terms that have high document frequency. IDF in BM25 solves this problem by adding a 1 value, therefore making it impossible to deliver a negative value.
BM25 will have a big impact on Solr and Elasticsearch, not only improving search results and accuracy with term frequency saturation.
Whitney Grace, May 10, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Semantics Made Easier
May 9, 2016
For fans of semantic technology, Ontotext has a late spring delight for you. The semantic platform vendor Ontotext has released GraphDB 7. I read “Ontotext Releases New Version of Semantic Graph Database.” According to the announcement, set up and data access are easier. I learned:
The new release offers new tools to access and explore data, eliminating the need to know everything about the dataset before start working with it. GraphDB 7 enables users to navigate their way through third-party and any other dataset regardless of data volumes, which makes it a powerful Big Data analytics tool. Ver.7 offers visual exploration of the loaded data schema – ontology, interactive query builder for better entity retrieval, and full support for RDF 1.1 allowing smooth import of a huge number of public Open Data as well as proprietary Linked Datasets.
If you want to have a Palantir-type system, check out Ontotext. The company is confident that semantic technology will yield benefits, a claim made by other semantic technology vendors. But the complexity challenges associated with conversion and normalization of content is likely to be a pebble in the semantic sneaker.
Stephen E Arnold, May 9, 2016
New Criminal Landscape Calls for New Approaches
May 9, 2016
The Oxford University Press’s blog discusses law enforcement’s interest in the shady side of the Internet in its post, “Infiltrating the Dark Web.” Writer Andrew Staniforth observes that the growth of crime on the Dark Web calls for new tactics. He writes:
“Criminals conducting online abuses, thefts, frauds, and terrorism have already shown their capacity to defeat Information Communication Technology (ICT) security measures, as well as displaying an indifference to national or international laws designed to stop them. The uncomfortable truth is that as long as online criminal activities remain profitable, the miscreants will continue, and as long as technology advances, the plotters and conspirators who frequent the Dark Web will continue to evolve at a pace beyond the reach of traditional law enforcement methods.
“There is, however, some glimmer of light amongst the dark projection of cybercrime as a new generation of cyber-cops are fighting back. Nowhere is this more apparent than the newly created Joint Cybercrime Action Taskforce (J-CAT) within Europol, who now provide a dynamic response to strengthen the fight against cybercrime within the European Union and beyond Member States borders. J-CAT seeks to stimulate and facilitate the joint identification, prioritisation, and initiation of cross-border investigations against key cybercrime threats and targets – fulfilling its mission to pro-actively drive intelligence-led actions against those online users with criminal intentions.”
The article holds up J-CAT as a model for fighting cybercrime. It also emphasizes the importance of allocating resources for gathering intelligence, and notes that agencies are increasingly focused on solutions that can operate in mobile and cloud environments. Increased collaboration, however, may make the biggest difference in the fight against criminals operating on the Dark Web.
Cynthia Murrell, April 9, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Wikipedia Relies on Crowdsourcing Once More
May 9, 2016
As a non-profit organization, the Wikimedia Foundation relies on charitable donations to fund many of its projects, including Wikipedia. It is why every few months, when you are browsing the Wiki pages you will see a donation bar pop to send them money. Wikimedia uses the funds to keep the online encyclopedia running, but also to start new projects. Engadget reports that Wikipedia is interested in taking natural language processing and applying it to the Wikipedia search engine, “Wikipedia Is Developing A Crowdsourced Speech Engine.”
Working with Sweden’s KTH Royal Institute of Technology, Wikimedia researchers are building a speech engine to enable people with reading or visual impairments to access the plethora of information housed in the encyclopedia. In order to fund the speech engine, the researchers turned to crowdsourcing. It is estimated that twenty-five percent, 125 million monthly users, will benefit from the speech engine.
” ‘Initially, our focus will be on the Swedish language, where we will make use of our own language resources,’ KTH speech technology professor Joakim Gustafson, said in a statement. ‘Then we will do a basic English voice, which we expect to be quite good, given the large amount of open source linguistic resources. And finally, we will do a rudimentary Arabic voice that will be more a proof of concept.’”
Wikimedia wants to have a speech engine in Arabic, English, and Swedish by the end of 2016, then they will focus on the other 280 languages they support with their projects. Usually, you have to pay to have an accurate and decent natural language processing machine, but if Wikimedia develops a decent speech engine it might not be much longer before speech commands are more commonplace.
Whitney Grace, May 9, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Semantic Search: Clustering and Heat Maps Explain Creativity
May 8, 2016
I know zero about semantics as practiced at big time universities. I know about the same when it comes to semantic search. With my background as a tabula rasa, I read “A Semantic Map for Evaluating Creativity.” According to the write up:
We present a semantic map of words related with creativity. The aim is to empirically derive terms which can be used to rate processes or products of computational creativity. The words in the map are based on association studies per for med by human subjects and augmented with words derived from the literature (based on human raters).
After considerable text processing and a dose of analytics, the paper states:
… There is an overlap in the set of words formed by the two methods, but there are also some differences. Further investigations could reveal how these methods are related and if they are both needed (as complements) to arrive at more objective procedures for the evaluation of computational (and human) creativity.
I await a mid tier consulting firm’s for fee study about the applications of this technology in determining which companies are creative. And what about government use cases; for example, which entry lever professional is most creative. Then there are academic applications; for instance, which professors are their most creative. Creative folks can create creative ways to understand creativity. Stay tuned.
Stephen E Arnold, May 8, 2016
European Cybersecurity Companies
May 8, 2016
We’ve run across an interesting list of companies at Let’s Talk Payments, “Europe’s Elite Cybersecurity Club.” The bare-bones roster names and links to 28 cybersecurity companies, with a brief description of each. See the original for the descriptions, but here are their entries:
SpamTitan, Gemalto, Avira, itWatch, BT, Sophos, DFLabs, ImmuniWeb, Silent Circle, Deep-Secure, SentryBay , AVG Technologies, Clearswift, ESNC, DriveLock, BitDefender, neXus, Thales, Cryptovision, Secunia, Osirium, Qosmos, Digital Shadows, F-Secure, Smoothwall, Brainloop, TrulyProtect, and Enorasys Security Analytics
It is a fine list as far as it goes, but we notice it is not exactly complete. For example, where is FinFisher’s parent company, Gamma International? Still, the list is a concise and valuable source for anyone interested in learning more about these companies.
Cynthia Murrell, May 8, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Microsoft Says That AI Is Stupid
May 7, 2016
I know there is a difference among:
- What senior managers believe about their minions’ innovations
- What marketers say about the technology the engineer wizards are crafting in the innovation microwave
- What “real” journalists angling for a job with some tailwind write
- What the reality of an innovation is, right now.
But these differences are essentially irrelevant. We are in the era of IBM Watson, Facebook-Google investing in smart software, and big universities doing cartwheels for research which raised nary en eyebrow 18 months ago.
Navigate to “Microsoft Research Chief: AI Is Still Too Stupid to Wipe Us Out (and Will Be for Decades).” I am okay with the notion that smart software is becoming more important. From my vantage point in rural Kentucky, I am aware of the marketing money available to those who would shill smart software. I know about the cash lust of venture outfits who are in search of the next big thing. I am aware that smart software works reasonably well when applied to advertising and Amazon-style recommendations.
I find the use of the word “stupid” interesting. I noted this passage which quotes a Microsoft guru in the artificial intelligence stuff:
“Yes, deep learning has achieved human-level performance in object recognition but what does that mean? It means the machine makes about the same number of errors as the human. “The reason the machine is as good as the human at this is because it can distinguish between 157 varieties of mushroom, whereas it makes all kinds of stupid mistakes that humans wouldn’t make.”
Why comment? Microsoft Tay made evident some flaws. Perhaps IBM Watson avoids public demonstrations like Tay to avoid making weaknesses vivid? Facebook and Google are angling to reduce costs and generate revenue. AI is one path to explore. But “stupid”? Interesting word.
Stephen E Arnold, May 7, 2016
How Hackers Hire
May 7, 2016
Ever wonder how hackers fill job openings, search-related or otherwise? A discussion at the forum tehPARADOX.COM considers, “How Hackers Recruit New Talent.” Poster MorningLightMountain cites a recent study by cybersecurity firm Digital Shadows, which reportedly examined around 100 million websites, both on the surface web and on the dark web, for recruiting practices. We learn:
“The researchers found that the process hackers use to recruit new hires mirrors the one most job-seekers are used to. (The interview, for example, isn’t gone—it just might involve some anonymizing technology.) Just like in any other industry, hackers looking for fresh talent start by exploring their network, says Rick Holland, the vice president of strategy at Digital Shadows. ‘Reputation is really, really key,’ Holland says, so a candidate who comes highly recommended from a trusted peer is off to a great start. When hiring criminals, reputation isn’t just about who gets the job done best: There’s an omnipresent danger that the particularly eager candidate on the other end of the line is actually an undercover FBI agent. A few well-placed references can help allay those fears.”
Recruiters, we’re told, frequently advertise on hacker forums. These groups reach many potential recruits and are often password-protected. However, it is pretty easy to trace anyone who logs into one without bothering to anonymize their traffic. Another option is to advertise on the dark web— researchers say they even found a “sort of Monster.com for cybercrime” there.
The post goes on to discuss job requirements, interviews, and probationary periods. We’re reminded that, no matter how many advanced cybersecurity tools get pushed to market, most attack are pretty basic; they involve approaches like denial-of-service and SQL injection. So, MorningLightMountain advises, any job-seeking hackers should be good to go if they just keep up those skills.
Cynthia Murrell, May 7, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Lexmark to Take Chinese Lessons
May 6, 2016
Dateline Harrod’s Creek, Kentucky. A buzz echoed in the hollow where I live. The message brought good news to Chinese language instructors. When Reuters distributed “Consortium Led by China’s Apex to Buy Lexmark for $3.6 Billion,” a demand for Chinese lessons rippled through the weeds growing around the pond filled with mine drainage.
The write up stated:
Printer maker Lexmark International Inc has agreed to be taken private by a group of investors led by China-based Apex Technology Co Ltd and PAG Asia Capital in a deal valued at $3.6 billion net of cash, the company said….Lexmark said the deal will help its efforts to penetrate the Asia Pacific market.
What if Apex makes Lexmark a money machine? Think about IBM. Lenovo snapped up the IBM PC business. Now the former IBM printer outfit will go across the international date line. What’s next for IBM in the sell off space? Perhaps an outfit in China will ignore the chunks which IBM has divested and go after the whole enchilada.
Big news from the Bluegrass state, but it will not overshadow the Kentucky Derby.
Stephen E Arnold, May 6, 2016
-
- Subscribe to Beyond Search
Feature archive
News archive
-
