Intelware: A Tricky Business
March 25, 2019
Short honk: I read “A New Age of Warfare”. The write up names specific companies like the NSO Group and DarkMatter. People are identified as well. Most coverage of intelligence software and systems is conducted in trade publications and at specialized conferences. The NYT may be sending a not-so-subtle alert that it wants to dig into software, systems, and business practices of highly specialized products and services. My hunch is that some companies and people will be eager to assist the NYT. Others may take a different approach. Worth monitoring how the Gray Lady moves forward. Unforeseen consequences ahead? Absolutely.
Stephen E Arnold, March 25, 2019
RedMonk and Its Assessment of IBM as an Open Source Leader
March 24, 2019
I read “The RedMonk Programming Language Rankings: January 2019.” The analysis was interesting and contained one remarkable assertion and one probably understandable omission. The guts of the report boiled down, in my opinion, to a reminder to job hunters. If you want to increase your chances of getting hired, know:
1 JavaScript
2 Java
3 Python
4 PHP
5 C#
But the surprising statement in the write up was this one:
IBM remains at the forefront of open source innovation.
Now the omission. If IBM is in the forefront, where is Amazon? The company has made an effort to support most of the widely used open source software. Plus, the company appears to be taking tactical steps to close or capture open source.
From my vantage point, Amazon is taking a more “innovative” approach to open source. Granted Amazon’s “approach” may be a milestone in the company’s enhanced walled garden approach to core software systems. IBM’s approach seems little more than Big Blue’s attempt to give back and convince the open source community that it is not the IBM of its mainframe heritage.
Stephen E Arnold, March 24, 2019
Silos Persist: GAO Analysis of DHS Asserts
March 23, 2019
Government reports are often filled with useful information. Some reports can be difficult to locate. A good example is GAP-19-210 “Homeland Security: Research & Development Coordination Has Improved, but Additional Actions Need to Track and Evaluate Project.” This report is online as of March 23, 2019, at this link: https://www.gao.gov/products/GAO-19-210. In order to obtain a copy, right click on the link and download the PDF. Rendering of the document in a browser is not reliable.
I think this findability issue provides a good example of the information sharing issues discussed in the 59 page report.
If you are interested in the structure of DHS, the report contains several current organization charts.
The information about the technologies in use for border control is one of the first lists of this type which I have seen recently. You can find these data in Appendix I: Overview of the Science Technology Directorate’s Research and Development Projects on pages 48 and following.
This is a useful document because future procurements are hinted at.
A quick heads up. If you look for the document at www.gao.gov, the document does not appear on the public facing Web site yet. Experimenting with the different options for locating public information, one selector returned a list of DHS related reports with the most recent document dated 2014.
Stephen E Arnold, March 23, 2019
Smart or Not So Smart Software?
March 22, 2019
I read “A Further Update on New Zealand Terrorist Attack.” The good news is that the Facebook article did not include the word “sorry” or the phrase “we’ll do better.” The bad news is that the article includes this statement:
AI systems are based on “training data”, which means you need many thousands of examples of content in order to train a system that can detect certain types of text, imagery or video. This approach has worked very well for areas such as nudity, terrorist propaganda and also graphic violence where there is a large number of examples we can use to train our systems. However, this particular video did not trigger our automatic detection systems. To achieve that we will need to provide our systems with large volumes of data of this specific kind of content, something which is difficult as these events are thankfully rare. Another challenge is to automatically discern this content from visually similar, innocuous content – for example if thousands of videos from live-streamed video games are flagged by our systems, our reviewers could miss the important real-world videos where we could alert first responders to get help on the ground.
Violent videos have never before been posted to Facebook? Hmmm.
Smart software, smart employees, smart PR. Sort of. The fix is to process more violent videos. Sounds smart.
Stephen E Arnold, March 22, 2019
Amazon and Video Advertising
March 22, 2019
DarkCyber monitors Amazon for policeware, not advertising. But the article “Amazon to Launch Mobile Ads, in a Threat to Google and Facebook” adds a bit of color to the otherwise drab Bezos bulldozer. Google and Facebook sell ads, but each is facing pushback from governments and users. Both firms may be dulling the edge of their targeting scalpels in order to appease antagonistic factions.
What does Amazon do? If the information in the Bloomberg (we don’t need sources for some articles about fiddled hardware) write up is accurate, Amazon:
has hit on a new way to grab a chunk of the $129 billion digital advertising market now dominated by Google and Facebook Inc.: sell video spots on the e-commerce giant’s smartphone shopping app.
Yep, video ads. The ground zero for demographics with short attention spans and a desire to squint at tiny screens.
The write up asserts:
For years, Amazon refrained from selling advertising space on its site for fear of disrupting the shopping experience. Instead it used price, product descriptions and consumer reviews to determine which products were most prominent on the page. The site is increasingly a pay-to-play platform, with the top of the page dedicated to the highest bidder, a shift that has helped boost Amazon’s profits.
How will the search and social media giants respond?
If recent actions are any indication, not in an effective manner. The Bezos bulldozer chugs forward in a measured, now predictable manner.
Stephen E Arnold, March 22, 2019
Instagram: Another Facebook Property in the News
March 22, 2019
Instagram (IG or Insta) has become an important social media channel. Here’s a quick example:
My son and his wife have opened another exercise studio in Washington, DC. How was the service promoted? Instagram.
Did the Instagram promotions for the new facility work? Yes, quite well.
The article “Instagram Is the Internet’s New Home for Hate” makes an attempt to explain that Facebook’s Instagram is more than a marketing tool. Instagram is a source of misinformation.
The write up states:
Instagram is teeming with these conspiracy theories, viral misinformation, and extremist memes, all daisy-chained together via a network of accounts with incredible algorithmic reach and millions of collective followers—many of whom, like Alex, are very young. These accounts intersperse TikTok videos and nostalgia memes with anti-vaccination rhetoric, conspiracy theories about George Soros and the Clinton family, and jokes about killing women, Jews, Muslims, and liberals.
We also noted this statement:
The platform is likely where the next great battle against misinformation will be fought, and yet it has largely escaped scrutiny. Part of this is due to its reputation among older users, who generally use it to post personal photos, follow inspirational accounts, and keep in touch with friends. Many teenagers, however, use the platform differently—not only to connect with friends, but to explore their identity, and often to consume information about current events.
Is it time to spend more time on Instagram? How do intelligence-centric software systems index Instagram content? What non obvious information can be embedded in a picture or a short video? Who or what examines content posted on the service? Can images with hashtags be used to pass information about possibly improper or illegal activities?
Stephen E Arnold, March 22, 2019
A Chat Search Feature for WhatsApp
March 21, 2019
Computers and anything with a hard drive or indexable database can be searched. Chat and text messages are stored are such devices, but good luck trying to find information with a quick search. WhatsApp is a popular message service and it might be on the verge of a breakthrough in chat technology: search. Mirror shares the story, “WhatsApp’s New Feature Will Make Searching Through Chats MUCH Easier.”
WhatsApp and other message services already have basic search functions that allow users to find specific messages, some keywords, and other information. The basic search still requires users to search through their messages to find information they cannot pull up with the function. That takes a long time and often does not yield successful results. WhatsApp will debut an advanced search mode that will outpace its basic search by a long run.
What will the new Whatsapp advanced search do?
“Advanced Search will let you filter your search, whether you’re looking for photos, links, audio, documents, GIFs or videos. The feature will also show your Search History. WABetaInfo explained: “If you tap a media file, for example Photos, WhatsApp will show all messages that contain an image!” Thankfully, your history an also be easily deleted, using a ‘Clear’ button. Your search results will also include a preview, meaning there’s no need to open the search result in order to see it.”
We believe this development could be an important one.
Whitney Grace, March 21, 2019
A Statistics Rebellion? One Can Only Hope
March 21, 2019
Yesterday I mentioned to a reporter than most smart software is “right” somewhere between 50 to 80 percent of the time. The reporter asked, “Does that mean results are incorrect half to one third of the time?”
My answer, “Probably worse.”
The reporter changed the subject. My hunch is that the hyperbole about the accuracy of smart software suggests that the systems are better than a human. Some may be better at some specific tasks.
In many cases, the number crunching chops down what a human must examine. In an age of data, chopping down what one has to examine is a very important task. For applications like online advertising, 70 percent accuracy is close enough to keep the advertiser semi happy and spending money to reach a target. For other applications like where will a bad actor commit a crime, the game is “close enough for horseshoes.”
Why talk about numbers? My observations, with which you are invited to disagree, are a prelude to my recommending that you read “Scientists Rise Up Against Statistical Significance.” Here a passage I underlined:
In 2016, the American Statistical Association released a statement in The American Statistician warning against the misuse of statistical significance and P values. The issue also included many commentaries on the subject. This month, a special issue in the same journal attempts to push these reforms further. It presents more than 40 papers on ‘Statistical inference in the 21st century: a world beyond P < 0.05’. The editors introduce the collection with the caution “don’t say ‘statistically significant’”. Another article with dozens of signatories also calls on authors and journal editors to disavow those terms. We agree, and call for the entire concept of statistical significance to be abandoned.
What if one is using a system which bakes in statistical procedures and locks them away from users? What if those procedures are introducing errors?
Tough questions for vendors of smart software.
Stephen E Arnold, March 21, 2019
Smart Software: Confused or Over Hyped?
March 21, 2019
I read “Lost in Translation: Osaka Subway Removes Website after Automated Program Produces Garbled Phrases.” The main point of the article is that smart software generated nonsense phrases. Here’s one example of an ad for a new TV program:
“Osaka Metro TV uploaded footage of city is the new born. Please visit the can’t usually see pretty CM filming behind the scenes!”
I have a different view. When I was in Japan, I noted slogans on pull overs and T shirts like this one:
My thought is that the wonky translation systems was trained on T short slogans crafted in Japan.
Stephen E Arnold, March 21, 2019
Intelligence Professionals: Data Caution Required?
March 21, 2019
Perhaps we rely too much on AI and machine learning in our respective industries. This is a lesson the intelligence world of NSA, CIA, and the like are beginning to understand. While the insights computer programs provide are illuminating, they are also subject to inaccuracies, too. We learned more from a recent Sapiens story, “The Science of Human Nature Has a Serious Problem.”
According to the story:
“But a growing body of research has raised concerns that many of these discoveries suffer from severe biases of their own. Specifically, the vast majority of what we know about human psychology and behavior comes from studies conducted with a narrow slice of humanity—college students, middle-class respondents living near universities, and highly educated residents of wealthy, industrialized, and democratic nations.”
The intelligence community and other investigative groups rely on smart software. The outputs from some systems may generate signals which can be off the mark. How will vendors respond?
Marketing may have to ride to the rescue.
Patrick Roland, March 21, 2019
-
- Subscribe to Beyond Search
Feature archive
News archive
-
