Machine Learning Changes the Way We Learn from Data

October 26, 2016

The technology blog post from Danial Miessler titled Machine Learning is the New Statistics strives to convey a sense of how crucial Machine Learning has become in terms of how we gather information about the world around us. Rather than dismissing Machine Learning as a buzzword, the author heralds Machine Learning as an advancement in our ability to engage with the world around us. The article states,

So Machine Learning is not merely a new trick, a trend, or even a milestone. It’s not like the next gadget, instant messaging, or smartphones, or even the move to mobile. It’s nothing less than a foundational upgrade to our ability to learn about the world, which applies to nearly everything else we care about. Statistics greatly magnified our ability to do that, and Machine Learning will take us even further.

The article breaks down the steps of our ability to analyze our own reality, moving from randomly explaining events, to explanations based on the past, to explanations based on comparisons with numerous trends and metadata. The article positions Machine Learning as the next step, involving an explanation that compares events but simultaneously progresses the comparison by coming up with new models. The difference is of course that Machine Learning offers the ability of continuous model improvement. If you are interested, the blog also offers a Machine Learning Primer.

Chelsea Kerwin, October 26, 2016
Sponsored by, publisher of the CyberOSINT monograph

Knowledge Management: Dazed and Confused?

October 16, 2016

I read an interview posted by TallyFox. If you are not familiar with the company, TallyFox provides a collaboration and content management system. The idea is that a company’s real and off site workers can share information. The company states on its LinkedIn page:

TallyFox’s intelligence platform, makes knowledge sharing fun and dynamic. With our proprietary algorithm SmartMatchPro, access to expertise is facilitated, collective knowledge becomes accessible, and you can benefit from it right now, anywhere in the world.

The TallyFox interview with Dr. Nancy Dixon (Common Knowledge, a non profit and a book) is interesting. I noted these factoids and assertions:

  • almost 50% of workers are virtual, or “distributed”
  • people who are communicating only virtually tend to lose the sense of purpose of what the organization is about.
  • A challenge is “to motivate our experts to share tacit knowledge to make the knowledge from inside of a project available to the team of another project.”
  • “Collective Sensemaking is a piece of the process which will show us how to take advantage of the virtual and still stay connected in a human way. We are doing it by crowdsourcing, by Innovation Jams, by Working Out Loud, and all of those ways are bringing back the Human Side into the Virtual.”
  • “People don’t offer their knowledge because they don’t know what the other person needs…”

Sounds good.It strikes me that Facebook’s Workplace may be encroaching on the collaboration segment. Does Facebook embrace knowledge management?

Stepping back: Knowledge management leaves me dazed and confused about what, how, where, and why? Perhaps knowledge management should become knowledge “Kumbaya” with people online and posting to Facebook while sitting around a Mac with a fireplace screensaver.

Stephen E Arnold, October 16, 2016

Improvements in News and Military Technology Coming

September 23, 2016

I read two stories. These stories seem unrelated. The first is “Defense Department Reaffirms Its Commitment to Venture Investing.” The second is “Facebook and Twitter Join Coalition to Improve Social Media Newsgathering.”

Let’s look at the short item about the US Department of Defense reaffirming its interest in funding new technology. In my forthcoming, Dark Web Notebook, I point to a Web page which contains a run down of more than 100 open source software components. The software does information collection and processing functions. But the main point is that the organizations creating the code is one of the more interesting lists of entities performing next generation innovation for the Department of Defense. The write up cited above states:

Not everyone is comfortable with a government entity backing what can be sensitive technologies (not to mention the privacy issues wrought by the NSA’s practices and deployment of new tech tools).

My view is that In-Q-Tel is a more visible entity than some of the Department of Defense activities. DoD, in fact, has been in the innovation far longer than In-Q-Tel. One might suggest that substantive innovation emerges from the DoD programs; for example, the DoD is the progenitor of the Internet. My view is that more disruption may be evident in what the DoD is funding than in what the In-Q-Tel organization is funding. The write up misses an important point in my opinion. DoD looks out the windshield of innovation and In-Q-Tel looks at the world via a rear view mirror. Case in point: funding open source software related to Dark Web actions. In-Q-Tel funding companies which often have been in existence for years prior to receiving an infusion of cash and some help making sales calls in the US government.

The second write up also underscores a need for change. The idea is that old fashioned approaches are not needed. New fangled approaches are the cat’s pajamas. The problem is that the new fangled methods make some interesting errors. To fix this, high profile social media companies are going to invent a fix via a coalition.

A method with practiced for news gathering exists. Traditional newspapers illustrate the method. The process works reasonably well. More accurately, the process worked when resources were available to employ individuals who conducted interviews and performed research.

The traditional method changed with software able to count who clicked on what, people with many digital friends, and systems which collect information and figure out what is important.

Now after some interesting mistakes, Internet giants are eager to improve what I call the millennial news method:

Channel 4 News, the Telegraph, the New York Times, Washington Post, BuzzFeed News, ABC News in Australia and Agence France-Presse are among more than 20 news organizations to have signed up to the partner network, which is being organized through Google-backed First Draft.

Now Facebook (big dog) and Twitter (starving dog) are in the game. The point is that the millennial methods appear to work. Unfortunately fake news and other oddities creep into the smart systems. The new methods also help foster tension between the remaining traditional news outfits and the comparative newcomers or disruptors.

The idea of teaming up to improve smart software is interesting. The goal, of course, is to obtain high value information at the lowest possible cost; that is, with the fewest number of humans as possible.

When I read these two articles, I noted three ideas which struck me as worth thinking about:

  1. Methods exist which work yet interest gravitates away from what works to a need to find a better, more innovative process
  2. The perception that traditional methods practiced by the Department of Defense and old school newspapers are less useful than “new” approach may slow down innovation or, even worse, get the focus fuzzy.
  3. The Silicon Valley fascination with the bright and shiny may produce wasteful, duplicate efforts.

Stephen E Arnold, September 23, 2016

Thunderstone Gets an Upgrade

September 1, 2016

Pokémon Go is the latest mobile gaming craze and all of the players want to have a Pikachu as their main Pokémon.  Eventually players will evolve their Pikachu into the more powerful Raichu using candy and stardust, but old school Pokémon gamers know that the true way to evolve a Pikachu is with a Thunderstone.  The hardest part of evolving a Pikachu, however, was finding the actual Thunderstone.  Compulsive searchers have their own difficulties trying to find their information and other related content in their systems.  There is a software search solution coincidentally named Thunderstone and it recently went through an upgrade: “Thunderstone Releases Version 16.”

Thunderstone’s newest release includes updates that improve search quality across the board: intranets, aggregators, and public facing Web sites.  There also are more authorization options for better security, including a central authentication service and negotiate Kerberos option.  Perhaps the biggest upgrade is the following:

Simplified crawl configuration

  • Sitemaps allowing easier crawling of sites where URLs are not easily determined from a crawl.
  • XML/XSL site support by applying stylesheets to sites that deliver content via XML and XSL instead of HTML; the searchable text is better identified.
  • Proxy Auto-config (PAC) file support which makes it easier to index and crawl enterprises composed of different networks with varying proxy rules: the same config files used by browsers may now be used at crawl time.

The Ajax crawlable URL scheme from Google is supported, allowing Ajax based dynamic sites that support it to be crawled and indexed more effectively.”

Thunderstone now packs a more powerful punch for search quality and returning results.  Now if only finding Cubone could be improved as well.

Whitney Grace, September 1, 2016
Sponsored by, publisher of the CyberOSINT monograph


Russia Boasts of Encryption Keys for Popular Social Messaging Apps

August 25, 2016

If Russia’s Federal Security Service is to be believed, they have devised a way to break through the encryption on some of the world’s biggest messaging apps. The International Business Times reports, “Russia Now Collecting Encryption Keys to Decode Information from Facebook, WhatsApp, and Telegram.” The initiative appears to be a response to pressure from the top; columnist Mary Ann Russon writes:

“In June, Russia passed a scary new surveillance law that demanded its security agencies find a way to conduct better mass surveillance, requiring all internet firms who provide services to citizens and residents in Russia to provide mandatory backdoor access to encrypted communications so the Russian government can know what people are talking about. If any of these internet companies choose not to comply, the FSB has the power to impose fines of up to 1 million rubles (£11,406)….

The article continued:

“The FSB has now updated its website declaring that it has indeed been able to procure a method to collect these encryption keys, although, cryptically, the agency isn’t saying how exactly it will be doing so. The notice on the FSB website simply declares that in order to ensure public safety and protect against terrorism, the FSB has found a ‘procedure of providing the FSB with a method necessary for decoding all received, sent, delivered, and chat conversations between users on messaging networks’ and that this method had been sent to the Ministry of Justice to approve and make provisions to amend federal law.”

At least the Russians are not coy about their efforts to spy on citizens. But, is this a bluff? Without the details, it is hard to say. We do know the government is holding out a carrot to foreign messaging companies—they can continue to operate within their borders if they have their services “certified” by a government-approved lab. Hmm. How much is the Russian messaging market worth to these companies? I suppose we shall see.

Cynthia Murrell, August 25, 2016

Another Day Another Possible Data Breach

August 19, 2016

Has the next Ashley Madison incident happened? International Business Times reports on breached information that has surfaced on the Dark Web. The article, breach: Passwords and sexual preferences of 40 million users up for sale on dark web, sheds some light on what happened in the alleged 40 million records posted on the The Real Deal marketplace. One source claims the leaked data was old information. Another source reports a victim who says they never had an account with The article states,

“The leak is the latest in a long line of dating websites being targeted by hackers and follows similar incidents at Ashley Madison, Mate1BeautifulPeople and Adult Friend Finder. In each of these cases, hundreds of thousands – if not millions – of sensitive records were compromised. While in the case of Ashley Madison alone, the release of information had severe consequences – including blackmail attempts, high-profile resignations, and even suicide. Despite claims the data is five years old, any users of are now advised to change their passwords in order to stay safe from future account exploitation.”

Many are asking about the facts related to this data breach on the Dark Web — when it happened and if the records are accurate. We’re not sure if it’s true, but it is sensational. The interesting aspect of this story is in the terms of service for The article reveals is released from any liability related to users’ information.


Megan Feil, August 19, 2016

Sponsored by, publisher of the CyberOSINT monograph

There is a Louisville, Kentucky Hidden /Dark Web meet up on August 23, 2016.
Information is at this link:


Finding Information Takes a Backseat to Providing a Comprehensive User Experience

July 20, 2016

The article titled An Intranet Success Story on BA Insight asserts that search is less about finding information than it is about user experience. In the context of Intranet networks and search, the article discusses what makes for an effective search engine. Nationwide Insurance, for example, forged a strong, award-winning intranet which was detailed in the article,

“Their “Find Anything” locator, navigation search bar, and extended refiners are all great examples of the proven patterns we preach at BA Insight…The focus for SPOT was clear.  It’s expressed in three points: Simple consumer-like experience, One-stop shop for knowledge, Things to make our jobs easier… All three of these connect directly to search that actually works. The Nationwide project has generated clear, documented business results.”

The results include Engagement, Efficiency, and Cost Savings, in the form of $1.5M saved each year. What is most interesting about this article is the assumption that UX experience trumps search results, or at least, search results are merely one aspect of search, not the alpha and omega. Rather, providing an intuitive, user-friendly experience should be the target. For Nationwide, part of that targeting process included identifying user experience as a priority. SPOT, Nationwide’s social intranet, is built on Yammer and SharePoint, and it is still one of the few successful and engaging intranet platforms.


Chelsea Kerwin, July 20, 2016

Sponsored by, publisher of the CyberOSINT monograph

There is a Louisville, Kentucky Hidden Web/Dark
Web meet up on July 26, 2016.
Information is at this link:

A Possible Goodbye to the Dark Web

June 7, 2016

Should the Dark Web be eradicated? An article from Mic weighs in with an editorial entitled, Shutting Down the Dark Web Is a Plainly Absurd Idea From Start to Finish. Where is this idea coming from? Apparently 71 percent of internet users believe the Dark Web “should be shut down”. This statistic is according to a survey of over 24,000 people from Canadian think tank Centre for International Governance Innovation. The Mic article takes issue with the concept that the Dark Web could be “shut down”,

“The Dark Net, or Deep Web or a dozen other names, isn’t a single set of sites so much as a network of sites that you need special protocols or software in order to find. Shutting down the network would mean shutting down every site and relay. In the case of the private web browser Tor, this means simultaneously shutting down over 7,000 secret nodes worldwide. The combined governments of various countries have enough trouble keeping the Pirate Bay from operating right on the open web, never mind trying to shut down an entire network of sites with encrypted communications and hidden IP addresses hosted worldwide.”

The feasibility of shutting down the Dark Web is also complicated by the fact that there are multiple networks, such as Tor, Freenet or I2P, that allow Dark Web access. Of course, there is also the issue, as the article acknowledges, that many uses of the Dark Web are benign or even to further human rights causes. We appreciated a similar article from Softpedia, which pointed to the negative public perception stemming from media coverage of the takedown child pornography and drug sales site takedowns. It’s hard to know what isn’t reported in mainstream media.


Megan Feil, June 7, 2016

Sponsored by, publisher of the CyberOSINT monograph

Turn to Unsplash for Uncommon Free Photos

June 7, 2016

Stock photos can be so, well, stock. However, Killer Startups points to a solution in, “Today’s Killer Startup: Unsplash.” Reviewer Emma McGowan already enjoyed the site for its beautiful free photos, with new ones posted every day. She especially loves that their pictures do not resemble your typical stock photos. The site’s latest updates make it even more useful. She writes:

“The new version has expanded to include lovely, searchable collections. The themes range from conceptual (‘Pure Color’) to very specific (‘Coffee Shops’). All of the photos are free to use on whatever project you want. I can personally guarantee that all of your work will look so much better than if you went with the usual crappy free options.

“Now if you want to scroll through beautiful images a la old-school Unsplash, you can totally still do that too. The main page is still populated with a seemingly never ending roll of photos, and there’s also a ‘new’ tab where you can check out the latest and greatest additions to the collection. However, I really can’t get enough of the Collections, both as a way to browse beautiful artwork and to more easily locate images for blog posts.”

So, if you have a need for free images, avoid the problems found in your average stock photography, which can range from simple insipidness to reinforcing stereotypes and misconceptions. Go for something different at Unsplash. Based in Montreal, the site launched in 2013. As of this writing, they happen to be hiring (and will consider remote workers).


Cynthia Murrell, June 7, 2016

Sponsored by, publisher of the CyberOSINT monograph

The Scottish Philosopher in Silicon Valley

June 6, 2016

When Alistair Duff, professor of information society and policy at Scotland’s Edinburgh Napier University, checked out Silicon Valley, he identified several disturbing aspects of the prevailing tech scene. The Atlantic’s Kevah Waddell interviews the professor in, “The Information Revolution’s Dark Turn.”

The article reminds us that, just after World War II, the idealistic “information revolution” produced many valuable tools and improved much about our lives. Now, however, the Silicon-Valley-centered tech scene has turned corporate, data-hungry, and self-serving. Or, as Duff puts it, we are now seeing “the domination of information technology over human beings, and the subordination of people to a technological imperative.”

Waddell and Duff discuss the professor’s Normative Theory of the Information Society; the potential for information technology to improve society; privacy tradeoffs; treatment of workers; workplace diversity; and his preference that tech companies (like Apple) more readily defer to government agencies (like the FBI). Regarding that last point, it is worth noting Duff’s stance against the “anti-statism” he believes permeates Silicon Valley, and his estimation that “justice” outranks “freedom” as a social consideration.

Waddell asks Duff what a tech hub should look like, if Silicon Valley is such a poor example. The professor responds:

“It would look more like Scandinavia than Silicon Valley. I’m not saying that we shouldn’t develop the tech industry—we can learn a massive amount from Silicon Valley….

“But what we shouldn’t do is incorporate the abuse of the boundary between work and home, we should treat people with respect, we should have integrated workforces. A study came out that only 2 percent of Google’s, Yahoo’s, and a couple of other top companies’ workforces were black. Twelve percent of the U.S. population is black, so that is not good, is it? I’m not saying they discriminate overtly against black people—I very much doubt that—but they’re not doing enough to change things.

“We need the best of Silicon Valley and the best of European social democracy, combined into a new type of tech cluster.

“There’s a book by Manuel Castells and Pekka Himanen called The Information Society and the Welfare State: The Finnish Model, which argues that you can have a different type of information society from the libertarian, winner-takes-all model pioneered in Silicon Valley. You can have a more human, a more proportioned, a tamer information society like we’ve seen in Finland.”

Duff goes on to say that the state should absolutely be involved in building the information society, a concept that goes over much better in Europe than in the U.S. He points to Japan as a country which has built a successful information society with guidance from the state. See the interview for more of Professor Duff’s observations.


Cynthia Murrell, June 6, 2016

Sponsored by, publisher of the CyberOSINT monograph

Next Page »