CyberOSINT banner

Luciad Data Visualization and Situational Awareness Is Like an Over Stimulated Google Maps

June 21, 2016

The promotional article on Luciad titled Luciad V2016 Puts Users at the Center of Technical Innovation discusses the data fusion product from the global software company emphasizing situational awareness systems for Aviation, Defense and Security markets. 50,000+ people have viewed the 3D browser technology via the web app launched in 2015 that shows the breathtaking capacity to track and visualize moving data in the form of 35,000 international flights. The article states,

“Luciad’s software components are designed for the creation of applications that tackle a range of tasks, from top-level strategy to tactical detail and mission planning to operations debriefing. By connecting directly to data sources, Luciad’s software not only analyzes and visualizes what is happening now, but also helps predict what will happen next – allowing users to act quickly and safely. “Connect, visualize, analyze, act” is both our method and our motto.”

The LuciadFusion technology product features include the ability to fuse and serve multi-dimensional and multi-layered formats as well as multi-dimensional raster data, which applies to weather data. If you thought Google Maps was cool, this technology will blow you away. The developers were very interested in the aesthetic quality of the technology, and richness of the imaging makes that focus crystal clear.
Chelsea Kerwin, June 21, 2016

Sponsored by, publisher of the CyberOSINT monograph

Public Opinion of Dark Web May Match Media Coverage

June 17, 2016

A new survey about the Dark Web was released recently. Wired published an article centered around the research, called Dark Web’s Got a Bad Rep: 7 in 10 People Want It Shut Down, Study Shows. Canada’s Center for International Governance Innovation surveyed 24,000 people in 24 countries about their opinion of the Dark Web. The majority of respondents, 71 percent across all countries and 72 percent of Americans, said they believed the “dark net” should be shut down. The article states,

“CIGI’s Jardine argues that recent media coverage, focusing on law enforcement takedowns of child porn sites and bitcoin drug markets like the Silk Road, haven’t improved public perception of the dark web. But he also points out that an immediate aversion to crimes like child abuse overrides mentions of how the dark web’s anonymity also has human rights applications. ‘There’s a knee-jerk reaction. You hear things about crime and its being used for that purpose, and you say, ‘let’s get rid of it,’’ Jardine says.”

We certainly can attest to the media coverage zoning in on the criminal connections with the Dark Web. We cast a wide net tracking what has been published in regards to the darknet but many stories, especially those in mainstream sources emphasize cybercrime. Don’t journalists have something to gain from also publishing features revealing the aspects the Dark Web that benefit investigation and circumvent censorship?


Megan Feil, June 17, 2016

Sponsored by, publisher of the CyberOSINT monograph

Dark Web Drug Sales Go on Despite One Marketplace Down

June 16, 2016

Another Dark Web drug marketplace has gone offline, at least for now. Vice’s Motherboard published an article that reports on this incident and offers insight into its larger implications in their piece, Dark Web Market Disappears, Users Migrate in Panic, Circle of Life Continues. Nucleus market mostly sold illegal drugs such as cocaine and cannabis. Now, the site is unresponsive and has made no announcements regarding downtime or a return. The article hypothesizes about why Nucleus is down,

“At the moment, it’s not totally clear why Nucleus’s website is unresponsive. It could be an exit scam—a scam where site administrators stop allowing users to withdraw their funds and then disappear with the stockpile of bitcoins. This is what happened with Evolution, one of the most successful marketplaces, in March 2015. Other examples include Sheep Marketplace, from 2013, and more recently BlackBank Market. Perhaps the site was hacked by a third party. Indeed, Nucleus claimed to be the targetof a financially motivated attack last year. Or maybe the administrators were arrested, or the site is just suffering some downtime.”

The Dark Web poses an interesting case study around the concept of a business lifecycle. As the article suggests, this graph reveals the brief, and staggered, lifetimes of dark web marketplaces. Users know they will be able to find their favorite vendors selling through other channels. It appears the show, and the sales, must go on.

Megan Feil, June 16, 2016

Sponsored by, publisher of the CyberOSINT monograph

Murdoch Wall Street Journal Factiva: Known Unknowns

June 10, 2016

That Donald Rumsfeld statement about known knowns, known unknowns, etc. Is back. The Wall Street Journal ran an ad for Factiva. You remember Factiva. It is the Dow Jones Information Service repositioned and renamed a number of times over the last 15 or 20 years.

If you are into for fee search, you will know about Factiva and its kissing cousins: LexisNexis (bring your legal client’s purchase order), CSA ProQuest Dialog (bring your library acquisition budget), and Ebsco (bring your credit card). For fee information services serve the professional searcher market. Most people — including Gen X and Millennials researchers — are happy with Google. Objective results every time.

The for-fee services are still around. Public library and university fund raising programs help pay for access. Some queries returning zero useful results can cost $100 or more. Hey, you didn’t know, right?

If you navigate to the June 2, 2016, Wall Street Journal, page A7 in my dead tree edition ran a full page ad for Factiva. The ad highlights a couple of pie charts. Here they are in a tough to read gray and blue motif. Users of commercial database services have really sharp eyes and don’t need high contrast text, right?

The first pie chart shows your life consumed with research. Notice how little time one has to eat lunch. Note what a tiny portion of one’s day is available for email, Facebook, talking with colleagues, making sales calls, printing, the youth soccer telephone tree.


Now look at the second chart.


Look at the many different tasks one can undertake in a single work day. One can, of course, “take lunch.” I eat lunch, but that’s because here in rural Kentucky, we “eat” a meal. We make decisions. Apparently in Factiva land one takes a meal and probably takes decisions.

Other tasks one can pursue when one has Factiva are:

  • Collaborating across departments
  • Advise colleagues
  • Stay on top of the news (Hey, it is part of that real journalism outfit owned by Mr. Murdoch. No bugging telephones, please.)
  • Create a company newsletter. (I assume this word is “blog”, a Snapchat, or a tweet, but I could be off base.)
  • Build powerful infographics. (Hmmm. I thought art types created infographics based on the data generated by a business intelligence system.)
  • Research. Yes via Factiva.

Now I know that I am really out of the flow. The diagram showing the different between Baby Boomers and Millennials created by ace research analyst Mary Meeker reminded me of the gulf between my demographic and the zippy millennials.


Slide 51 from the Meeker, State of the Internet report.

The main point for me is that I possess zero of the attributes of millennials. I don’t earn to spend. I am retired. I conserve to pay for the old age home which I believe millennials call “opportunities for bingo.”

But the best part of the Factiva ad is the copy. I know words. Those nifty pie charts were the cat’s pajamas, weren’t they?

Here’s the guts of the message:

Spend your day working, not searching. Factiva’s reputable sources, flexible search and powerful insights provide access to thousands of quality, licensed, news and information sources in 28 languages. Know unknowns. [Emphasis added]

If Ms. Meeker is correct in her research and the supporting information from Hillhouse Capital and dozens of what appear to be primary sources and many hours of online searching commercial and Web resources — messaging apps are where the future is. Oh, there are videos too, but the takeaway is that traditional methods of getting digital information are in the same spot newspapers were yesterday.

The ad warrants several questions:

  • Why does it have to be so darned big? Maybe small ads in the Wall Street Journal are ignored?
  • How many of the Wall Street Journal’s readers are information specialists trained in the use of commercial online services? Judging from the Special Library Association’s challenges, I would suggest that the ad would have made sense to the corporate information specialist working in 1986, not 2016.
  • What’s with the wonky pie charts? When I worked at a commercial database company, I don’t recall meeting any online users who spent the bulk of every day online. There were reference interviews (remember them, millennials?), culling the outputs from dot matrix printers, and planning search strategies before going online and whacking away.

Mr. Rumsfeld’s statement about knowns and unknowns emerged from his brush with the murky world of government related information. If he were to use Factiva today, would he have modified this famous statement:

There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don’t know. But there are also unknown unknowns. There are things we don’t know we don’t know.

Perhaps Factiva, like IBM Watson, is easier to describe than turn an information search system into a lean, mean, money making machine? I would suggest that the answer for decades has been an unknown unknown.

Stephen E Arnold, June 10, 2016

Weakly Watson: Unusual Watson Applications

June 9, 2016

The change in leadership at IBM Watson is a bit like the weather. One does not know how the weekend will turn out. I read “5 Unusual Things You Can Do with IBM’s Watson.” I must admit that I have missed the full page ads with weird made up chemical symbols suggesting Watson’s combinatorial magic. I also have missed the “Watson cures cancer” write ups. I always wonder how that project is coming along.

In the unusual write up, I noted the five things; to wit:

  1. Create a “custom” order for granola.
  2. Shop for clothes.
  3. Find a bottle of wine. [Shades of Endeca’s long standing example!]
  4. Ask health questions. [When I worked at Ziff in the 1990s, we had a health reference center which performed the same trick. Libraries loved the system. Doctors, not so much.]
  5. Check into a Hilton and ask about bus routes. [Uber, anyone?]

My hunch is that IBM wants to make darned certain it is in the race for smart software. Okay, IBM Watson with its open source technology, home brew scripts, and acquired technology is really big in artificial intelligence. I give up already.

Custom granola? A slam dunk. Help me shop for clothes? My wife may have some thoughts about that. These five items comprise compelling use cases for someone I assume. Oh, when I check into a hotel, I think Uber, not bus routes. Ever try to take a bus in Xian, China?

Stephen E Arnold, June 3, 2016

BA Insight Asserts Government Transformation for Information Access

June 6, 2016

This is a bold, bold assertion. In my limited experience with the US government’s entities, I can attest that certain systems and methods do not change. That obviously does not match the marketing message from BA Insight, a search and retrieval vendor with a Microsoft focus and competence in all sorts of interesting buzzwords.

Navigate to “BA Insight and MD Tech Solutions Join Forces to Transform the Way Government Agencies Find and Work with Content.” I read:

BA Insight today announced a partnership with MD Tech Solutions, a technology services company located in Fredericksburg, Virginia specializing in Custom SharePoint Development, Administration, Design and Architecture.  As a reseller of BA Insight’s software portfolio, MD Tech can now provide its customers with a solution that quickly connects SharePoint users to the essential knowledge they need to be productive, while providing an internet-like search experience that users will love.

Love in the government entities with which I had experience when I labored in the vineyards in Congress, the executive branch, and some other outfits was, in my experience, in short supply. There were numerous search and retrieval systems. There were legacy systems which did magic things to information. Anyone remember the baked in search system with SharePoint? Maybe Fast Search? What about the components in Oracle? Palantir? IBM i2?

I find the notion of transforming the government interesting. Even more fascinating is the notion of users loving a search system. Love in the government. Hmmm.

Stephen E Arnold, June 6, 2016

eBay Struggles with Cluttered, Unstructured Data, Deploys Artificial Intelligence Strategy

May 24, 2016

The article on Forbes titled eBay’s Next Move: Artificial Intelligence To Refine Product Searches predicts a strong future for eBay as the company moves further into machine learning. For roughly six years eBay has been working with Expertmaker, a Swedish AI and analytics company. Forbes believes that eBay may have recently purchased Expertmaker. The article explains the logic behind this logic,

“One of the key turnaround goals of eBay is to encourage sellers to define their products using structured data, making it easier for the marketplace to show relevant search results to buyers. The acquisition of Expertmaker should help the company in this initiative, given its expertise in artificial intelligence, machine learning and big data.”

The acquisition of Expertmaker should allow for a more comprehensive integration of eBay’s “noisy data.” Expertmaker’s AI strategy is based in genetics research, and has made great strides in extracting concealed value from data. For eBay, a company with hundreds of millions of listings clogging up the platform, Expertmaker’s approach might be the ticket to achieving a more streamlined, categorized search. If we take anything away from this, it is that eBay search currently does not work very well. At any rate, they are taking steps to improve their platform.

Chelsea Kerwin, May 24, 2016

Sponsored by, publisher of the CyberOSINT monograph

The Kardashians Rank Higher Than Yahoo

May 20, 2016

I avoid the Kardashians and other fame chasers, because I have better things to do with my time.  I never figured that I would actually write about the Kardashians, but the phrase “never say never” comes into play.  As I read Vanity Fair’s “Marissa Mayer Vs. ‘Kim Kardashian’s Ass” : What Sunk Yahoo’s Media Ambitions?” tells a bleak story about the current happenings at Yahoo.

Yahoo has ended many of its services, let go fifteen percent of staff, and there are very few journalists left on the team.  The remaining journalists are not worried about producing golden content, they have to compete with a lot already on the Web, especially “Kim Kardashian’s ass” as they say.

When Marissa Mayer took over Yahoo as the CEO in 2012, she was determined to carve out Yahoo’s identity as a tech company.  Mayer, however, wanted Yahoo to be media powerhouse, so she hired many well-known journalists to run specific niche projects in popular areas from finance to beauty to politics.  It was not a successful move and now Yahoo is tightening its belt one more time.  The Yahoo news algorithm did not mesh with the big name journalists, the hope was that their names would soar above popular content such as Kim Kardashian’s ass.  They did not.

Much of Yahoo’s current work comes from the Alibaba market.  The result is:

“But the irony is that Mayer, a self-professed geek from Silicon Valley, threw so much of her reputation behind high-profile media figures and went with her gut, just like a 1980s magazine editor—when even magazine editors, including those who don’t profess to “get” technology, have long abandoned that practice themselves, in favor of what the geeks in Silicon Valley are doing.”

Mayer was trying to create a premiere media company, but lower quality content is more popular than top of the line journalists.  The masses prefer junk food in their news.


Whitney Grace, May 20, 2016
Sponsored by, publisher of the CyberOSINT monograph

Affinio and the Differences between Useful Data and Fanciful Data

May 17, 2016

I read “Understanding the Cultural Differences Between NASCAR and Formula One Fans [Analysis].” The write up is in a blog post from Affinio. The company describes itself in this way:

Marketing Intelligence that leverages the social graph to understand today’s customer.

The information in the  write up presents clusters of interest between the two fan bases for each of these motor sports. F1 consists of clusters labeled this way:


To illustrate the differences, Affinio presents a visualization of the Nascar audience:


The labels strike me as unhelpful; for example, Cluster 14, Cluster 6, etc.

The top interests of the two audiences consist of a collage of small images. I am not sure what each image represents.


Equally unhelpful is the word clouds for each of the audiences; for example:


The map showing the geographic area where F1 is popular focuses on a global scale with a centroid in Western Europe. The absence of a hot spot in the Middle East was puzzling. Is Australia as large an F1 market as the UAE in terms of money spent on F1 activities?


The map for the Nascar market depicts only the US of A. My question, “Why not show a global map?”


Thinking about this analysis, I have several questions:

  1. A list of dot points would get the message across in a more efficient, possibly less confusing way would it not?
  2. What is analyzed? It seems that the single actionable fact is that the F1 market is global and the Nascar market is local.
  3. What are the data sets used for the analysis?
  4. Why are terms like “Cluster 14” used instead of words?

The most important data from my uninformed vantage point is the money generated by the two types of motor racing.

My hunch is that the Affino write up wanted to show off visualizations, not substantive and actionable data analysis. In short, is this marketing or is it substance? I will leave the answer to you, gentle reader.

Stephen E Arnold, May 17, 2016

IBM Uses Watson Analytics Freebie Academic Program to Lure in Student Data Scientists

May 6, 2016

The article on eWeek titled IBM Expands Watson Analytics Program, Creates Citizen Data Scientists zooms in on the expansion of the IBM  Watson Analytics academic program, which was begun last year at 400 global universities. The next phase, according to Watson Analytics public sector manager Randy Messina, is to get Watson Analytics into the hands of students beyond computer science or technical courses. The article explains,

“Other examples of universities using Watson Analytics include the University of Connecticut, which is incorporating Watson Analytics into several of its MBA courses. Northwestern University is building Watson Analytics into the curriculum of its Predictive Analytics, Marketing Mix Models and Entertainment Marketing classes. And at the University of Memphis Fogelman College of Business and Economics, undergraduate students are using Watson Analytics as part of their initial introduction to business analytics.”

Urban planning, marketing, and health care disciplines have also ushered in Watson Analytics for classroom use. Great, so students and professors get to use and learn through this advanced and intuitive platform. But that is where it gets a little shady. IBM is also interested in winning over these students and leading them into the data analytics field. Nothing wrong with that given the shortage of data scientists, but considering the free program and the creepy language IBM uses like “capturing mindshare among young people,” one gets the urge to warn these students to run away from the strange Watson guy, or at least proceed with caution into his lair.

Chelsea Kerwin, May 6, 2016

Sponsored by, publisher of the CyberOSINT monograph


« Previous PageNext Page »