November 20, 2016
If you want a quick look at what visualizations to use for use cases, you may find “An Overview of Text Mining Visualizations Possibilities with R on the CETA Trade Agreement.” The article focuses on trade agreement data, but the graphics provide a darned good refresher about visualization options. One caveat: Some of the links in the write up do not work. Nevertheless, we found the illustrations and commentary helpful.
Stephen E Arnold, November 20, 2016
September 18, 2016
When I knew people at the original Malcolm Forbes Forbes, I learned that stories were meticulously researched and edited. I read “Advanced Analytics: Insights Produce New Wealth.” I was baffled, but, I thought, that’s the point.
The main point of the write up pivots on the assertion that an “insight” converts directly to “wealth.” I am not sure about the difference between old and new wealth. Wealth is wealth in my book.
The write up tells me:
Data is the foundation that allows transformative, digital change to happen.
The logic escapes me. The squirrels in Harrod’s Creek come and go. Change seems to be baked into squirreldom. The focus is “the capitalist tool,” and I accept that the notion of changing one’s business can be difficult. The examples are easy to spot: IBM is trying to change into a Watson powered sustainable revenue machine. HP is trying to change from a conglomeration of disparate parts into a smaller conglomeration of disparate parts. OpenText is trying to change from a roll up of old school search systems into a Big Data wealth creator. Tough jobs indeed.
I learned that visualization is important for business intelligence. News flash. Visualization has been important since a person has been able to scratch animals on a cave’s wall. But again I understand. Predictive analytics from outfits like Spotfire (now part of Tibco) provided a wake up call to some folks.
The write up informs me:
While devices attached to the Internet of Things will continue to throw out growing levels of structured data (which can be stored in files and databases), the amount of unstructured data being produced will also rise. So the next wave of analytics tools will inevitably be geared to dealing with both forms of information seamlessly, while also enabling you to embed the insights gleaned into the applications of your choosing. Now that’s innovation.
Let’s recap. Outfits need data to change. (Squirrels excepted.) Companies have to make sense of their data. The data come in structured and unstructured forms. The future will be software able to handle structured and unstructured data. Charts and graphs help. Once an insight is located, founded, presented by algorithms which may or may not be biased, the “insights” will be easy to put into a PowerPoint.
BAE Systems’ “Detica” was poking around in this insight in the 1990s. There were antecedents, but BAE is a good example for my purpose. Palantir Technologies provided an application demo in 2004 which kicked the embedded analytics notion down the road. A click would display a wonky circular pop up, and the user could perform feats of analytic magic with a mouse click.
Now Forbes’ editors have either discovered something that has been around for decades or been paid to create a “news” article that reports information almost as timely as how Lucy died eons ago.
Back to the premise: Where exactly is the connection between insight and wealth? How does one make the leap from a roll up of unusual search vendors like Information Dimension, BRS, Nstein, Recommind, and my favorite old time Fulcrum Technologies produce evidence of the insight to wealth argument. If I NOT out these search vendors and focus on the Tim Bray SGML search engine, I still don’t see the connection. Delete Dr. Bray’s invention. What do we have? We have a content management company which sells content management as governance, compliance, and other consulting friendly disciplines.
Consultants can indeed amass wealth. But the insight comes not from Big Data. The wealth comes from selling time to organizations unable to separate the giblets from the goose feathers. Do you know the difference? The answer you provide may allow another to create wealth from that situation.
One doesn’t need Big Data to market complex and often interesting software to folks who require a bus load of consultants to make it work. For Forbes, the distinction between giblets and goose feathers may be too difficult to discern.
My hunch is that others, not trained in high end Manhattan journalism, may be able to figure out which one can be consumed and which one can ornament an outfit at an after party following a Fashion Week showing.
Stephen E Arnold, September 18, 2016
August 15, 2016
Palantir Technologies has developed a fondness for Silk. Silk is an interactive data visualization company. You can read about the announcement in “CIA-Backed Palantir Just Bought This Entire Startup Team.” If the write up is correct, Silk is an “acquihire” play, not a product play. I learned:
Employees of the graph and chart-making platform will directly join Palantir, leaving the Silk platform behind running on its own. Meanwhile, the Silk team will “work on even bigger and more important data problems”
A British newspaper described Silk as a “Tumblr for data.” Silk’s technology allowed a person viewing an interactive Silk-generated visualization to point and click to explore the data. A Silk user can flip between a map or a traditional bar chart, also with a click.
- Palantir wants to add to its secret sauce with some visual exploration spice. The wheel menu was hot years ago, but the shelf life of “wow” can be short
- Palantir has designs on the commercial sector, which makes sense. Even though Palantir has government work, the banks and pharma companies may have a quicker buy cycle to go with their desire for instant analysis
- Buying a company to get people is one way to deal with the shortage of certain types of technical and management talent.
Palantir competitors like IBM i2 Analyst’s Notebook have been, in my opinion, less agile in moving their systems toward the burgeoning millennial decision makers.
Stephen E Arnold, August 15, 2016
May 17, 2016
Check out this valuable cultural archive, highlighted by Open Culture in the piece, “Discover Europeana Collections, a Portal of 48 Million Free Artworks, Books, Videos, Artifacts & Sounds from across Europe.” Writer Josh Jones is clearly excited about the Internet’s ability to place information and artifacts at our fingertips, and he cites the Europeana Collections as the most extensive archive he’s discovered yet. He tells us the works are:
“… sourced from well over 100 institutions such as The European Library, Europhoto, the National Library of Finland, University College Dublin, Museo Galileo, and many, many more, including contributions from the public at large. Where does one begin?
“In such an enormous warehouse of cultural history, one could begin anywhere and in an instant come across something of interest, such as the the stunning collection of Art Nouveau posters like that fine example at the top, ‘Cercle Artstique de Schaerbeek,’ by Henri Privat-Livemont (from the Plandiura Collection, courtesy of Museu Nacional d’Art de Catalynya, Barcelona). One might enter any one of the available interactive lessons and courses on the history of World War I or visit some of the many exhibits on the period, with letters, diaries, photographs, films, official documents, and war propaganda. One might stop by the virtual exhibit, ‘Photography on a Silver Plate,’ a fascinating history of the medium from 1839-1860, or ‘Recording and Playing Machines,’ a history of exactly what it sounds like, or a gallery of the work of Swiss painter Jean Antoine Linck. All of the artifacts have source and licensing information clearly indicated.”
Jones mentions the archive might be considered “endless,” since content is being added faster than anyone could hope to keep up with. While such a wealth of information and images could easily overwhelm a visitor, he advises us to look at it as an opportunity for discovery. We concur.
Cynthia Murrell, May 17, 2016
February 22, 2016
The underlying data? Important, yeah, but the action is Hollywood style graphics. Taking a page from the Palantir game plan, SAP is getting with the visual sizzle program. Navigate to “SAP Buys All the Pretty Data Firm Roambi.” The article states:
The data prettifier’s angle is it that displays data using deliciously slick and dynamically updating charts, graphs and sliders that are native apps for iOS and Android. Roambi’s front ends tap into back ends including Excel, SQL Server, Cognos, Box, Salesforce and – yes – SAP.
Special effects matter in videos, Web pages, and business analytics.
What if the analyst gets the underlying data out of joint? What if the person using the graphic output does not understand what analytic choices were made to give the visual some zing?
What? Who worries about details? It is the visual snap that crackles.
Stephen E Arnold, February 22, 2016
November 28, 2015
Want to make a snappy visualization to impress your manager or a one star general? Navigate to “Top 5 Visualisation Tools” and explore the five recommendations. These systems output some Hollywood-style chart. Just remember to know where a particular data point came from and how the number was produced. Well, if you are briefing a CEO or a four star general, you might not have to stick to close to the facts. Just make each chart shout, “Good news.”
Here are the five systems the write up explains and illustrates:
- Gephi. Yep, free to use with a couple of caveats
- Tom Sawyer Perspectives. Not for the Huck Finns eager to kick back on a raft
- Keylines. You too can do geospatial integration
- Linkurious. Sharpen your query language skills
- GraphX. Open source and Spark what could be more wonderful?
PowerPoint away. Just remember to make sure you can answer the question, “Where did that come from?”
Stephen E Arnold, November 28, 2015
October 18, 2015
The write up, sourced from ESRI, is interesting because it describes visualizations without displaying any visualizations. Navigate to “The Central Intelligence Atlas: how BI Is Going Visual.” I expected some narrative and then examples of visualizations.
Now the write up references outfits like Google and its Maps. I know what these look like, and I have seen mash ups which display data on Google Maps. But, like the supporters of old fashioned radio say, use your imagination. Okay, I did.
I noticed a couple of statements in the write up which quivered my imaginative Jello.
First, I highlighted in J-E-L-L-O green:
once we are talking about things that exist and happen in the real physical world, there is a significant dimension that a BI system doesn’t typically make use of to find essential correlations between disparate datasets – location. Fortunately, there is a complementary enterprise technology that can fill that gap right now. It’s called Enterprise GIS – Geographical Information Systems – and many businesses are waking up to the fact that visualization using interactive digital maps and applying ‘spatial analysis’, as it’s called, can produce significant results. Recognizing this, forward-thinking BI vendors including Microstrategy and IBM Cognos are starting to embed limited GIS capabilities into their enterprise platforms.
Yep, putting data on maps. I visualize presentations like those from Geofeedia. Here’s what that outfit’s approach looks like:
Source: Geofeedia, 2015
I find this type of example helpful.
I also noted this passage, which I highlighted in what I call cool blue:
Spatial analysis isn’t only useful to businesses with physical property and assets however. Mitsui Sumitomo Insurance Underwriting at Lloyd’s Limited (MSIG), for example, has created a dynamic online atlas of hundreds of potential hazards that could impact insured properties and businesses around the world, using GIS. The technology enables it to monitor hazards at specific locations, ranging from tornados and wildfires to terrorist activities.
What I visualized was this overlay of bomber identification on a satellite image:
Source: David Reeths, “IHS Jane’s Satellite Imagery Analysis: Use of Imagery Intelligence within OSINT,” 2014, page 15
I agree that mash ups of maps and data are useful. I would suggest that examples, not just word descriptions based on marketing lingo, help make the ideas more interesting. But content marketing does what it can.
Stephen E Arnold, October 18, 2015
October 12, 2015
I read “Lux: Useful Sankey Diagram on AI.” A Sankey diagram, according to Sankey Diagrams a “Sankey diagram says more than 1,000 pie charts.” The assumption is, of course, that a pie chart presents meaningful data. In the energy sector you can visual flows in complex systems. It helps to have numbers when one is working towards a Sankey map, but if real data are not close at hand, one can fudge up some data.
Here’s the Sankey diagram in the write up:
You can see an almost legible version at this link.
What the diagram suggests is that certain information access and content processing functions flow into data mining, machine learning, and statistics. If you are a fan of multidimensionality, the arrow of time may flow in the reverse direction; that is from data mining, machine learning, and statistics to affective computing, cognitive computing, computational discovery, image and video analytics, language translation, navigation, recommender systems, and speech recognition.
The intermediary state, tinted a US currency green provides intermediating operations or conditions; for example, anomaly detection, collaborative filtering, computer eavesdropping, computer vision, pattern recognition, NLP, path planning, clustering, deep learning, dimensionality reduction, networks graphic models, online reinforcement learning, pattern similarity, probabilistic modeling, regression, and, my favorite, search algorithms.
The diagram, like the wild and crazy chemical imagery for Watson, seems to be a way to:
- Collect a number of discrete operations
- Arrange the operations into some orderly framework
- Allow the viewer to perceive relationships or the potential for relationships among the operations.
In short, skip the wild and crazy presentations by search and content processing vendors about how search enables broader and, hence, more valuable activities. Search is relegated to an entry in the intermediating column of the Sankey diagram.
My thought is that some folks will definitely love the idea that the many different specialties of content processing can be presented in a mandala which invites contemplation and consideration.
The diagram makes clear that when a company wants to know what one can do with the different and often clever operatio0ns one can perform with content, the answer may be, “Make a poster and hang it on the wall.”
In terms of applications, the chart makes quite explicit that some clever team will have to put the parts in order. Does this remind you of building a Star Wars character from Lego blocks.
The construct is the value, not the individual enabling blocks.
Stephen E Arnold, October 12, 2015
August 17, 2015
I read “CI Radar Delivers New Competitive Intelligence Coverage of the Data Visualization Market.” In the story which explains a tracking and monitoring tool from a competitive intelligence firm was a little chunk of information. The story contains a list of the players which the competitive intelligence firm considers important in the Hollywoodization of analytic system outputs. Who loves a great chart? Certainly generals, mid tier consultants, and MBA students.
Here’s the list of data visualization players:
- Adobe (ah, the magic of the creative cloud)
- Advizor Solutions
- Afs Technologies
- Centrifuge Systems
- ClearStory Data
- Dell (visualization and not laptops?)
- iDashboards (maybe free for academics?)
- Inetsoft Technology
- Infor (I think of this outfit as a CRM vendor)
- Informatica (now owned by Permira)
- Information Builders
- International Business Machines (IBM) (which unit of IBM?)
- Jinfonet Software
- Logi Analytics
- Microsoft (my goodness)
- OpenText (is this the Actuate or the Talend acquisition?)
- Panorama Software
- Pentaho (don’t forget this is Hitachi)
- Phocas Software
- Pyramid Analytics
- Salesforce (a surprise to me)
- SAP (interesting?)
- SAS (also interesting?)
- Splunk (a bit of a surprise)
- Tableau Software
- Teradata (Is this Rainstor, ThinkBig or another chunk of acquired technology?)
- TIBCO (is this Spotfire?)
I would point out that some of the key players in the law enforcement and intelligence community are not included. Why would a consulting firm want to highlight the companies which are pioneering next generation, dynamic, interactive, and real time visualization tools. Although incomplete from my vantage point, how long will it be before Forrester, Gartner, and other mid tier firms roll out a magic wave rhomboid explaining what these companies are doing to be “players”?
Stephen E Arnold, August 17, 2015
March 5, 2015
Here we go again! Another brand new year and it is time to predict where data will take us. For the past few years it has been all about the big data and while it has a solid base, other parts of the data science are coming into the limelight. While LinkedIn is a social network for professionals, one can also read articles on career advice, hot topics, and new trends in fields. Kurt Cagle is a data science expert and has written on the topic for over ten years. His recent article, “Ten Trends In Data Science In 2015” from December was posted on LinkedIn.
He calls the four data science areas the Data Cycle: analysis, awareness, governance, and acquisition. From Cagle’s perspective, 2014 saw big data has matured, data visualization software is in high demand, and semantics is growing. He predicts 2015 will hold much of the same:
“…with the focus shifting more to the analytics and semantic side, and Hadoop (and Map/Reduce without Hadoop) becoming more mainstream. These trends benefit companies looking for a more comprehensive view of their information environment (both within and outside the company), and represent opportunities in the consulting space for talented analysts, programmers and architects.”
Data visualization is going to get even bigger in the coming year. Hybrid data stores with more capabilities will become more common, semantics will grow even larger and specializing companies will be bought up, and there will be more competition for Hadoop. Cable also predicts work be done on a universal query language and data analytics are moving beyond the standard SQL.
His ending observations explain that data silos will be phased into open data platforms, making technology easier not just for people to use but also for technology to be compliant with each other.