FOIA Suit Seeks Details of Palantirs Work with ICE

March 21, 2018

Well, this should be interesting. The Electronic Privacy Information Center (Epic.org) has announced, “EPIC FOIA- EPIC Sues for Details of Palantir’s Government Systems.” The brief write-up reports the watchdog’s complaint requesting information on the relationship between data-analysis firm Palantir and the Immigration and Customs Enforcement agency (ICE). The announcement specifies:

The federal agency contracted with the Peter Thiel company to establish vast databases of personal information, and to make secret determinations about the opportunities for employment, travel, and also who is subject to criminal investigations. EPIC is seeking the government contracts with Palantir, as well as assessments and other related documents. The ICE Investigative Case Management System and the FALCON system pull together vast troves of personal data from across the federal government. EPIC wrote in the complaint, ‘Palantir’s “big data” systems raise far-reaching privacy and civil liberties risks.’

Palantir’s role in creating “risk assessment” scores for travelers (US citizens and non-citizens alike) was revealed through an earlier FOIA lawsuit from EPIC. It would be interesting to see what information the organization is able to shake loose.

Cynthia Murrell, March 21, 2018

Big Data and Smart Software: A Volatile Mixture?

March 19, 2018

For several years big data and artificial intelligence have been running on parallel tracks. Once in a while they cross over, but mostly they kept independent of one another. But that is poised to change, as we saw from a recent press release from Lucidworks, “Lucidworks Launches Fusion 4 With Operationalized AI and Portable Applications.”

According to the piece, their AI and big data are coming together because:

“Our customers are global organizations who demand a reactive and flexible platform that lets them adapt to hybrid run-time environments including on premise, private cloud, and public cloud infrastructures. With Fusion 4, we’ve brought that portability to application development so customers can create and run apps that best fit their security and operational constraints.”

This is picking up steam in a major way. Multiple companies are offering mashups of big data and AI and the results run the gamut. Forbes recently ran a list of 30 free sources for such tools. Not only are they becoming more available, there is a lot of evidence that this combo is disrupting the normal patterns of business and life. Expect more from this pairing because our world is waiting for a new explosion of AI and data.

Beyond Search wants to point out that the use of data from services like Facebook by third parties can have unexpected consequences. Those facilitating volatile compounds may find themselves walking a knife edge. Will that “work”?

Patrick Roland, March 19, 2018

Bigquery Equals Big Data Transfers for Google

March 16, 2018

Google provides hundreds of services for its users; these include YouTube, AdWord, DoubleClick Campaign Manager, and more.  Google, however, is mainly used as a search engine and all of the content on its other services are fed into the search algorithm so they can be queried.  In order for all of the content to be searchable, it needs to be dumped and mined.  That requires a lot of push power, so what does Google use?  According to Smart Data Collective, Google uses the, ““Big Query Service: Next Big Thing Unveiled By Google On Big Data”.“”

Google and big data have not been in the news together for a while, but the BigQuery Data Transfer Service shows how it is moving away from SaaS.  How exactly does this work?

According to a Google’s blog post, the new service automates the migration of data from these apps in BigQuery in a scheduled and managed manner. So good so far, the service will support data transfers from AdWords, DoubleClick Campaign Manager, DoubleClick for Publishers, and YouTube Content and Channel Owner Reports and so forth. As soon as the data gets to BigQuery, users can begin querying on the immediate basis. With the help of Google Cloud Dataprep, users cannot only clean and prep the data for that analysis but also further think of analyzing other data alongside that information kept in BigQuery.

The data moves from the apps within 24 hours and BigQuery customers can schedule their own data deliveries so they occur regularly.  Customers who already use BigQuery are Trivago and Zenith.

The article turns into a press release for other services Google provides related to machine learning and explains how it is the leading company in the industry.  It is simply an advertisement for cloud migration and yet another Google service.

Whitney Grace, March 16, 2018

Visualization Aims to Be Huge in 2018

March 9, 2018

Al the data in the world won’t do you much good if users can’t visualize it. This has been a fact for the computer since Steve Jobs was working out of a garage. But with today’s onslaught of big data, it’s more important than ever. Luckily, it’s going to be huge in the coming year, according to a recent Business Wire article, “IHS Markit Identifies the Top Eight Tech Trends for 2018.”

According to the story, the two best trends are:

Trend #5: Ubiquitous video

The growing use of screens and cameras across multiple consumer- and enterprise-device categories, along with increasingly advanced broadcast, fixed and mobile data networks, is powering an explosion in video consumption, creation, distribution and data traffic. More importantly, video content is increasingly expanding beyond entertainment into industrial applications for medical, education, security and remote controls, as well as digital signage.

Trend #6: Computer vision

The increasing importance of computer vision is directly tied to the mega-trend of digitization that has been playing out in the industrial, enterprise and consumer segments. The proliferation of image sensors, as well as improvements in image processing and analysis, are enabling a broad range of applications and use cases including industrial robots, drone applications, intelligent transportation systems, high-quality surveillance, and medical and automotive.

Perhaps nowhere will this intersection of big data and visualization be bigger than with AI. Experts are ready for artificial intelligence to become user-friendly and they all say it’ll be through visualization. Just wait to see what the new year brings.

Patrick Roland, March 9, 2018

Governance: Now That Is a Management Touchstone for MBA Experts

February 27, 2018

I read “Unlocking the Power of Today’s Big Data through Governance.” Quite a lab grown meat wiener that “unlocking,” “power,” “Big Data,” and “governance” statement is that headline. Yep, IDG, the outfit which cannot govern its own agreements with the people the firm pays to make the IDG experts so darned smart. (For the back-story, check out this snapshot of governance in action.)

Image result for wishful thinking

What’s the write up with the magical word governance about?

Instead of defining “governance,” I learn what governance is not; to wit:

Data governance isn’t about creating a veil of secrecy around data

I have zero idea what this means. Back to the word “governance.” Google and Wikipedia define the word in this way:

Governance is all of the processes of governing, whether undertaken by a government, market or network, whether over a family, tribe, formal or informal organization or territory and whether through the laws, norms, power or language of an organized society.

Okay, governing. What’s governing mean? Back to the GOOG. Here’s one definition which seems germane to MBA speakers:

control, influence, or regulate (a person, action, or course of events).

The essay drags out the chestnuts about lots of information. Okay, I think I understand because Big Data has been touted for many years. Now, mercifully I assert, the drums are beating out the rhythm of “artificial intelligence” and its handmaiden “algos,” the terrific abbreviation some of the marketing jazzed engineers have coined. Right, algos, bro.

What’s the control angle for Big Data? The answer is that “data governance” will deal with:

  • Shoddy data
  • Incomplete data
  • Off point data
  • Made up data
  • Incorrect data

Presumably these thorny issues will yield to a manager who knows the ins and outs of governance. I suppose there are many experts in governance; for example, the fine folks who have tamed content chaos with their “governance” of content management systems or the archiving mavens who have figured out what to do with tweets at the Library of Congress. (The answer is to not archive tweets. There you go. Governance in action.)

The article suggests a “definitive data governance program.” Right. If one cannot deal with backfiles, changes to the data in the archives, and the new flows of data—how does one do the “definitive governance program” thing? The answer is, “Generate MBA baloney and toss around buzzwords.” Check out the list of tasks which, in my experience, are difficult to accomplish when resources are available and the organization has a can-do attitude:

  • Document data and show its lineage.
  • Set appropriate policies, and enforce them.
  • Address roles and responsibilities of everyone who touches that data, encouraging collaboration across the organization.

These types of tasks are the life blood of consultants who purport to have the ability to deliver the near impossible.

What happens if we apply the guidelines in the Governance article to the data sets listed in “Big Data And AI: 30 Amazing (And Free) Public Data Sources For 2018.” In my experience, the cost of normalizing the data is likely to be out of reach for most organizations. Once these data have been put in a form that permits machine-based quality checks, the organization has to figure out what questions the data can answer with a reasonable level of confidence. Getting over these hurdles then raises the question, “Are these data up to date?” And, if the data are stale, “How do we update the information?” There are, of course, other questions, but the flag waving about governance operates at an Ivory Tower level. Dealing with data takes place with one’s knees on the ground and one’s hands in the dirt. If the public data sources are not pulling the hay wagon, what’s the time, cost, and complexity of obtaining original data sets, validating them, and whipping them into shape for use by an MBA?

You know the answer: “This is not going to happen.”

Here’s a paragraph which I circled in Oscar Mayer wiener pink:

One of the more significant, and exciting, changes in data governance has been the shift in focus to business users. Historically, data has been a technical issue owned by IT and locked within the organization by specific functions and silos. But if data is truly going to be an asset, everyday users—those who need to apply the data in different contexts—must have access and control over it and trust the data. As such, data governance is transforming from a technical tool to a business application. And chief data officers (CDOs) are starting to see the technologies behind data governance as their critical operating environment, in much the same way SAP serves CFOs, and Salesforce supports CROs. It is rare to find an opportunity to build a new system of record for a market.

Let’s look at this low calorie morsel and consider some of its constituent elements. (Have you ever seen wieners being manufactured? Fill in that gap in your education if you have not had the first hand learning experience.)

First, business users want to see a pretty dashboard, click on something that looks interesting in a visualization, and have an answer delivered. Most of the business people I know struggle to understand if the data in their system is accurate and limited expertise to understand the mathematical processes which churn away to display an “answer.”

The reference to SAP is fascinating, but I think of IBM-type systems as somewhat out of step with the more sophisticated tools available to deal with certain data problems. In short, SAP is an artifact of an earlier era, and its lessons, even when understood, have been inadequate in the era of real time data analysis.

Let me be clear: Data governance is a management malarkey. Look closely at organizations which are successful. Peer inside their data environments. When I have looked, I have seen clever solutions to specific problems. The cleverness can create its own set of challenges.

The difference between a Google and a Qwant, a LookingGlass Cyber and IBM i2, or Amazon and Wal-Mart is not Big Data. It is not the textbook definition of “governance.” Success has more to do with effective problem solving on a set of data required by a task. Google sells ads and deals with Big Data to achieve its revenue goals. LookingGlass addresses chat information for a specific case. Amazon recommends products in order to sell more products.

Experts who invoke governance on a broad scale as a management solution are disconnected from the discipline required to identify a problem and deal with data required to solve that problem.

Few organizations can do this with their “content management systems”, their “business intelligence systems,” or their “product information systems.” Why? Talking about a problem is not solving a problem.

Governance is wishful thinking and not something that is delivered by a consultant. Governance is an emergent characteristic of successful problem solving. Governance is not paint; it is not delivered by an MBA and a PowerPoint; it is not a core competency of jargon.

In Harrod’s Creek, governance is getting chicken to the stores in the UK. Whoops. That management governance is not working. So much in modern business does not work very well.

Stephen E Arnold, February 27, 2018

Can Presscoin Keep News from Going Fake

February 23, 2018

Fake news is a topic that has everyone on all sides of the aisle concerned. However, One innovative idea merging fake news and big data might have solved the problem. We learned more from a recent The Next Web story, “PresscoinL The Largest Crowdfunding Effort to Address the News Crisis.”

According to the story:

As the GDP of the PressCoin economy grows, the value of tokens will rise.

And blockchain keeps it honest. Our hope is to grow a decentralized media network reaching 100 mln engaged ‘users/prosumers’ in five years time, and to meaningfully address 10 percent of the news media industry within a couple of years of that.

PressCoin’s business strategy rests on these legs:

  • Shared Design Philosophy–  Open Collaboration, Partnership, and Decentralization
  • Shared Technology Infrastructure– Seamless media technology cloud, underlying big-data systems, advanced APIs for engagement and analytics
  • Shared Business Services– Consumer Data Intelligence, Monetization, Enterprise Sales, Ecosystem Partnerships, Strategic Relations
  • Shared Developer Network– This fertile playground for agile experiments in the news/media/journalism sphere
  • Shared Fiat/Crypto Financial Services Infrastructure– Built on Cointype
  • Shared Venture Arm– Foster disruption within the ecosystem

While PressCoin is just getting off the ground, big data is already sorting out whether fake news can be trusted. One interesting use has been the story of how fake news might not have had that big an impact on the 2016 election. This could be the dawn of some very insightful times.

Patrick Roland, February 23, 2018

Big Data and Net Freedom in China Make a Complicated Relationship

February 21, 2018

One of China’s hottest new app uses a big data engine, unlike anything most of us can imagine, however, that horsepower is getting the company in trouble. We learned more in a recent Slashdot piece, “Toutiao, One of China’s Most Popular News Apps, is Discovering the Risks Involved in Giving People Exactly What They Want Online.”

It actually pulls from a New York Times article and says:

Now the company is discovering the risks involved, under China’s censorship regime, in giving the people exactly what they want. The makers of the popular news app Jinri Toutiao unveiled moves this week to allay rising concerns from the authorities.

Last week, the Beijing bureau of China’s top internet regulator accused Toutiao of “spreading pornographic and vulgar information” and “causing a negative impact on public opinion online,” and ordered that updates to several popular sections of the app be halted for 24 hours. In response, the app’s parent company, Beijing Bytedance Technology, took down or temporarily suspended the accounts of more than 1,100 bloggers that it said had been publishing “low-quality content” on the app. It also replaced Toutiao’s “Society” section with a new section called “New Era,” which is heavy on state media coverage of government decisions.

Toutiao is the vanguard of a growing movement in China. For years, citizens knew they were being tracked by the government, but now are beginning to demand privacy. We certainly hope they can get there but are mighty skeptical. Good luck!

Patrick Roland, February 21, 2018

Palantir: Accused of Hegelian Contradictions

January 29, 2018

I bet you have not thought about Hegel since you took that required philosophy course in college. Well, Hegel and his “contradictions” are central to “WEF 2018: Davos, Data, Palantir and the Future of the Internet.”

I highlighted this passage from the essay:

Data is the route to security. Data is the route to oppression. Data is the route to individual ideation. Data is the route to the hive mind. Data is the route to civic wealth. Data is the route to civic collapse.

Thesis, antitheses, synthesis in action I surmise.

The near term objective is synthesis. I assume this is the “connecting the dots” approach to finding what one needs to know.

I learned:

The stakes for big data couldn’t be bigger.

Okay, a categorical in our fast changing, diverse economic and political climate. Be afraid seems to be the message.

Palantir’s point of operations in Davos is described in the write up as “a pimped up liquor store.” Helpful and highly suggestive too.

The conclusion of the essay warranted a big red circle:

So next time you hear the names Palantir or Alex Karp, stop what you’re doing and pay attention. The future – your future – is under discussion. Under construction. This little first draft of history of which you’ve made it to the end (congratulations and thanks) – the history of data – is of a future that will in time come to be seen for what it is: digital that truly matters.

Several observations:

  • The author wants me to believe that Palantir is not a pal.
  • The big data thing troubles the author because Palantir is one of the vendors providing next generation information access.
  • The goal of making Palantir into something unique is best accomplished by invoking Fancy Dan ideas.

I would suggest that knowledge about companies like Gamma Group FinFisher, Shoghi, Trovicor, and some other interesting non US entities might put Palantir in perspective. Palantir has an operational focus; some of the other vendors perform different information services.

Palantir is an innovator, but it is part of a landscape of data intercept and analysis organizations. I could make a case that Palantir is capable but some companies in Europe and the East are actually more technologically advanced.

But these outfits were not at Davos. Why? That’s a good question. Perhaps they were too busy with their commercial and government work. My hunch is that a few of these outfits were indeed “there”, just not noticed by the expert who checked out the liquor store.

Stephen E Arnold, January 29, 2019

We Are Without a Paddle on Growing Data Lakes

January 18, 2018

The pooling of big data is commonly known as a “data lake.” While this technique was first met with excitement, it is beginning to look like a problem, as we learned in a recent Info World story, “Use the Cloud to Create Open, Connected Data Lakes for AI, Not Data Swamps.”

According to the story:

A data scientist will quickly tell you that the data lake approach is a recipe for a data swamp, and there are a few reasons why. First, a good amount of data is often hastily stored, without a consistent strategy in place around how to organize, govern and maintain it. Think of your junk drawer at home: Various items get thrown in at random over time, until it’s often impossible to find something you’re looking for in the drawer, as it’s gotten buried.

This disorganization leads to the second problem: users are often not able to find the dataset once ingested into the data lake.

So, how does one take aggregate data from a stagnant swamp to a lake one can traverse? According to Scientific Computing, the secret lies in separating the search function into two pieces, finding and searching. When you combine this thinking with Info World’s logic of using the cloud, suddenly these massive swamps are drained.

Patrick Roland, January 18, 2018

 

 

Google Tries Like Crazy to End Extreme Content Controversy

January 16, 2018

Google is having a tough time lately. When it purchased YouTube few thought extremist videos and wonky children’s programming would be its most concerning headaches. But their solutions remain strained, as we discovered in a recent Verge story, “YouTube Has Removed Thousands of Videos from Extremist Cleric Anwar Al-Awlaki.”

Google removed hundreds of al-Awalaki’s videos in 2010 which directly advocated violence, following the conviction of Roshonara Choudhry, a radicalized follower who stabbed British MP Stephen Timms earlier that year. At the time, a YouTube spokesperson cited the site’s guidelines against inciting violence. But al-Awalaki posted tens of thousands of other videos, and in subsequent years, was cited as an influence in other notable terrorist attacks at Fort Hood, the Boston Marathon, San Bernardino, and Orlando, Florida.

This comes on the heels of another Verge story with a similar issue, “YouTube Says it Will Crack Down on Bizarre Videos Targeting Children.”

We’re in the process of implementing a new policy that age restricts this content in the YouTube main app when flagged,” said Juniper Downs, YouTube’s director of policy. “Age-restricted content is automatically not allowed in YouTube Kids.” YouTube says that it’s been formulating this new policy for a while, and that it’s not rolling it out in direct response to the recent coverage.

Google is trying to do better, but it seems like they are fighting off an avalanche with a snow shovel. Luckily, as Washington Post points out, the United States leads the world in terms of big data. One can hope that a solution lies in their somewhere, but good luck predicting what it will be.

Patrick Roland, January 17, 20186

Next Page »

  • Archives

  • Recent Posts

  • Meta