Palantir Technologies: Recycling Day Old Hash
July 31, 2017
I read “Palantir: The Special Ops Tech Giant That Wields As Much Real World Power as Google.” I noticed these hot buttons here:
“Special ops” for the Seal Team 6 vibe. Check.
“Wields” for the notion of great power. Check.
“Real world.” A reminder of the here and now, not an airy fairy digital wonkiness. Check.
“Google.” Yes. Palantir as potent as the ad giant Google. Check.
That’s quite a headline.
The write up itself is another journalistic exposé of software which ingests digital information and outputs maps, reports, and visualizations. Humans index too. Like the i2 Analyst Notebook, the “magic” is mostly external. Making these Fancy Dan software systems work requires computers, of course. Humans are needed too. Trained humans are quite important, essential, in fact.
The Guardian story seems to be a book review presented as a Gladwell-like revisionist anecdote. See, for example, Done: The Secret Deals That Are Changing Our World by Jacques Peretti (Hodder & Stoughton, £20). You can buy a copy from bookshop.theguardian.com. (Online ad? Maybe?)
Read the Palantir story which stuffed my Talkwalker alert with references to the article. Quite a few bloggers are recycling the Guardian newspaper story. Buzzfeed’s coverage of the Palo Alto company evoked the same reaction. I will come back to the gaps in these analyses in a moment.
The main point of the Guardian’s July 30, 2017, story strikes me as:
Palantir tracks everyone from potential terrorist suspects to corporate fraudsters…child traffickers, and what they refer to as subversives. But it all done using prediction.
Right. Everyone! Potential terrorist suspects! And my favorite “all”. Using “prediction” no less.
Sounds scary. I am not sure the platforms work with the type of reliability that the word “all” suggests. But this is about selling books, not Palantir and similar companies’ functionality, statistical methods, or magical content processing. Confusing Hollywood with reality is easy today: At least for some folks.
Palantir licenses software to organizations. Palantir is an “it,” not a they. The company uses the lingo of its customers. Subversives is one term, but it is more suggestive in my opinion than “bad actor,” “criminal,” “suspect,” or terrorist.” I think the word “tracks” is pivotal. Palantir’s professionals, like Pathfinder, look at deer tracks and nails the beastie. I want to point out that “prediction”—partly the Bayesian, Monte Carlo, and Markovian methods pioneered by Autonomy in the mid 1990s—is indeed used for certain processes. What’s omitted is that Palantir is just one company in the content processing and search and retrieval game. I am not convinced that its systems and methods are the best ones available today. (Check out Recorded Future, a Google and In-Q-Tel funded company for some big league methods. And there are others. In my CyberOSINT book and my Dark Web Notebook I identify about two dozen companies providing similar services. Palantir is one, admittedly high profile example, of next generation information access providers.
The write up does reveal at the end of the article that the Guardian is selling Jacque Peretti’s book. That’s okay. What’s operating under the radar is a book promo that seems to be one thing but is, in the real world, a nifty book promotion.
In closing, the information presented in the write up struck me as a trifle stale. I am okay with collections of information that have been assembled to make it easy for a reader to get the gist of a system quickly. My Dark Web Notebook is a Cliff’s Notes about what one Tor executive suggests does not exist.
When I read about Palantir, I look for information about:
- Technical innovations within Gotham and Palantir’s other “products”
- Details about the legal dust up between i2 and Palantir regarding file formats, an issue which has some here and now relevance with the New York police department’s Palantir experience
- Interface methods which are designed to make it easier to perform certain data analysis functions
- Specifics about the data loading, file conversion, and pre-processing index tasks and how these impact timeliness of the information in the systems
- Issues regarding data reconciliation when local installs lose contact with cloud resources within a unit and across units
- Financial performance of the company as it relates to stock held by stakeholders and those who want the company to pursue an initial public offering
- What are the specific differences among systems on offer from BAE, Textron, and others with regards to Palantir Gotham?
Each time I read about Palantir these particular items seem to be ignored. Perhaps these are not sufficiently sexy or maybe getting the information is a great deal of work? The words “hash” and “rehash” come to my mind as one way to create something that seems filling but may be empty calories. Perhaps a “real journalist” will tackle some of the dot points. That would be more interesting than a stale reference to special effects in a star vehicle.
NB. I was an adviser to i2 Group Ltd., the outfit that created the Analyst’s Notebook.
Stephen E Arnold, July 31, 2017
Deep Text: A Review of an Important New Book by Tom Reamy
July 5, 2017
You have reached a special page. To continue, click here.
Stephen E Arnold: Dark Web Use Expected to Increase
June 27, 2017
Author predicts filtering and other restrictions on the open Internet will push more users toward secret encrypted platforms
Despite stepped-up efforts by federal and local law enforcement agencies, the Dark Web and the contraband markets that thrive there will continue to grow in the coming years. That’s the conclusion shared by author and consultant Stephen E Arnold in his new book Dark Web Notebook, a practical guide for law enforcement, intelligence, and corporate security personnel.
Arnold was a featured speaker this month at the at the Techno Security & Digital Forensics Conference in Myrtle Beach, SC, where he explained how the Dark Web operates and how investigators can assume a secret identity and acquire the necessary tools to explore hidden online communities.
In his lectures, Arnold noted that governments in the UK, China, Egypt, and other countries are implementing tighter restrictions on electronic communications while private companies such as Facebook and Google are implementing more content filtering. The result, he said, is that more activities that are normally conducted on the “surface web” are migrating to the encrypted platforms offered by the Dark Web. At the same time, private citizens are adopting apps and other encryption tools to protect their personal privacy.
“It’s like squeezing a sponge or a tube of toothpaste,” Arnold said. “The substance has to go somewhere. The Dark Web will become more attractive to people who want to conduct their affairs in secret and that will make the Dark Web an increasing area of interest to law enforcement, security and intelligence professionals.”
The Dark Web is also flashing on the radar of state and local law enforcement agencies. A June 10 article in the New York Times (https://www.nytimes.com/2017/06/10/business/dealbook/opioid-dark-web-drug-overdose.html) described how police in the resort town of Park City, UT, had to learn about the Dark Web after two 13-year-old boys died after taking a synthetic opioid called “Pinky” that was purchased on the Dark Web.
“This terrible tragedy is an example of how the influence of the Dark web is spreading into small town in the US and around the world. Law enforcement personnel at all levels are going to need to understand how this new environment functions if they hope to protect the citizens in their communities,” Arnold said.
Although much has been written about the Dark Web, Arnold’s book is unique in that it was created specifically for investigators. It presents step-by-step instructions that help investigators safely explore the digital underworld. The book is available as a 250-page digital download for $49 at https://gum.co/darkweb.
In addition to his lectures, Arnold also offers a hands-on tutorial for small groups of qualified participants. In these sessions, participants use a secure computer to create a “legend” identity and explore the Dark We. They also learn how to acquire other digital tools and services including Bitcoin, the digital currency favored by Dark Web vendors.
Arnold will present his lectures and tutorial again at the 2017 Techno Security & Digital Forensics Conference September 18-20 in San Antonio. Information about the conference is available at www.technosecurity.us.
About the Book
The 250-page book includes a tour of the Dark Web with profiles of vendors and markets, explanations of access tools such as the Tor browser, reviews of services for searching Dark Web content, research resources available on the public Internet, and details about commercial solutions.
Detailed descriptions of each chapter are posted at http://xenky.com/darkwebnotebook.
The book is available as a digital download for $49 at https://gum.co/darkweb.
About the Author
Stephen E Arnold’s career includes work at Halliburton Nuclear Utility Services and Booz, Allen & Hamilton. He built and sold several stary-up ventures including The Point Internet Service. He and his team built the online system for the Threat Open Intelligence Gateway (TOSIG) for the US government. He is the author of The Google Legacy (2005), Google Version 2 (2007), and Google: The Digital Gutenberg (2009). In 2015, he published CyberOSINT: Next Generation Information Access. He is a Summa cum laude graduate of Bradley University and he has completed work on his PhD at the University of Illinois.
About ArnoldIT
Arnold Information Technology conducts research into online information and services. Founded by Stephen E Arnold in 1991, the company has provided technology services to organizations worldwide.
Ric Manning, June 27, 2017
Google Books Is Not Violating Copyright
November 12, 2015
Google Books was controversial the moment it was conceived. The concept is simple and effective though: books in academic libraries are scanned and snippets are made available online. People have the ability to search Google Books for specific words or phrases, then they are shown where it is contained within a book. The Atlantic wrote, “After Ten Years, Google Books Is Legal” about how a Second Circuit judge panel ruled in favor of Google Books against the Authors Guild.
The panel ruled that Google Books fell under the terms of “Fair Use,” which as most YouTubers know, is the ability to use a piece of copyrighted content within a strict set of rules. Fair usage includes works of parody, academic works, quotations, criticism, or summarization.
The Authors Guild argued that Google Books was infringing upon its members copyrights and stealing potential profits, but anyone knows that too much of a copyright is a bad thing. It places too many limitations on how the work can be used, harming the dissemination of creative and intellectual thought.
“’It gives us a better senses of where fair use lies,” says Dan Cohen, the executive director of the Digital Public Library of America. They “give a firmer foundation and certainty for non-profits…Of all the parts of Judge Leval’s decision, many people I talked to were happiest to see that it stressed that fair use’s importance went beyond any tool, company, or institution. ‘To me, I think a muscular fair use is an overall benefit to society, and I think it helps both authors and readers,’ said Cohen.”
Authors do have the right to have their work copyright and make a profit off it, which should be encouraged and a person’s work should not be given away for free. There is a wealth of information out there, however, that is kept under lock and key and otherwise would not be accessed with a digital form. Google Books only extends a book’s reach, speaking from one who has relied on it for research.
Whitney Grace, November 12, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Scribd Obtains Social Reading
July 22, 2015
Access to books and other literary material has reached an unprecedented high. People can download and read millions of books with a few simple clicks. Handheld ebook readers are curtailing the sales of printed book, but they also are increasing sales of digital books. One of the good things about ebooks is bibliophiles do not have to drive to a bookstore or get waitlisted on the library. Writers also can directly sell their material to readers and potentially by pass having to pay agents and publishers.
It occurred to someone that bibliophiles would love to have instant access to a huge library of books, similar to how Netflix offers its customers an unending video library. There is one and it is called Scribed. Scribd is described as the Netflix of books, because for a simple $8.99 bibliophiles can read and download as many books as they wish.
The digital landscape is still being tested by book platforms and Scribd has increased its offerings. VentureBeat reports Scribd’s newest business move in: “Scribd Buys Social Reading App Librify.” Librify is a social media reading app, offering users the opportunity to connect with friends and sharing their reading experiences. It is advertised as a great app for book clubs.
“In a sparse press release, Scribd argues Librify’s “focus on the social reading experience” made the deal worthwhile. The news arrives at a heated time for the publishing industry, as Amazon, Oyster, and others all fight to be the definitive Netflix for books — all while hawking remarkably similar products.”
Netflix has its own rivals: Hulu, Amazon Prime, Vimeo, and YouTube, but it offers something different by creating new and original shows. Scribd might be following a similar business move, by offering an original service its rivals do not have. Will it also offer Scribd only books?
Whitney Grace, July 22, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Books about Math
January 21, 2015
We’ve run across a list of books that should interest to anyone who would like to understand more about mathematics. These are not textbooks with which to expand our math skills, but rather volumes that take a look at the mathematics field itself. Blogger Kelly J. Rose shares “5 Insanely Great Books About Mathematics You Should Read.” Rose writes:
“I’ve been asked over and over for good books about mathematics for a layperson, someone who hasn’t taken advanced courses in university and is more simply interested in learning about what math is, and some of the more interesting historical figures and results from mathematics. Ironically, when you are a mathematics major at Waterloo, you get the opportunity in 4th year to take a course on the history of mathematics and you get introduced to a few really good books that start to explain the mindset and philosophy behind mathematics and not simply just the theorems and proofs. Here are the 5 books about I most recommend to those who want to understand the mathematical mind and philosophy.”
A few highlights: for a comprehensive history of the field, there’s A History of Mathematics by Carl B. Boyer. For an understanding of what it is like to live the life of a mathematician, it seems Rose cannot recommend The Mathematical Experience by Philip J. David and Reuben Hersh highly enough. Then there’s Proofs and Refutations by Imre Lakatos; Rose says this is likely the most advanced book on his list, yet calls it a quick read. He prescribes it to anyone considering a career in mathematics. Check out the post for more recommendations.
Cynthia Murrell, January 21, 2015
Sponsored by ArnoldIT.com, developer of Augmentext
How to Turnaround a Failing Company
September 17, 2013
Jonathan H. Lack has been an associate of ArnoldIT since 1996. His new monograph is Plan to Turn Your Company Around in 90 Days. We recommend this practical and pragmatic guide for managers struggling with shifting economic winds.
Mr. Lack said:
“Every company’s financial and operational situation, culture, and dynamics are different. However, the fundamentals of operating any business and the problems to which many companies are vulnerable are not that unique. This entire book is based on firsthand experience if helping different types of companies work through very similar problems.”
HighGainBlog said:
This book is written for businesses large and small as well as for CEOs, board members, and managers. Lack’s expertise comes from his role as principal for ROI Ventures, which specializes in turning companies around. He also has 20 years of experience in management and strategic planning. This expertise shines through as he offers sound advice ranging from effectively managing cash-flow to managing staff. We highly recommend this book to drowning professionals looking for a lifeline as well as those interested in injecting new life into their business and gain a few valuable insights along the way.
Plan to Turn Your Company Around in 90 Days is available for purchase online at Amazon.com or at Apress.com under ISBN13: 978-1-4302-4668-8. Order a copy if you are involved in search, content processing, and analytics. This industry sector faces increased cost of sales, long sales cycles, hard-to-control costs, and challenging revenue targets.
Stephen E Arnold, September 17, 2013
Enterprise Search by White Is Vital Tool in Business Search Management
January 24, 2013
Martin White, information management consultant and Managing Director of Intranet Focus Ltd., is one of the leading experts on enterprise search and information access. White has published seven books on topics surrounding information consultancy and enterprise search applications. His most recent publication, Enterprise Search: Enhancing Business Performance, focuses on how to plan and implement a managed search environment in your corporation. The book explains how to meet both the needs of your business and your employees.
White makes a clear business case for search, emphasizing the need to evaluate current search systems and the creation of a support team. The book is well organized and easy to read, with a thorough preface giving an overview of chapters and topics as well as simplified summaries at the end of each chapter. This style makes White’s recent book a great tool for the busy professional.
Chapter 12 is a recommended starting place, listing twelve critical success factors. White states that if you don’t meet at least eight of these twelve, which include investing in a search support team, getting the best out of your current investment in search, and providing location-independent search, then you definitely need the contents of this book.
In Chapter 10, titled Managing Search, White expands on the idea of managing a search support team:
“Implementing search should never be ‘a project’. The work of ensuring that users continue to have high levels of search satisfaction will never come to a close. Each week, and perhaps even most days, there will be something that needs attention. The role of the search support team is not just to be reactive but to anticipate when changes to the search application need to be made, or to identify a training requirement that will address an issue that is just starting to show up on the search logs and user satisfaction surveys.”
Most organizations are not prepared for the rate of growth of information that they are experiencing. White does a great job dissecting the need for enterprise search and then giving you the tools to successfully manage your system, based on far more than just available technology. The section on the future of enterprise search, Chapter 11, stood out to me. White makes an excellent case for why this topic can no longer be ignored.
Additional features include a thorough glossary, lists of books and blogs on information retrieval and enterprise search, and resources for further reading. The book is available here from O’Reilly Media in eBook and print formats. Highly recommended.
Andrea Hayden, January 24, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
The Internet: Wonderland or Wasteland?
February 7, 2012
For years the Internet has increasingly become the go-to source for information, networking, shopping, and socializing. You can read the news on any major news outlet, check the weather for the day, send messages to friends and family, and buy anything your heart desires, all before getting out of bed or having your first cup of coffee, and all at the touch your fingertips.
The Internet can easily be considered the greatest medium for information, and it has completely transformed the way we communicate, the way we shop, and even the way we learn. While the Internet can certainly be considered a convenience, is there ever a point when too much information or too much access is a problem?
In the recent report “Are We on Information Overload?” posted on Salon.com, Thomas Rogers interviews David Weinberger, author of recently published Too Big to Know, on the impact the Internet has on knowledge today and if the vastness of readily available information is too much for our minds to handle. According to Weinberger:
Ask anybody who is in any of the traditional knowledge fields, and she or he will very likely tell you that the Internet has made them smarter. They couldn’t do their work without it; they’re doing it better than ever before, they know more; they can find more; they can run down dead ends faster than ever before. In the sciences and humanities, it’s hard to find somebody who claims the Internet is making him or her stupid, even among those who claim the Internet is making us stupid….Curiosity can lead you to lots of bad directions. It can steer you wrong and waste your time, but it is fundamental. We need it more than anything else if we’re going to try to understand our world. Now we have a medium that is as broad as our curiosity.
I really don’t know if I can say that the Internet has made me smarter (especially with the constant flow of nonsense I expose myself to on a regular basis on social networking sites), but it’s certainly made doing my job easier on many occasions.
The information you can find on the Internet is limitless, and if you can’t find something, you have the ability to put it there so the next person can find it. As Weinberger suggests, it is a great tool for the curious mind. If the curious mind suffers from lack of focus, though, then I would imagine overload is a definite possibility.
Stacey Duwe, February 7, 2012
Sponsored by Pandia.com
==
New Oracle E-Book
February 3, 2012
Oracle brews a data latté with cinnamon you may enjoy if you if you suck down this new book. The Salvatoreyc blog presents for download, “Oracle XSQL: Combining SQL, Oracle Text, XSLT and Java to Publish Dynamic Web Content.” The product description states:
Welcome to the exciting world of eXtended Structured Query Language (XSQL). ‘Oracle XSQL: Combining SQL, Oracle Text, XSLT and Java to Publish Dynamic Web Content’ presents a complete approach to building XML Web applications and Web services with XSQL, Oracle Text, SQL, XSLT, and Java from data found in Oracle databases. Companion Web site contains the code examples in the book as well.
Sounds helpful. The download is only available to Media Search members, and the download page takes you right to their signup sheet. Naturally, there is a fee.
Cynthia Murrell, February 3, 2012
Sponsored by Pandia.com
-
- Subscribe to Beyond Search
Feature archive
News archive
-
