Specialized Technology: Why Processing Talk Can Be Helpful to Anyone
May 7, 2021
Some specialized services companies have provided cheat sheets for audio and video intercepts. I heard that this technology was under wraps and available only to those with certain privileges. Not any longer.
An outfit at Wordcab.com can perform what once was an intelligence function for anyone with Internet access, content, and a way to pay. Navigate to Wordcab.com and sign up. The company says:
Automagically summarize all your internal meetings. Wordcab creates detailed, natural-language summaries of all your meetings and sales calls. So you can focus on people, not paper.
Thumbtypers will thrill with the use of the word “automagically.” The service can ingest a Zoom recording and generate a summary. The outputs can be tweaked, but keep in mind, this is smart software, not Maxwell Perkins reincarnated as your blue pencil toting digital servant. There’s an API so the service can be connected to whizzy distributed services and, if you have a copy of Palantir Gotham-type software, you can do some creative analysis.
The idea is that the smart software can make an iPhone toting bro or bro-ette more efficient.
The key point is that once was a secret capability is now available to anyone with an Internet connection. And to those who don’t think there is useful information in TikTok-type services. Maybe think again?
Stephen E Arnold, May 7, 2021
Did You Know You Had a LexID? No. Worth Checking Maybe
April 22, 2021
With ICE’s contract with Thomson Reuters’ CLEAR expiring, The Intercept reports, “LexisNexis to Provide Giant Database of Personal Information to ICE.” Apparently the company could not resist the $16.8 million contract despite downplaying its ties to the agency in the past. Once focused on providing data to legal researchers and law firms, reduced sales compelled LexisNexis to branch into serving law enforcement. The firm will be supplying Homeland Security agents with billions of records that aggregate data from sources both public and private, like credit histories, bankruptcy records, license plate photos, and cell phone subscriber info. Naturally, these profiles also come with analytics tools. Reporter Sam Biddle writes:
“It’s hard to wrap one’s head around the enormity of the dossiers LexisNexis creates about citizens and undocumented persons alike. While you can at least attempt to use countermeasures against surveillance technologies like facial recognition or phone tracking, it’s exceedingly difficult to participate in modern society without generating computerized records of the sort that LexisNexis obtains and packages for resale. The company’s databases offer an oceanic computerized view of a person’s existence; by consolidating records of where you’ve lived, where you’ve worked, what you’ve purchased, your debts, run-ins with the law, family members, driving history, and thousands of other types of breadcrumbs, even people particularly diligent about their privacy can be identified and tracked through this sort of digital mosaic. LexisNexis has gone even further than merely aggregating all this data: The company claims it holds 283 million distinct individual dossiers of 99.99% accuracy tied to ‘LexIDs,’ unique identification codes that make pulling all the material collected about a person that much easier. For an undocumented immigrant in the United States, the hazard of such a database is clear.”
Biddle notes that both LexisNexis and Thomson Reuters are official data partners of Palantir, which insists it is not, itself, a data company. It is, however, a crucial partner to law enforcement agencies at all levels across the US, as well as the security departments at several corporations. The firm supplies its clients, including ICE, with huge datasets, analysis tools, and consultants to help organizations track anyone of interest. Despite these partnerships, both Thomson Reuters and LexisNexis have largely escaped the controversy that has surrounded Palantir.
Biddle has trouble reconciling LexisNexis’ new contract with its insistence it is actually on the side of detainees because it supplies them with access to an e-library of legal materials. For its part, the firm takes pains to note the contract complies with President Biden’s Executive Order 13993, which revised immigration enforcement policies and DHS interim guidelines. We are reminded, though, that despite the new occupant of the Oval Office, those running ICE remain the same. It is their hands into which this astounding trove of personal data is being delivered.
Cynthia Murrell, April 22, 2021
Amazon: Putting Eyes on Humans
February 17, 2021
Amazon may have a new driver at the controls of the Bezos bulldozer, but the big orange machine keeps pushing monitoring technology. “Amazon’s Driver Monitoring App Is an Invasive Nightmare” does not like the system the online bookstore uses to keep an eye on human delivery drivers. The write up states:
Mentor is made by eDriving, which describes the app on its website as a “smartphone-based solution that collects and analyzes driver behaviors most predictive of crash risk and helps remediate risky behavior by providing engaging, interactive micro-training modules delivered directly to the driver in the smartphone app.”
From my tumble down shack in rural Kentucky, the Bezos bulldozer seems to be using technology from an outfit called eDriving. There are several options available to the online bookstore. Amazon can continue to pay eDriving. Amazon can clone the system. Amazon can acquire the company, people, or technology.
Based on my on-going research into Amazon’s surveillance capabilities, the enhanced cameras, the online hook to the AWS mothership, and the use of third-parties to nudge monitoring forward is still in its early days. Amazon moves slowly and in a low profile way. Most law enforcement and intelligence organizations observe Amazon the way a tourist does a turtle in the Galapagos: Check out where the turtle is after breakfast and then note that the darned thing moved behind a rock a few fee away by noon. No big deal. Turtles move, right? Turtles are not gazelles, right?
Several observations:
- Amazon chugs along in a sprightly manner behind the curtain separating public use of a system like Mentor
- Amazon time makes it difficult for some observers to note significant change in a system or technology
- The trick to figuring out where Amazon is headed in surveillance systems is to step back and observe the suite of systems.
What does one learn?
How about Amazon as the plumbing for many of the widely used policeware and intelware systems? Who knew that Palantir Technologies is a good Amazon customer? Maybe not IBM which inked a deal with the chipper Denver based “ride ‘em cowboy” policeware firm.
How useful would Amazon’s monitoring technology be if connected to a Palantir content intake system? My guess is that it would be quite useful, and it would require the Amazon cloud to work. What’s that mean for cloud competitors like Google, IBM, and Microsoft?
Amazon’s policeware and intelware approach is a lock in dream. Where could a Mentor-type system be useful to investigators?
Sorry. I can’t think of a single use case. Ho ho ho.
Stephen E Arnold, February 17, 2021
IBM: Emphasizing the Big in Big Blue Quantum Computing
February 12, 2021
Did you know a small outfit in China is selling a person quantum computer. Discover Magazine reveals this in “A Desktop Quantum Computer for Just $5,000.” This means quantum computers will be crunching Excel spreadsheets for those with terminal spreadsheet fever.
But one must think big. I read “IBM Promises 100x Faster Quantum Computers through New Software Foundations.” The write up explains that Big Blue has gone big, quantumly speaking, of course:
IBM unveiled on Wednesday improvements to quantum computing software that it expects will increase performance of its complex machines by a factor of 100, a development that builds on Big Blue’s progress in making the advanced computing hardware. In a road map, the computing giant targeted the release of quantum computing applications over the next two years that will tackle challenges such as artificial intelligence and complex financial calculations. And it’s opening up lower level programming access that it expects will lead to a better foundation for those applications.
Imagine how much better Watson will perform with more quantum horsepower at its disposal.
But there’s more. The write up explains in a content marketing manner:
IBM is working on increasing the number of qubits in its quantum computers, from 27 in today’s “Falcon” to 1,121 in its “Condor” systems due in 2023. IBM expects in 2024 to investigate a key quantum computing technology called error correction that could make qubits much more stable and therefore capable, Jay Gambetta, IBM’s quantum computing vice president, said in a video.
And the source of this revelation? IBM, of course. The future is just two years away. Sounds good. Now how about revenue growth, explaining how the Palantir tie up will work, and when Watson will deliver on that promise of a billion in revenue from cognitive computing?
Stephen E Arnold, January 12, 2021
Open Source Software: The Community Model in 2021
January 25, 2021
I read “Why I Wouldn’t Invest in Open-Source Companies, Even Though I Ran One.” I became interested in open source search when I was assembling the first of three editions of Enterprise Search Report in the early 2000s. I debated whether to include Compass Search, the precursor to Shay Branon’s Elasticsearch reprise. Over the years, I have kept my eye on open source search and retrieval. I prepared a report for an the outfit IDC, which happily published sections of the document and offering my write ups for $3,000 on Amazon. Too bad IDC had no agreement with me, managers who made Daffy Duck look like a model for MBAs, and a keen desire to find a buyer. Ah, the book still resides on one of my back of drives, and it contains a run down of where open source was getting traction. I wrote the report in 2011 before getting the shaft-a-rama from a mid tier consulting firm. Great experience!
The report included a few nuggets which in 2011 not many experts in enterprise search recognized; for instance:
- Large companies were early and enthusiastic adopters of open source search; for example Lucene. Why? Reduce costs and get out of the crazy environment which put Fast Search & Transfer-type executives in prison for violating some rules and regulations. The phrase I heard in some of my interviews was, “We want to get out of the proprietary software handcuffs.” Plus big outfits had plenty of information technology resources to throw at balky open source software.
- Developers saw open source in general and contributing to open source information retrieval projects as a really super duper way to get hired. For example, IBM — an early enthusiast for a search system which mostly worked — used the committers as feedstock. The practice became popular among other outfits as well.
- Venture outfits stuffed with oh-so-technical MBAs realized that consulting services could be wrapped around free software. Sure, there were legal niceties in the open source licenses, but these were not a big deal when Silicon Valley super lawyers were just a text message away.
There were other findings as well, including the initiatives underway to embed open source search, content processing, and related functions into commercial products. Attivio (formed by former super star managers from Fast Search & Transfer), Lucid Works, IBM, and other bright lights adopted open source software to [a] reduce costs, [b] eliminate the R&D required to implement certain new features, and [c] develop expensive, proprietary components, training, and services.
Mobile and Social Media Users: Check Out the Utility of Metadata
January 15, 2021
Policeware vendors once commanded big, big bucks to match a person of interest to a location. Over the last decade prices have come down. Some useful products cost a fraction of the industrial strength, incredibly clumsy tools. If you are thinking about the hassle of manipulating data in IBM or Palantir products, you are in the murky field of prediction. I have not named the products which I think are the winners of this particular race.
Source: https://thepatr10t.github.io/yall-Qaeda/
The focus of this write up is the useful information derived from the deplatformed Parler social media outfit. An enterprising individual named Patri10tic performed the sort of trick which Geofeedia made semi famous. You can check the map placing specific Parler uses in particular locations based on their messages at this link. What’s the time frame? The unusual protest at the US Capitol.
The point of this short post is different. I want to highlight several points:
- Metadata can be more useful than the content of a particular message or voice call
- Metadata can be mapped through time creating a nifty path of an individual’s movements
- Metadata can be cross correlated with other data. (If you attended one of my Amazon policeware lectures, the cross correlation figures prominently.)
- Metadata can be analyzed in more than two dimensions.
To sum up, I want to remind journalists that this type of data detritus has enormous value. That is the reason third parties attempt to bundle data together and provide authorized users with access to them.
What’s this have to do with policeware? From my point of view, almost anyone can replicate what systems costing as much as seven figures a year or more from their laptop at an outdoor table near a coffee shop.
Policeware vendors want to charge a lot. The Parler analysis demonstrates that there are many uses for low or zero cost geo manipulations.
Stephen E Arnold, January 15, 2021
DarkCyber for January 12, 2021, Now Available
January 12, 2021
DarkCyber is a twice-a-month video news program about online, the Dark Web, and cyber crime. You can view the video on Beyond Search or at this YouTube link.
The program for January 12, 2021, includes a featured interview with Mark Massop, DataWalk’s vice president. DataWalk develops investigative software which leapfrogs such solutions as IBM’s i2 Analyst Notebook and Palantir Gotham. In the interview, Mr. Massop explains how DataWalk delivers analytic reports with two or three mouse clicks, federates or brings together information from multiple sources, and slashes training time from months to several days.
Other stories include DarkCyber’s report about the trickles of information about the SolarWinds’ “misstep.” US Federal agencies, large companies, and a wide range of other entities were compromised. DarkCyber points out that Microsoft’s revelation that bad actors were able to view the company’s source code underscores the ineffectiveness of existing cyber security solutions.
DarkCyber highlights remarkable advances in smart software’s ability to create highly accurate images from poor imagery. The focus of DarkCyber’s report is not on what AI can do to create faked images. DarkCyber provides information about how and where to determine if a fake image is indeed “real.”
The final story makes clear that flying drones can be an expensive hobby. One audacious drone pilot flew in restricted air zones in Philadelphia and posted the exploits on a social media platform. And the cost of this illegal activity. Not too much. Just $182,000. The good news is that the individual appears to have avoided one of the comfortable prisons available to authorities.
One quick point: DarkCyber accepts zero advertising and no sponsored content. Some have tried, but begging for dollars and getting involved in the questionable business of sponsored content is not for the DarkCyber team.
Finally, this program begins our third series of shows. We have removed DarkCyber from Vimeo because that company insisted that DarkCyber was a commercial enterprise. Stephen E Arnold retired in 2017, and he is now 77 years old and not too keen to rejoin the GenX and Millennials in endless Zoom meetings and what he calls “blatant MBA craziness.” (At least that’s what he told me.)
Kenny Toth, January 12, 2021
LinkedIn Reveals Disinterest in Search and Retrieval
December 7, 2020
LinkedIn does quite a bit of info-ramming when either one of my team or I log in to the Microsoft social media system. Here’s the graphic displayed when we were checking to see if our automated posts from this blog were appearing:
The eight “cards” tell me about LinkedIn Groups in which I may have an interest. The little boxes reveal a small amount of information about the content access topics in which the unemployed, the consultants cruising for gigs, and the self-promoters have an interest.
The table below presents some of the data in this graphic in tabular form. No, I did not use Excel 365 connected to Teams. Sorry, Mother Microsoft. I still recall Bob. (You remember Bob, don’t you, gentle reader?)
| LinkedIn Group Name | Number of LinkedIn Followers |
| Data Science Central | 374,694 |
| Association for Intelligent Information Management | 27,861 |
| Scientific, Technical, Medical Publishing Group | 12,253 |
| Data & Text Analytics Professionals | 12,503 |
| Special Libraries Asso. | 15,191 |
| Semantic Web | 15,098 |
| Semantic Technologies Group | 3,772 |
| Enterprise Search & Discovery | 624 |
LinkedIn does not reveal the hard count for its total number of registered humans, the number of human users who log on to the system once per week, or the number of paying human users. Hence, figuring out the percentage of LinkedIn members interested in these groups is a difficult task akin to predicting the share price of Palantir Technologies on January 1, 2022.
An outfit called Oberlo reports with confidence that LinkedIn has 660 million users. Close enough for horseshoes.
The table below presents the percentage of these LinkedIn users interested in each the groups suggested to me:
| LinkedIn Group Name | Percentage of LinkedIn Members Interested in These Topics |
| Data Science Central | 0.0567718182% |
| Association for Intelligent Information Management | 0.0042213636% |
| Scientific, Technical, Media Publishing Group | 0.0018565152% |
| Data & Text Analytics Professionals | 0.0018943939% |
| Special Libraries Asso. | 0.0023016667% |
| Semantic Web | 0.0022875758% |
| Semantic Technologies Group | 0.0005715152% |
| Enterprise Search & Discovery | 0.0000945455% |
Eyeballing my math, surely there are errors. How can such a compelling subject as Enterprise Search & Discovery appeal to 0.0000945455 percent of the LinkedIn members.
What’s interesting is that an astounding 0.0042213636 percent of the LinkedIn membership are pulled to the Association for Intelligent Information Management.
And the semantic topics. Magnetic indeed.
What’s the analysis suggest? Anyone looking for a job in enterprise search may want to spin their expertise a different way.
Stephen E Arnold, December 7, 2020
Why Investigative Software Is Expensive
December 3, 2020
In a forthcoming interview, I explore industrial-strength policeware and intelware with a person who was Intelligence Officer of the Year. In that review, which will appear in a few weeks, the question of cost of policeware and intelware is addressed. Systems like those from IBM’s i2, Palantir Technologies, Verint, and similar vendors are pricey. Not only is there a six or seven figure license fee, the client has to pay for training, often months of instruction. Plus, these i2-type systems require systems and engineering support. One tip off of to the fully loaded costs is the phrase “forward deployed engineer.” The implicit message is that these i2-type systems require an outside expert to keep the digital plumbing humming along. But who is responsible for the data? The user. If the user fumbles the data bundle, bad outputs are indeed possible.
What’s the big deal? Why not download Maltego? Why not use one of the $100 to $3,000 solutions from jazzy startups by former intelligence officers? These are “good enough”, some may assert. One facet of the cost of industrial strength systems available to qualified licensees is a little appreciated function: Dealing with data.
“Keep Data Consistency During Database Migration” does a good job of explaining what has to happen in a reliable, consistent way when one of the multiple data sources contributes “new” or “fresh” data to an intelware or policeware system. The number of companies providing middleware to perform these functions is growing. Why?
Most companies wanting to get into the knowledge extraction business have to deal with the issues identified in the article. Most organizations do not handle these tasks elegantly, rapidly, or accurately.
Injecting incorrect, stale, inaccurate data into a knowledge centric process like those in industrial strength policeware causes those systems to output unreliable results.
What’s the consequence?
Investigators and analysts learn to ignore certain outputs.
Why? The outputs can be more serious than a flawed diagram whipped up by an MBA who worries only about the impression he or she makes on a group of prospects attending a Zoom meeting.
Data consistency is a big deal.
Stephen E Arnold, December 2, 2020
Amazon and the Cyber Security Industrial Complex
November 24, 2020
This is probably no big deal. Cyber security, threat intelligence, and wonky proprietary tools from startups populated by retired or RIFed intel officers are a big business. I was asked by a “real news” reporter, “How big?” I dutifully sent links to companies selling market forecasts for global cyber security revenues. How big were these numbers? Acquisition big. The hypothesis I have formulated is that when wild and crazy market size projections fly like hungry sparrows, there is a revenue problem. Specifically there are too many sparrows chasing available bugs and bread crumbs. That’s why Blackberry is in the cyber security business. Why LookingGlass stepped away from Cyveillance. That’s why Dark Web indexes of bad actors’ Crime as a Service offerings are a dime a dozen.
It is, therefore, no surprise that the write up “Trend Micro integrates with AWS Network Firewall” explains that Amazon is continuing to add to its pool of 65,000 plus partners. Many of these outfits like Palantir Technologies are in the cyber intelligence and cyber threat business. Bad actors beware.
The write up reports:
Trend Micro’s built-in IPS intelligence will inspect traffic for malicious intent so that the firewall can stop threats before they get a foothold in a virtual private cloud. Together, AWS and Trend Micro offer a simple, scalable service with reliable protection that does not require any infrastructure management.
What’s the hook? Here’s the statement I circled with an Amazon happy face:
Trend Micro’s threat intelligence will be available free with easy deployment for AWS Network Firewall customers.
What do I make of free cyber security services? No much but I hear the Bezos bulldozer pulling into the cyber intelligence and security services shopping mall. Roll up or roll over time for the cheerful orange machine with a big smile painted on the cab.
Stephen E Arnold, November 24, 2020
-
- Subscribe to Beyond Search
Feature archive
News archive
-
