Synthetic Content: A Challenge with No Easy Answer
January 30, 2023
Open source intelligence is the go-to method for many crime analysts, investigators, and intelligence professionals. Whether social media or third-party data from marketing companies, useful insights can be obtained. The upside of OSINT means that many of its supporters downplay or choose to sidestep its downsides. I call this “OSINT blindspots”, and each day I see more information about what is becoming a challenge.
For example, “As Deepfakes Flourish, Countries Struggle with Response” is a useful summary of one problem posed by synthetic (fake) content. What looks “real” may not be. A person sifting through data assumes that information is suspect. Verification is needed. But synthetic data can output multiple instances of fake information and then populate channels with “verification” statements of the initial item of information.
The article states:
Deepfake technology — software that allows people to swap faces, voices and other characteristics to create digital forgeries — has been used in recent years to make a synthetic substitute of Elon Musk that shilled a crypto currency scam, to digitally “undress” more than 100,000 women on Telegram and to steal millions of dollars from companies by mimicking their executives’ voices on the phone. In most of the world, authorities can’t do much about it. Even as the software grows more sophisticated and accessible, few laws exist to manage its spread.
For some government professionals, the article says:
problematic applications are also plentiful. Legal experts worry that deepfakes could be misused to erode trust in surveillance videos, body cameras and other evidence. (A doctored recording submitted in a British child custody case in 2019 appeared to show a parent making violent threats, according to the parent’s lawyer.) Digital forgeries could discredit or incite violence against police officers, or send them on wild goose chases. The Department of Homeland Security has also identified risks including cyber bullying, blackmail, stock manipulation and political instability.
The most interesting statement in the essay, in my opinion, is this one:
Some experts predict that as much as 90 per cent of online content could be synthetically generated within a few years.
The number may overstate what will happen because no one knows the uptake of smart software and the applications to which the technology will be put.
Thinking in terms of OSINT blindspots, there are some interesting angles to consider:
- Assume the write up is correct and 90 percent of content is authored by smart software, how does a person or system determine accuracy? What happens when a self learning system learns from itself?
- How does a human determine what is correct or incorrect? Education appears to be struggling to teach basic skills? What about journals with non reproducible results which spawn volumes of synthetic information about flawed research? Is a person, even one with training in a narrow discipline, able to determine “right” or “wrong” in a digital environment?
- Are institutions like libraries being further marginalized? The machine generated content will exceed a library’s capacity to acquire certain types of information? Does one acquire books which are “right” when machine generated content produces information that shouts “wrong”?
- What happens to automated sense making systems which have been engineered on the often flawed assumption that available data and information are correct?
Perhaps an OSINT blind spot is a precursor to going blind, unsighted, or dark?
Stephen E Arnold, January 30, 2023
The LaundroGraph: Bad Actors Be On Your Toes
January 20, 2023
Now here is a valuable use of machine learning technology. India’s DailyHunt reveals, “This Deep Learning Technology Is a Money-Launderer’s Worst Nightmare.” The software, designed to help disrupt criminal money laundering operations, is the product of financial data-science firm Feedzai of Portugal. We learn:
“The Feedzai team developed LaundroGraph, a self-supervised model that might reduce the time-consuming process of assessing vast volumes of financial interactions for suspicious transactions or monetary exchanges, in a paper presented at the 3rd ACM International Conference on AI in Finance. Their approach is based on a graph neural network, which is an artificial neural network or ANN built to process vast volumes of data in the form of a graph.”
The AML (anti-money laundering) software simplifies the job of human analysts, who otherwise must manually peruse entire transaction histories in search of unusual activity. The article quotes researcher Mario Cardoso:
“Cardoso explained, ‘LaundroGraph generates dense, context-aware representations of behavior that are decoupled from any specific labels.’ ‘It accomplishes this by utilizing both structural and features information from a graph via a link prediction task between customers and transactions. We define our graph as a customer-transaction bipartite graph generated from raw financial movement data.’ Feedzai researchers put their algorithm through a series of tests to see how well it predicted suspicious transfers in a dataset of real-world transactions. They discovered that it had much greater predictive power than other baseline measures developed to aid anti-money laundering operations. ‘Because it does not require labels, LaundroGraph is appropriate for a wide range of real-world financial applications that might benefit from graph-structured data,’ Cardoso explained.”
For those who are unfamiliar but curious (like me), navigate to this explanation of bipartite graphs. The future applications Cardoso envisions include detecting other financial crimes like fraud. Since the researchers intend to continue developing their tools, financial crimes may soon become much trickier to pull off.
Cynthia Murrell, January 20, 2022
The Intelware Sector: In the News Again
January 13, 2023
It’s Friday the 13th. Bad luck day for Voyager Labs, an Israel-based intelware vendor. But maybe there is bad luck for Facebook or Meta or whatever the company calls itself. Will there be more bad luck for outfits chasing specialized software and services firms?
Maybe.
The number of people interested in the savvy software and systems which comprise Israel’s intelware industry is small. In fact, even among some of the law enforcement and intelligence professionals whom I have encountered over the years, awareness of the number of firms, their professional and social linkages, and the capabilities of these systems is modest. NSO Group became the poster company for how some of these systems can be used. Not long ago, the Brennan Center made available some documents obtained via legal means about a company called Voyager Labs.
Now the Guardian newspaper (now begging for dollars with blue and white pleas) has published “Meta Alleges Surveillance Firm Collected Data on 600,000 Users via Fake Accounts.” the main idea of the write up is that an intelware vendor created sock puppet accounts with phony names. Under these fake identities, the investigators gathered information. The write up refers to “fake accounts” and says:
The lawsuit in federal court in California details activities that Meta says it uncovered in July 2022, alleging that Voyager used surveillance software that relied on fake accounts to scrape data from Facebook and Instagram, as well as Twitter, YouTube, LinkedIn and Telegram. Voyager created and operated more than 38,000 fake Facebook accounts to collect information from more than 600,000 Facebook users, including posts, likes, friends lists, photos, comments and information from groups and pages, according to the complaint. The affected users included employees of non-profits, universities, media organizations, healthcare facilities, the US armed forces and local, state and federal government agencies, along with full-time parents, retirees and union members, Meta said in its filing.
Let’s think about this fake account thing. How difficult is it to create a fake account on a Facebook property. About eight years ago as a test, my team created a fake account for a dog — about eight years ago. Not once in those eight years was any attempt to verify the humanness or the dogness of the animal. The researcher (a special librarian in fact) set up the account and demonstrated to others on my research team how the Facebook sign up system worked or did not work in this particularly example. Once logged in, faithful and trusting Facebook seemed to keep our super user logged into the test computer. For all I know, Tess is still logged in with Facebook doggedly tracking her every move. Here’s Tess:
Tough to see that Tess is not a true Facebook type, isn’t it?
Is the accusation directed at Voyager Labs a big deal? From my point of view, no. The reason that intelware companies use Facebook is that Facebook makes it easy to create a fake account, exercises minimal administrative review of registered user, and prioritizes other activities.
I personally don’t know what Voyager Labs did or did not do. I don’t care. I do know that other firms providing intelware have the capability of setting up, managing, and automating some actions of accounts for either a real human, an investigative team, or another software component or system. (Sorry, I am not at liberty to name these outfits.)
Grab your Tum’s bottle and consider these points:
- What other companies in Israel offer similar alleged capabilities?
- Where and when were these alleged capabilities developed?
- What entities funded start ups to implement alleged capabilities?
- What other companies offer software and services which deliver similar alleged capabilities?
- When did Facebook discover that its own sign up systems had become a go to source of social action for these intelware systems?
- Why did Facebook ignore its sign up procedures failings?
- Are other countries developing and investing in similar systems with these alleged capabilities? If so, name a company in England, France, China, Germany, or the US?
These one-shot “intelware is bad” stories chop indiscriminately. The vendors get slashed. The social media companies look silly for having little interest in “real” identification of registrants. The licensees of intelware look bad because somehow investigations are somehow “wrong.” I think the media reporting on intelware look silly because the depth of the information on which they craft stories strikes me as shallow.
I am pointing out that a bit more diligence is required to understand the who, what, why, when, and where of specialized software and services. Let’s do some heavy lifting, folks.
Stephen E Arnold, January 13, 2023
Cyber Investigators: Feast, Famine, or Poisoned Data in 2023
January 11, 2023
At this moment in time, the hottest topic among some cyber investigators is open source intelligence or OSINT. In 2022, the number of free and for-fee OSINT tools and training sessions grew significantly. Plus, each law enforcement and intelligence conference I attended in 2022 was awash with OSINT experts, exhibitors, and investigators eager to learn about useful sites, Web and command line techniques, and intelware solutions combining OSINT information with smart software. I anticipate that 2023 will be a bumper year for DYOR or do your own research. No collegial team required, just a Telegram group or a Twitter post with comments. The Ukraine-Russia conflict has become the touchstone for the importance of OSINT.
Over pizza, my team and I have been talking about how the OSINT “revolution” will unwind in 2023. On the benefit side of the cyber investigative ledger, OSINT is going to become even more important. After 30 years in the background, OSINT has become the next big thing for investigators, intelligence professionals, entrepreneurs, and Beltway bandits. Systems developed in the US, Israel, and other countries continue to bundle sophisticated analytics plus content. The approach is to migrate basic investigative processes into workflows. A button click automates certain tasks. Some of the solutions have proven themselves to be controversial. Voyager Lab and the Los Angeles Police Department generated attention in late 2021. The Brennan Center released a number of once-confidential documents revealing the capabilities of a modern intelware system. Many intelware vendors have regrouped and appear to be ready to returned to aggressive marketing of their systems, its built-in data, and smart software. These tools are essential for certain types of investigations whether in US agencies like Homeland Security or in financial crime investigations at FINCEN. Even state and city entities have embraced the mantra of better, faster, easier, and, in some cases, cheaper investigations.
Another development in 2023 will be more tension between skilled human investigators and increasingly smarter software. The bean counters (accountants) see intelware as a way to reduce the need for headcount (full time equivalents) and up the amount of smart software and OSINT information. Investigators will face an increase in cyber crime. Some involved in budgeting will emphasize smart software instead of human officers. The crypto imbroglio is just one facet of the factors empowering online criminal behavior. Some believe that the Dark Web, CSAM, and contraband have faded from the scene. That’s a false idea. In the last year or so, what my team and I call the “shadow Web” has become a new, robust, yet hard-to-penetrate infrastructure for cyber crime. Investigators now face an environment into which a digital Miracle-Gro has been injected. Its components are crypto, encryption, and specialized software that moves Web sites from Internet host to Internet host in the click of a mouse. Chasing shadows is a task even the most recent intelware systems find difficult to accomplish.
However, my team and I believe that there is another downside for law enforcement and a major upside for bad actors. The wide availability of smart software capable of generating misinformation in the form of text, videos, and audio. Unfortunately today’s intelware is not yet able to flag and filter weaponized information in real time or in a reliable way. OSINT advocates and marketers unfamiliar with the technical challenges of ignoring “fake” information downplay the risk of weaponized or poisoned information. A smart software system ingesting masses of digital information can, at this time, learn from bogus data and, therefore, output misleading or incorrect recommendations. In 2023, poisoned data continue to derail many intelware systems as well as traditional investigations when insufficient staff are available to determine provenance and accuracy. Our research has identified 10 widely-used mathematical procedures particularly sensitive to bogus information. Few want to discuss these out-of-sight sinkholes in public forums. Hopefully the reluctance to talks about OSINT blindspots will fade in 2023.
The feast? Smart software. Masses of information.
The famine? Funds to expand the hiring of full time (not part time) investigators and the money needed to equip these professionals with high-value, timely instruction about tools, sources, pitfalls, and methods for verification of data.
The poison? The ChatGPT and related tools which can make anyone with basic scripting expertise into a volcano of misinformation.
Let me suggest four steps to begin to deal with the feast, famine, and poison challenges?
First, individuals, trade groups, and companies marketing intelware to law enforcement and intelligence entities stick to the facts about their systems. The flowery language and the truth-stretching lingo must be decreased. Why do intelware vendors experience brutal churn among licensees? The distance between the reality of the system and the assertions made to sell the system.
Second, procurement processes and procurement professionals must become advocates for reform. Vendors often provide “free” trials and then work to get “on the budget.” The present procurement methods can lead to wasted time, money, and contracting missteps. Outside-the-box ideas like a software sandbox require consideration. (If you want to know more about this, message me.)
Third, consulting firms which are often quick to offer higher salaries to cyber investigators need to evaluate the impact of their actions on investigative units. There is no regulatory authority monitoring the behavior of these firms. The Wild West of cyber investigator poaching hampers some investigations. Legislation perhaps? More attention from the Federal Trade Commission maybe? Putting the needs of the investigators ahead of the needs of the partners in the consulting firms?
Fourth, a stepped up recruitment effort is needed to attract investigators to the agencies engaged in dealing with cyber crime. In my years of work for the US government and related entities, I learned that government units are not very good at identifying, enlisting, and retaining talent. This is an administrative function that requires more attention from individuals with senior administrative responsibilities. Perhaps 2023 will generate some progress in this core personnel function.
Don’t get me wrong. I am optimistic about smart software. I believe techniques to identify and filter weaponized information can be enhanced and improved. I am confident that forward leaning professionals in government agencies can have a meaningful impact on institutionalized procedures and methods associated with fighting cyber crime.
My team and I are committed to conducting research and sharing our insights with law enforcement and intelligence professionals in 2023. My hope is that others will adopt a similar “give back” and “pay it forward” approach in 2023 in the midst of feasts, famines, and poisoned data.
Thank you for reading. — Stephen E Arnold, January 11, 2023
Palantir Makes Clear That Its Aggressively Marketed Systems May Not Work as Advertised
December 21, 2022
The real journalists at the Wall Street Journal has made painfully clear that Palantir’s smart software and sophisticated platform for functioning like the seeing stone in Lord of the Rings does not work.
You can read the real news analysis in “Palantir Misfires on Revenue Tied SPAC Deals.” The main point of the write up is that Palantir, equipped with proprietary technology and oodles of seeing stone expert, lost a great deal of money quickly.
The article says:
The bets have backfired.
So what? No big deal. Tens of millions gone, maybe hundreds of millions. The bigger loss is the exposure of the shortcomings of smart software. What did Palantir’s spokesperson say:
The market has turned an it is now clear that these investments were unsuccessful. It was a bet on a group of early stage companies that, with the benefit of hindsight, we wish we did not make.
But Palantir’s marketing since the firm open for intelligence analysis in 2003 or almost two decades ago has pitched the system’s ability to reveal what ordinary intelware cannot identify. In my files, I have some Palantir marketing material. Here’s an example:
Who doesn’t want data sovereignty? ©Palantir Technologies
Several observations:
- The Palantir management team presumably had access to Gotham and other Palantir technology. But the Palantir system did deliver massive financial losses. Some seeing stone.
- In my opinion, Palantir made big bets in order to get a big payoff so that the company’s financial strength and the excellence of its smart software would be evident. What’s evident is that even Palantir’s software and its wizards cannot get the Palantir systems to be right about “bets.”
- Intelware and policeware vendors typically sell to government and selected financial services customers. Converting intelligence software tuned to the needs of a three letter agency has not worked in the past, and it is now evident Palantir may be failing in its commercial push now.
- Intelware works because no matter how slick the intelware is, governments also rely on old fashioned methods before taking action.
- Palantir’s technology is almost 20 years old, based on open source, and highly derivative. There are better, faster, and cheaper options available from Palantir’s competitors.
Net net: Palantir has embraced full throttle marketing. The company has done some interesting things regarding the IBM Analysts Notebook file formats. Palantir’s investment were, in my opinion, investments which made it attractive to the recipients of Palantir’s funds to become Palantir customers. As I write this, Palantir’s marketing is chugging along, but Palantir’s share price is a stellar $6.43 a share. A blind seeing stone? Hmmmm. Good question.
Stephen E Arnold, December 21, 2022
Hello, Lawmakers in Greece. Have You Heard about Open Source Software?
December 15, 2022
I read a story from an outfit which makes quoting one of the stories risky business. The write up in question is “As Wiretap Claims Rattle Government, Greece Bans Spyware.” The article presents as real news — allegedly the old fashioned kind when newspapers were arbiters of truth via stringers — that Greece outlaws what it calls commercial spyware. For a number of years, I have used the term “intelware” to describe the specialized services and software provided to government agencies by commercial enterprises and open source developers.
The article does the normal handwaving associated with products and services which have been available since the mid 19th century. Those early systems chugged along within products from Bell, Systems Development Corporation, and others. I have found the bland names fascinating. Systems Development Corporation? What could be better? If you read Jill Lepore’s techno-noir history, you will know more than you ever wanted to know about Simulmatics. There’s a descriptive company name for you, right?
What happens when a government bans specialized services and software? Some interesting things; for example, it may be tough to know when warships from a friendly country are converging on a critical island. What if a country on Greece’s border gets frisky with its Soviet era tanks and artillery? The answer is, “License those specialized software and systems. Now!”
In terms of the ban on commercial intelware, what’s Greece going to do with the open source version of Maltego or one of dozens of other tools which can ingest digital content and output useful facts. What happens when one of those open source intelware tools requires an extension of functions?
The answer is to hire a consulting firm, hopefully not one affiliated with a certain jewelry store in Athens, to create bespoke code. Once that’s done, won’t government entities use these tools to protect citizen and monitor potential threats?
The answer is, “You bet your life.” The secret word is “politicians.” I am not sure of Greek’s elected officials or the people reporting on the world of intelware understand the difference between handwaving and getting a particular job done.
And the story. Oh, objective and an example of publicizing the considered viewpoints of elected officials.
Stephen E Arnold, December 15, 2022
Intelware Explained: On Reddit, Not the Gray Lady
December 9, 2022
Goodness gracious, real media is chasing the intelware sector. Nothing like a slow reaction to a specialized services sector that is what? – 25 or more years old? Yeah, real news.
I want to direct your attention to a Reddit post by FjorgVanDerPlorg. You can — at least as of December 9, 2022, at 740 US Eastern time — read his quite useful summary of how intelware pivots around a certain government’s investments in surveillance and information gathering systems.
Here’s the link. Due to the importance of the information in FjorgVanDerPlorg’s post, I have a holiday gift for you. My research team has summarized the Reddit post as a series of dot points just like those for which some blue chip advisory firms charge big bucks.
Very useful article because:
- Entities are identified
- Source of technologies identified
- Use cases referenced.
Who will pay attention to FjorgVanDerPlorg? Some with it real journalists who are now covering an interesting story related to specialized software and services. Speedy. Sure. It’s only been three decades or more since intelware became available to certain government entities.
Stephen E Arnold, December 2022
Three Constants: Death, Taxes, and NSO?
December 8, 2022
I know the special action is interesting to some. Plus, there’s a volcanic eruption outputting. And there is the Twitter saga, the NGX drama, and exciting World Cup. (Did Spain lose to Japan to avoid Seleção Brasileira? Of course not.)
But poking through the PR fumes and rising near the flocks of legal eagles circling for prey is the NSO Group. Navigate to “Why We’re Suing NSO Group.” You will learn that El Faro, a real news outfit in the pace-setting Republic of Salvador, and its taking action against NSO Group. The company has become the touchstone for allegedly unlawful surveillance of individuals.
The write up asserts:
Beginning in June 2020, at least 22 people associated with El Faro were the targets of spyware attacks. Over a period of about 18 months, their iPhones were accessed remotely and surreptitiously, their communications and activities monitored, and their personal data stolen. Many of these attacks occurred when the journalists were communicating with confidential sources, and reporting on abuses by the Salvadoran government.
The legal action is described this way:
the Knight Institute filed suit against NSO Group on behalf of 15 of the El Faro employees whose iPhones were infected with Pegasus spyware….Our complaint explains that NSO Group’s development and deployment of the spyware violated, among other laws, the Computer Fraud and Abuse Act, which prohibits accessing computers without authorization. We argue that our case belongs in a U.S. court because the spyware attacks violated U.S. law, because they were intended to deter journalism that is important to hundreds of thousands of American readers, and because NSO Group’s development and deployment of Pegasus involved deliberate and sustained attacks on the U.S. infrastructure of U.S. technology companies—including Apple, which itself sued NSO Group last year, contending that the spyware manufacturer had damaged its business and harmed its users.
Death, taxes, and NSO—Are these three constants of modern life?
Stephen E Arnold, December 8, 2022
Google and Crypto: Solana Should Anyone Ask
November 18, 2022
I read “Google Cloud Just Became a Solana Validator.” The article explains what Google has chosen to reveal to those who follow the company via “real” journalists; namely:
Google’s cloud computing division Google Cloud announced on Saturday that it’s now running a validator on the Solana blockchain, and will soon add features aimed at welcoming Solana developers and node runners.
No big deal. Amazon has blockchain-related services and a handful of patents pertaining to its digital currency inventions. No big deal either.
The write up says:
Google Cloud also announced it’s now indexing Solana data and adding it to its BigQuery data warehouse, a move that will “make it easier for the Solana developer ecosystem to access historical data.” The feature will launch in the first quarter of 2023, Mittal said. Mittal added that Google Cloud is bringing its credits program to “select startups in the Solana ecosystem” with up to $100,000 in Cloud Credits available for applicants.
Ah, more functionality.
What’s not in the write up? How about deanonymization functionality?
Stephen E Arnold, November 18, 2022
An Interesting NSO Related Action
November 9, 2022
In what sounds like the idea for a thriller/drama miniseries, The Times of Israel states that; “Former NSO CEO And Ex-Chancellor of Austria Establish New Cybersecurity Startup.” Sebastian Kurz, former Austrian chancellor, and ex-CEO of NSO Group Shalev Hulio established the new cybersecurity company Dream Security.
Hulio and Kurz formed Dream Security to protect critical infrastructures, such as energy, water, and oil facilities from cyber attacks. Dream Security will begin building a market in Europe. Kurz and Hulio raised $20 million in pre-seed funds from investors led by Dove Frances, who is an Israeli-American venture capitalist founder of the Group 11 investment firm. Other investors include entrepreneurs from the Israeli cybersecurity industry and early NSO Group investor Adi Shalev.
Founder Former Wayout Group CEO Gil Dolev will join Dream Security’s initial team.
Kurz and Hulio are concerned with infrastructures from their past work:
“Kurz told the publication that as Austrian chancellor, he ‘witnessed many attacks on governments as well as on manufacturing plants and energy installations, most of which were not published in the media. This has far-reaching implications for supply chains as well as regular energy supplies and public services such as water and hospitals.’
Hulio told Bloomberg he was leaving ‘the intelligence side, offensive side if you want, and move to the defensive side. We saw that the biggest challenge the cyber world is dealing with is critical infrastructure.’ He said the new company would focus on European markets ‘because I currently think that they have the biggest threats right now because of the geopolitical situation.’”
Both men’s reputations are covered with black marks . Kurz left politics because he was accused of a corruption scandal. At NSO Group, Hulio oversaw the development of the Pegasus spyware. Pegasus has been used by countries with poor human rights records to spy on “rabble rousers.” Apple and Facebook are pursuing lawsuits against NSO Group for breaking into their products and violating the terms of use. The European Union is investigating the use of Pegasus by its critics and the US Commerce Department blacklisted the company, then limited access to US components and technology.
Israel is also tightening restrictions on its cybersecurity companies. The number of countries that can buy Israeli cyber technology went from 100 down to 37.
It appears Dream Security is attempting to skirt Israeli restrictions by building a new company in Europe. The leaders are preaching they want to help people by protecting their infrastructures, but it would not be surprising if their plans were more nefarious.
Whitney Grace, November 9, 2022
-
- Subscribe to Beyond Search
Feature archive
News archive
-
