DYOR and OSINT Vigilantes
November 7, 2022
DYOR is an acronym used by some online investigators for “do your own research.” The idea is that open source intelligence tools provide information that can be used to identify bad actors. Obviously once an alleged bad actor has been identified, that individual can be tracked down. The body of information gathered can be remarkably comprehensive. For this reason, some law enforcement, criminal analysts, and intelligence professionals have embraced OSINT or open source intelligence as a replacement for the human-centric methods used for many years. Professionals understand the limitations of OSINT, the intelware tools widely available on GitHub and other open source software repositories, and from vendors. The most effective method for compiling information and doing data analysis requires subject matter experts, sophisticated software, and access to information from Web sites, third-party data providers, and proprietary information such as institutional knowledge.
If you are curious about representative OSINT resources used by some professionals, you can navigate to www.osintfix.com and click. The site will display one of my research team’s OSINT resources. The database the site pulls from contains more than 3,000 items which we update periodically. New, useful OSINT tools and services become available frequently. For example, in the work for one of our projects, we came across a useful open source tool related to Tor relays. It is called OrNetStats. I mention the significance of OSINT because I have been doing lectures about online research. Much of the content in those lectures focuses on open source and what I call OSINT blind spots, a subject few discuss.
The article “The Disturbing Rise of Amateur Predator=Hunting Stings: How the Search for Men Who Prey on Underage Victims Became a YouTube Craze” unintentionally showcases another facet of OSINT. Now anyone can use OSINT tools and resources to examine an alleged bad actor, gather data about an alleged crime, and pursue that individual. The cheerleading for OSINT has created a boom in online investigations. I want to point out that OSINT is not universally accurate. Errors can creep into data intentionally and unintentionally. Examples range from geo-spoofing, identifying the ultimate owner of an online business, and content posted by an individual to discredit a person or business. Soft fraud (that is, criminal type actions which are on the edge of legality like selling bogus fashion handbags on eBay) is often supported by open source information which has been weaponized. One example is fake reviews of restaurants, merchants, products, and services.
I urge you to work through the cited article to get a sense of what “vigilantes” can do with open source information and mostly unfiltered videos and content on social media. I want to call attention to four facets of OSINT in the context of what the cited article calls “predator-hunting stings”:
First, errors and false conclusions are easy to reach. One example is identifying the place of business for an online service facilitating alleged online crime. Some services displace the place of business for some online actors in the middle of the Atlantic Ocean or on obscure islands with minimal technical infrastructure.
Second, information can be weaponized to make it appear that an individual is an alleged bad actor. Gig work sites allow anyone to spend a few dollars to have social media posts created and published. Verification checks are essentially non-existent. One doesn’t need a Russia- or China-system intelligence agency; one needs a way to hire part time workers usually at quite low rates. How does $5 sound.
Third, the buzz being generated about OSINT tools and techniques is equipping more people than ever before to become Sherlock Holmes in today’s datasphere. Some government entities are not open to vigilante inputs; others are. Nevertheless, hype makes it seems that anything found online is usable. Verification and understanding legal guidelines remain important. Even the most scrupulous vigilante may have difficulty getting the attention of some professionals, particularly government employees.
Fourth, YouTube itself has a wide range of educational and propagandistic videos about OSINT. Some of these are okay; others are less okay. Cyber investigators undergo regular, quite specific training in tools, sources, systems, and methods. The programs to which I have been exposed include references to legal requirements and policies which must be followed. Furthermore, OSINT – including vigilante-type inputs – have to be verified. In my lectures, I emphasize that OSINT information should be considered background until those data or the items of information have been corroborated.
What’s the OSINT blind spot in the cited article’s report? My answer is, “Verification and knowledge of legal guideless is less thrilling than chasing down an alleged bad actor.” The thrill of the hunt is one thing; hunting the right thing is another. And hunting in the appropriate way is yet another.
DYOR is a hot concept. It is easy to be burned.
Stephen E Arnold, November 7, 2022
OSINT Is Popular. Just Exercise Caution
November 2, 2022
Many have embraced open source intelligence as the solution to competitive intelligence, law enforcement investigations, and “real” journalists’ data gathering tasks.
For many situations, OSINT as open source intelligence is called, most of those disciplines can benefit. However, as we work on my follow up to monograph to CyberOSINT and the Dark Web Notebook, we have identified some potential blind spots for OSINT enthusiasts.
I want to mention one example of what happens when clever technologists mesh hungry OSINT investigators with some online trickery.
Navigate to privtik.com (78.142.29.185). At this site you will find:
But there is a catch, and a not too subtle one:
The site includes mandatory choices in order to access the “secret” TikTok profile.
How many OSINT investigators use this service? Not too many at this time. However, we have identified other, similar services. Many of these reside on what we call “ghost ISPs.” If you are not aware of these services, that’s not surprising. As the frenzy about the “value” of open source investigations increases, geotag spoofing, fake data, and scams will escalate. What happens if those doing research do not verify what’s provided and the behind the scenes data gathering?
That’s a good question and one that gets little attention in much OSINT training. If you want to see useful OSINT resources, check www.osintfix.com. Each click displays one of the OSINT resources we find interesting.
Stephen E Arnold, November 2, 2022
Wonderful Statement about Baked In Search Bias
October 12, 2022
I was scanning the comments related to the HackerNews’ post for this article: “Google’s Million’s of Search Results Are Not Being Served in the Later Pages Search Results.”
Sailfast made this comment at this link:
Yeah – as someone that has run production search clusters before on technologies like Elastic / open search, deep pagination is rarely used and an extremely annoying edge case that takes your cluster memory to zero. I found it best to optimize for whatever is a reasonable but useful for users while also preventing any really seriously resource intensive but low value queries (mostly bots / folks trying to mess with your site) to some number that will work with your server main node memory limits.
The comment outlines a facet of search which is not often discussed.
First, the search plumbing imposes certain constraints. The idea of “all” information is one that many carry around like a trusted portmanteau. What are the constraints of the actual search system available or in use?
Second, optimization is a fancy word that translates to one or more engineers deciding what to do; for example, change a Bayesian prior assumption, trim content based on server latency, filter results by domain, etc.
Third, manipulation of the search system itself by software scripts or “bots” force engineers to figure out what signals are okay and which are not okay. It is possible to inject poisoned numerical strings or phrases into a content stream and manipulate the search system. (Hey, thank you, search engine optimization researchers and information warfare professionals. Great work.)
When I meet a younger person who says, “I am a search expert”, I just shake my head. Even open source intelligence experts display that they live in a cloud of unknowing about search. Most of these professionals are unaware that their “research” comes from Google search and maps.
Net net: Search and retrieval systems manifest bias, from the engineers, from the content itself, from the algorithms, and from user interfaces themselves. That’s why I say in my lectures, “Life is easier if one just believes everything one encounters online.” Thinking in a different way is difficult, requires specialist knowledge, and a willingness to verify… everything.
Stephen E Arnold, October 12, 2022
US Federally Funded Research: Open Access, Folks
September 7, 2022
In a surprise announcement, reports Ars Technica, “US Government to Make All Research it Funds Open Access on Publication.” The new policy was issued by the Office of Science and Technology Policy (OSTP) at the end of August. We expect this will be a windfall for researchers—in and outside the US. Though the US government is believed to be the world’s largest funder of scientific research, only those paying for subscriptions to academic journals have had access to many (most?) publicly funded studies. Writer John Timmer notes this constraint has loosened in recent years as a result of increased open-access journals and, especially with COVID-19 research, a trend toward preprints. We learn:
“Some people involved in scientific publishing worried that these trends would undercut the finances of the entire publishing industry, while others hoped to push them to open up all scientific publishing. This tension played out in the halls of Congress, where competing legislation would mandate or block open access to federal research. A truce of sorts was reached during the Obama administration. For federally funded research, publishers had two choices: either make the publication open access from the start or have subscription-only access for a year before opening things up. Government-sponsored repositories were opened to host copies of papers that weren’t made open access on the publisher’s site. In the intervening time, there has been a lot of growth in open access journals, and many subscription journals allowed authors to pay a fee to immediately open published papers. Most subscription journals also offered COVID-related papers as open access without any additional fees. OSTP has apparently decided that these adjustments have prepared the industry to survive even greater access levels.”
One provision requires a digital identifier, like a DOI, for all data and documentation. The policy memorandum argues the benefits of open access became apparent during the pandemic, when it accelerated researchers’ understanding of the virus and the development of a vaccine. Acting head of the OSTP Alondra Nelson expects the change will lead to gains across society. She stated:
“When research is widely available to other researchers and the public, it can save lives, provide policymakers with the tools to make critical decisions, and drive more equitable outcomes across every sector of society.”
Publishers have some time to pivot—the policy goes fully into effect in 2026. The article notes they could still make a buck from these papers by creating versions with added features like integrated graphics / videos or cross-references to other studies. Will that be enough to sooth ruffled feathers?
Cynthia Murrell, September 7, 2022
Deep Fakes: Alarming Predictions Made Real
August 25, 2022
Here is one disturbing way deep fakes can provide bad actors with a new money-making opportunity. Reporting on a growing, Google-play enabled problem, Rest of World tells us “Mexican Scam Loan Apps Will Edit Your Face onto X-Rated Photos and Send Them to Your Family.” Yes, at least one victim had such faked images sent to her contacts, including her minor daughter, along with the claim she had turned to prostitution to pay off her loan. In their odious efforts to collect more funds than they lent in the first place, these outfits will also dox victims and harass with ceaseless threatening phone calls. Reporter Erika Lilian Contreras writes:
“Rest of World identified 94 loan apps listed as possibly related to doxxing activities across various Mexican cyber police departments; 35 of these are available on Google’s Play store, the app store on Android devices, which account for 80.87% of mobile internet traffic in Mexico. According to victims, government officials, digital rights activists, and platforms, gaps in Mexican law allow these lenders to continue to push scams on app stores and leave victims with no clear avenue to restitution or justice. Consumer education and criminal investigations that come after a crime has occurred are the only ways to currently combat this new type of digital financial fraud, but none of these efforts has stopped the apps from appearing in app stores, according to digital security experts who spoke to Rest of World.”
Mexico’s financial services regulators say it is not their problem since these loan apps are not registered financial institutions. This leaves the national and local cybersecurity agents who, so far, have been unable to keep the problem from growing. Bad actors know a good opportunity when they see one. The article notes:
“While the Mexican government can do little and the tech companies platforming scam loan apps won’t take responsibility, the onus has fallen onto civil society. … However, most activism is limited to educating Mexicans about the risks posed by scam loan apps.”
It would be nice if Google would supply even an ounce of prevention here as it finally did in India. That, however, was only after the Reserve Bank of India forcefully drew about 600 scam-credit apps to its attention.
We wonder: where will deep fake technology be weaponized next? We think we know one knock on effect: Open source intelligence will be eroded or poisoned.
Cynthia Murrell, August 25, 2022
TikTok: Is Joe Rogan the Person to Blow the Whistle on Chinese Surveillance?
August 3, 2022
TikTok has been around since 2015 as A.me and Douyin. If you want to scrape below the shiny surface of the TikTok rags-to-riches story, there something called Musical.ly which surfaced in 2014. In 2018, the Musical.ly management team decided that selling to ByteDance was a super great idea. Then TikTok was created to entertain and log data. Few talk about the link to certain entities in the Chinese political structure. Even fewer think that short videos were bad. Sure, there were allegations of self harm, addiction, erosion of self worth, and students who preferred watching vids pumped at them by a magical algorithm. Nobody, including some Silicon Valley real news people with an inflated view of their intellectual capabilities said, “Yo, TikTok is a weaponized content delivery and surveillance system.” Nope. Just cute videos. What’s the problemo?
Who is now concerned about TikTok? The NSA? The CIA? The badge-and-gun entities in the US Federal government? Well, maybe. But the big voice is now a semi-real sports event announcer. “Joe Rogan Warns Americans about TikTok: China Knows Every … Thing You type.” Hey, Joe, don’t forget psychographic profiling to identify future insider operators, please.
The article reports:
Rogan listed the other data being collected by the popular platform. “‘User agent, mobile carrier, time zone settings, identifiers for advertising purpose, model of your device, the device system, network type, device IDs, your screen resolution and operating system, app and file names and types,’” he said. “So all your apps and all your file names, all the things you have filed away on your phone, they have access to that.” He continued: “‘File names and types, keystroke patterns or rhythms.’”
Hot intel, Mr. Rogan.
Where did this major news originate? From Mr. Rogan’s wellness infused research?
Nope. He read the terms of service.
The estimable newspaper pointed out:
… the tech news site Gizmodo reported that leaked internal documents from TikTok showed the extent to which the app sought to “downplay the China association.” The documents, labeled “TikTok Master Messaging” and “TikTok Key Messages,” detail the social media giant’s public relations strategy during a period of mounting scrutiny from regulators and lawmakers over its parent company ByteDance and its ties to the Chinese Community Party.
Gizmodo? Is this Silicon Valley type “real news” outlet emulating Cryptome.org?
According to the cited New York Post story:
TikTok has pledged to “publish insights about the covert influence operations we identify and remove from our platform globally to show how seriously we take attempts to mislead our community.”
That sounds good just like a cyber security firm’s PowerPoint deck. Talk, however, is not action.
Maybe Mr. Rogan can use his ring announcer voice to catch people’s attention? I am not sure some of the TikTok lovers will listen or believe what Mr. Rogan discovered in the super stealthy terms of service for TikTok.
That’s real open source intel. Put Mr. Rogan on a panel at the next OSINT conference, please. I mean TikTok has a 10 year history and it seems to be quite new to some folks.
Stephen E Arnold, August 3, 2022
UK Organization to Harness Open Source Intelligence
June 30, 2022
Technical innovations over the last decade or so have empowered civilians with tools and information once the strict purview of government agencies. Now the war in Ukraine has prompted a new effort to harness that tech, we learn from the BBC article, “New UK Centre Will Help Fight Information War.” Those behind The Centre for Emerging Technology and Security (CETaS), based at the Alan Turing Institute, have noticed the efforts of Open Source Intelligence (OSINT) enthusiasts are proving effective against Russia’s disinformation campaign (outside of Russia anyway). The new center hopes to develop and channel this expertise. Reporter Gordon Corera writes:
“US and UK governments have been active in using open-source information to be able to talk publicly about what their secret sources are indicating. But this type of information is most powerfully used by those outside government to reveal what is really happening on the ground. On the evening of 23 February, graduate students in Monterey, California, who had been using publicly available satellite imagery to watch Russian tanks on the border with Ukraine, saw Google Maps showing a traffic jam inching towards the Ukrainian border. They tweeted that a war seemed to have started, long before any official announcement.”
Since the invasion, others have used OSINT to illuminate possible war crimes and counter Russian propaganda. For several years, many have considered Russia to be ahead of the tech information game with its weaponization of social media and hacking prowess. According to a pair of anonymous UK officials, however, the balance has shifted since the war began thanks to the skilled use of OSINT resources. Imagine what could be achieved if only such efforts were focused by a dedicated organization. The article continues:
“Harnessing new technology to maintain an edge is part of the new center’s mission. This could include fields like automated recognition of military vehicles from satellite imagery or social media, allowing human experts to spend their time on trickier problems. Tools are already allowing greater translation and interpretation of foreign language material. Artificial Intelligence can also be used to reveal patterns in behavior or language that indicate the presence of an organized disinformation network on social media. Dealing with these challenges at speed is one of the ambitions for the center which aims to build a community that can keep pace with the growing amount of data and tools to exploit it.”
But will the center be able to overcome the barriers? Intelligence agencies face not only regulatory and technical restrictions on what data they can use, but also a bias against information from beyond their institutions. We wonder whether the trend toward pay-to-play OSINT resources help or hurt the cause.
Cynthia Murrell, June 30, 2022
Google: Is The Ad Giant Consistently Inconsistent?
June 21, 2022
Not long ago, the super bright smart software management team decided that Dr. Timnit Gebru’s criticism of the anti-bias efficacy was not in sync with the company’s party line. The fix? Create an opportunity for Dr. Gebru to find her future elsewhere. The idea that a Googler would go against the wishes of the high school science club donut selection was unacceptable. Therefore, there’s the open window. Jump on through.
I recall reading about Google’s self declared achievement of quantum supremacy. This was an output deemed worthy of publicizing. Those articulating this wild and crazy idea in the midst of other wild and crazy ideas met the checklist criteria for academic excellence, brilliant engineering, and just amazing results. Pick out a new work cube and soldier on, admirable Googler.
I know that the UK’s Daily Mail newspaper is one of the gems of online trustworthiness. Therefore, I read “Google Engineer Warns the Firm’s AI Is Sentient: Suspended Employee Claims Computer Programme Acts Like a 7 or 8-Year-Old and Reveals It Told Him Shutting It Off Would Be Exactly Like Death for Me. It Would Scare Me a Lot.” (Now that’s a Googley headline! A bit overdone, but SEO, you know.)
The write up states:
A senior software engineer at Google who signed up to test Google’s artificial intelligence tool called LaMDA (Language Model for Dialog Applications), has claimed that the AI robot is in fact sentient and has thoughts and feelings.
No silence of the lambda in this example.
The write up adds:
Lemoine worked with a collaborator in order to present the evidence he had collected to Google but vice president Blaise Aguera y Arcas and Jen Gennai, head of Responsible Innovation at the company dismissed his claims. He was placed on paid administrative leave by Google on Monday [June 6, 2022 presumably] for violating its confidentiality policy.
What do these three examples suggest to me this fine morning on June 12, 2022?
- Get shown the door for saying Google’s smart software is biased and may not work as advertised and get fired for saying the smart software works really super because it is now alive. Outstanding control of corporate governance and messaging!
- The Google people management policies are interesting? MBA students, this is a case example to research. Get the “right” answer, and you too can work at Google. Get the wrong answer, and you will not understand the “value” of calculating pi to lots of decimal places!
- Is the objective of Google’s smart software to make search “work” or burn through advertising inventory? If I were a Googler, I sure wouldn’t write a paper on this topic.
Ah, the Google.
Stephen E Arnold, June 21, 2022
Open Source: Dietary Insights
May 5, 2022
One of the more benign news briefs about Russia these days concerns the eating habits of the country’s secret police. The Verge explains how delivery apps revealed Russian law enforcement’s food preferences: “Data Leak From Russian Delivery App Shows Dining Habits Of The Secret Police.” A massive data leak from Yandex Food, a large food delivery service in Russia, contained names, addresses, phone numbers, and delivery instructions related to the secret police.
Yandex Food is a subsidiary of the Russian search engine of the same name. The data leak occurred on March 1 and Yandex blamed it on the bad actions of one of its employees. The leak did not include users’ login information. The Roskomnadzor, the Russian government agency responsible for mass media, threatened Yandex with a 100,000 ruble fine and it also blocked a map containing citizen and secret police data.
Bellingcat researchers were investigating leads on the poisoning of Alexey Navalny, the Russian opposition leader. They searched the Yandex Food database collected from a prior investigation and discovered a person who was in contact with Russia’s Federal Security Service (FSB) to plan Navalny’s poisoning. The individual used his work email to register with Yandex Food. They also searched for phone numbers linked to Russia’s Main Intelligence Directorate (GRU). Bellingcat found interesting information in the leak:
“Bellingcat uncovered some valuable information by searching the database for specific addresses as well. When researchers looked for the GRU headquarters in Moscow, they found just four results — a potential sign that workers just don’t use the delivery app, or opt to order from restaurants within walking distance instead. When Bellingcat searched for FSB’s Special Operation Center in a Moscow suburb, however, it yielded 20 results. Several results contained interesting delivery instructions, warning drivers that the delivery location is a military base. One user told their driver “Go up to the three boom barriers near the blue booth and call. After the stop for bus 110 up to the end,” while another said ‘Closed territory. Go up to the checkpoint. Call [number] ten minutes before you arrive!’”
The most scandalous information leaked from the Yandex Food breach was information about Putin’s former mistress and their “suspected daughter.”
While it is hilarious to read about Russian law enforcement’s eating habits, it is alarming when the situation is applied to the United States. Imagine all of the information DoorDash, Grubhub, Uber Eats, and other delivery services collect on customers. There was a DoorDash data leak in 2019 that affected 4.9 million people and it was much larger than the Yandex Food leak.
Whitney Grace, May 5, 2022
Simple, Fair Digital Markets: Saddle Up, Don Quixote
March 25, 2022
Who knew that I would continue to reference the very long, very weird book I had to read in the seventh grade? Yet here I am: Don Quixote, slayer of windmills, a trusted sidekick, and a study horse.
“Europe Agrees New Law to Curb Big Tech Dominance” explains that the proud animal and adept rider is ambling from the barn after decades of training. Tally ho! The write up says:
Under the Digital Markets Act (DMA), giants such as Google and Apple will be forced to open up their services and platforms to other businesses. Major technology firms have long faced criticism that they use their market dominance to squeeze out competition.
Now that certain US technology outfits are dominant, what’s the fix? I suppose one could dismount and paint the windmills a different color. Where would one locate a color? How about Googling? Alternatively one might consult a Facebook group. And there is the ever objective Amazon, complete with fake reviews and odd ball videos showing a functioning product? Amazing.
Outfitted like the elegant Don, the trusted source of information reports:
The EU wants to give users more choice over how people send messages. The new rules would require that technology make their messaging services interoperable with smaller competitors.
As the rider, cohorts, and snorting animals charge at their targets, will the companies be fungible. Might they prove to be chimera?
At least one of the evil entities is Googzilla? Despite its age, the creature still has teeth, lots of teeth, and lawyers, lots of lawyers.
Stephen E Arnold, March 25, 2022
-
- Subscribe to Beyond Search
Feature archive
News archive
-
