From the Desk of Captain Obvious: How Image Recognition Mostly Works
July 8, 2019
Want to be reminded about how super duper image recognition systems work? If so, navigate to the capitalist’s tool “Facebook’s ALT Tags Remind Us That Deep Learning Still Sees Images as Keywords.” The DarkCyber teams knows that this headline is designed to capture clicks and certainly does not apply to every image recognition system available. But if the image is linked via metadata to something other than a numeric code, then images are indeed mapped to words. Words, it turns out, remain useful in our video and picture first world.
Nevertheless, the write up offers some interesting comments, which is what the DarkCyber research team expects from the capitalist tool. (One of our DarkCyber team saw Malcolm Forbes at a Manhattan eatery keeping a close eye on a spectacularly gaudy motorcycle. Alas, that Mr. Forbes is no longer with us, although the motorcycle probably survives somewhere unlike the “old” Forbes’ editorial policies.
Here’s the passage:
For all the hype and hyperbole about the AI revolution, today’s best deep learning content understanding algorithms are still remarkably primitive and brittle. In place of humans’ rich semantic understanding of imagery, production image recognition algorithms see images merely through predefined galleries of metadata tags they apply based on brittle and naïve correlative models that are trivially confused.
Yep, and ultimately the hundreds of millions of driver license pictures will be mapped to words; for example, name, address, city, state, zip, along with a helpful pointer to other data about the driver.
The capitalist tool reminds the patient reader:
Today’s deep learning algorithms “see” imagery by running it through a set of predefined models that look for simple surface-level correlative patterns in the arrangement of its pixels and output a list of subject tags much like those human catalogers half a century ago.
Once again, no push back from Harrod’s Creek. However, it is disappointing that new research is not referenced in the article; for example, the companies involved in Darpa Upside.
Stephen E Arnold, July 8, 2019
The Middle East and Facial Recognition
June 12, 2019
How many times has science fiction been called stuff and nonsense, but the genre has actually predicted many things that are commonplace today? One thing that used to be make believe is facial recognition technology. US right advocates have successfully banned the technology in some parts of the country, but facial recognition developers are taking their creations to “friendlier” locals. Buzz Feed News shares where in the article, “Facial Recognition Technology Is Facing A Huge Backlash In The US. But Some Of The World’s Biggest Tech Companies Are Trying To Sell It In The Gulf.”
While the US is saying no way, Chinese and American facial recognition purveyors take their wares to Dubai. The biggest sellers are IBM, Hikvision, and Huawei. In the US, opposers to the technology state it could be used for social control, but Dubai is located in the United Arab Emirates where citizens are more under the government’s thumb. Hacking software is already used to spy on political dissidents, potential criminals, and journalists. While Dubai is heralded as a futuristic city, it is still in the heart of fundamentalist Islam territory. Theocracies are not known to be tolerant of “unreligious” behaviors.
“Police in Dubai have begun rolling out an ambitious program, dubbed Oyoon, the Arabic word for “eyes,” that will implement facial recognition and analysis driven by artificial intelligence across the city. Police say the program will reduce crime as well as traffic accidents. An analysis of hundreds of government procurement and regulatory documents make clear the scope of Dubai’s high-tech policing ambitions, showing the police have sought video analytics platforms meant to record and analyze people’s faces, voices, behavior, and cars in the time it takes to do a Google search. And a review of dozens of company marketing materials and interviews with officials show global tech giants are eager to provide the police with the technology they are seeking.”
Dubai police favor facial and voice recognition technology and use it to monitor potential threats through a central command center. There have already been three hundred arrests with the technology. Several UAE government agencies support using the technology to monitor its citizens. Like any sort of technology, it can be used for good or bad.
Dubai has the most political prisoners per capita I the world and the UAE prides itself on keeping order.
“‘They focus on preventative surveillance,’ said Joe Odell, a campaigner at the International Campaign for Freedom in the UAE. It’s about control to prevent street mobilizations through establishing a wide-reaching surveillance state, where they can nip anything in the bud before it even happens. They’ve spent millions of pounds on that.’”
The UAE does not like anyone that opposes its government and goes after even the most peaceful protesters. It is an authoritarian government armed with technology that is so strange it can only be true. Here is some advice: do not do anything stupid to anger the UAE if you visit.
Whitney Grace, June 12, 2019
Facial Recognition: In China, Deployed. In the US, Detours
April 9, 2019
Amazon faces push back for its facial recognition system Rekognition. China? That is a different story.
Chinese authorities seem to be fond of re-education camps and assorted types of incarceration facilities. China is trying to become the recognized (no pun intended) technology capital of the world. Unlike Chile and Bolivia which have somewhat old school prison systems, the Chinese government is investing money into its prison security systems. Technode explains how Chinese upgraded its security system in, “Briefing: Chinese VIP Jail Uses AI Technology To Monitor Prisoners.”
One flagship for facial recognition is China’s Yancheng Prison, known for imprisoning government officials and foreigners. The facility has upgraded its security system with a range of surveillance technology. The new surveillance system consists of a smart AI network with cameras and hidden sensors that are equipped with facial recognition, movement analysis The system detects prisoners’ unusual behavioral patterns, then alerts the guards and it is included in daily reports.
Yancheng Prison wants to cut down on the number of prison breaks, thus the upgrade:
“Jointly developed by industry and academic organizations including Tianjin-based surveillance technology company Tiandy, the system is expected to provide blanket coverage extending into every cell, rendering prison breaks next to impossible. The company is also planning to sell the system to some South American countries for jails with histories of violence and security breaches. The use of technology to monitor prisoners prompted concern over negative effects on prisoners’ lives and mental state from one human behavior expert who also suggested that some prisoners may look find ways to exploit the AI’s weaknesses.”
China continues to take steps to put technology into use. The feedback to the engineers who develop these systems can make adjustments. Over time, China may become better at facial recognition than almost any other country.
Whitney Grace April 9, 2019
Observation from Orbit: Gaining Traction
February 11, 2019
The day has arrived. ZeroHedge tells us the “‘Largest Fleet of Satellites in Human History’ Set to Revolutionize Space-Based Spying.” Writer Tyler Durden tells us about Planet Labs, an aerospace firm out of San Francisco that has launched almost 300 satellites for the express purpose of imaging specified sections of the Earth, on demand. We observe that Amazon is also into satellites now, and Google Alphabet and its Loon unit are making “we love satellites too” noises.
Though most of Planet Labs’ customers are currently agricultural companies seeking snapshots of their immense fields, the NGA is a noteworthy exception. We’re told:
“Their most important customer is the National Geospatial Intelligence Agency (NGA) – the government body responsible for analyzing satellite photos from its 2.7 million square-foot headquarters south of Washington D.C. staffed with 14,500 employees. ‘I’m quite excited about capabilities such as what Planet’s putting up in space,’ says NGA director Robert Cardillo. …
We also noted:
“NGA’s capabilities are of course top secret, however they have been collecting the bulk of their images from three multi-billion dollar satellites the size of a city bus, according to satellite tracker Ted Molczan- who uses giant binoculars.”
Contrast that description to that of Planet’s satellites, which are the size of a loaf of bread. If you’re curious, navigate to the article for that image. (You can also see a photo of those giant binoculars in action.) Is the world ready for this level of satellite surveillance? In my humble opinion, anyone surprised by this development has not been paying close attention. Founded in 2010, Planet Labs was organized by former NASA scientists.
Amazon’s edged toward satellite management services. Perhaps there is a connection?
Cynthia Murrell, February 11, 2019
FBI Photo Recognition: Mixed Views
January 31, 2019
Get used to debates like this in the future: law enforcement agencies develop new technology that helps track criminals more efficiently, but the courts and the public doubt its validity. This is not a new argument, but seems to be entering into a new arena, as we discovered from a recent ProPublica story, “The FBI Says its Photo Analysis is Scientific Evidence—Scientists Disagree.”
The story has several interesting spins:
“FBI examiners have tied defendants to crime pictures in thousands of cases over the past half-century using unproven techniques, at times giving jurors baseless statistics to say the risk of error was vanishingly small. Much of the legal foundation for the unit’s work is rooted in a 22-year-old comparison of bluejeans.”
From bluejeans to facial recognition software, this is the latest frontier for collecting evidence and it is healthy that it is met with skepticism. We are reminded of how fingerprints were disregarded as not being scientific evidence in the 1800s. Over time, thanks to rigorous testing and patience, the world at large began to trust this evidence. We foresee that being the case with photo analysis, once it is able to meet the standards of the scientific community.
Patrick Roland, January 31, 2019
Facial Recognition and Image Recognition: Nervous Yet?
November 18, 2018
I read “A New Arms Race: How the U.S. Military Is Spending Millions to Fight Fake Images.” The write up contained an interesting observation from an academic wizard:
“The nightmare situation is a video of Trump saying I’ve launched nuclear weapons against North Korea and before anybody figures out that it’s fake, we’re off to the races with a global nuclear meltdown.” — Hany Farid, a computer science professor at Dartmouth College
Nothing like a shocking statement to generate fear.
But there is a more interesting image recognition observation. “Facebook Patent Uses Your Family Photos For Targeted Advertising” reports that a the social media sparkler has an invention that will
attempt to identify the people within your photo to try and guess how many people are in your family, and what your relationships are with them. So for example if it detects that you are a parent in a household with young children, then it might display ads that are more suited for such family units. [US20180332140]
While considering the implications of pinpointing family members and linking the deduced and explicit data, consider that one’s fingerprint can be duplicated. The dupe allows a touch ID to be spoofed. You can get the details in “AI Used To Create Synthetic Fingerprints, Fools Biometric Scanners.”
For a law enforcement and intelligence angle on image recognition, watch for DarkCyber on November 27, 2018. The video will be available on the Beyond Search blog splash page at this link.
Stephen E Arnold, November 18, 2018
Amazon Image Recognition: Industrial Parts
August 9, 2018
Amazon has come out with a handy little app for iOS that could save users some time and frustration. SiliconAngle reports, “Amazon Rolls Out Part Finder to Help Find Nuts and Bolts and More.” Reasonably dubbed Part Finder, the app uses image recognition to identify that little screw, bolt, washer, or nut so you can go get more of the (exact) same. Writer James Farrell tells us:
“Amazon said it identifies about 100 types of fasteners, which ‘represents thousands, if not millions of parts.’ You will, however, have to take the part and place it on a white surface next to a penny, presumably to help scale the object. Then some tilting of the phone will have to be done until the camera is aligned correctly, something the app will tell you. You might also be asked for some additional information. Although the app uses augmented reality technology, Part Finder actually employs computer vision technology. Does it work well? Early reviewers of the app have said it takes a bit of getting used to, or that it’s a great idea that just doesn’t work well enough.”
Farrell observes that Amazon released the app to little fanfare several weeks ago. No word on when Part Finder will be available for Android, but Amazon has said they plan to expand its repertoire well beyond fasteners to all manner of replacement parts.
Cynthia Murrell, August 9, 2018
Amazon Joins Visual Search Parade
July 30, 2018
Text search is long past done as a frontier. Verbal search is already being nailed down by more startups and tech giants than you can tell Alexa to shake a stick at. So, the new frontier? It’s visual search and, you guessed it, one of the biggest names in the industry is already working their way in, as we discovered in a recent Fortune story, “Snapchat and Amazon are Working On Visual Search Feature.”
According to the story:
“Snap appears to be laying the groundwork for a partnership with e-commerce giant Amazon. “According to TechCrunch, a version of Snapchat being developed for Android phones includes code for a new feature called “Visual Search” that can use Snapchat’s camera to send images of a product or a barcode scan to Amazon, which then display search results.”
Amazon is not alone, however. Microsoft is also developing a visual search tool that can simply look at items and begin shopping for them. The controversy about the accuracy of Amazon’s Rekognition system may inhibit some of Amazon’s plans for image centric features and functions. I I search for a product with my mobile phone and Amazon returns matches which are incorrect, what happens to consumer confidence?
Error rates are likely to matter, probably more when looking for a shirt than when trying to figure out which elected official is a bad actor. Shirts are different. Bad actors not so much, some may suggest.
Patrick Roland, July 30, 2018
Microsoft Tweaks Bing Thing
June 26, 2018
Microsoft, do you know that many non-Internet savvy people use Bing? Perhaps that is too much of a generalization, but experience will tell you that Google does deliver better results. It seems, however, that Microsoft has made a decent Bing upgrade, says the eWeek article, “Microsoft Uses Intel FPGAs For Smarter Bing Searches.”
Google has a feature where you type in a search term and it will spit out a small, informative blurb about it. Bing is “copying” that idea, so Microsoft added Intel field programmable gate arrays (FPGAs) to make the search engine smarter. This endeavor is based on the deep learning acceleration platform Project Brainwave. The FPGAs allow Bing to gather information from multiple sources and spit out an information tidbit:
“‘Intel’s FPGA chips allows Bing to quickly read and analyze billions of documents across the entire web and provide the best answer to your question in less than a fraction of a second,’ wrote Microsoft representatives in a blog post. ‘Intel’s FPGA devices not only provide Bing the real-time performance needed to keep our search fast for our users, but also the agility to continuously and quickly innovate using more and more advanced technology to bring you additional intelligent answers and better search results.’”
Bing is also using the new FPGAs to translate jargon, working on a how-to answer feature, and upgrading image search with an object detection tool. eBay and Pinterest may be exploring similar functionality.
Whitney Grace, June 26, 2018
Facial Recognition for a Certain Type of Bro
April 11, 2018
Male white privilege is a topic that pervades social and cultural discourse, but according to The Seattle Times the bias exists in facial recognition technology, “Facial-Recognition Technology Works Best If You’re A White Guy, Study Says.” AI’s ability to recognize people is improving more and more each day. The technology’s developers improve the technology by feeding AI data that help it learn to discern between physical differences such as gender, skin color, facial features, and other traits. It seems, however, that the data groups are overwrought with white men.
Apparently facial recognition software is 99 percent accurate in identifying white men, but the darker a person’s skin is the more errors that arise. MIT researcher Joy Buolamwini discovered the disparities and said it was a reflection of real word biases. The AI is only as smart as the people that program it:
“In modern artificial intelligence, data rules. AI software is only as smart as the data used to train it. If there are many more white men than black women in the system, it will be worse at identifying the black women. One widely used facial recognition data set was estimated to be more than 75 percent male and more than 80 percent white, according to another research study.”
Another alarming factor is that facial recognition and related technologies have a high adoption rate, such as companies that use them to target social media ads and automated decisions such as hiring people and money lending. Do not forget that law enforcement officials are relying more on the technology and minorities are more likely to singled out in databases.
While this information is disparaging, it makes a bigger issue out of something that can be easily remedied. Yes, the data is skewed towards white males, because, based on statistics, more white men work in the technology field so they draw on data they have ready access to. It is the same with the genetics field, European and Asian genes are more accurately represented than African DNA, because these countries are more developed than the mother continent. To resolve this conundrum, they need to start feeding facial recognition technology data with more females and people with darker skin. It is probably not that hard to find the data, just visit social media or an image library, then download away.
Whitney Grace, April 11, 2018
-
- Subscribe to Beyond Search
Feature archive
News archive
-
