Facial Recognition and Image Recognition: Nervous Yet?

November 18, 2018

I read “A New Arms Race: How the U.S. Military Is Spending Millions to Fight Fake Images.” The write up contained an interesting observation from an academic wizard:

“The nightmare situation is a video of Trump saying I’ve launched nuclear weapons against North Korea and before anybody figures out that it’s fake, we’re off to the races with a global nuclear meltdown.” — Hany Farid, a computer science professor at Dartmouth College

Nothing like a shocking statement to generate fear.

But there is a more interesting image recognition observation. “Facebook Patent Uses Your Family Photos For Targeted Advertising” reports that a the social media sparkler has an invention that will

attempt to identify the people within your photo to try and guess how many people are in your family, and what your relationships are with them. So for example if it detects that you are a parent in a household with young children, then it might display ads that are more suited for such family units. [US20180332140]

While considering the implications of pinpointing family members and linking the deduced and explicit data, consider that one’s fingerprint can be duplicated. The dupe allows a touch ID to be spoofed. You can get the details in “AI Used To Create Synthetic Fingerprints, Fools Biometric Scanners.”

For a law enforcement and intelligence angle on image recognition, watch for DarkCyber on November 27, 2018. The video will be available on the Beyond Search blog splash page at this link.

Stephen E Arnold, November 18, 2018

Amazon Image Recognition: Industrial Parts

August 9, 2018

Amazon has come out with a handy little app for iOS that could save users some time and frustration. SiliconAngle reports, “Amazon Rolls Out Part Finder to Help Find Nuts and Bolts and More.” Reasonably dubbed Part Finder, the app uses image recognition to identify that little screw, bolt, washer, or nut so you can go get more of the (exact) same. Writer James Farrell tells us:

“Amazon said it identifies about 100 types of fasteners, which ‘represents thousands, if not millions of parts.’ You will, however, have to take the part and place it on a white surface next to a penny, presumably to help scale the object. Then some tilting of the phone will have to be done until the camera is aligned correctly, something the app will tell you. You might also be asked for some additional information. Although the app uses augmented reality technology, Part Finder actually employs computer vision technology. Does it work well? Early reviewers of the app have said it takes a bit of getting used to, or that it’s a great idea that just doesn’t work well enough.”

Farrell observes that Amazon released the app to little fanfare several weeks ago. No word on when Part Finder will be available for Android, but Amazon has said they plan to expand its repertoire well beyond fasteners to all manner of replacement parts.

Cynthia Murrell, August 9, 2018

Amazon Joins Visual Search Parade

July 30, 2018

Text search is long past done as a frontier. Verbal search is already being nailed down by more startups and tech giants than you can tell Alexa to shake a stick at. So, the new frontier? It’s visual search and, you guessed it, one of the biggest names in the industry is already working their way in, as we discovered in a recent Fortune story, “Snapchat and Amazon are Working On Visual Search Feature.”

According to the story:

“Snap appears to be laying the groundwork for a partnership with e-commerce giant Amazon. “According to TechCrunch, a version of Snapchat being developed for Android phones includes code for a new feature called “Visual Search” that can use Snapchat’s camera to send images of a product or a barcode scan to Amazon, which then display search results.”

Amazon is not alone, however. Microsoft is also developing a visual search tool that can simply look at items and begin shopping for them. The controversy about the accuracy of Amazon’s Rekognition system may inhibit some of Amazon’s plans for image centric features and functions. I I search for a product with my mobile phone and Amazon returns matches which are incorrect, what happens to consumer confidence?

Error rates are likely to matter, probably more when looking for a shirt than when trying to figure out which elected official is a bad actor. Shirts are different. Bad actors not so much, some may suggest.

Patrick Roland, July 30, 2018

Microsoft Tweaks Bing Thing

June 26, 2018

Microsoft, do you know that many non-Internet savvy people use Bing? Perhaps that is too much of a generalization, but experience will tell you that Google does deliver better results. It seems, however, that Microsoft has made a decent Bing upgrade, says the eWeek article, “Microsoft Uses Intel FPGAs For Smarter Bing Searches.”

Google has a feature where you type in a search term and it will spit out a small, informative blurb about it. Bing is “copying” that idea, so Microsoft added Intel field programmable gate arrays (FPGAs) to make the search engine smarter.  This endeavor is based on the deep learning acceleration platform Project Brainwave.  The FPGAs allow Bing to gather information from multiple sources and spit out an information tidbit:

“‘Intel’s FPGA chips allows Bing to quickly read and analyze billions of documents across the entire web and provide the best answer to your question in less than a fraction of a second,’ wrote Microsoft representatives in a blog post. ‘Intel’s FPGA devices not only provide Bing the real-time performance needed to keep our search fast for our users, but also the agility to continuously and quickly innovate using more and more advanced technology to bring you additional intelligent answers and better search results.’”

Bing is also using the new FPGAs to translate jargon, working on a how-to answer feature, and upgrading image search with an object detection tool. eBay and Pinterest may be exploring similar functionality.

Whitney Grace, June 26, 2018

Facial Recognition for a Certain Type of Bro

April 11, 2018

Male white privilege is a topic that pervades social and cultural discourse, but according to The Seattle Times the bias exists in facial recognition technology, “Facial-Recognition Technology Works Best If You’re A White Guy, Study Says.” AI’s ability to recognize people is improving more and more each day. The technology’s developers improve the technology by feeding AI data that help it learn to discern between physical differences such as gender, skin color, facial features, and other traits. It seems, however, that the data groups are overwrought with white men.

Apparently facial recognition software is 99 percent accurate in identifying white men, but the darker a person’s skin is the more errors that arise. MIT researcher Joy Buolamwini discovered the disparities and said it was a reflection of real word biases. The AI is only as smart as the people that program it:

“In modern artificial intelligence, data rules. AI software is only as smart as the data used to train it. If there are many more white men than black women in the system, it will be worse at identifying the black women. One widely used facial recognition data set was estimated to be more than 75 percent male and more than 80 percent white, according to another research study.”

Another alarming factor is that facial recognition and related technologies have a high adoption rate, such as companies that use them to target social media ads and automated decisions such as hiring people and money lending. Do not forget that law enforcement officials are relying more on the technology and minorities are more likely to singled out in databases.

While this information is disparaging, it makes a bigger issue out of something that can be easily remedied. Yes, the data is skewed towards white males, because, based on statistics, more white men work in the technology field so they draw on data they have ready access to. It is the same with the genetics field, European and Asian genes are more accurately represented than African DNA, because these countries are more developed than the mother continent. To resolve this conundrum, they need to start feeding facial recognition technology data with more females and people with darker skin. It is probably not that hard to find the data, just visit social media or an image library, then download away.

Whitney Grace, April 11, 2018

The Metropolitan Museum of Art Tackles Image Search: A Missed Block Halts the Speeding Researcher

March 23, 2018

The Met Wins The War to Get Online

For a few decades, art and history museums have been struggling with their online presences. The experience of seeing a Jpeg of a painting or sculpture is not the same as seeing it in person. That’s true. But there is one area where museums are holding a lot of valuable data and just now it’s starting to be searchable. We discovered this recently when the Metropolitan Museum of Art ‘s database “MetPublications.”

According to the page:

“MetPublications includes a description and table of contents for most titles, as well as information about the authors, reviews, awards, and links to related Met titles by author and by theme. Current book titles that are in-print may be previewed and fully searched online, with a link to purchase the book. The full contents of almost all other book titles may be read online, searched, or downloaded as a PDF.”

This includes over five hundred books about various exhibits that have spanned the last five decades. These slim volumes, usually released in conjunction with various exhibits, is fully searchable and a huge score for art lovers and historians. Previously, it was seen as too daunting and, potentially impossible. As far back as 2002 Computer Weekly was bemoaning the fact that museums had missed the digital boat. Turns out museums like the Met didn’t miss the boat, it’s just that their ship sails a little more slowly than the white knuckle world of Silicon Valley.

Stepping back, Beyond Search has noted that image collections remain difficult to use. Browsing often works best. Searches can be frustrating. Results rendering sluggish. Interfaces confusing. Even commercial image search systems are challenging. For example, locate the Google “Life Magazine” collection. Now try to find the image from the 1950s showing a child jumping from a tank in the side yard of the Smithsonian Museum. Impossible, right? (I know. The kid was my much younger self, and my family kept the page from the magazine but it was misplaced. Before my father died, he wanted me to locate that image. Fail even with three of my researchers beavering away.)

Consequently useful reference resources fall short of the mark. That often makes a museum visit necessary. And getting access to certain content remains difficult.

Stephen E Arnold, March 23, 2018

Google: Image Search Accuracy

February 16, 2018

Google image search is a hot topic. I wanted to test the functionality of the system  because Google killed the “view image” option. Google really wants to be best friends with copyright holders. I took this image of myself:


I loaded it into the Google image “search by image” function. Yep, that’s the little camera graphic for those of my gentle readers who do not understand Google’s wonderful iconography.

Here’s what Google delivered as “similar” and “matching” images.


Notice anything interesting?

I am flattered that Google thinks I look like the female singer Mpume.

With precise image matching like this, Google may want to cease development of its system. What do you think about the thumbnail images. Yep, just like me. Oh, did I mention I am Caucasian? But if Google sees me differently, I go with Google. The company’s “algorithms” are the dope.

By the way, if you want the “old” image search function, try this link.

Stephen E Arnold, February 16, 2018

Google Images Staring down Some Steep Competition

January 5, 2018

When we are looking for photos online Google Images has become a sort of shorthand for tracking down pics quick. The folks in Mountain View don’t want you to think much about its shortcomings. However, that topic is bubbling up to the surface, as we saw in a recent Free Technology for Teachers story, “5 Good Alternatives to Google Image Search.”

According to the story:

Google Images tends to be the default image search tool of students and adults who haven’t been introduced to better options. Google Images is convenient, but it’s not the best place for students to find images that are in the public domain or images that have been labeled with a Creative Commons license.

One they recommend is:

Unsplash offers a huge library of images that are either in the public domain or have a Creative Common license. If you or your students are using Google Slides, the Unsplash add-on for Google Slides makes it easy to quickly take images from Unsplash and add them to your slides. Watch my video embedded below to see how the Unsplash add-on for Google Slides works.

This should be a wakeup call for Google. The tech giant seems to have a new balloon popped every day. We love competition and we love leaders reinventing themselves to better meet client needs. We consider this to be a win-win no matter how you slice it.

Patrick Roland, January 5, 2018

Google Supports Outraged Scholars

October 2, 2017

Google has taken issue with a recent list from the Campaign for Accountability (CfA), TechCrunch reports in, “Google Responds to Academic Funding Controversy—with a GIF.” Writer Frederic Lardinois reports that the CfA recently released a list of policy experts and academics who, they say, had received Googley dollars last year. The only problem—many who found themselves on the list dispute their inclusion, saying they had not received any funding from Google or, if they had, it was unrelated to the work the CfA specified. Google issued a response, supporting the protesting experts and academics as well as defending its support of researchers in general. The company also struck back; the article explains:

And in a direct attack on CfA, Google also notes that while the group advocates for transparency, its own corporate funders remain in the shadows. The only backer we know of is Oracle, which is obviously competing with Google in many areas. The group has also recently taken on SolarCity/Tesla. In its blog post, Google also argues that ‘AT&T, the MPAA, ICOMP, FairSearch and dozens more’ fund similar campaigns.

Google later created a GIF in response to requests for elaboration. It shares a series of tweets from some of the affected scholars, in which they detail just where the CfA went wrong in each of their cases. Lardinois continues:

It’s not often that a company like Google makes its own GIF in response to a request for comment, but I gather this goes to show that Google wants to move on from this discussion and let the academics speak for themselves. While the CfA’s methods are less than ideal, there are legitimate questions about how even small amounts of funding can influence research.

So far, Lardinois notes, public discussion on how funding can influence research have centered around pharmaceuticals. He projects it will soon grow, however, to include policy research as tech companies ramp up their funding programs

Cynthia Murrell, October 2, 2017

Searching for a Disguised Face? Some Progress

September 9, 2017

I read “Meeting the Disguised Face Challenge via Deep Convolutional Network.” My interest was piqued because I thought I had seen references to a breakthrough in facial recognition when the subjects of interest were wearing disguises.

I noted these comments in the write up:

  • The method: “Deep convolutional networks are software creations organized into interconnected layers, much like the visual cortex, the part of the brain that processes visual information.”
  • The accuracy: “The fewer facial key points it can see, the worse the software is at recognizing a person in a photo. It’s also thrown off by busy backgrounds, so can only identify a person wearing a cap, glasses and scarf 43 per cent of the time if they’re standing in front of a complicated background.”
  • Work around: “Wearing a rigid mask that covers the whole face, for example, would give current facial recognition systems nothing to go on. And other researchers have developed patterned glasses that are specially designed to trick and confuse AI facial recognition systems.”

To sum up: Promising but real time facial recognition of people walking through an airport entrance remains a challenge. In addition to the computational demands, the false positives can quickly consume available resources.

Stephen E Arnold, September 9, 2017

« Previous PageNext Page »

  • Archives

  • Recent Posts

  • Meta