Facebook: Search Images by the Objects They Contain

July 3, 2017

Has Facebook attained the holy grail of image search? Tech Crunch reports, “Facebook’s AI Unlocks the Ability to Search Photos by What’s in Them.” I imagine this will be helpful to law enforcement.

A platform Facebook originally implemented to help the visually impaired, Lumos (built on top of FBLearner Flow), is now being applied to search functionality across the social network. With this tool, one can search using keywords that describe things in the desired image, rather than relying on tags and captions. Writer John Mannes describes how this works:

Facebook trained an ever-fashionable deep neural network on tens of millions of photos. Facebook’s fortunate in this respect because its platform is already host to billions of captioned images. The model essentially matches search descriptors to features pulled from photos with some degree of probability. After matching terms to images, the model ranks its output using information from both the images and the original search. Facebook also added in weights to prioritize diversity in photo results so you don’t end up with 50 pics of the same thing with small changes in zoom and angle. In practice, all of this should produce more satisfying and relevant results.

Facebook expects to extrapolate this technology to the wealth of videos it continues to amass. This could be helpful to a user searching for personal videos, of course, but just consider the marketing potential. The article continues:

Pulling content from photos and videos provides an original vector to improve targeting. Eventually it would be nice to see a fully integrated system where one could pull information, say searching a dress you really liked in a video, and relate it back to something on Marketplace or even connect you directly with an ad-partner to improve customer experiences while keeping revenue growth afloat.

Mannes reminds us Facebook is operating amidst fierce competition in this area. Pinterest, for example, enables users to search images by the objects they contain. Google may be the furthest along, though; that inventive company has developed its own image captioning model that boasts an accuracy rate of over 90% when either identifying objects or classifying actions within images.

Cynthia Murrell, July 3, 2017

 

Giffying All the Way to Profits

June 7, 2017

Giphy, the GIF search engine has secured $150 funding at $600 million valuations. What started as a web crawler is on its way to profitability.

Business Insider in an article titled Inside the GIF Factory: How Giphy Plans to Build a Real Business by Animating the Internet says:

Giphy isn’t profitable yet. In fact, the company doesn’t even have a reliable means of generating revenue at this point. But now that GIFs are an ingrained aspect of online behavior, the company is hard at work drafting a blueprint to turn its popular service into a money-making business.

Though there are multiple ways to monetize GIFs, a mainstay of personal messages and online forums and social media networks, Alex Chung, the founder is yet to find a way to monetize it. Giphy can be integrated into various communication tools for inserting reaction GIFs into comments. Internet users also flock to the website to get entertained. The website claims to have 150 million users daily. With that kind of user base, it would not be difficult for the company to turn profitable.

Vishal Ingole, June 7, 2017

Snapchat Introduces Search Feature

May 29, 2017

Photo-sharing app Snapchat is late to the search game, but it has now arrived. The Daily Mail reports, “Snapchat Introduces a ‘Universal Search’ Feature: Tool Lets You Create Groups and Find New People to Follow.” Writer Abigail Beall explains:

Snapchat’s universal search bar hopes to address an issue some users had with the photograph-sharing app – the difficulty in finding new people to follow and gaining a large following. Previously, the only way people could gain a following was by sharing their username, or Snapcode, outside of the app. The new search bar, that will always be present at the top of the app, will allow people to find users easily through searching, discovering and groups. …

 

The new feature also lets users create groups, to combine snaps. Previously, boxes for finding specific conversations, accounts to follow and Stories or Discover channels were all in different places.

The tool was implemented for some Android users in mid-January, with availability to all Android and iOS users to follow “soon.” Beall notes the development was predicted by some last August after Snapchat acquired Vurb, a mobile search startup founded in 2011 and based in San Francisco.

Snap Inc., Snapchat’s parent company, bills itself as a camera company that is reinventing the camera. The company has acquired nine other enterprises since its founding in 2011. Snap is now selling (through their special vending machines!)  Spectacles, sunglasses with a camera on each temple that, of course, link right in with Snapchat.

Cynthia Murrell, May 29, 2017

Search Pinterest Pictures Without Pinterest

April 25, 2017

Pinterest is the beloved social media network, where users can post pictures, make comments, get decorating ideas, and recipes.  However, Recode tells us about a new implausible Google Chrome extension: “Pinterest Will Now Let You Search For Products Using Any Image You Find Online-Without Visiting Pinterest.”  Pinterest just launched a new Google Chrome extension that allows users to save images seen online as they browse.  The extension will work like this:

The new tool lets you select an item in any photograph online, and ask Pinterest to surface similar items using its image recognition software.  For example: If you see an image of sunglasses you like on Nordstrom.com, you could use the extension to browse similar glasses from Pinterest without ever leaving Nordstrom’s website.  If you click on one of the search results, you’ll then be taken to Pinterest.

Pinterest wants to leverage itself as an image search engine for all images, in real life and on the Internet.  Evan Sharp, Pinterest co-founder, said that users, should not “..have to put their thoughts into words to find great ideas.”  Visual search technology already exists, but only on Pinterest’s Web site.

Whitney Grace, April 25, 2017

Chipping Away at Social Content with Pictures

February 27, 2017

Analytics are catching up to content. In a recent ZDNet article, Digimind Partners with Ditto to Add Image Recognition to Social Media Monitoring, we are reminded images reign supreme on social media. Between Pinterest, Snapchat and Instagram, messages are often conveyed through images as opposed to text. Capitalizing on this, an intelligence software company Digimind has announced a partnership with Ditto Labs to introduce image-recognition technology into their social media monitoring software called Digimind Social. We learned,

 “The Ditto integration lets brands identify the use of their logos across Twitter no matter the item or context. The detected images are then collected and processed on Digimind Social in the same way textual references, articles, or social media postings are analysed. Logos that are small, obscured, upside down, or in cluttered image montages are recognised. Object and scene recognition means that brands can position their products exactly where there customers are using them. Sentiment is measured by the amount of people in the image and counts how many of them are smiling. It even identifies objects such as bags, cars, car logos, or shoes.”

 It was only a matter of time before these types of features emerged in social media monitoring. For years now, images have been shown to increase engagement even on platforms that began focused more on text. Will we see more watermarked logos on images? More creative ways to visually identify brands? Both are likely and we will be watching to see what transpires.

 Megan Feil, February 27, 2017

 

Pinterest Offers the Impulse Shopper a Slice of Wonderfulness

February 20, 2017

How about point-and-click impulse buying? Sound good? Pinterest has merged looking at pictures with spending money for stuff.

Navigate to “Pinterest’s New ‘Lens’ IDs Objects and Helps You Buy Them.” I know that I spend hours looking at pictures on Pinterest. When I see wedding snapshots and notice a pair of shoes to die for, I can buy them with a click… almost. My hunch is that some children may find Pinterest buying as easy as Alexa Echo and Dot buying.

I learned:

[Pinterest] announced a new feature called Lens, which will enable people to snap a picture of an item inside the Pinterest app. The app will then suggest objects it thinks are related. Think Shazam but for objects, not music. Surfacing the products will make it easier for people to take action, according to Pinterest. That could include everything from making a purchase to cooking a meal.

One of Pinterest’s wizards (Evan Sharp) allegedly said:

“Sometimes you spot something out in the world that looks interesting, but when you try to search for it online later, words fail you.” The new technology, Sharp said, “is capable of seeing the world the way you do.”

Isn’t the consumerization of no word search a life saver? Now I need a new gown to complement my size 11 triple E high heels. There’s a bourbon tasting in Harrod’s Creek next week, and I have to be a trend setter before we go squirrel hunting.

Stephen E Arnold, February 20, 2017

Blippar: Your Phone May Recognize You, Not Just a Product

January 4, 2017

I read “Blippar AI Visual Search Engine Recognizes Faces in Real Time.” The main point of the write up is that you can point your phone at something, and the phone will recognize that thing or person. The flip side is that if your phone has a camera which can see you, your phone makes it easy for “someone” to recognize you. Isn’t that special? Blippar info is at this link.

I learned:

Blippar expanded its augmented reality visual search browser on Tuesday to recognize faces in real time with a simple smartphone camera and return information about that person.

The write up talks about how consumers will drool over this feature. My thought was, “Gee, wouldn’t that function be useful for surveillance purposes?”

The write up included this statement:

The feature allows users to point the camera phone at any real person or their image in a picture on television and the Blippar app returns information about the person from the company’s database filled with more than three billion facts. Real-time facial recognition is the latest tool, amidst expansion in artificial intelligence and deep-learning capabilities.

Yep. Just another “tool.”

Blippar includes a feature for humans who want to be recognized:

For public figures, their faces will be automatically discovered with information drawn from Blipparsphere, the company’s visual knowledge Graph that pulls information from publicly accessible sources, which was released earlier this year. Public figures can also set up their own AR Face Profile. The tool enables them to engage with their fans and to communicate information that is important to them by leveraging their most personal brand — their face.  Users also can create fact profiles — Augmented Reality profiles on someone’s face, which users create so they can express who they are visually.Users can view each other’s profiles that have been uploaded and published and can add pictures or YouTube videos, as well as AR moods and much more to express themselves in the moment.

Why not convert existing images to tokens or hashes and then just match faces? Maybe not. Who would want to do this to sell toothpaste?

Stephen E Arnold, January 4, 2017

Google Buys Image Search: Invention Out

December 23, 2016

I read “Google Buys Shopping Search Startup to Make Images More Lucrative.” The Alphabet Google thing has been whacking away at image search for more than a decade. I have wondered why the GOOG’s whiz kids cannot advance beyond fiddling with the interface. Useful ways to slice and dice images are lacking at Google, but other vendors have decided to build homes on the same technical plateau. Good enough is the watchword for most information search and retrieval systems today.

The news that the Google is buying yet another outfit comes as no surprise. Undecidable  Labs, founded by a denizen of Apple, wants to make it easy to see something and buy it.

Innovation became very hard for the Alphabet Google thing once it had cherry picked the low hanging fruit from research labs, failed Web search systems, and assorted disaffected employees from search, hardware, and content processing companies.

Now innovation comes from buying outfits that are nimble, think outside the Google box, and have something that is sort of real. According to the write up:

The acquisition suggests that Google, the largest unit of Alphabet Inc., is making further moves to tie its massive library of online image links with a revenue stream.

eBay is paddling into the same lagoon. The online flea market wants to make it easy for me to spot a product I absolutely must have, right now. Click it and be transported to an eBay page so I can buy that item. Google seems to be thinking along a similar line, just without the “old” Froogle.com system up and running. Google’s angle will make an attempt to hook a search into a product sale. Think of Google as an intermediary or broker, not a digital store with warehouses. Yikes, overhead. No way at the GOOG. Not logical, right?

Earlier efforts around online commerce have delivered mixed results at Google. The company’s mobile payments have yet to see significant pickup. Its comparison shopping service, which facilitates online purchases within search results, has growing traction with advertisers, according to external estimates.

Perhaps one asset for the GOOG is that the founder is Cathy Edwards. I wonder if she wears blue jeans and a black turtle neck. What are the odds she uses an Apple iPhone?

Stephen E Arnold, December 23, 2016

Physiognomy for the Modern Age

December 6, 2016

Years ago, when I first learned about the Victorian-age pseudosciences of physiognomy and phrenology, I remember thinking how glad I was that society had evolved past such nonsense. It appears I was mistaken; the basic concept was just waiting for technology to evolve before popping back up, we learn from NakedSecurity’s article, “’Faception’ Software Claims It Can Spot Terrorists, Pedophiles, Great Poker Players.”  Based in Isreal, Faception calls its technique “facial personality profiling.” Writer Lisa Vaas reports:

The Israeli startup says it can take one look at you and recognize facial traits undetectable to the human eye: traits that help to identify whether you’ve got the face of an expert poker player, a genius, an academic, a pedophile or a terrorist. The startup sees great potential in machine learning to detect the bad guys, claiming that it’s built 15 classifiers to evaluate certain traits with 80% accuracy. … Faception has reportedly signed a contract with a homeland security agency in the US to help identify terrorists.

The article emphasizes how problematic it can be to rely on AI systems to draw conclusions, citing University of Washington professor and “Master Algorithm” author Pedro Domingos:

As he told The Washington Post, a colleague of his had trained a computer system to tell the difference between dogs and wolves. It did great. It achieved nearly 100% accuracy. But as it turned out, the computer wasn’t sussing out barely perceptible canine distinctions. It was just looking for snow. All of the wolf photos featured snow in the background, whereas none of the dog pictures did. A system, in other words, might come to the right conclusions, for all the wrong reasons.

Indeed. Faception suggests that, for this reason, their software would be but one factor among many in any collection of evidence. And, perhaps it would—for most cases, most of the time. We join Vaas in her hope that government agencies will ultimately refuse to buy into this modern twist on Victorian-age pseudoscience.

Cynthia Murrell, December 6, 2016

 

Is Sketch Search the Next Big Thing?

December 5, 2016

There’s text search and image search, but soon, searching may be done via hand-drawn sketching. Digital Trends released a story, Forget keywords — this new system lets you search with rudimentary sketches, which covers an emerging technology. Two researchers at Queen Mary University of London’s (QMUL) School of Electronic Engineering and Computer Science taught a deep learning neural network to recognize queries in the form of sketches and then return matches in the form of products. Sketch may have an advantage surpassing image search,

Both of those search modalities have problems,” he says. “Text-based search means that you have to try and describe the item you are looking for. This is especially difficult when you want to describe something at length, because retrieval becomes less accurate the more text you type. Photo-based search, on the other hand, lets you take a picture of an item and then find that particular product. It’s very direct, but it is also overly constrained, allowing you to find just one specific product instead of offering other similar items you may also be interested in.

This search technology is positioning itself to online retail commerce — and perhaps also only users with the ability to sketch? Yes, why read? Drawing pictures works really well for everyone. We think this might present monetization opportunities for Pinterest.

Megan Feil, December 5, 2016

Next Page »

  • Archives

  • Recent Posts

  • Meta