The Metropolitan Museum of Art Tackles Image Search: A Missed Block Halts the Speeding Researcher
March 23, 2018
The Met Wins The War to Get Online
For a few decades, art and history museums have been struggling with their online presences. The experience of seeing a Jpeg of a painting or sculpture is not the same as seeing it in person. That’s true. But there is one area where museums are holding a lot of valuable data and just now it’s starting to be searchable. We discovered this recently when the Metropolitan Museum of Art ‘s database “MetPublications.”
According to the page:
“MetPublications includes a description and table of contents for most titles, as well as information about the authors, reviews, awards, and links to related Met titles by author and by theme. Current book titles that are in-print may be previewed and fully searched online, with a link to purchase the book. The full contents of almost all other book titles may be read online, searched, or downloaded as a PDF.”
This includes over five hundred books about various exhibits that have spanned the last five decades. These slim volumes, usually released in conjunction with various exhibits, is fully searchable and a huge score for art lovers and historians. Previously, it was seen as too daunting and, potentially impossible. As far back as 2002 Computer Weekly was bemoaning the fact that museums had missed the digital boat. Turns out museums like the Met didn’t miss the boat, it’s just that their ship sails a little more slowly than the white knuckle world of Silicon Valley.
Stepping back, Beyond Search has noted that image collections remain difficult to use. Browsing often works best. Searches can be frustrating. Results rendering sluggish. Interfaces confusing. Even commercial image search systems are challenging. For example, locate the Google “Life Magazine” collection. Now try to find the image from the 1950s showing a child jumping from a tank in the side yard of the Smithsonian Museum. Impossible, right? (I know. The kid was my much younger self, and my family kept the page from the magazine but it was misplaced. Before my father died, he wanted me to locate that image. Fail even with three of my researchers beavering away.)
Consequently useful reference resources fall short of the mark. That often makes a museum visit necessary. And getting access to certain content remains difficult.
Stephen E Arnold, March 23, 2018
Google: Image Search Accuracy
February 16, 2018
Google image search is a hot topic. I wanted to test the functionality of the system because Google killed the “view image” option. Google really wants to be best friends with copyright holders. I took this image of myself:
I loaded it into the Google image “search by image” function. Yep, that’s the little camera graphic for those of my gentle readers who do not understand Google’s wonderful iconography.
Here’s what Google delivered as “similar” and “matching” images.
Notice anything interesting?
I am flattered that Google thinks I look like the female singer Mpume.
With precise image matching like this, Google may want to cease development of its system. What do you think about the thumbnail images. Yep, just like me. Oh, did I mention I am Caucasian? But if Google sees me differently, I go with Google. The company’s “algorithms” are the dope.
By the way, if you want the “old” image search function, try this link.
Stephen E Arnold, February 16, 2018
Google Images Staring down Some Steep Competition
January 5, 2018
When we are looking for photos online Google Images has become a sort of shorthand for tracking down pics quick. The folks in Mountain View don’t want you to think much about its shortcomings. However, that topic is bubbling up to the surface, as we saw in a recent Free Technology for Teachers story, “5 Good Alternatives to Google Image Search.”
According to the story:
Google Images tends to be the default image search tool of students and adults who haven’t been introduced to better options. Google Images is convenient, but it’s not the best place for students to find images that are in the public domain or images that have been labeled with a Creative Commons license.
One they recommend is:
Unsplash offers a huge library of images that are either in the public domain or have a Creative Common license. If you or your students are using Google Slides, the Unsplash add-on for Google Slides makes it easy to quickly take images from Unsplash and add them to your slides. Watch my video embedded below to see how the Unsplash add-on for Google Slides works.
This should be a wakeup call for Google. The tech giant seems to have a new balloon popped every day. We love competition and we love leaders reinventing themselves to better meet client needs. We consider this to be a win-win no matter how you slice it.
Patrick Roland, January 5, 2018
Google Supports Outraged Scholars
October 2, 2017
Google has taken issue with a recent list from the Campaign for Accountability (CfA), TechCrunch reports in, “Google Responds to Academic Funding Controversy—with a GIF.” Writer Frederic Lardinois reports that the CfA recently released a list of policy experts and academics who, they say, had received Googley dollars last year. The only problem—many who found themselves on the list dispute their inclusion, saying they had not received any funding from Google or, if they had, it was unrelated to the work the CfA specified. Google issued a response, supporting the protesting experts and academics as well as defending its support of researchers in general. The company also struck back; the article explains:
And in a direct attack on CfA, Google also notes that while the group advocates for transparency, its own corporate funders remain in the shadows. The only backer we know of is Oracle, which is obviously competing with Google in many areas. The group has also recently taken on SolarCity/Tesla. In its blog post, Google also argues that ‘AT&T, the MPAA, ICOMP, FairSearch and dozens more’ fund similar campaigns.
Google later created a GIF in response to requests for elaboration. It shares a series of tweets from some of the affected scholars, in which they detail just where the CfA went wrong in each of their cases. Lardinois continues:
It’s not often that a company like Google makes its own GIF in response to a request for comment, but I gather this goes to show that Google wants to move on from this discussion and let the academics speak for themselves. While the CfA’s methods are less than ideal, there are legitimate questions about how even small amounts of funding can influence research.
So far, Lardinois notes, public discussion on how funding can influence research have centered around pharmaceuticals. He projects it will soon grow, however, to include policy research as tech companies ramp up their funding programs
Cynthia Murrell, October 2, 2017
Searching for a Disguised Face? Some Progress
September 9, 2017
I read “Meeting the Disguised Face Challenge via Deep Convolutional Network.” My interest was piqued because I thought I had seen references to a breakthrough in facial recognition when the subjects of interest were wearing disguises.
I noted these comments in the write up:
- The method: “Deep convolutional networks are software creations organized into interconnected layers, much like the visual cortex, the part of the brain that processes visual information.”
- The accuracy: “The fewer facial key points it can see, the worse the software is at recognizing a person in a photo. It’s also thrown off by busy backgrounds, so can only identify a person wearing a cap, glasses and scarf 43 per cent of the time if they’re standing in front of a complicated background.”
- Work around: “Wearing a rigid mask that covers the whole face, for example, would give current facial recognition systems nothing to go on. And other researchers have developed patterned glasses that are specially designed to trick and confuse AI facial recognition systems.”
To sum up: Promising but real time facial recognition of people walking through an airport entrance remains a challenge. In addition to the computational demands, the false positives can quickly consume available resources.
Stephen E Arnold, September 9, 2017
Helpful Search Operators for Google Users
July 31, 2017
We have found a resource that can help readers google like never before: GoogleGuide’s article is titled simply, “Search Operators.” Unsatisfied with the information she found at Google’s website, mathematician and search enthusiast Nancy Blachman started GoogleGuide to enlighten us all on advanced Google Search methods. In “Search Operators,” she and colleague Jerry Peek educate us on one exacting approach. They write:
The following is an alphabetical list of the search operators. This list includes operators that are not officially supported by Google and not listed in Google’s online help. Note: Google may change how undocumented operators work or may eliminate them completely. Each entry typically includes the syntax, the capabilities, and an example.
The article leads with a table listing the search operators next to the relevant Google service: Web search, image search, groups, etc., which can be cross-referenced with the alphabetical list. Operator functions include useful tasks like searching for specific pages by title, discovering who has linked to a certain website and restricting searches by file type. The team even concludes with a set of exercises for practice with the operators. Check it out to make your internet searches even more efficient.
Cynthia Murrell, July 31, 2017
Facebook: Search Images by the Objects They Contain
July 3, 2017
Has Facebook attained the holy grail of image search? Tech Crunch reports, “Facebook’s AI Unlocks the Ability to Search Photos by What’s in Them.” I imagine this will be helpful to law enforcement.
A platform Facebook originally implemented to help the visually impaired, Lumos (built on top of FBLearner Flow), is now being applied to search functionality across the social network. With this tool, one can search using keywords that describe things in the desired image, rather than relying on tags and captions. Writer John Mannes describes how this works:
Facebook trained an ever-fashionable deep neural network on tens of millions of photos. Facebook’s fortunate in this respect because its platform is already host to billions of captioned images. The model essentially matches search descriptors to features pulled from photos with some degree of probability. After matching terms to images, the model ranks its output using information from both the images and the original search. Facebook also added in weights to prioritize diversity in photo results so you don’t end up with 50 pics of the same thing with small changes in zoom and angle. In practice, all of this should produce more satisfying and relevant results.
Facebook expects to extrapolate this technology to the wealth of videos it continues to amass. This could be helpful to a user searching for personal videos, of course, but just consider the marketing potential. The article continues:
Pulling content from photos and videos provides an original vector to improve targeting. Eventually it would be nice to see a fully integrated system where one could pull information, say searching a dress you really liked in a video, and relate it back to something on Marketplace or even connect you directly with an ad-partner to improve customer experiences while keeping revenue growth afloat.
Mannes reminds us Facebook is operating amidst fierce competition in this area. Pinterest, for example, enables users to search images by the objects they contain. Google may be the furthest along, though; that inventive company has developed its own image captioning model that boasts an accuracy rate of over 90% when either identifying objects or classifying actions within images.
Cynthia Murrell, July 3, 2017
Giffying All the Way to Profits
June 7, 2017
Giphy, the GIF search engine has secured $150 funding at $600 million valuations. What started as a web crawler is on its way to profitability.
Business Insider in an article titled Inside the GIF Factory: How Giphy Plans to Build a Real Business by Animating the Internet says:
Giphy isn’t profitable yet. In fact, the company doesn’t even have a reliable means of generating revenue at this point. But now that GIFs are an ingrained aspect of online behavior, the company is hard at work drafting a blueprint to turn its popular service into a money-making business.
Though there are multiple ways to monetize GIFs, a mainstay of personal messages and online forums and social media networks, Alex Chung, the founder is yet to find a way to monetize it. Giphy can be integrated into various communication tools for inserting reaction GIFs into comments. Internet users also flock to the website to get entertained. The website claims to have 150 million users daily. With that kind of user base, it would not be difficult for the company to turn profitable.
Vishal Ingole, June 7, 2017
Snapchat Introduces Search Feature
May 29, 2017
Photo-sharing app Snapchat is late to the search game, but it has now arrived. The Daily Mail reports, “Snapchat Introduces a ‘Universal Search’ Feature: Tool Lets You Create Groups and Find New People to Follow.” Writer Abigail Beall explains:
Snapchat’s universal search bar hopes to address an issue some users had with the photograph-sharing app – the difficulty in finding new people to follow and gaining a large following. Previously, the only way people could gain a following was by sharing their username, or Snapcode, outside of the app. The new search bar, that will always be present at the top of the app, will allow people to find users easily through searching, discovering and groups. …
The new feature also lets users create groups, to combine snaps. Previously, boxes for finding specific conversations, accounts to follow and Stories or Discover channels were all in different places.
The tool was implemented for some Android users in mid-January, with availability to all Android and iOS users to follow “soon.” Beall notes the development was predicted by some last August after Snapchat acquired Vurb, a mobile search startup founded in 2011 and based in San Francisco.
Snap Inc., Snapchat’s parent company, bills itself as a camera company that is reinventing the camera. The company has acquired nine other enterprises since its founding in 2011. Snap is now selling (through their special vending machines!) Spectacles, sunglasses with a camera on each temple that, of course, link right in with Snapchat.
Cynthia Murrell, May 29, 2017
Search Pinterest Pictures Without Pinterest
April 25, 2017
Pinterest is the beloved social media network, where users can post pictures, make comments, get decorating ideas, and recipes. However, Recode tells us about a new implausible Google Chrome extension: “Pinterest Will Now Let You Search For Products Using Any Image You Find Online-Without Visiting Pinterest.” Pinterest just launched a new Google Chrome extension that allows users to save images seen online as they browse. The extension will work like this:
The new tool lets you select an item in any photograph online, and ask Pinterest to surface similar items using its image recognition software. For example: If you see an image of sunglasses you like on Nordstrom.com, you could use the extension to browse similar glasses from Pinterest without ever leaving Nordstrom’s website. If you click on one of the search results, you’ll then be taken to Pinterest.
Pinterest wants to leverage itself as an image search engine for all images, in real life and on the Internet. Evan Sharp, Pinterest co-founder, said that users, should not “..have to put their thoughts into words to find great ideas.” Visual search technology already exists, but only on Pinterest’s Web site.
Whitney Grace, April 25, 2017
-
- Subscribe to Beyond Search
Feature archive
News archive
-
