Facebook’s AI lets you search pictures by what’s in them, no tags needed
If like most people you don’t take the time to tag and name all of your photos, then you know that searching for a specific picture can be a nightmare. Thanks to Facebook Inc. and artificial intelligence, that problem may soon be a thing of the past.
Facebook has been investing heavily in artificial intelligence research over the last few years, especially when it comes to computer vision. Today the social network announced an update that will allow users to search for images by content rather than by their name or tags.
“In other words, in a search for ‘black shirt photo,’ the system can ‘see’ whether there is a black shirt in the photo and search based on that, even if the photo wasn’t tagged with that information,” explained Joaquin Quiñonero Candela, director of engineering at Facebook.
The new feature takes advantage of Facebook’s Lumos AI, which uses computer vision to recognize people, animals and a number of other objects in a photo. Lumos can even take things step further by recognizing what is actually happening in a photo, so in addition to recognizing that a person is in the image, Lumos could also determine whether that person is walking, riding a horse, playing an instrument and so on.
According to Candela, the technology developed for Lumos has a wide range of applications beyond simply improving image searches.
“More than 200 visual models have been trained and deployed on Lumos by dozens of teams, for purposes such as objectionable-content detection, spam fighting, and automatic image captioning,” Candela explained. “The applications are wide-reaching, with everyone from our Connectivity Labs to Search to the Accessibility team using the technology.”
Facebook built Lumos using its FBLearner Flow platform, a general-purpose machine learning pipeline that the company created so that its engineers could implement multiple AI projects without having to provision the necessary hardware each time. According to Facebook, there are currently more than 1.2 million AI experiments running on FBLearner Flow each month, and all of these projects, including Lumos, are able to access the never ending stream of data coming out of the social network and its many apps.
Although Facebook is making some impressive progress with Lumos and similar projects, the social network is not the only tech giant working on computer vision. Alphabet Inc. has been developing its own picture-scanning AI to boost image searches, and the company boasted last year that its Inception v3 AI could caption the content of an image with 93.9 percent accuracy. Inception v3 made headlines again recently as the basis for a new AI designed to help doctors catch skin cancer early.
Image courtesy of Facebook
Since you’re here …
… We’d like to tell you about our mission and how you can help us fulfill it. SiliconANGLE Media Inc.’s business model is based on the intrinsic value of the content, not advertising. Unlike many online publications, we don’t have a paywall or run banner advertising, because we want to keep our journalism open, without influence or the need to chase traffic.The journalism, reporting and commentary on SiliconANGLE — along with live, unscripted video from our Silicon Valley studio and globe-trotting video teams at theCUBE — take a lot of hard work, time and money. Keeping the quality high requires the support of sponsors who are aligned with our vision of ad-free journalism content.
If you like the reporting, video interviews and other ad-free content here, please take a moment to check out a sample of the video content supported by our sponsors, tweet your support, and keep coming back to SiliconANGLE.