At Google I/O, Google senior vice president Prabhakar Raghavan announced new enhancements for its Lens multisearch tool, which lets you conduct a search with just an image and a couple of words.
A new mode, called “near me,” will let users take a photo of an object and then find results locally. As Raghavan explained, you’ll be able to take a photo of a dish and then search for restaurants that serve that specific food. Google will then display a list of relevant restaurants near you. To make this feature happen, Google scans relevant photos from websites, as well as those posted by reviewers, and then matches them to the one you uploaded. Near me will be available in English later this year and will expand to more languages “over time.”
Google is also rolling out something called scene exploration. This will allow users to pan their camera and then enter a search phrase about the objects in front of them. When explaining the feature, Raghavan used the example of trying to find a nut-free chocolate bar in a supermarket. You’ll be able to scan an entire shelf of chocolate bars and then see overlays that provide “helpful insights,” like reviews about each object. We think Raghavan’s description of the feature sums this up quite nicely: “This is like having a supercharged Ctrl+F for the world around you.”
:format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/23453709/google_multisearch_scene_exploration.png)
The search giant first rolled out multisearch in April, but its purpose was primarily tied to shopping and searching for instructions. For instance, you can take a screenshot or picture of a dress you like and then type in the name of a color. Google’s multisearch will then show a list of results with similar-looking dresses in that color. Or you can take a photo of a specific type of plant and look up “care instructions” to find relevant information.
But, as Lou Wang, Google’s director of product management, hinted at during its release, Google’s multisearch feature can be used for a lot more, and it looks like that additional functionality is starting to arrive.
To use multisearch, open up the Google app on Android or iOS, and then hit the Lens icon on the right side of the search bar. From there, you can upload a photo or screenshot from your gallery — or snap a picture of an object in front of you — to begin your search. Then, swipe up on the screen and select “Add to your search” to search for the photo alongside a relevant word or phrase.
Update May 11th, 1:51PM ET: Updated to add additional context from Prabhakar Raghavan.
Update May 11th, 2:12PM ET: Updated to add Google’s relevant blog post and additional information related to the new features.
