Google is expanding its multi-search capabilities with local results.

At Google I/O, Google Senior Vice President Prabhakar Raghavan announced new enhancements to the Lens Multisearch tool that allows you to perform searches with just images and a few words. A new mode called “Near Me” allows users to take a picture of an object and then find the results locally.

As Raghavan explains, you can take a picture of a dish and then search for restaurants that serve it. Google will then display a list of relevant restaurants nearby. To implement this feature, Google scans relevant photos from nearby locations and then matches them with the photos you upload. “Around me” is initially available in English and will expand “over time” to more languages.

Google is also releasing a feature called Scene Navigation. This allows the user to pan the camera and then enter a search phrase for an object in front. When describing the feature, Raghavan used the example of finding a nut-free chocolate bar in a supermarket. You can scan the entire shelf of chocolate bars and then view overlays that provide “useful insights” such as reviews of each object. I think Raghavan’s description of the function sums it up quite nicely.

The search giant first launched multi-search in April, but its purpose was primarily related to shopping and guided search. For example, you can take a screenshot or photo of the dress you want, then enter a color name. A multi-search on Google then brings up a list of results for similar-looking dresses of that color. Alternatively, you can take pictures of certain types of plants and look for “care instructions” to find relevant information.

But as Lou Wang, Google’s director of product management, hinted at the time of launch, Google’s multi-search feature could be used much more, and it appears that additional features are starting to arrive.

To use multi-search, open the Google app on Android or iOS, then tap the lens icon to the right of the search bar. From there, you can start your search by uploading a photo or screenshot from your gallery, or by taking a picture of an object in front of you. Then swipe up on the screen and select “Add to Search” to search for photos with related words or phrases. It’s still not entirely clear how Google will integrate scene navigation and my surroundings into its interface.

Updated at 1:51 PM EST: Updated to add additional context from Prabhakar Raghavan.

Leave a Comment