Google Reveals Search Live: An AI Tool Powered by Gemini That Utilizes Your Phone’s Camera


Google is introducing its AI-driven Search Live feature in the U.S. on Wednesday, enabling mobile users to access real-time information through voice commands and their phone camera. Formerly part of Google Search Labs, Search Live is now accessible on the Google app for both iOS and Android.

Search Live is integrated into Google’s AI Mode search tool, which is powered by Gemini, allowing users to pose questions and receive conversational responses with the help of AI. It features a conversational voice capability and enables Gemini to visualize what users are seeing. By sharing their camera feed with Google, users can obtain real-time insights based on what appears in the frame.

Announced in June, the Search Live feature previously required an opt-in via Google Labs. Now, all U.S. users can utilize it by tapping the Live button located beneath the search bar in the Google app, enabling them to pose questions vocally or share their camera view.

Moreover, Search Live will be available through Google Lens. Users can select the Live icon within the app to initiate their conversation, as camera sharing in Google Lens is automatically enabled.

Mashable had the opportunity to test the feature prior to its official launch. During a live demonstration with Google, Gemini promptly recognized objects, products, games, and more, engaging in dialogue with the user about these items.

In our observations, even with five to ten objects in the camera’s perspective, Gemini accurately identified and responded to inquiries related to them, regardless of whether the items were oriented upside down.

Google showcased scenarios where Search Live could prove beneficial, such as repairing a malfunctioning appliance or understanding the rules for a board game lacking a manual. The camera feed can also assist with technology troubleshooting, offering guidance for a new home theater configuration. With voice commands, Search Live provides a hands-free experience.

The official launch of Search Live and its prominent placement within the Google app emphasize Google’s commitment to AI Mode and AI-enhanced search capabilities over its traditional search offerings. This occurs amid persistent concerns regarding the precision of AI chatbots and hallucinations, particularly in relation to AI Overviews.

Currently, Search Live is available solely in English, but Google aims to introduce support for additional languages and regions in the future.