4 new AI-powered Pixel and Android accessibility updates

Working with the disability community to advance accessibility innovation is a core part of our mission to build for everyone. Here’s a look at our latest accessibility features that use AI — from Pixel Camera updates to more widely available captions.

Get more suggestions to capture the perfect shot

Use Guided Frame to take great selfies and photos

Guided Frame, built with and for people with blindness or low vision, lets you take great photos and selfies with your Pixel Camera. Guided Frame will give spoken guidance to help you not only get your face in the frame, but also position the camera angle just right. Prompts will tell you to tilt your face up or down, or pan left to right before the camera auto-captures the photo. It will also announce that “lighting is low,” so you’ll know to move your phone to find a better shot. The latest update to Guided Frame also delivers improved object recognition, intelligent face filtering in group photos, and better focus on subjects even in complex scenes.

Previously, Guided Frame was only accessible through TalkBack, Android’s screen reader. Now you can also turn the feature on right from the camera settings, making it even easier for those who don’t use screen readers.

Find the information you need with your Pixel Camera

Different influencers with disabilities share their experience on using Magnifier



10:25

Last year we partnered with the low-vision community to release Magnifier, a Pixel-exclusive app, that uses your phone’s camera to help you zoom into the world around you. Now, we’re using AI to help you do even more with Magnifier.

  • Search for specific words in your surroundings. This is especially helpful if you’re looking for your flight’s departure time at the airport or trying to find a vegetarian dish on a menu.
  • Use a picture-in-picture format so you can see both the bigger context of what you’re looking at along with the details on your screen. For example, if you’re at a deli counter and want to take a closer look at the menu board, snap a photo and use picture-in-picture to check out all of your options without losing your place.
  • Choose the best lens for the moment in your app settings, whether you’re zooming in with a macro lens to read the fine print or busting out the wide-angle lens to get more context.
  • Put the camera on you! Turn on Selfie Illumination to use your front-facing camera as a mirror for your touch-up.

Use Magnifier to locate text in your surroundings

Magnifier is available on Google Play for Pixel 5 and up.1

Turn on a new Live Transcribe mode — just for foldable phones

Live Transcribe on Android helps over a billion people get real-time transcriptions of speech and sounds around them. Based on community feedback, we learned that it would be helpful if all the speakers in a conversation could see their own transcriptions while using Live Transcribe. So today, we're launching a new dual-screen mode for foldable phones, including the new Pixel 9 Pro Fold. With dual-screen mode, you can easily set your phone in a tabletop posture on any surface for better visibility of transcriptions. Now everyone around the table can follow the conversation — whether you’re attending a meeting or having a dinner conversation with friends.

Side by side of a Pixel fold in full screen mode on the left with Live Transcribe dual mode selected and on the right the other side of the phone showing the same transcription

Live Transcribe Dual Mode enables everyone to follow along with the conversation

Use Live Caption and Live Transcribe in even more languages and places

With Live Caption, you can get real-time captions for anything that has sound on your Android device across all your apps. Today we’re adding Korean, Polish, Portuguese, Russian, Chinese, Turkish and Vietnamese to Live Caption, expanding the availability of the feature.

We’re also making these seven languages available on-device for Live Transcribe. This new update will make it possible to use Live Transcribe in up to 15 languages even when you don’t have service or Wi-Fi, like when you’re on a subway or have your phone on airplane mode. Live Transcribe will continue to work in over 120 languages with an internet connection.

Two phones showcase Live Caption's available languages and live Caption in Korean being used on a social media post

Use Live Caption, now with even more languages, to get captions across all your apps

We’ll continue working with the disability community and use advances in AI to make the world a more accessible place. Check out our What’s New in Google Accessibility video series to learn about new accessibility launches.

Blog Article: Here

  • Related Posts

    60 of our biggest AI announcements in 2024

    Recap some of Google’s biggest AI news from 2024, including moments from Gemini, NotebookLM, Search and more.

    Our remedies proposal in DOJ’s search distribution case

    Today we filed Google’s remedies proposal based on the actual findings in the Court’s decision — our agreements with partners to distribute search.

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    You Missed

    Announcing CodeQL Community Packs

    60 of our biggest AI announcements in 2024

    60 of our biggest AI announcements in 2024

    Our remedies proposal in DOJ’s search distribution case

    Our remedies proposal in DOJ’s search distribution case

    How Chrome’s Autofill can drive more conversions at checkout

    How Chrome’s Autofill can drive more conversions at checkout

    The latest AI news we announced in December

    The latest AI news we announced in December

    OpenAI’s latest o1 model now available in GitHub Copilot and GitHub Models

    OpenAI’s latest o1 model now available in GitHub Copilot and GitHub Models