Google has updated Live Captions on Android by adding "emotion to your captions" to make it more useful. The feature, Expressive Captions, builds upon the pain that captions often don't convey the nuances of language and sound, including emphasis, tone and personality.
Captions or subtitles have been around for over half a century now. They were initially meant to assist TV audiences with hearing difficulty or loss but later became popular among people in general. One survey found that about 70% of Gen Z viewers in the US use subtitles often, for instance, when they are in a loud public space, on the subway, or to understand better what is being said.
Expressive Captions has landed alongside the Google Pixel December feature drop, which brings new camera features and expands Gemini's capabilities. It is another AI-powered feature from the search giant meant to communicate things like tone, volume, environmental cues, and human noises.
For instance, Expressive Captions can capitalize the text to tell users about the intensity of speech. It can identify sounds like sighing, gasping, and grunting and label additional noises in the foreground and background, such as applause and cheers.
"These small things make a huge difference in conveying what goes beyond words, especially for live and social content that doesn’t have pre-loaded or high-quality captions," Google said in a blog post, adding that Expressive Captions is a collaborative effort between Android and Google DeepMind teams.
The feature has started rolling out in the US, with support only for the English language. It can work natively on any Android device running Android 14 or later that has Live Caption. It can work across apps and make up for the lack of pre-loaded captions in live streams and social content. Moreover, the feature uses on-device processing to generate captions in real-time and even works when aeroplane mode is enabled.
1 Comment - Add comment