Apple Watch Series 12 and Ultra 4 are getting two surprisingly futuristic new tricks: Live Rewind and Siri Recap.
Live Rewind is being marketed as a way to rewind the world around you. Actually, it can only turn the previous 15 seconds of a conversation into a text snippet. Still, it has the potential to help people remember something that’s just been said.
And Siri Recap quietly takes notes during your conversations then uses Apple Intelligence to create concise summaries you can refer to later. The goal is to help people stay focused on the moment.
They can be quite useful, but these Audio Intelligence features make privacy advocates nervous. Fortunately, Apple really locked them down hoping to prevent them from being abused.
The benefits of Apple Watch Live Rewind
During the Apple Watch portion of the Surprise and Shine event on Wednesday, Apple demoed using Live Rewind at a restaurant. A server quickly rattled off a lengthy list of daily specials, and a customer then used their Apple Watch to turn that rush of words into text they they can easily scroll through to make a decision (see image above).
“This is great to recall a book recommendation, a colleague’s brilliant idea or something you didn’t hear the first time,” said Ron Huang, Apple VP, Sensing and Connectivity.
Activate Live Rewind with a double press of the Digital Crown on a compatible Apple Watch to show the previous 15 seconds of a conversation as text.
Summarize your life with Siri Recap
Image: Apple
Siri Recap takes Apple’s new ambient-listening approach a step further. When enabled, Apple Watch can listen during conversations and use Apple Intelligence to create a high-level summary, complete with a title and key points.
The idea is to let you stay focused on a meeting, appointment or conversation instead of worrying about taking notes.
“It uses ambient listening to take high-level notes of conversations like takeaways from a team meeting or parent-teacher conference,” said Huang.
Audio Intelligence features alarm privacy hawks
Because Live Rewind is activated after the words it transcribes have already been spoken, the Apple Watch has to be constantly monitoring everything said near it.
The same goes for Siri Recap. In order to create the summaries, it must be listening.
It’s no surprise that privacy advocates are raising red flags about the Audio Intelligence features. But Apple built in quite a few privacy safeguards.
Apple locked down Live Rewind to prevent abuse
First off, the Apple Watch wearer must activate these features in Settings — they are off by default. So it’s up to the user to decide whether to use them.
For Live Rewind, the user can not access the temporary audio recording, only a text transcription. And Apple can never access it.
The watch’s S11 chip includes a hardware-isolated Secure Exclave where the temporary audio buffer is held. When Live Rewind is activated, the relevant audio is securely transferred to the Secure Exclave on the paired iPhone for transcription. The raw audio is then deleted. The process happens entirely on the Watch and iPhone, not a remote server.
Plus, Apple also built a privacy warning into Live Rewind. Activating the feature produces an audible chime, even when the Apple Watch is in silent mode or connected to headphones. The display also shows an animation and microphone indicator so people nearby can see that the feature has been activated.
Siri Recap also takes privacy into account
The process is similar for Siri Recap. Apple says it doesn’t create a permanent audio recording, and the user can’t even get a full transcript of the conversation, only the summary.
The raw audio gets transferred from the Apple Watch’s Secure Enclave to be processed into bullet points inside the iPhone’s Secure Exclave then deleted. The audio is inaccessible to the user and Apple.
Neither of these Audio Intelligence features name the people involved in conversations. Siri Recap uses “Speaker A” and “Speaker B,” though the actual recap may contain names if these are mentioned in the conversation. Live Rewind doesn’t do attribution at all.
Siri Recap summaries can be saved in the Siri app, but they automatically disappear after seven days unless the user chooses to keep them.
An example of Apple Watch Live Rewind and Siri Recap privacy
Let’s consider an example of someone trying to abuse Live Rewind and Siri Recap to see how the privacy protections work.
Cousin Daryl hears Cousin Larry call Aunt Hildegard “an old besom.” He uses his Apple Watch to capture the phrase and goes off to tattle to their aunt.
But Larry doesn’t have a recording of Cousin Daryl’s voice. Or anything that labels the insult as coming from Daryl. All he has is some text on his Apple Watch screen that Larry could have easily faked. Not very convincing.
Siri Recap wouldn’t help Larry either. All it might say is that someone called Speaker A was criticizing Aunt Hildegard, but there’s no way to know who. That person might even be Larry himself.
Requires the latest hardware
Apple promises that Siri Recap and Live Rewind arrive in beta later this year. But take note: they’ll only work on Apple Watch Series 12 or Apple Watch Ultra 4, not earlier models.
Plus, an iPhone that supports Apple Intelligence must be paired with the Apple Watch for either of these Audio Intelligence features to be available.
