Among the big new features Apple revealed with the Apple Watch Series 12 and Apple Watch Ultra 4 were a few that pricked the ears of those with privacy concerns. Apple's Audio Intelligence is a suite of AI-powered features that use the Apple Watch to listen in on what's happening around you, so that Siri can provide more proactive advice, as well as help you remember what it was you just heard.
The most controversial of these features were Siri Recap, which allows your watch to listen in on conversations you have throughout the day, and uses AI to take notes and generate summaries, and Live Rewind, which lets you go back 15 seconds and view a transcript of what was just said.
The idea behind this is that Apple can "jog your memory" on events from your day, and make sure you can stay present in the moment. Naturally there are a lot of people who aren't happy about the idea that someone's Apple Watch could be listening in on conversations without warning.
Apple has anticipated this, and when we spoke to John Ternus, the new Apple CEO, and Greg Joswiak, Apple's senior vice president of worldwide marketing, they revealed their approach to Recap's privacy at length.
"Whenever we do anything, we always do it with an eye towards privacy and security," Ternus said. "It's something we take very seriously and approach very thoughtfully.
On the subject of Siri Recap, Ternus pointed out that it's been possible to record audio and create transcripts of what people have said for a while — both on smartwatches and phones.
Ternus makes it clear that recap does not produce a recording or a transcript, with Jozwiak adding that Apple can't access what doesn't exist. "If we don't have a recording, it means no one else has the recording. It can't get hacked. It can't get delivered to somebody."
It sounds unrealistic, but Jozwiak does confirm that Apple had to develop a whole new chip to make Recap possible in this manner. In the process, he sees it as a feature that allows people to concentrate on what they're doing, while being able to generate "higher level notes" that aren't indiscriminately documenting whatever sensitive information is being said.
"We wanted to do this in a way that people feel comfortable using it, said Ternus. "And people around the user feel comfortable about its existence."
Siri Recap's privacy features
Apple has also posted a sizable explainer discussing all the different privacy features Recap incorporates.
The big privacy feature here is that any audio that the Watch hears is sent to Apple's Secure Enclave on the watch itself or your paired iPhone. Apple claims that it can't be accessed by the user, Apple, apps or even the operating system itself.
The audio also isn't recorded or stored, though the processed data is summarized and formatted by Apple's Private Cloud Compute. Apparently this involves condensed versions of the original conversation that are less than half their length, reducing the contents to the core, rather than a full-length transcript.
While the Private Cloud Compute data is not accessible by Apple or anyone else, it does still mean your data is being processed on the cloud rather than just on your device. Which may be too risky for some of you to agree with.
While the Private Cloud Compute data is not accessible by Apple or anyone else, it does still mean your data is being processed on the cloud rather than just on your device.
Other features include a lack of speaker attribution, which means nobody will be directly quoted or attributed to. Siri is also going to censor what it hears, and will automatically omit anything it deems harmful or sensitive. Apple says this includes language promoting self-harm, hate speech, financial data, authentication and personally identifying data and "government-assigned identifiers." So it won't be making a record of your social security number.
Summaries are automatically deleted after seven days unless you save them to the Siri app. Like all Siri data, those saved summaries can then be deleted from all synced devices at any time — though you can also export them to other apps, including Apple Notes. Syncing is also encrypted and done via iCloud, meaning only your own trusted devices can access those summaries.
Recaps aren't running all the time, and you do need to opt in to actually set them up. How they record depends on you, with Apple promising you can set up a schedule or manually toggle it on and off in the Control Center.
Because it's an AI feature, Apple has the usual caveat explaining that the summaries aren't going to be 100% accurate all the time. That doesn't fill me with a whole lot of confidence, especially after the fiasco with Apple News summaries that had to be recalled from iOS.
Live Rewind
If you've ever been pulled in two directions at once, you've inevitably had to ask "what did you just say?" to someone. Live Rewind is meant to help jog your short-short-term memory, by displaying a text snippet of the last 15 seconds of conversation when you double-press the Digital Crown on your Apple Watch.
To address privacy concerns around this feature, Apple says that these text snippets "disappear permanently from your Apple Watch approximately 30 seconds after the Apple Watch display dims," unless you purposely save them.
This doesn't fix everything
Apple suggests that people "be mindful" of everyone around them, especially if there are sensitive conversations happening. Though that may be giving people too much credit.
There are plenty of people out there that are considerate of other people's thoughts and opinions, but there are also plenty who aren't. They either forget recap is running, or straight up do not care that other people may not like it.
Admittedly, both Ternus and Jozwiak have a point: there are features that offer this sort of thing without any emphasis on privacy. Then again, it is more likely that people take issue with the implied always-listening approach Recap takes. I personally feel it would make a lot more sense to restrict it to manual control.
Whichever way you look at it, though, I guess it's still a better option than seeing people pull out their phones to video every little unusual thing they come across.
Follow Tom's Guide on Google News and add us as a preferred source to get our up-to-date news, analysis, and reviews in your feeds. Subscribe to Tom's Guide on YouTube and follow us on TikTok. Finally, you can visit our dedicated Tom's Guide Savings Squad hub for expert help on getting the best products for less.
More from Tom's Guide
