With Apple Watch that eavesdrops on conversations, what’s at stake is public perception
I’ve published more than a couple of pieces on this blog trying to raise awareness about the risks of viral photo-editing apps, like FaceApp back in 2019 [pt_BR].
We are curious beings. The promise of looking into a magic mirror that shows us old, as a different gender, or in another era is nearly irresistible — a force that makes many people ignore the warning signs, even the obvious ones.
This kind of thing used to be more sporadic and required some effort to take part in — at the very least, downloading a new app —, and a bigger leap of faith in unknown apps.
The arrival of ChatGPT’s image generator changed that.
The first hit was the Ghibli-studio trend. In early September, the new craze (at least here in Brazil) was photos with a 1980s aesthetic.
In practical terms, ChatGPT and those questionable apps of yore are no different in how they treat the content submitted to them. OpenAI’s privacy policy, in fact, is explicit about what was once only speculated due to lack of information. It confirms that data submitted by users — including images — is retained and used for various purposes, among them training new artificial intelligences:
As noted above, we may use Content you provide us to improve our Services, for example to train the models that power ChatGPT.
OpenAI, the company behind ChatGPT, although full of issues of its own, is several levels above the old viral photo-editing apps in terms of brand recognition and trust. So much so that hardly anyone talks today about the risks of sending your photos there.
The ends are the same, and so are the circumstances (going viral on social media). What changes is public perception. ChatGPT normalized what used to be met with some unease and a bit of apprehension. Now, nobody cares.
On LinkedIn, Pedro Burgos posted this provocation [pt_BR, loosely translated]:
I find it cute how people who automatically send every single photo they take to iCloud and Google Photos suddenly get scared of joining the [1980s] trend over privacy risks, scams, or whatever.
And worse, there are influencers cashing in on people’s fear.
Let folks be happy posting fun AI photos, people.
I replied to him that this is a false equivalence.
Both iCloud and Google Photos do use algorithms to recognize and group photos of specific people (faces), offer more precise search, and provide other conveniences to organize, retrieve, and edit images — but extrapolating from that to say it’s the same thing ChatGPT does is a whole different ballgame. There are fundamental differences between the two situations.
Both companies, Apple and Google, state in help documents that material stored by users in their photo services is not used to train models, AIs, or other applications.
From Apple:
[…] Apple does not access your photos or videos, and does not use them for research and development.
Photos uses on-device machine learning to deliver a personalized experience, organize photos and videos in ways that are meaningful to you, and power features like Sharing Suggestions, Memories, the People & Pets Album, and Featured Photos. […]
Photos additionally uses on-device analysis to recognize the faces of people in photos, and groups them together in the People & Pets Album. […]
From Google:
- Your personal data in Google Photos is never used for ads.
- Responses aren’t reviewed by humans, unless you give feedback, or to address abuse or harm (this is rare).
- We don’t train any generative AI models outside of Google Photos with your personal data in Google Photos.
Apple even offers end-to-end encryption for iCloud. It’s opt-in, but it’s there and it’s very easy to turn it on.
The features these apps offer are not rocket science. Even free-software projects with budgets infinitesimally small compared to Apple’s and Google’s, such as digiKam and Immich, offer facial recognition and other “smart” features. Which, obviously, is very different from claiming that both use photos to “train AI.”
***
Normalizing behaviors is a superpower of big tech that we don’t give the attention it deserves. That’s why I’ve been enjoying (and trying to contribute to) a timid, incipient shift in attitude on this front.
I spent a few weeks hitting hard at Meta’s smart glasses — the ones perverts use to record women without their knowledge or consent.
Are there legitimate uses for high-definition cameras hidden in glasses that look like any other pair? Certainly. Still, critics’ logic holds that the balance of normalizing them is negative. A world in which this kind of product becomes popular and stops shocking people would be an (even more) surveilled, (even more) oppressive world. And let’s be honest, it’s not as if we were suffering from a shortage of images. Quite the opposite. Privacy aside, we would be better off with fewer cameras, not more.
Last Wednesday 9th, Apple announced new Apple Watches with the ability to listen all the time. Of the four features that rely on those ever-on microphones, two are frightening — Siri Recap and Live Rewind. In the company’s own words:
- Siri Recap transforms your conversations into high-level notes, so you can stay focused and present. Each Siri Recap includes a title, a summary, and key points that you can review in the Siri app on your Apple Watch or iPhone. You can turn Siri Recap on or off at any time from Control Center or choose where and when it takes notes, such as only at work or never at night.
- Did someone say something you couldn’t hear in a loud restaurant or share cooking instructions you didn’t quite catch? Just double-click the Digital Crown, and Live Rewind transcribes what was said in the last 15 seconds. You can ask Siri about the content of the transcript or save it to the new Siri app to revisit later.
Apple spent precious minutes of its big annual September keynote and published a lengthy document explaining all the care the company took to “protect the privacy” of users, highlighting the mandatory deletion of raw audio after processing.
And… it doesn’t matter. The message that sticks in people’s minds is that the Apple Watch has become a constantly listening device. And although the new feature only works on the just-announced watches (Series 12 and Ultra 4), the distrust extends to every Apple Watch in use, since they are visually indistinguishable.
In an announcement lasting just a few minutes, Apple threw in the trash a decade plus effort of trying to convince people that their iPhones aren’t listening to their conversations to serve ads hours later.
One could argue, along the lines of Pedro’s provocation, that phones have had active microphones for ages, used to recognize wake words (“Hey Siri,” “Ok Google”). To just let people be happy recording everything and everyone around them.
To which I would reply, once again, that it’s not the same thing, for equally obvious reasons.
Beyond the reputational stain on one of Apple’s strengths (privacy), there is the fear that these watches will help normalize the constant recording of audio. Until now, only gizmos made by startups or AI companies carried that proposition. Marginal products, with no appeal to the general public and, therefore, ignorable.
Apple’s entry into the business of covertly recording conversations works as a validation of a very wrong idea. Those startups got free advertising, and Apple’s rivals got a green light to launch their own always-on microphones. To make matters worse, no matter how hard Apple worked to ensure the privacy of the spy microphone on its watches, smaller or less scrupulous companies won’t take the same care.
The only chance of containing these perverse intrusions is a broad, loud rejection of them. As happened with the late Google Glass and the glassholes and, to some extent, has been happening with Meta’s pervert glasses. The problem is that this time, on the other side stands the most powerful consumer products company on the planet.