Whenever you use a video conferencing application, you are sending your audio data to the company hosting the services. And, according to a new study, that means everything of your audio data. This includes voice and background noise whether you are streaming or muted.
Researchers at the University of Wisconsin-Madison investigated “many popular apps” to determine how well video conferencing apps capture data while users use the software’s built-in “mute” button.
According to a university press release, their findings were substantial:
They used runtime binary analysis tools to trace raw audio in popular video conferencing applications when audio traveled from the application to the computer’s audio driver and then to the network while the application was on. mute.
They found that all of the apps they tested occasionally collected raw audio data when muted, with one popular app gathering information and providing data to its server at the same rate whether the microphone was muted or not.
Unfortunately, as this research is unpublished, we are unable to confirm the specific applications tested. So, for now, we can’t name and shame them.
However, the effectiveness of this document is not necessarily in doubt due to the fact that it was accepted at the 2022 Symposium on Privacy Enhancing Technologies. We’ll just have to wait and see who gets named when the paper comes out in June.
However, this does not mean that we cannot draw conclusions. According to the researchers, this data could be used to extract meaningful information. And, with a little machine learning, that information can paint an incredibly vivid picture of a user’s reality — again, even with your microphone muted in the app.
The research team was able to determine what specific sound was being sent during the tests, and by extrapolating this data, they were able to predict the inferences big tech might make.
Of course, big tech uses AI to analyze everything. So the researchers built their own algorithms to study the data. What they found was amazing.
According to the abstract of the unpublished article:
Using the network traffic we intercept en route to the telemetry server, we implement a proof-of-concept background activity classifier and demonstrate the feasibility of inferring ongoing background activity during a meeting – cooking, cleaning, typing, etc. We achieved 81.9% macro accuracy on identifying six common background activities using intercepted outgoing telemetry packets when a user is muted.
In other words: graduate students at the University of Wisconsin-Madison were able to create machine learning models that could determine what a teleconferencing app user was doing while their microphone was muted with greater than 80% accuracy.
File under: Imagine what Google or Microsoft could do with their massive AI models. Ouch!
Neural’s point of view: Should we be worried? Yes. Absolutely. Should I stop using these apps? No, because it’s not really an option for everyone.
Our real concern is that big tech doesn’t care if users know they’re being recorded even when muted, or it never stopped thinking about users would like to care. Either way, it shows a worrying level of detachment from the user experience.
The unspoken mantra of big tech is “data at all costs”, and that only proves how thirsty the beast is. There’s no good reason not to explicitly tell users in large print that the mute button doesn’t stop their audio streaming to the server.
Fortunately, you have options. If you really want to mute your audio stream, you need to double mute. If you’re lucky enough to own a headset that has a physical mute button, use it to mute your headset, then use the app button as an extra layer of mute.
If your headset doesn’t have a physical mute button, or you’re using a built-in microphone to communicate, you’ll need to mute the system by disabling your microphone from your operating system’s system settings, as well as disabling the app.
Ultimately, it’s unclear exactly what big tech is doing with this data. The scope of the study did not involve investigating big tech – and we’ll update this article if Zoom, Microsoft or Google come back to us with a statement.
But, whatever happens, the fact that it is collected under such misleading circumstances is a major source of concern.
Forcing users into operating system menus to ensure they achieve a minimum of privacy is an anti-user policy. And, what’s more, it demonstrates how much more sensitive our audio data can be than our video data.
As the study’s lead author, Kassem Fawaz, said in the university’s press release:
With a camera, you can turn it off or even put your hand on it, and no matter what you do, no one will be able to see you. I don’t think it exists for microphones.
Don’t forget to mute the sound, friends. You will probably forget to double reactivate from time to time, but the compromise prevents Google, Microsoft and everyone else from training their machines on the ambient sounds of your privacy.