This is probably a three years old problem now, but I don’t see any discussion about this so I’m gonna start the topic, TL;DR: apps on any Android/iOS phone could use accelerometer to eavesdrop on “trigger words” at near-100% accuracy w/out ur consent.
TIL that the Android (and also iOS) currently doesn’t limit the permission to access the accelerometer unlike how it restricts mic and camera permissions, i.e. any installed app can read it at any time without that “Allow to access ?” popup;
And I just read about a 2020 research paper, Learning-based Practical Smartphone Eavesdropping with Built-in Accelerometer, which explored the possibility to eavesdrop on users without requesting for mic access, by training a deep learning model to infer sound waves of people talking from accelerometer input.
[PDF] Learning-based Practical Smartphone Eavesdropping with Built-in Accelerometer | Semantic Scholar (This is a Chinese research paper, I’ll take it with a grain of salt,)
It states that sound waves can trigger a slight change in the accelerometer reading, which is visible on a waveform diagram, as demonstrated at 6:00 of the video Are apps eavesdropping on our conversations? (Chinese, Bilibili)
As there is no permission management for the accelerometer currently, any installed app has the potential to do this, normally the user has no defense. The paper states that based on their model, recognition accuracy can reach 99% in moderate or low noise environments, and even 100% in noise-free environments! The cost of acquiring this data is low, as simple trigger words like “Hey Siri” can be set up, and input from many people saying the trigger word can be used to train the model and achieve high accuracy.
After I saw the above on a channel I watch, I did a web search for “accelerometer eavesdropping” and found out that research on this topic has been done by Chinese researchers 2-3 years ago, while lately Western researchers also demonstrated this: EarSpy: Spying On Phone Calls Via Ear Speaker Vibrations Captured By Accelerometer - Slashdot
There is naturally two questions that come from this research paper - Is this method already being exploited in the wild by proprietary, big-brother apps to profile user interests or even target specific groups? And how can we protect ourselves?
I don’t have a very good answer for either of the questions; For the first question, I believe that Chinese apps (e.g. TikTok, AliExpress, Weibo i.e. China’s Twitter) might have already pioneered the use of this, because the Chinese firms probably already applied this technology when they hired researchers to research it and publish the result. Many privacy-minded Chinese social media users have already felt it, something getting accurately recommended by algorithm after they talked about it orally despite never searching for it. So if you have TikTok installed it might be eavesdropping on you Right NOW using this method. As for western big-brother apps such as Facebook, Instagram, and Twitter, they might also be using this, less likely than the Chinese apps though.
You could say I’m too paranoid and you’re probably right, but if an eavesdropping technology can achieve so high accuracy and so low cost with just a little training, and it is also unregulated, I don’t see why companies won’t want to exploit it. The thing is you can’t be 100% sure if your apps are committing this, because extracting the accelerometer data on both Android and iOS currently doesn’t require any permissions, thus no asking for permissions and no logs in the system App Privacy Report/Privacy dashboard. The only way to know is to reverse engineer these apps.
The comments of the video Are apps eavesdropping on our conversations? (Chinese, Bilibili) is filled with people saying that they have this experience - apps on their phone without microphone access suggesting a product they discussed orally. If you have encountered this too plz let me know below.
Secondly, how can we protect ourselves?
It is worth mentioning here that eavesdropping through accelerometer, although stealthy, cannot achieve the accuracy of using microphone. It is not (yet) realistic to achieve full TTS with this method, only trigger words can be set up. For example a car app has to first set up the trigger word “Model 3” and continuously to monitor the accelerometer in the background; then it gets triggered when you talk about cars with a friend irl and perhaps mentioned “Model 3” once. And then the car app starts showing Tesla ads and you feel that the app’s recommendation is hella accurate.
Completely prohibiting apps to read the accelerometer is hard as mobile OSs currently don’t give us this option. If your Android has the option to globally disable the accelerometer in the developer options, that’s best, but not every phone has this option, I can’t find this in my Pixel developer options. It might also cause inconveniences like apps unable to distinguish between landscape and portrait orientation. If you’re rooted, using the XPrivacyLua module to disable the accelerometer for specific apps could work, but it requires putting trust into an additional module’s developer instead of simply trusting Android system to do its job, and XPrivacyLua is recently discontinued and won’t support future Android versions. The best method I can think of is to avoid using proprietary apps completely or to always force stop them after use. Of course, if you have any better suggestions, let me know.
I believe the only way that really works to protect users is for systems like Android and iOS to add a new permission for acceleration readings. Apps should have to ask for this permission before they read accelerometer value. Really this should have been the case from day one, as except fitness apps, compass apps and racing games I can’t think of any legitimate reason for apps to read the accelerometer.
