AI for Privacy Utilization - TLDR a privacy policy

General
1

I know there is some manner of distrust towards the efficacy of AI / LLMs, but honestly I’ve found a use case that is quite solid.

I am utilizing Proton’s Lumo for this, at which I simply ask the following question

You are an experienced legal consultant tasked with analyzing the privacy policy for anything that would flag for surveillance capitalism. Identify areas of risk associated with a degree of [high, medium, low] confidence towards the likelihood towards exploitation. Further, take note of how much this likely compares with other services’ policies with a degree of [close , somewhat , distant]. Communicate assumptions towards confidence levels with justified explanations. Do not assume maliciousness or benevolence, focus on facts. With this information, provide a table which summarizes the risk, the confidence band, the justification towards that risk, possible mitigations and remediations against this, and the similarity band to other services.
{{URL}}

At which I get a completely digested policy with possible mitigations towards said policy. I don’t actually assume high / medium / low is actually of that risk, but I mainly value the possible directions and ideas I can use to determine if its worth it to continue forward.

Example utilizing Tuta's Privacy Policy

Privacy Statement | Tuta

Risk Area Confidence Band Justification Possible Mitigations Similarity to Other Services
End-to-End Encryption Claims Low Document states user data encrypted such that Tutao GmbH has no access to unencrypted data. This fundamentally limits ability to monetize content. Verify technical implementation independently (security audits, source code review) Close to Proton Mail; distant from Gmail/Outlook/Yahoo which scan emails for ad targeting
Metadata Collection Medium Email addresses of senders/recipients, timestamps, and subject line data stored unencrypted. Necessary for service but creates communication graph metadata. Request metadata deletion after delivery; minimize metadata in communications Somewhat close to Proton Mail; closer to baseline industry minimum; distant from services collecting message content for targeting
Device-Level Random ID Tracking Medium Single random ID shared by all accounts on one device, transmitted with usage statistics. While anonymous, creates persistent device fingerprint enabling cross-account correlation within Tuta ecosystem. Disable usage statistics in settings; clear local ID periodically Somewhat distant from Facebook/Google device graphs (they link to real identities); close to other privacy-focused services using anonymous telemetry
IP Address Processing (Abuse Prevention) Low IPs stored anonymized for abuse prevention and fraud detection. No customer IPs in server logs per statement. Use VPN to mask IP; periodic account credential rotation Distant from ISPs/proxies that log full IPs indefinitely; close to standard hosting security practices
Campaign Attribution Tracking Medium Cryptographic hash of IP + user agent stored with campaign ID for 30 days. Allows marketing ROI measurement while preventing direct inference of IP. Use private browsing or privacy-focused referral links; disable tracking cookies Somewhat close to Google Ads conversion tracking; distant from traditional ad networks linking to personally identifiable information
Payment Data Collection High* Collects full banking details (IBAN/BIC, account holder), credit card data, PayPal username for billing. This creates financial records regardless of data sovereignty claims. Use cryptocurrency payments if available (Tuta supports crypto historically); third-party payment processors Close to any paid service requiring direct payment; distinguishable from services accepting cash/vouchers
Legal Disclosure Obligations High* States binding to provide content, traffic, and inventory data under valid court orders. Jurisdictional vulnerability given German/EU legal framework. Legal counsel consultation; use jurisdiction-divergent backup systems Close to all EU-based services; varies significantly from offshore jurisdictions claiming absolute confidentiality
Usage Statistics Voluntary Opt-In Low Requires explicit prior consent; user-controlled revocation; data anonymized with inability to draw conclusions about users. Simply opt-out in settings if desired Distant from Google Analytics/opt-out-required model; close to truly privacy-preserving analytics
No Third-Party Selling Clause Low Explicitly states There will be no sale of data. Strong signal against core surveillance capitalism revenue model. None needed beyond trust in commitment Distant from Meta/Google whose core business model relies on advertiser-accessible user data
Server Log Retention (7 Days) Low Limited 7-day retention for diagnostics and abuse. Short window reduces historical reconstruction capability. Periodic account activity monitoring; assume minimal forensic traceability exists Close to security best practices; shorter than many cloud providers retaining logs 30-90+ days

*High confidence here indicates HIGH CERTAINTY of the stated risk—not necessarily high severity relative to alternatives.

2 Likes
2

I have to counter-flag two issues:

  1. LLMs are not lawyers.
  2. Surveillance capitalism is structurally embedded within and outside the Internet, not cleanly isolated within privacy policies for your convenience.
3

LLMs are not caveman, but you can make them speak like cavemen by saying they are one. Telling them their role is not saying they are the thing but to prime the inference to output tokens more aligned with that role. This is quite a common way to utilize LLMs.

I don’t mention surveillance capitalism because their output will be picture perfect on it, rather it primes the output tokens to lean into weights associated with that, and surveillance capitalism is strongly associated with privacy protection.

2 Likes
4

It is funny (a little scary, perhaps?) how often people think that telling an LLM that it is an expert in a field grants it magical fairy powers of understanding that it was never programed with rather than, as you say, a means of formatting the output. I usually tend to skip role assignment just because I’m ignorant enough in most things that I need the answer dumbed down to my level.

Honestly, though, I like using AI in the manner you suggest - looking at something I’ve been working on with “eyes” that are biased differently from my own, and the response will frequently turn up keywords I can search for understanding I didn’t have before.

1 Like
5

For better or worse, I significantly utilize LLMs for my job. While I agree some prompting begins to equate technomancy and incantations, it often boils down to evaluating the output of prompts for the same task as a means to evaluate effectiviness.

In short, treat every prompt as though it’s a leading question. But instead of just a leading question, it’s quite literally a leading style influencing output. If you ask for the top 5 worst things on something, it will forcibly provide 5 things even if it doesn’t make sense.

1 Like
6

This reminds me of a similar tool that uses LLM (probably ChatGPT) for privacy policy analysis. The developer has put it on ice, for now. https://edit.tosdr.org/ is still manual.

There are tools for generating privacy policies & terms of service on the fly. This means for quite a number of businesses, privacy policy does not reflect the actual data handling policies for the company.

1 Like
7

Use large language models however you want to use them.

I’m never using them like this.

1 Like
8

No one said you had to :man_shrugging: unless you’ve got more to contribute as to why such a thing has no merit for your use case

5 Likes
9

Well don’t get me wrong, I didnt say you were forcing it on me. I said to use LLMs however you want to, and then I shared that I wasn’t going to use them as you would use them. You seem to be under the impression that I am making the opposite claim? that you shouldn’t be using it that way?

2 Likes
10

Its getting a bit too meta. TLDR; if you’ve got opposing views, sharing why helps inform readers

2 Likes
11

You are unlikely to obtain a comprehensive understanding of the scope of privacy policies that you … inspect? … in this way. You are just rolling the dice on whether or not the stochastic algorithm generates a summary that is usefully precise. And the only way to verify it did would be to … read the privacy policy.

In general, LLMs are not good summarizers.

You will probably feel like the LLM must have provided a good summary, because you can’t actually certify what it generates. This is a prime issue for LLMs: they are only useful if you can tell when they are doing a bad job, but you can only tell when they are doing a bad job by… doing the job yourself to compare.

2 Likes
12

I had obviously misinterpreted the original intent of the use case. I read it with blinders on imposed by the fact that I have been working on my own privacy policy. I was thinking of using it to proofread and formalize the policies I have established, and I like it very much for this.

Despite some of the concerns raised by others about the competence of LLMs to evaluate preexisting privacy policy pages, which I think are wholly justified, I still think there is value in taking this approach. So long as it is treated as supplemental to ones own due diligence rather than a replacement for it. I’m old; I have old eyes and a shite memory. And I miss things that AI may catch.

2 Likes