I just got a chance to read this blog post, and I love it so I wanted to share in case someone missed it. It really sums up what I have been saying about these sorts of sites in offhand comments myself for a while, but @soatok actually takes the time to write down his thoughts in detail which is far more useful
I literally just yesterday saw someone (on YouTube, not here) point to a useless āVPN comparison tableā as an example of someone else supposedly taking the time to āthoroughly testā and be āobjectiveā about VPNs lol
It also makes me glad we (Privacy Guides) never pursued this whole making-a-checklist thing. At some point I wanted to do sort of a comparison table based on our recommendations (which to be fair is the opposite of what most people do: make a comparison table and then draw their recommendations from it), because people fucking love their comparison tables and I knew itād be shared widely. Unfortunately, Privacy Guides is still a very small community compared to others in this space, and I always want to change that.
Ultimately, I could not come up with a way to make one that communicates our recommendations in a way thatās both accurate and doesnāt appear deceptive. If thereās one thing crappy companies are good at, itās checking boxes (and getting people accustomed to expect certain boxes via marketing, as he alludes to with the ājurisdictionā point in the post).
One final thing I will add. I do think there is one exception: when a comparison table compares a single metric. Hereās a beautiful example of this from @yaelwrites, where the comparison table exclusively compares āwhat sites does this service remove data fromā and not much else:
(I know he was not really referring to lists like this with his blog post in the first place, but I wanted an excuse to share it anyways lol)
As a somewhat related sidenote, I noticed that deceptive checklists can consist of entire websites. Like, scammy versions of our own recommendations with little to no criteria on why they should be included. Since many checklist-esque websites are essentially paid advertising to shady services, I canāt agree with this sentiment enough. Bonus points if the website has āhackerā graphics and neon green headers
Shady VPN services and crypto exchanges asides, these websites often take advantage of SEO to get their website spread around folks new to the community. I donāt know how much website traffic they get but it must be substantial enough.
Iām glad we avoided this reputation but it is SO difficult to gain community trust. Even minor things like not including comparison tables can help so much.
Yeah and what they decide to include or leave out is a big area where their subjective opinion bleeds through. If you look at almost any product page they have a heavily biased checklist of cherry-picked features that their app supports and other apps donāt, which are almost always technically accurate but they donāt tell the full picture.
I think weāre pretty open that we arenāt an objective source of info and we try to show what criteria weāre going by and what we value in the different categories, hopefully weāre open enough about our biases. Thereās never really going to be a way to say one thing is objectively better than something else.
I donāt think checklists (or comparison tables) are inherently bad, just the ones used for self-promotion. They could be used as a tool for people to make decisions but never something to be relied on.
I think comparison tables could be okay if theyā¦
Are made by a neutral entity not affiliated with anything included in the comparison
Include explanations of each metric
There arenāt any unnecessary metrics or too many metrics
One product doesnāt dominate all other products included, either nothing meets all the criteria, or more than one product meets all the criteria
Remain consistently up-to-date
An example of what I mean (this may be inaccurate)
