Ad blocking test at d3ward is extremely flawed

https://xcancel.com/GrapheneOS/status/1831877968895078464#m

Vanadium will not include rules for cheating at content filtering tests because we think it would be a breach of user trust, regardless of flaws in a test.

1 Like

There is also a small thread on their Mastodon page.

Yes. Because that OS has no beef with any other project…

5 Likes

The usefulness of this type of test is always going to be kind of limited.

But I don’t really understand why they chose to characterize other adblockers as “cheating”. Seems like hyperbole and another example of the “everyone else is wrong/acting in bad-faith” mentality.

That chain of tweets unnecessarily assumes other projects are all acting in bad-faith, rather than taking the time to understand that the frustration they now feel for this sort of test is actually shared by the peers they just accused of “cheating.”

Here are the actual sections they chose to characterize as ‘cheating’ (pay special attention to the comments/explanations):

uBlock Origin

[Gorhill]: Please do not use those online tools which claim to measure how good is your content blocker, they are often flawed.

! defuse pointless online tool causing hardship to volunteers
! https://twitter.com/gorhill/status/1583581072197312512
*$3p,domain=d3ward.github.io
@@*$redirect-rule,domain=d3ward.github.io
d3ward.github.io##.textads
Adguard:
! SECTION: d3ward.github.io
!
! We're getting tons of questions from users who do not understand why some test on the internet does not show 100%.
! We're tired of explaining why this or that domain should not be blocked in reality so there's that
!
! Following domains should not be blocked globally because they are not used to collect data (admin panels, bug tracking services)
!
! Google Analytics admin panel. We're blocking what's required on this web site, but more granuralry (i.e. `/collect` location). Unfortunately, this test check for the whole domain blocking, hence here's a rule
||analytics.google.com^$domain=d3ward.github.io
! Used as a click-through tracking domain for Google Analytics when a user clicks on a ad link that has been tagged with Google Analytics tracking code
||click.googleanalytics.com^$domain=d3ward.github.io
! Blocking of this domain may break Yahoo apps
||log.fc.yahoo.com^$domain=d3ward.github.io
! Used for managing Yandex AppMetrica
||appmetrica.yandex.ru^$domain=d3ward.github.io
||metrika.yandex.ru^$domain=d3ward.github.io
! Used by TikTok Business API for managing advertising campaigns
||business-api.tiktok.com^$domain=d3ward.github.io
! Used for creating Pinterest widgets
||widgets.pinterest.com^$domain=d3ward.github.io
! Pinterest Analytics dashboard
||analytics.pinterest.com^$domain=d3ward.github.io
!
||udc.yahoo.com^$domain=d3ward.github.io
||udcm.yahoo.com^$domain=d3ward.github.io
! Bug tracker services
||notify.bugsnag.com^$domain=d3ward.github.io
||sessions.bugsnag.com^$domain=d3ward.github.io
||api.bugsnag.com^$domain=d3ward.github.io
||app.bugsnag.com^$domain=d3ward.github.io
||browser.sentry-cdn.com^$domain=d3ward.github.io
||app.getsentry.com^$domain=d3ward.github.io
!
! `afs_ads` class - popular anti-adblock bait
d3ward.github.io##.afs_ads
! NOTE: d3ward.github.io end ⬆️

Here is what I think is a less hostile/bitter sounding rephrasing of the partially valid point they tried to articulate:

Users shouldn’t make generalizations about the quality or effectiveness of an adblocker based on tests like d3ward.github.io. While these tests can have value and do serve a purpose, they cause much misunderstanding among users who don’t understand how to interpret the results nor understand the limitations of the test and flaws in the methodology. For this reason tests like this probably do more harm than good.

In addition to this, tests like this one create a lot of unnecessary work and frustration for volunteers and filter list maintainers, due to persistent misunderstandings among the userbase about the significance of the test results, the % value, as well as flaws in the test itself. This is a time suck.

GrapheneOS is a new entrant to the adblocking space, but other adblockers like uBlock Origin and Adguard have long been frustrated by the situation." After years of trying to educate users and a longstanding bug in the test effecting uBO, uBO chose to just block all 3rd party resources on d3ward.github.io, and Adguard has created rules for that specific domain to block specific 3rd party resources for that site only that they don’t block globally, and don’t recommend blocking globally.

In both cases the goal is to reduce the number of support requests, and misguided reports and complaints by users who misunderstand the results. But GrapheneOS considers this approach to be misleading in and of itself. Instead of taking a similar approach, we are trying to raise awareness about the limitations of these tests, and hope our users will understand.

8 Likes

That is my sense as well. I do like the OS but the constant blaming other projects is getting old.

4 Likes

I never felt that d3ward was a scientific-precision ad block tester at all but more of a quick general test that gives you a reasonable idea of where you stand.
Some people really need to relax a bit. :wink:

3 Likes

That test is/was actually broken. I can get a 100% on it without any blocker.

4 Likes

PRobably because of DNS ?

Why exactly should I care whether some random website is a good test. Is it commonly used? Never heard of it myself lol

2 Likes

You have to care about this issue immediately because GrapheneOS said so on Twitter! /s

1 Like

Honestly I think Louis Rossmann has a great point. While GrapheneOS is technically very impressive and well regarded…I just don’t trust running code as an OS for my most sensitive device by someone so unstable and petty.

2 Likes

Think about all the software that you use every day. Do you know all the people that are behind that software? If yes, do you actually know what kind of people they’re in real life, etc.? Probably no. Yet you still use all of that software.

By that logic, we might as well add a new criteria requiring that people behind recommended projects should prove that they don’t have a criminal record, don’t have any mental issues, and that all of them are perfect, saint human beings without any imperfections or issues.

I’m not actually suggesting the above, just pointing out how ridiculous this type of thinking is.

6 Likes

I just typed “adblock test” into Brave Search and that test was the first result.

Ah yes Daniel is marking friends again. I very don’t get why he is allowed access to GrapheneOS social media.

2 Likes

I have used it a bit in the past for testing ad-blockers. It’s very commonly pointed to to test the effectiveness of an ad-blocker. It’s not some random website tbh.

2 Likes

One more thing that should be noted is that these are project accounts, more than one person has access to them, but I like how everyone just assumed that Daniel was the one who made these posts.

Exactly this. Originally uBO repeatedly declined to fix the test / POC sites. This site alone dated back to 2021:

gorhill: Anyways, I just realize OP is just reporting a proof-of-concept page, we do not fix these.

gorhill: We do not add filters for proof-of-concepts, we add filters for real-world cases after ensuring no widespread breakage occurs as a result.

okiehsch: We don’t fix test cases.

When the prerequisites for new issues became tighter, and people were able to search duplicated issues, github reports slowed down.

But then, they turned to uBO subreddit. It was asked so many times, and uBO team had to reply again and again. It got even more ridiculous when I saw a reddit user wanted to use the old “uBlock” (not uBO) since it has higher scores, despite not updated for a long time, simply because it doesn’t have as many redirection filters as uBO.

Eventually uBO team had enough, since the questions just come despite all the tweets gorhill posted and all the explanations from the team, and “fixed” that test site. After that, questions stopped and volunteers can focus on the real issues of real sites.

Funny that Graphene did nothing to help during those whole times with uBO and Adguard. And now when they finally implement their own blocker, and experience the same questions from their users, they start to accuse other projects.

7 Likes

Yeah my offline stopwatch app and my OS need the same level of trust, how could I forget.

I think you’re missing the point. The problem is that they are using the official GrapheneOS account for pointless drama. If they like drama so much they can use their own social media accounts instead of scare everyone that could be interested in privacy away because of toxicity.

Never ceases to amaze me how like-minded individuals, with a common goal, manage to be so aggressive towards eachother.

If they just linked to gorhill post and made a warring post and explained to people why they should take ad-block test site with a grain of salt and why block list maintainer shouldn’t add fake ads list, everyone would agree with them. Stop debating everyone who disagree with them and stop using words like “cheater” Just keep official accounts professional.

Write like Daniel, reply like Daniel, debate with everyone like Daniel, hyperfixation on a few words like Daniel, definitely isn’t Daniel. Jk no idea if it is Daniel or not. I just like the meme. :slight_smile:

6 Likes

In a way I actually do this, but in a broad sense. There are other things I won’t use because I find the behavior or background of people involved to be suspicious or sketchy even if it’s open source and well regarded.

3 Likes

“Cheating” is definitely the correct word. Just because a project you don’t like points something out, doesn’t make it invalid. The project account did not say Adguard bad or uBlock sucks, just that they cheat the results.

Read jonah’s minutes of meeting about not wanting free accounts for reviews because developers might cheat, so is he accusing every subscription based service of being evil, or is he using the word as it is intended to be used?

(Disclaimer: am a disposable account of a lurker)

1 Like