What should we require of VPN providers on macOS?

[Overall-Bet3743]

@jonah

As noted here and in the PR, this change has nothing to do with Proton VPN in the first place, so your conclusion does not really follow the facts of the matter. Otherwise, I see your perspective on the rest of your post.

“This has nothing to do with ProtonVPN” - really? The change would never have occurred if it wasn’t pointed out that ProtonVPN does not meet the criteria for recommendation. The change specifically accommodates ProtonVPN’s kill switch failure on macOS.

@dngray

What they really mean is we need to prioritize #1863 as well as investigate some hardware solutions like those provided by GL.inet.

The reality is there are many reasons for leaks see: Your VPN Kill Switch Won’t Stop All Leaks they’re all pretty crap besides offloading to a routing device they’re all pretty bad. Ultimately if you want to depend on it, that’s your option.

That’s great that you will prioritize investigating hardware solutions for kill switch, but please understand that this is not an option for everyone, and not everyone needs a 100% kill switch.

Yes, OS-level kill switches are not perfect. Yes, it is particularly difficult to make a kill switch work on macOS. But not everyone needs perfection.

As I keep stating, people have different threat levels - recommendations ought to account for this. Privacy Guides already lists “Best case” features for VPNs and amongst them you have:

Kill switch on all major platforms with highly configurable options (enable/disable on certain networks, on boot, etc.)

Great! So that is a “best case”, and hardware solutions can be recommended in the “best case”. What I am talking about, and what you keep ignoring, is that we should have reliable minimum criteria. Do you not think that at minimum the VPNs you recommend shouldn’t leak your IP during server switches?

@dngray

Other than this decision to follow Apple’s recommended practices they’re a lot better than many of the competitors out there who don’t even provide any source code to their applications.

That’s great that they are a lot better than many competitors, so are Mullvad and IVPN. Doesn’t mean you should carve out exceptions for them.

Now regarding “Apple’s recommended practices” - I already replied to you in the Remove ProtonVPN thread:

@Overall-Bet3743

@dngray

It is not a favored VPN provider, and now that I’ve looked into it in more detail, its obvious why ProtonVPN chose to do it the supported Apple Way rather than relying pf which has other known issues. Apple very clearly say not to use pf for widely distributed applications.

How so is it obvious? The TN3165 documentation you linked to was first publish in February 2024, yet there are reported cases of this exact network switching issue that date back 3+ years (source) (source). So this issue was occurring before Apple provided this guidance to developers and is thus not a result of said guidance.

@dngray

Nothing specific, beyond “it doesn’t happen with pf, but it happens with Network Extensions”. Nobody managed to dig up that Apple technical notice (TN3165) which actually points out very clearly pf is deprecated with wording like:

This is completely irrelevant, operating systems evolve, but to say that a privacy provider should sit back and not fix blatant issues because the current functioning method will eventually be deprecated is ridiculous, especially when other providers are putting in the hard work to resolve the issue. Again - if Mullvad VPN can bypass this issue, then it is NOT a platform issue, how do you not understand this?

I can only say the same thing so many times, so I would encourage you to fully read the replies rather than skim through them.

I will however add, that when ProtonVPN initially launched their kill switch for macOS in 2019, they did so using Packet Filter (source), specifically because they knew that it could not work with Apple’s API:

Implementing a Kill Switch (as defined above) required us to work around certain limitations within Apple’s native VPN infrastructure, specifically that it does not allow an app to fully block network traffic outside of the VPN connection on an Apple device. To resolve this, we have created a helper application to generate a packet filter. Now, whenever you connect to a VPN server with Kill Switch enabled, the packet filter blocks all external network communications except for those routed through the VPN server you are currently connected to. Since all your network traffic is restricted to the VPN server, if connection to the VPN server is lost, all Internet traffic is stopped immediately and your data is never exposed. This workaround of Apple’s network stack allows us to achieve what was previously impossible on macOS.

The fact that people on reddit were complaining about this over a year before Apple’s guidance on Packet Filter came out demonstrates that following said guidance is not why their kill switch stopped working.

You just saw some Tech Note from Apple and decided that this is why they don’t use Packet FIlter. In reality, that’s just a handy excuse since they never invested enough engineering effort to ensure that it would work.

@dngray

The other thing we’re also aware of, is tightening the criteria to the point where we only have one VPN provider really means we will have to remove the VPN section. As it is there are dangerously few decent VPN providers out there. We’d like to add more to the page.

This issue is specific to macOS. You can remove your recommendation for ProtonVPN on macOS and leave it for other platforms.