What they really mean is we need to prioritize #1863 as well as investigate some hardware solutions like those provided by GL.inet.
The reality is there are many reasons for leaks see: Your VPN Kill Switch Won't Stop All Leaks they’re all pretty crap besides offloading to a routing device they’re all pretty bad. Ultimately if you want to depend on it, that’s your option.
Other than this decision to follow Apple’s recommended practices they’re a lot better than many of the competitors out there who don’t even provide any source code to their applications.
The other thing we’re also aware of, is tightening the criteria to the point where we only have one VPN provider really means we will have to remove the VPN section. As it is there are dangerously few decent VPN providers out there. We’d like to add more to the page.
As noted here and in the PR, this change has nothing to do with Proton VPN in the first place, so your conclusion does not really follow the facts of the matter. Otherwise, I see your perspective on the rest of your post.
“This has nothing to do with ProtonVPN” - really? The change would never have occurred if it wasn’t pointed out that ProtonVPN does not meet the criteria for recommendation. The change specifically accommodates ProtonVPN’s kill switch failure on macOS.
That’s great that you will prioritize investigating hardware solutions for kill switch, but please understand that this is not an option for everyone, and not everyone needs a 100% kill switch.
Yes, OS-level kill switches are not perfect. Yes, it is particularly difficult to make a kill switch work on macOS. But not everyone needs perfection.
As I keep stating, people have different threat levels - recommendations ought to account for this. Privacy Guides already lists “Best case” features for VPNs and amongst them you have:
Kill switch on all major platforms with highly configurable options (enable/disable on certain networks, on boot, etc.)
Great! So that is a “best case”, and hardware solutions can be recommended in the “best case”. What I am talking about, and what you keep ignoring, is that we should have reliable minimum criteria. Do you not think that at minimum the VPNs you recommend shouldn’t leak your IP during server switches?
That’s great that they are a lot better than many competitors, so are Mullvad and IVPN. Doesn’t mean you should carve out exceptions for them.
Now regarding “Apple’s recommended practices” - I already replied to you in the Remove ProtonVPN thread:
I can only say the same thing so many times, so I would encourage you to fully read the replies rather than skim through them.
I will however add, that when ProtonVPN initially launched their kill switch for macOS in 2019, they did so using Packet Filter (source), specifically because they knew that it could not work with Apple’s API:
Implementing a Kill Switch (as defined above) required us to work around certain limitations within Apple’s native VPN infrastructure, specifically that it does not allow an app to fully block network traffic outside of the VPN connection on an Apple device. To resolve this, we have created a helper application to generate a packet filter. Now, whenever you connect to a VPN server with Kill Switch enabled, the packet filter blocks all external network communications except for those routed through the VPN server you are currently connected to. Since all your network traffic is restricted to the VPN server, if connection to the VPN server is lost, all Internet traffic is stopped immediately and your data is never exposed. This workaround of Apple’s network stack allows us to achieve what was previously impossible on macOS.
The fact that people on reddit were complaining about this over a year before Apple’s guidance on Packet Filter came out demonstrates that following said guidance is not why their kill switch stopped working.
You just saw some Tech Note from Apple and decided that this is why they don’t use Packet FIlter. In reality, that’s just a handy excuse since they never invested enough engineering effort to ensure that it would work.
This issue is specific to macOS. You can remove your recommendation for ProtonVPN on macOS and leave it for other platforms.
I can only tell you what is true from our perspective. If we made any change to accommodate a specific provider, the change was made specifically to accommodate IVPN on iOS, which people seemed to feel was reasonable. We are still deciding what to do about macOS, for example in this thread as well as the thread about their marketing, so Proton has not been let off the hook so to speak.
If you still disagree that is fine, but there is nothing else we will be able to add to this discussion, so we need to leave this here before we start a cycle of repeating ourselves.
I think the question remains whether Apple provides a mechanism to do this, which I still have to look into:
If Apple does not provide a way to do this then it seems like our answer will be no, we won’t require VPN clients to do this on macOS. I can’t tell you with certainty though until we actually figure out this in detail. I need to do more digging since I haven’t seen anyone post any answers this question since I asked it yesterday.
This is functionally identical to adding a warning about macOS to the Proton listing. Do you disagree? It is impossible for us to completely remove it from our website while also leaving the recommendation for other platforms.
imo, VPN clients must be required to use macOS provided (pf) or iOS/macOS assisted (includeAllNetworks / enforceRoutes) killswitches, regardless.
Do 1p apps (Apple’s own apps) bypass iOS’ “Always-on VPN” setup (which is what Enterprise-grade VPNs use)? Any references?
Are these exceptions for “Apple-related services” documented by Apple? I only see they mention 4 exceptions, and none of those call out “Apple services” directly.
This is an incredibly long thread, could someone please explain to a non technical Mac user , what the issue is with vpn’s on Mac OS and which if any are recommended.
I have tor browser on all my machines so it’s more best options for use with say brave browser. I have a proton account, can use Windscribe and AdGuard or even nym
I understand where this perspective is coming from, and I partially agree with it. Implementing the VPN and kill switch at the router level can indeed avoid some of the client-side quirks and leaks that can occur due to operating system limitations.
However, in practice this approach can also be quite impractical for many users. A lot of people don’t actually want every device on their network to be permanently behind a VPN or in a strict kill switch state.
For example, some services and websites are VPN-hostile or block VPN traffic entirely. In those cases users often need the flexibility to temporarily bypass the VPN for certain devices or tasks. If the kill switch is enforced at the router level, that flexibility becomes much harder to manage unless the router supports fairly advanced per-device or per-network routing rules.
Because of that, while router-level VPNs can be very robust from a leak-prevention standpoint, many users still rely on client-side implementations so they can selectively enable or disable the VPN depending on the situation.
Somehow, I still feel that an enforced platform and a VPN client that delivers some type of state of art on this area would make the usability a bit more approachable. The solution for those things that we are discussing might still be out there in development waiting for some creative engineers to create what we need. Of course it would help if platforms like the macOS would be a bit more modular and open source so communities could come up with solutions.
In the end, IMHO, this whole discussion turned up to be a very good and interesting learning for me, and I believe for many, that I don’t see in this forum for quite a while.
This feels obvious so I am surprised you did not touch on it but, I would point out that you do not need to have every device under a VPN just because you have the VPN on the router level instead. This is what VLANs and policy based routing is for.
All the PG tools in the Router Firmware - Privacy Guides have forums and other resources that provide very accessible guides on how to achieve this.
That’s a fair point, and you’re right that VLANs and policy-based routing can allow devices to be routed differently even when the VPN is configured at the router level.
My concern was less about whether it is technically possible, and more about how practical it is for typical users.
While VLANs and PBR can definitely solve this problem, setting them up usually requires fairly advanced router configuration (multiple networks, routing rules, firewall zones, etc.). Many people are still using ISP routers or consumer mesh systems where this level of control is not available or is quite difficult to configure.
There is also the flexibility aspect. With a client VPN application, users can usually toggle the VPN on or off instantly when they run into VPN-hostile services like banking sites or certain streaming platforms. With router-level VPN setups, even when PBR is used, that kind of temporary control often requires switching networks, changing routing rules, or logging into the router.
So I agree that router-level VPNs can be very robust from a leak-prevention standpoint, but the trade-off seems to be usability and flexibility for many users. That is my point.
@anonymous549 I’d like to please ask you to not leave replies like this, where you are essentially only pointing out arguments you’ve already made (as opposed to What should we require of VPN providers on macOS? - #71 by anonymous549 where you pointed out something new/different). Just asking someone if they really believe what they are saying is pointless.
The community can read the posts from both you and @Cyber-Typhoon and make up their mind on their own without a comment like this. This is going to help us avoid some repeated back and forth that nobody besides you two will benefit from.
I’ve read the entirety of all 5 threads related to removing Proton VPN and I’d like to validate my understanding as the issue seem technical.
First, from reading the threads, I believe the below is what most agrees on:
First, is the above technically true? In other words, do you agree technically with the above? @jonah@dngray
If so, I don’t understand why we’re still recommending ProtonVPN. I’m on Windows and will still use ProtonVPN after this whole fiasco, because it seems work fine on it.
If PG is hard-bent on keeping it for whatever reason, then at the very least explain all the intricacies of a kill switch, when and where they work, and why.
Taking the extreme approach (installing kill-switch at the router level for instance) is clearly not what users want from reading all the threads and doesn’t help the common folk. From other threads as well, most users do not want to install a VPN on their router for different valid reasons.
From my understanding, using a VPN’s kill-switch is not bulletproof (like on OSboot) and this should simply be explained in the VPN page.
Edit: Apologies, I just saw the change for macOS on the actual website, and personally, I think this change is fine. Hopefully, it won’t take a public outcry for a change next time as it leaves a sour taste. Sorry if this is blunt
Leaving this for others who might not have seen it yet:
Kill switch feature provides poor protections on macOS
Proton VPN’s kill switch on macOS does not block any traffic when you intentionally disconnect from the VPN, including when you disconnect by switching servers. You should not make any sensitive connections while the VPN is turned off, nor when switching servers. It is only designed to prevent traffic leaks in the case of an unexpected VPN disconnection, which is still a useful feature to have, but it does not provide the same level of protection as a kill switch that blocks all traffic when the VPN is turned off.
Additionally, system crashes may occur on Intel-based Macs when using the VPN kill switch. If you require this feature, and you are using a Mac with Intel chipset, you should consider using another VPN service.
There’s a whole bunch of things that you can do with low-level services on macOS that just aren’t supported as APIs by Apple. PF is one example of this. PF is not suitable for use in third-party products because there’s no way to arbitrate the PF rules between the various folks who might be using PF
For context, the user’s initial question to the Apple engineer was about how to ensure that no traffic leaks outside the VPN tunnel because they wanted to migrate from Packet Filter to Network Extension.
we're interpreting TN3165 as a signal that pf should be considered deprecated
Well, we can’t deprecate something that we’ve never officially supported as an API (-: But, yes, our position is that, if you’re building Mac networking products using legacy techniques then you should move to a Network Extension provider. That includes PF, but other things as well, including manually manipulating UTUN interfaces and modifying the routing table ‘behind the back’ of configd.
may not be available in the next major macOS release
We’ve made no announcement about timelines. However, our experience is that products using these legacy techniques tend to experience compatibility problems as we evolve the system, so it’d be better to move off them sooner rather than later.
It appears that Apple considers Packet Filter to be a legacy technique that may experience compatibility problems in future. However, the notion that Packet Filter will be deprecated is incorrect since it was not supported as an API in the first place.
Now, regarding the Network Extension framework, and in particular the includeAllNetworks issue. That same Apple engineer, states the following:
My go-to reference for this sort of stuff is Routing your VPN network traffic [1]. This outlines two options for preventing ‘escapes’:
For a full tunnel, set includeAllNetworks and then set exceptions via the various excludeXYZ options.
He then addresses the user’s other question:
be able to block any traffic while the tunnel is not yet connected
Honestly, I’m not sure what the best path forward is for that requirement. I’m gonna research that and get back to you.
…
My initial thought on this front was to create a packet tunnel provider that starts and stays started regardless of the state of the underlying tunnel. The system will then route traffic for the destination network to you, and you can just drop it if the tunnel is down.
The drawback to this approach is that there isn’t a good way to start the tunnel at boot. We actually discussed this on the forums recently — see this thread — and there are approaches that work but nothing that I’d consider to be ‘good’.
An alternative approach is to create an NE filter data provider and have it and your packet tunnel provider cooperate to implement this feature. That is, when a new flow arrives at the filter, check whether the tunnel is up and, if it isn’t, block the flow.
Note that, if you use sysex packaging, which you really should, both NE providers run in the same process and thus this sort of cooperation is easy. It can be something as simple as a global variable (with ån appropriate mutex, of course).
Now, I will let others interpret this / verify - but the initial user did mark this answer as solving their answer. So I assume this means there is some possibility of not exposing IP addresses between server switches.
So, then, why is ProtonVPN leaking IP addresses on macOS during server switches?
I did a cursory investigation into this and believe that the issue is caused by their poor implementation of the Network Extension in macOS.
Disclaimer: This is my hypothesis, and I am not 100% confident it is correct. I am very open to being wrong here. I am not a software engineer or network engineer, so i cannot validate this. A lot of these findings come from my discussion with an AI code reviewer
This image shows the sequence of events that take place when you switch servers on ProtonVPN on macOS.
In essence, what the macOS client is doing is disconnecting from the tunnel, waiting >0.7 seconds, and connecting to a new tunnel. The macOS client fully depends on includeAllNetworks as the kill switch with no other backups.
Now, I am not 100% certain why the IP leak is occurring, but I think there are 2 possibilities:
includeAllNetworks might require there to be a tunnel in order to work. By entering a NEVPNConnection .disconnected state, includeAllNetworks may not be able to block the connection.
Apple documentation:
> source: The NEVPNConnection class provides methods for starting and stopping the VPN programmatically.
> source: NEVPNStatus.disconnected - The VPN is disconnected.
ProtonVPN’s macOS client calls exit(0) on the WireGuard system extension, which hosts the NEPacketTunnelProvider subclass. This exit(0)method of stopping the tunnel is used for both server disconnects and server switches and prevents includeAllNetworks from functioning correctly and blocking the user’s connection.
The apparent reason for using this exit(0)method is due to an old Apple bug that would crash NEPacketTunnelProvider that has seemingly since been fixed. It seems that ProtonVPN would have unscrupulously implemented a proposed fix that had this side effect.
Interestingly, this current behavior for macOS is flagged as “legacy”. For iOS, there is a ConnectionFeatureflag that does not result in the >0.7 second delay between tunnel switches. Instead, the tunnel is switched instantly. It is unclear to me if this resolves the kill switch issue, but it is an improvement.
Irrespective of the actual reason for the IP leak during server switching, one of my findings of the code was that ProtonVPN fully relies on includeAllNetworks alone with zero fallback in case of failure.
Again, I’m not fully confident with this assessment. Nevertheless, It seems that a lot of developers have been having issues with includeAllNetworks as demonstrated by the various bugs described on Apple’s developer forums. Its difficult through pure research to get a full understanding of the Network Extension’s expected behavior. I can see why many VPN providers have decided to outright reject it.
Given all of this, I think its understandable that other VPN providers may prefer to rely on Packet Filter and completely avoid Network Extension. My impression has been the it is Network Extension that is “hacky” rather than the other way around.
This is also my impression and exactly what I speculated is happening in the video we just uploaded, so good to know
I also shared having this exact thought when I was thinking about ways Proton could potentially mitigate this problem this week. I’m not sure why they aren’t doing this, but maybe they have a reason?
It doesn’t seem like there’s a reason not to do this based on the rest of this quote. And It’s good to know that an Apple engineer suggested this approach.
I think it is also important to remember not everyone has this option because they do not control the router they use daily - i.e. students living on campus. Students face a lot of surveillance and censorship from their schools.
Just on that you can always have a router you do control behind a router that you do not. That is practically how the internet is.
The reality of it is, irrespective to user preference or wishes, a router doing the routing into the VPN when the packets come from a particular source (your Mac) will always be more full proof and less prone to failure than a rules based firewall trying to allow the VPN with some exceptions.
The reason is because it’s okay for the router to be allowed full access, and then just select the traffic from the device and put it into the VPN.
