What should we require of VPN providers on macOS?

Site Development
84

Disclaimer: I am not a software engineer or network engineer. I can’t guarantee that what I write below is 100% accurate.

I have investigated the kill switch implementation for the following VPN providers on macOS and iOS:

macOS:

Mullvad VPN : Has no kill switch “option”. The kill switch is always active. Zero use of Network Extension. Entirely relies on Packet Filter. Includes an optional Advanced Kill Switch (“Lockdown mode”) that blocks all non-VPN traffic when VPN is disabled and has a LaunchDaemon to block all non-VPN traffic on system boot*.

IVPN: Has no kill switch “option”. The kill switch is always active. Zero use of Network Extension. Entirely relies on Packet Filter. Includes an optional Advanced Kill Switch (“Always-on firewall”) that blocks all non-VPN traffic when VPN is disabled and has a LaunchDaemon to block all non-VPN traffic on system boot*.

ProtonVPN: Has a kill switch option that can be toggled on or off. Uses Network Extension and does not use Packet Filter. Entirely relies on includeAllNetworks to act as kill switch with no fallbacks. It appears that their kill switch is poorly implemented, which is the reason why server switching leaks the user’s IP address. Does not have an Advanced Kill Switch. (see my previous in-depth comment on macOS ProtonVPN)

*True macOS kill switch deficiency: Whilst Network Extension quirks can be bypassed by using Packet Filter, macOS has no way of 100% enforcing network blocks on boot - there is a window of time between the machine powering on and the VPN’s LaunchDaemon starting. As such, the guidance from MullvadVPN is to disconnect the network before rebooting the device.

iOS:

MullvadVPN: Does not advertise a kill switch:

> Q: Does the app have a kill switch?

The Mullvad app uses the “on-demand VPN” function in iOS which acts as a kill switch when the VPN is connected. It should not leak traffic (with some exceptions) as our VPN always appears as being “up”. It is not using “includeAllNetworks” - see Why we still don’t use includeAllNetworks.

However, in practice, the app does employ strategies to prevent IP address leaks. It uses Network Extension with PacketTunnelProvider and WireGuard. When an .error state occurs, the app configures a dummy WireGuard tunnel that effectively blocks all traffic while keeping the tunnel extension process alive. When switching servers, a .reconnecting state occurs (note, no .disconnected state occurs during server switching), the tunnel does not disconnect (this means there should be no IP leak but I haven’t tested this in practice).

It is possible that the WireGuard tunnel fails, the user will receive the following message:

“Unable to configure the tunnel for error state. Traffic blocking may not be active.”

this is explicitly stated in the code :

// If we can’t configure the error state tunnel (e.g., setNetworkSettings fails),
// log it but don’t propagate the error. We’re already in error state.
// The system will remain in error state without traffic blocking via WireGuard.

However, it does not necessarily mean that the user’s IP address is exposed as setTunnelNetworkSettings should route the traffic into a stopped WireGuard adapter. However, this is not enforced at a kernel level, so cannot be guaranteed.

The app explicitly does not use includeAllNetworks in production builds and its ‘known issues’ section states:

We have determined that from a security and privacy standpoint, in relation to the Mullvad VPN app, TunnelVision (CVE-2024-3661) and TunnelCrack LocalNet (CVE-2023-36672 and CVE-2023-35838) are virtually identical.

The Mullvad VPN iOS app is unfortunately vulnerable to these attacks. The only solution we know against these leaks on iOS is to enable a flag called includeAllNetworks in iOS VPN terminology. This flag has not been compatible with our app, so we have not been able to turn it on. But work is being done in order to change the app so includeAllNetworks can be used, and this attack can be mitigated.

This affects all versions of the iOS app on all versions of iOS.

Timeline

In debug builds specifically, the iOS app does have includeAllNetworks (source).

The implementation of includeAllNetworks in the debug build, however, does not remove the WireGuard kill switch, it just enforces the fallback at the kernel-level, resolving the above stated issue.

IVPN: Disabled kill switch on iOS16+ devices as they claimed they would (announcement) (source code).

A caveat: Specifically only when “Block LAN traffic” is enabled, includeAllNetworks will be used for all iOS devices.
excludeLocalNetworks = false and includeAllNetworks = true
So although no kill switch is advertised, technically, some level of a kill switch does exist if “Block LAN traffic” is enabled. This should also (hypothetically) prevent IP leaks during VPN server switches.

ProtonVPN: Has a kill switch option that can be toggled on or off. The implementation uses Network Extension with includeAllNetworks

Generally speaking, with regards to includeAllNetworks I do not think that this can by itself be considered a “kill switch”. In Apple’s own documentation they state:

includeAllNetworks - A Boolean value that indicates whether the system sends most network traffic over the tunnel.

In response to a developer’s complaint that includeAllNetworks was switched from “all” to “most” network traffic, an apple engineer stated :

The way I have always thought of this property is that it allows your tunnel to define a sweeping set of destination addresses without having to manually define all of these routes in your packet tunnel configuration.

Both Mullvad and IVPN have stated their reasoning as to why they do not use includeAllNetworks:

Mullvad:

What follows is a deeply technical walkthrough of our challenges with the includeAllNetworks flag. If you care not for the technical details, the short answer is - if we were to enable the flag today, the app would work fine until it would be updated via the AppStore, at which point the system would lose all network connectivity. The most intuitive way of fixing this is to restart the device. As far as we know, there is no way for our app to detect and in any way help work around this behavior.

So it seems that Mullvad is waiting on certain fixes from Apple before being able to use includeAllNetworks but given their debug releases this would not replace their WireGuard kill switch, just add to it.

IVPN:

The iOS VPN bypass issue was initially discovered in iOS 13.3.1. To resolve the problem, Apple introduced the ‘includeAllNetworks’ feature in iOS 14+, which was designed to force all network traffic through the VPN tunnel.

Recent tests conducted by security researchers revealed that on iOS 16.1+ devices, network traffic to Apple’s servers still leaks outside the VPN tunnel, even when ‘includeAllNetworks’ is enabled.

I would like to briefly tie in this comment to @jonah ‘s post regarding Proton’s marketing.

ProtonVPN markets its macOS “kill switch” incorrectly by stating that it does not reveal IP addresses during network switches when it in fact does.

ProtonVPN also markets that it has a kill switch on iOS whereas neither IVPN nor MullvadVPN make this claim.

IVPN specifically removed their iOS kill switch because they found includeAllNetworks to not meet their definition of a kill switch and did not want to mislead their customers. In fact, even when they do enable includeAllNetworksfor “Block LAN traffic”, they specifically say that this is not a kill switch.

MullvadVPN is still waiting for includeAllNetworks to be compatible with their app. They seemingly have not introduced it as standalone as they do not consider it to be a kill switch by itself. From the debug build of the app, it appears that they plan to continue using WireGuard as the main kill switch, with includeAllNetworksas the fall back.

So it seems to me that neither IVPN nor MullvadVPN would consider ProtonVPN’s macOS or iOS “kill switch” to actually be one, even if it was properly implemented with the Network Extension.

I think this speaks volumes as to the standards that IVPN and MullvadVPN apply to themselves and the care that they place to not mislead and potentially harm customers that rely on their services… ProtonVPN not so much.

85

The main issue with this theory is that I can’t find any other VPN apps on iOS which seem to operate in this way, they all appear to pull down the packet tunnel shown to the OS when switching servers, even though the “server” the tunnel that the OS shows it’s connecting to is 127.0.0.1.

86

What iOS VPNs are you looking at specifically?

88

Wouldn’t the most effective solution to deal with this problem be to use a router with a VPN? The kill switch would then work at the router level.

89

Should Privacy Guides really be recommending generally that everyone set up their VPN connection on their router when you demonstrably can implement a kill switch on macOS anyway?

90

Its not really a solution. Its more just avoiding the issue.

@lyricism why wouldn’t both methods be included? It seems to me, like depending on the user one or the other could be preferred but neither is objectively a better option. At least thats how it seems to me but, I have a very amatuer understanding of this topic.

91

I don’t see where anything I wrote could possibly be construed as saying using a VPN via your router should be recommended against. This thread is about PG’s requirements of VPN providers on macOS due to the kill switch implementation failures specific to ProtonVPN, and the person I was responding to proposed simply recommending that you set up the connection on your router as the sole option if you want a killswitch, which is both not technically necessary and unreasonable to expect of everyone who looks to PG for guidance.