Very interrestting. They fixed the vulnerability of overflows, and a Windows specific one.
Other vulnerabilities are privacy-related, including minor leaks of the tunnel. This has already been remedied on Linux and they asked Google to fix it on Android.
Last words
Mullvad is very happy with the quality of the audit performed by X41 D-Sec. X41 managed to find issues in our code that previous audits missed, which shows that there is great benefit in having audits performed by different companies. This is not meant as criticism against the previous audit companies. The app is too big to realistically look into every aspect and detail in a few weeks. We have always had the explicit tactic to use a different third party auditor for every audit, to get different sets of eyes from people with different skills and mindsets every time.
They really won me over when they fixed the memory bug reported by GOS users immediately and openly, then pushed the upstream go to also notice it as an issue. Nice to see another audit, feels like they have excellent attitude towards security overall.
Slightly off topic but how does PG evaluate if an auditor, such as X41, is trustworthy or not?
I have no doubts that this is a legit audit but, anytime audits come up I do wonder what the process is for telling if the audit meets the “security audits from a reputable third-party firm.” requirement?
Maybe it’s just me, but I feel as though the word reputable almost speaks for itself? If an auditing company has been around for a while and demonstrates an ability to find serious vulnerabilities in the software they review, then they would be considered trustworthy. It would be difficult for an incompetent actor to present themselves as a competent one in this context. Hopefully, that helps.
Maybe I am misunderstanding what you mean but from what I can tell most users would not be able to name two auditing companies let alone evaluate based on reputation. In my mind, none of these companies have reputations that speak for themselves.
I am not sure that’s true, auditing is a notorious marketing tool in the VPN space. Take PureVPN and their “always on” audit from KPMG as an example.
How many users are able to look at KPMG and X41 and have any clue which is a “reputable” service?
I feel there has been a misunderstanding on my part. In terms of auditing whether a VPN provider keeps logs, I don’t have a good answer for what makes an auditing company trustworthy. I was referring to audits like the subject of this topic, which audit the security of a company’s products/services. In that case, I still feel that demonstrating an ability to deliver results indicates they are at least somewhat competent. This specific audit caught potential vulnerabilities which had been present for years despite previous audits, which for me adds to this company’s credibility.
Yeah. I meant how does PG tell if the company providing the audit is reputable? It seems like something that would be important to the VPN criteria.
My assumption is most users just read the conclusion at the end of the audit and have no clue if the audit itself is technically sound, which means there needs to be some level of trust in the company providing the audit.
That is true for people who read the entire report too. The entire report could just be fabricated and/or full of half-truths and there would be no difference in knowledge between the one who read it entirely and the one who just saw the conclusions.
There is no way for a person to check of the audit was technically sound unless you are the one doing the audit.
Its mostly a web of trust thing, at least for me. I trust certain researchers, these researchers trust certain projects, those projects trust certain auditors, so I then trust those auditors. Or I read follow some researchers who have published stuff I understand (so I know they are good faith actors), and thus trust the orgs they endorse/work for. For critical apps, I try to see and understand relevant bits of code like where they are doing their encryption, manifests for irregular permissions, etc. but it is mostly based on trust.
Looking at past clients and types of discoveries also helps. If they consistently find no issues everywhere, then they are probably a rubber stamp org used for certifications. If they find interesting issues and get endorsement like the one X41 got from a trusted org like Mullvad, they gain more legitimacy.
Also what happened above, looks like a minefield with all the reported comments
Sounds reasonable to me. I think from a criteria perspective it would be nice to hear from @staff about how they do it but, this sounds reasonable.
I do wonder if its appropriate to re-evaluate these groups every so often and if there is a process to do that. A company like Kaspersky Labs comes to mind, where they have all the accomplishments you could want from a company in their field but could also be considered untrustworthy.
It has always been interesting to me that there is seemingly so little known about auditing companies for most users while audits are a minimum criteria for VPNs, so it always interested me on how PG looks at those companies.
I agree. What is PG’s view is more important than what I do.
Should be done, can’t think of a way to do it without delegating audit of audit companies to PG, which I don’t think anyone would be comfortable with. Would be good to hear what others think.
Off topic (Kaspersky)
Kaspersky has always been a trusted name at least for me. It is weird to distrust someone who has actively helped other companies and countries uncover and protect against cyber attacks (including russian and chinese), and has been contributing to security over years now. It is same as the folks who keep harping about Apple and Google being backdoored. I mean where do people draw the line? Is Qualcomm compromised? Are chinese manufactured silicon chips compromised? Are rare earth metals from Australia or Canada compromised? Meh imo
Was not going to respond, as I agree with you and I have probably veered the original post a bit off topic but, I do want to say your view is just as important and I think anytime someone offers how they evaluate an issue, gives perspective to the rest of the community.
Would also be interested in seeing if someone like @ruihildt could provide perspective on how Mullvad chooses an auditor.
I would love to be a Mullvad customer. But unfortunately that’s out of the question for me because they don’t support port forwarding. That’s what I call a poor feature set.
Their customer support, for me personally, was super accomodating and even gave me a refund and extra time on my subscripton even though I had said I was leaving as port forwarding was a need.
Proton, another PG reccomendation, offers it if you need it.
You can also use Quantum to really reduce the annoyance of their ephemeral port forwarding implementation.
I think if you are willing to look outside of PG reccomendations, AirVPN offers the best implementation of port forwarding while also being a reasonable choice in terms of privacy. Just to be transparent, I used AirVPN for 18 months after moving on from Mullvad, before switching to Proton this year.
