We have not introduced any “blocking standards” that specifically exclude Skiff products.
I would like to make some mention of the audit that did occur in the description. I think this would be a good thing in Skiff’s favor if I did that. To do that, I’d like some description of what it contained, what was found, what commit was audited etc.
I took a look at the Skiff source code, is that it appears this not an active development repository, that means the source in skiff-org/skiff-mail, is almost certainly not what is in production. That means the “open source” claim is really a kind of, it might have been at one point in time.
Would I be right in assuming that those libraries are not available in the source repository? Or are they somewhere else?
about the mails, without discussing new criteria, we should already wait a few months, for what is mentioned here: New Email Services Recommendation: Skiff - #15 by nishil
Not all servers support DNSSEC and DANE yet. Even if they are backup servers, I think it’s better to wait until the problems mentioned here are corrected.
But it doesn’t concern pages/drive in the meantime.
This is a copy of the typed-envelopes repository. I completely agree with the commenter above as well. It’s confusing for any user to have a list of recommended software when some elements of the list are held to different and higher standards, and some are held to lower standards. That’s why we focused on compliance with all criteria.
This is a good question. There is functionality to rotate encryption keys for Skiff Drive and Pages files as people may be unshared. However, this is a bit ineffective for files because they are static, so, if you previously had access, you could have downloaded them.
I believe the question raised by @anon20402919 is whether the criteria apply to all email servers or only primary email servers. Amazon SES configuration may be outside of Skiff’s control, but they don’t seem to meet our email server criteria even though Skiff’s own servers appear to.
So, Skiff Mail is not open source, the only open source component is the encryption library at skiff-org/typed-envelopes, right? Just trying to wrap my head around this
Skiff-mail is open source. We don’t update it every day yet. Many open source products work this way. ProtonMail was not open source until version 2. Signal develops major features, such as their in-app stories and payments, outside of their open source repo before publishing it back in there.
Again: Why are the standards different for us?
We can remove SES as a backup. But, now that we built email export (prioritized because of this thread), you can inspect headers yourself and see it is not used.
Do you have a planned release schedule of any kind? My understanding is that projects that typically develop in this manner still publish their source code before releasing even if the actual work isn’t being committed in the open. I know Skiff Mail has had multiple releases since the skiff-org/skiff-mail repo was initially published.
Because your product is different than other products we currently recommend in numerous ways. We have not had to deal with the question of backup email servers before for example, because none of the other providers we recommend outsource their backup MX to a third-party.
I’m sure Amazon SES isn’t used in normal operation, because it’s set at a lower priority. I’m not telling you that your use of Amazon SES for sure means that you don’t meet the criteria, I’m telling you that I don’t know whether we should hold backup mail servers to the same criteria or not, and that’s something we need to discuss further amongst ourselves. Is the future risk that the less-secure (than your own servers) Amazon SES might be used for incoming email in case of a Skiff outage or misconfiguration significant enough for us to be concerned about it? I have no idea at the moment.
Of course, just removing Amazon SES as a backup entirely would certainly make that discussion easier for us. I wouldn’t assume to know what’s possible for you to do.
The core point here is that we do satisfy the minimum and recommended criteria, and adding a backup mailserver was done because it seems like a correct engineering practice to build fallbacks. In fact, many of our users complain that Proton and Tutanota have biweekly or monthly outages where emails aren’t received.
You may be largely correct on releases, but we have not released anything major. We made an update when Skiff Calendar was launched, which was the last major release inside Mail.
52.88.57.209 on inbound-smtp.skiff.com seems to be down as we speak, so maybe it is a good idea for you to have backups.
Anyways, I think you’re missing the point that @dngray has been telling you in this thread actually, which is that we are not obligated to tell people to use your service based on the completion of a checklist. A number of people have asked you questions here that I don’t believe have been addressed, and that certainly is a factor for consideration. Scrolling up through this thread, here’s a few I want to remind you about:
I think we’ve muddied the waters in this thread here with all this “GDPR” talk, which is a controversial subject. Ignoring legalities entirely, our position is still that this is not something we expect a privacy-oriented service to use recovery emails for without consent.
We also can’t make recommendations based on future promises, and I think there are things you’ve committed to that we want to see completed:
That audit, see above.
Exporting email folders, you said you were open to that, and I think that is an important issue to us, because data control/liberation is one of our fundamental values. Our position is that privacy is ultimately about a person controlling their own data, and portability is an important aspect of that, either in the form of standards-based export formats, or supporting standards like IMAP/SMTP (which I realize isn’t possible in your case without bridge software like Proton Bridge, so I wouldn’t suggest that).
Finally, now that I’m looking at Skiff Mail myself, I have my own question: Would you ever consider removing this “Encrypted” security indicator from Skiff Mail?
Positive security indicators are generally considered to be mostly useless, and I’d rather see no indicator at all for emails that will be sent with TLS, and a red warning indicator for emails being sent to a domain that does not support TLS. I think that given Skiff Mail’s marketing, people will assume a padlock next to a recipient implies End-to-End Encryption.
The green End-to-End Encrypted mark on communications between Skiff users is still perfectly reasonable.
Audit - discussed at length. Trail of Bits has audited our entire code base, of every product, twice. Because of this thread, we are asking them to write a blog post. Their work costs hundreds of thousands of dollars, and I don’t see any audit reports for Cryptee, for example, and Proton simply publishes attestation.
Exporting folders. We built EML export because of this thread. Exporting folders seems completely unrelated.
Recovery emails are not being used without consent. There is a bug with sending mail to new Skiff Mail users (we never had email until 8 months ago), but they are free product updates with unsubscribe links. All products you recommend send product updates.
The encrypted comment is inappropriate. Do you see the badge that Proton adds? We thought that “encrypted” was far less misleading, because the Proton badge makes it sound end-to-end encrypted. As you write, “people will assume a padlock next to a recipient implies End-to-End Encryption” - in that case, the Proton badge is far more misleading.
I don’t feel like you all have any obligation to add our products, but they clearly stand the test of the criteria for Mail, Pages, Drive, and Calendar. In some of those categories, they also seem to be far more trustworthy than the alternatives.
I mean all of this with the best of intentions: I deeply want there to be incredibly technical, high quality guides for secure products. And, it is great to have companies work with you to improve their products and the criteria. But, it is easy to lose faith when we are doing that - working through the issues and meeting the criteria.
Firstly, if you’re going to take personal offense to my replies here, then I’m happy to leave this thread to @dngray instead.
Secondly, I don’t know how you can argue with a straight face that putting “Encrypted” next to non-E2EE recipients in your email composer, and “Thread is encrypted” above non-E2EE threads is less misleading than Proton’s messaging, which is no lock at all next to non-E2EE recipients in their composer, and a tooltip next to saved emails that specifically says “Stored with…” to avoid wrongly communicating that the message was transmitted with E2EE, which it was not. To be clear, I’m not super happy with Proton’s UX here either, but I disagree that they are worse.
I was giving you the benefit of the doubt before by just inquiring about it, because I can see why you went with these indicators, and they’re not completely unreasonable by any means, but if you can’t even understand what my perspective on this is in the first place, then we might have an issue.
I appreciate that you built individual email exports, it’s great. I already explained why exporting folders is completely related to this discussion:
I think that is an important issue to us, because data control/liberation is one of our fundamental values. Our position is that privacy is ultimately about a person controlling their own data, and portability is an important aspect of that, […] in the form of standards-based export formats […]
A lack of bulk exports effectively means that those mailboxes are tied to Skiff Mail.
We tested this 1 month ago, so I don’t think this only applies to Skiff users from before the launch of Skiff Mail. If you think we are wrong and you do not send product updates to the recovery email specified for new Skiff Mail users, then I will test this with a new account again.
I don’t believe any of our recommended providers send product updates to account recovery emails. I will test this out to confirm this is the case.
I don’t know what you’re asking, EteSync is not something we currently recommend. Are you asking why it was originally added half a decade or so ago?
I appreciate your prompt responses here a lot. I’m trying to be overly transparent here to give you clarity about our current perspective, not to be adversarial.
I will see if we can switch to using the exact copy Proton is using. I disagree with your analysis completely - and note that “zero knowledge” and “zero access” are terms that security engineers deeply dislike.
It is completely possible to export folders. Simply select each email and export them. My main issue is that we are consistently held to completely different criteria than other recommended products. Tutanota also lacks bulk export features - see https://www.reddit.com/r/tutanota/comments/z0h9ms/export_all_emails/. We’ve done exhaustive research on all of these features and providers.
We’ve made a good faith effort to address and respect every concern and idea here. Unfortunately, it doesn’t feel like there is any consistency in your criteria or recommendations.
The issue is it’s open source in name but not in nature. The source repositories you do have are copies of code at some point in time. The whitepaper (2022) suggests:
Open-sourcing enables everyone to review, use, and contribute to Skiff’s products - making our community stronger and furthering our mission.
This statement in a whitepaper reads with thick coats of marketing paint. External contributions aren’t really going to happen while the community cannot see any kind of activity or development cycle.
Typically what happens is advanced users will try development branches, while testing for bugs/regressions they may have personally. Others may contribute specific features they want. For that matter there doesn’t appear to be any build instructions either.
Just to be clear about my above comment:
The criteria on the site is very much an entry level bar (must pass to even consider discussion), it isn’t however exhaustive. Some things are more important than others.
We’re not a site that provides “free marketing”, we do evaluation when a product is to be considered, the criteria is very much a first round of scrutiny.
It is impossible to include everything in the criteria because there are a many ways in which a product can have issues. The requirements for products which offer E2EE are more stringent, because there are higher levels of privacy expected.
Their work may cost a lot, but it’s not really useful to anyone but people evaluating the quality of your product. We did suggest that Proton’s attestation reports were sufficient as they provide some information about what and when it took place. While Cryptee doesn’t have an audit, it is developed in public.
Our future direction is that we very much want to tighten the criteria to audited products as we have for the instant messenger page. The reason is because we anticipate more projects to offer E2EE and we want to separate those which don’t adhere to best practices/quality from those that do.
There are many products which come and go, and we like to avoid removing things when we can.
There is a general expectation that recovery emails are used for recovery and not for marketing. No product I’ve seen sends marketing emails to the recovery address, and that includes the likes of Gmail.
Proton Mail does however show in the compose window a tip that links users to their Encryption lock meaning article. Also in that screenshot there is a major keyword there stored which indicates it is talking about how it is stored on Proton’s servers. They are also very clear about what is encrypted.
In contrast when compared to Skiff’s articles such as How to password protect an email it feels highly SEO optimized without any actual information of help. A good portion of the article talks about Outlook and Gmail, rather than how to achieve security with your own product. I touched on that earlier in this thread.
Just to be clear here, what I’m trying to avoid is the criteria being used as a checklist with minimum effort and then radio silence after being listed. I’m not saying that’s necessarily what is going to happen here though.
The purpose of EML export is to give control to users over their data. If they have many emails they’re not going to want to individually export each one, that essentially equates to a dark pattern, preventing them from leaving or having control.
You can still export entire folders of mails. While not ideal they do have other features you do not, such as password protected emails for external users.
I think you are missing the point of auditing. Auditors look at far more than just a codebase. They look at secrets management, infrastructure, software design process, employee security practices, and more. “Developed in the open” is not that. Security engineers would never advise someone to use unaudited, supposedly E2EE products.
If you say the requirements are higher for E2EE products, I’d think that a formal audit is a hard requirement. Also, Cryptee uses Google authentication. I’ve seen posts here about that and would think it is disqualifying as well.
So, it’s still confusing where/what the criteria are. I hope that makes sense.
