What do you guys think about chutes.ai and especially their TEE models?
On first glance all of it looks solid but I’m not sure if I’m missing something. If I understand it correctly their TEE models could be great for privacy?
Let’s say for example that I’m using the model GLM-5 TEE from Chutes. From what I understand they use NVIDIAs official remote attestation to check if the hardware/GPU of the miner is secure, what it claims to be, not tampered with and using the TEE as intended.
Is that implementation is done correctly? And am I understanding it correctly that TEE blocks reading the prompt from the GPU memory, dumping the VRAM content, log the requests and blocks modifying the model and code?
I got the feeling that I’m missing something crucial otherwise I don’t know why nobody else on here is talking about it since 3$/month for 300 requests daily for very good security/privacy sounds great.
I just went to the site and on first look it looks like something I would run away.
Their website is full of buzzwords without any meaning.
For example:
“Breakthrough Serverless Compute for AI, At Scale.” What the actual fuck … the word “serverless” is like saying a car without a car. It doesn’t make sense and feels more like the PR is just putting words on it, to sound cool and fancy.
The service might be good, however I would be already scared away from such buzzwording.
Might be, but then serverless is just wrong. The right word would be decentralized.
And the one of the few reasons to obfuscate the meaning of words in such way is to sound fancy, new and go with the hype.
Not to provide an actual benefit for the user. Which is something that I don’t like and which is a red flag from me.
Fair. I’m personally more concerned with the technicalities and to be honest often even don’t really read the startpage completely and just go to the technical documentation so I completely missed that until you pointed that out.
I stumbled across them on Reddit once again and thought I might check them out. I’ve seen their name before in OpenCode, OpenRouter and various different platforms so they seem to be somewhat established.
We collect information you provide directly to us, such as when you create an account, use our services, or contact us for support.
Account Information: When you create an account, we collect your name, email address, and other contact information.
Usage Data: We collect information about how you use our services, including API requests, model usage, and platform interactions. We do not collect the content of your requests.
Technical Information: We automatically collect certain technical information, including IP addresses, browser type, device information, and log data.
Payment Information: We collect payment information when you subscribe to our services, though this is processed securely by our payment providers.
We do not sell, trade, or otherwise transfer your personal information to outside parties except as described in this policy.
Service Providers: We may share your information with third-party service providers who assist us in operating our platform.
Legal Requirements: We may disclose your information if required by law or if we believe such action is necessary to comply with legal obligations.
Business Transfers: In connection with any merger, sale of company assets, or acquisition of all or a portion of our business.
They admittedly collect a bunch of PII. Any big tech data aggregation company could gain access to that information if they bought them out tomorrow
I created an account with them for testing and never had to provide anything else besides a made up username. Not even an e-mail address or something similar which is imo not bad.
You can also pay using TAO (which you can buy with XMR) or by using a code which you can buy on another website making it even harder for them to track you.
So in reality if you log in using the Tor browser, pay using TAO and only use the service via OpenCode using a proxy they basically have nothing on you?
The only things they should know is when you send queries and some other metadata like size etc. This matters of course but realistically speaking how bad is it in comparison to other AI providers?
I guess that is basically my main question: How good are they in comparison to other AI providers and if there are better ones which and why?
Please don’t answer “just run LLMs locally bro”. Models that a consumer can run locally are currently way behind and for me personally not good enough.
In terms of inference providers that offer models running in TEEs, Chutes is the cheapest and has the largest model catalogue by far. The only other option I can think of off the top of my head for you to compare to is Tinfoil.sh (since it seems like you want API access).
There are others discussed on this forum like Maple (which just uses Tinfoil as their upstream provider anyway), Lumo and Confer (neither of which provide an API), and then spammers like Phala that use bot accounts on Reddit to promote themselves.
Personally would have no problem using Chutes, and from what I’ve seen from them on X, they have reasonable takes regarding privacy. It seems like you also read their privacy policy and came to the same conclusion, so I’d just stick with it.
Thanks person with the questionable username! I’ll check out tinfoil.sh
Do you think the technical implementation of Chutes is solid from what you know? I haven’t seen any problems with it but it sounds a bit too good especially for the price.
I think Confer and Lumo might be a great tools in the future but feel a bit unfinished rn.
To be frank I haven’t looked into this enough to give you a good answer. I’d also like to note there are very few people who would be able to give you accurate information on this. Especially on this forum where it seems to be the case that most are vehemently anti-AI for one reason or another.
In my opinion, your best bet would be to read about the topic on your own and work through Chutes’/Tinfoil’s documentation, sending them questions if you have any.
A trusted execution environment (TEE) is a secure area of a main processor. It helps the code and data loaded inside it be protected with respect to confidentiality and integrity. Data confidentiality prevents unauthorized entities from outside the TEE from reading data, while code integrity prevents code in the TEE from being replaced or modified by unauthorized entities, which may also be the computer owner itself as in certain DRM schemes described in Intel SGX.
My initial question was basically if anyone knows if it’s done correctly. Yes, I could’ve said THROUGH audits from reputable 3rd parties for example.
You could also verify it through
real-time, public access to TD Quotes, NVIDIA attestations, and full IMA software manifests so third parties can independently validate the environment and running code.
as Chutes says in their blog post. I thought because there are so many smart people here that I might’ve missed an audit, some architectural design flaws, or some other information.
Sometimes people on here (at least in the past) had very nice insights, inputs or once or twice even insider knowledge. That is why people ask questions in here in the first place.
