What are the best AI options right now?

Questions
1

So, I want to preface, that I despise AI, how it was made, and where it is taking societies.

I also need to use it for my work. That’s just how it is now.

What are my options? AFAIK there is Lumo, Confer, DDG chat, and Brave chat. I’ve used DDG, and its not bad functionally. But as I understand it, isn’t DDG overselling the privacy thing here a bit? They basically act as a proxy to whatever ai service you call. And while maybe that protects you directly, it doesn’t prevent the query itself from being logged by the AI.

Furthermore, the recent NYT case basically means that any query sent through DDG has to be logged by the AI provider anyway, as I understand it.

Yeah, I realize self hosting is the best. But have you seen the price of RAM lately??.?

So what are my options?

2

If you need to use it for work, I’m guessing you have authorization from your employer to use it? Do they not themselves provide access to AI tools you are authorized to use?

You should always remember to never share any PII with any AI tool/platform in your prompts.

That said, I think the best option is NanoGPT. You can access and purchase it privately and even anonymously should you choose to. You also get access to almost literally every AI model under the sun and more. You have an option for subscription I think but you can also use it pay as you go.

It’s literally the best option and I have more than a few times said all the same aforementioned things on the forum in other posts so you can research the same question that has also been asked here before which can provide you some more context.

4

While all those features sound nice, I was reading up on NanoGPT; how does what they do offer a real privacy edge compared to the aforementioned duck ai? Nanogpt acts as a proxy to providers and they merely ask providers not to train their models.on your data. That’s it. It seems even weaker that DDGs protections, who at least has contracts.

In particular, how does that compete with confer, which is supposed to have E2EE?

5

I think AI and privacy cannot go together but there attempts at folks making this happen. As of today, there is still not good enough solution that would make me call any AI or platform private. AI cannot be be private because it will need info you enter to then spit out the answer from its model.

What you can as of today only achieve is access and usage of AI to be as private and secure as possible provided you don’t use any PII like I said. So, that’s how using AI can be private but there is no private AI if you ask me.

Also, Duck has no where near the quality of models available than Nano so even trying to compare the two is futile if you ask me.

6

You can run AI locally and then it’s guaranteed to be private. :slight_smile:

The hardware that can run the good models is expensive though and then you’ll be limited to only use the open sourced models from hugging face. The absolute easiest/best normie way of doing this is to use a Mac with a lot of RAM as their metal architecture combines your VRAM + RAM which is godly compared to Linux/Windows where you’re limited by just your GPU’s VRAM.

3 Likes
7

I really don’t agree with this. Local AI is about as private as you can really get (agentic AI is a whole nother can of worms but let’s assume we’re just talking about a chatbot). For server-side AI, TEE support is a huge deal in providing privacy for your requests. Homomorphic encryption, while not practical for now, would provide E2EE for your AI chats, and of course tech like OHTTP can be used to help protect your IP address from being associated with your requests. There’s really a lot that can be done. It’s important to hold companies to account and try to get them to implement better privacy features when they can.

5 Likes
8

OKay… good points.

Then what is the best AI option right now that OP is asking for?

9

I meant, of course apart from local AI. That’s obvious.

10

I’d probably say confer is looking good, Apple has been using their Private Cloud Compute the whole time so that’s a pretty good option as well if you have Apple devices (although no chatbot yet). Google announced a similar thing but it doesn’t cover everything yet. Those are the main ones I’m aware of for now.

There’s not really a good option for homomorphic encryption yet, it’s just too slow for now.

11

And which models does Confer use? Cause that’s a heavy price tag considering the next best thing which I would still say is NanoGPT.

Plus, no private way to pay for Confer so that’s not good either.

12

I did a dive on confer’s code a couple of days ago: Confer: end-to-end encryption for AI chats - #14 by lone-cloud Confer proxies

I haven’t logged in to the hosted confer web app to see what they’re actually exposing there, but anything from OpenAI or TogetherAI is technically super easy for them to add. Another question is how much do they upcharge on top of those subscription fees.

1 Like
13

Thanks for sharing. I missed that thread. Been training my brain to look over anything that says AI. It’s literally everywhere and I am getting tired.

I’ll read up on it now.

14

The list of–interesting to me–privacy centered or privacy-adjacent cloud hosted AI options right now is:

  • Providers offering TEE / Confidential Compute
  • Non-TEE / trust based privacy
    • Duck.ai
    • Proton Lumo
    • Brave Leo
  • Whatever Apple’s muddling their way through
On DDG / Lumo / Brave

But as I understand it, isn’t DDG overselling the privacy thing here a bit? They basically act as a proxy to whatever ai service you call. And while maybe that protects you directly, it doesn’t prevent the query itself from being logged by the AI.

Maybe slightly, but the contracts they’ve made stipulate that conversations will not be used to train models and no data will be kept beyond 30 days (with exceptions for abuse/fraud and compliance iirc).

Furthermore, the recent NYT case basically means that any query sent through DDG has to be logged by the AI provider anyway, as I understand it.

As I understand it that specifically applies to OpenAI hosted models. It’s a consideration if you use these models, but if you use open models, I don’t think it’s relevant.

Fwiw, Lumo and Brave are not prevented from logging queries also. In all of these cases we are relying on trust in the service provider(s) and whoever is physically hosting the models.

My understanding is that those providers who offer TEE implementations can significantly minimize–but not fully eliminate–the trust component. I won’t pretend to have a deep understanding of the technical benefits and limitations of this approach though.

On Nano-GPT's privacy considerations

Just to be clear, nano-gpt offers both models that run from trusted execution environments and standard models, only a subset ~10-20% of the models have TEE implementations available. If you are using the standard models–especially the proprietary ones–you can pay privately but that is just one piece of the puzzle. The chats are not confidential.

I do think that Nano-GPT is a pretty attractive option if you stick with the TEE models. I like that it’s pay-as-you-go (pay per token), I like that they offer various open models, I like that they accept Monero payments, and I like that they offer TEE as an option.

It’s the same basic mechanism as Confer afaict (encrypted in transit + processed in a trusted execution environment.

Have you been able to figure out what models confer is using and where they are hosted?

Also is there anything that gives Confer’s approach an edge over others using confidential compute, or is this just the service you chose ot mention/are most familiar with? (others using confidential compute include Maple AI, PrivateMode AI, and Nano-GPT)

Afaict when I briefly tested, the webapp didn’t specify which model is being used or offer any control over that. It was a super simple UI (pretty similar look to open-webui if you are familiar, but a bit more basic), no settings, few to no choices, quite barebones.

I’m having login issues with their passkey implementation so I can’t login to refresh my memory at the moment.

1 Like
15

I have no idea no, I’m mainly interested in the privacy tech.

I just forgot about those other ones. I do think Confer’s use of passkeys as the encryption key is really cool and innovative, I’m not sure if anyone else is doing that.

2 Likes
16

You don’t need a lot of RAM to run smaller models. This IBM version has enterprise safeguards and is trained on exclusively public domain data. I use Lumo day to day and find it good enough.