İPhone 15 a16.
The biggest Qwen model (Qwen2.5 3B(Quantisation5)) is sometimes a bit slower to load. But apart from that the app is working fine.
Interesting. How was the speed with 3B models?
I guess it’s better to stick with SLMs. When I have time, I will PR to add pocketpal and also explain what you can and can’t do on a smartphone AI.
On Pixel 9, I found Llama 3 8B speed to be OK actually. Like it’s not super fast but it basically is a fast as you can read.
Probably better to stick with Q4. The generation speed is OK then?
Yes. It approximately takes 10 seconds to load and generation speed is 8-10 tokens per second.
1 Like
I really like Opera’s Aria AI. I was able to create a fake account with a cock.li address and now use the service via a VPN.
The Aria AI is almost as strong as ChatGPT (I use it for business, or have to)
1 Like
Brave Leo and DuckDuckGo AI exist.
1 Like
Hugging chat is free to use and has a decent privacy policy.
For Firefox users, it can be used in browser just as Leo, with the recent updates.
2 Likes
Personally I’m using o1/o1-mini from OpenAI. I know It’s shitt* for privacy but I’m not aware of any other AI chatbot with good privacy and high performance for complex tasks and high reasoning (and that’s what I need in my studies). The last version of OpenAI is very, very good.
So efficient that It’s disturbing.
For common tasks I’m sure there are some chatbot respecting your privacy though
1 Like
Will this be also available for Firefox derivates? Is it fully Open Source?
Yes, and not really. Probably open source but only allows conenction to proprietary chatbots, not local one. (AFAIK)
That depends on the fork/derivative maintainer’s decision. The UI is open source, but it connects to proprietary models, except for hugging face.
1 Like
- And Mistral
- Additionally, for semi-advanced users, Firefox includes the ability to connect to your own locally hosted model OR to connect to a custom cloud provider of your choice. (these settings are found in about:config)
I’ve used Firefox’s AI sidebar with:
- Locally hosted open and open-ish models (e.g. Mistral Nemo, Llama 8B, ) [1]
- Cloud hosted open models from various hosting providers [2]
- I currently have it set to Duck.ai, and I use Llama 3 70B. Previously I was using locally hosted Llama 8B, but my system is too underpowered for that to be convenient.
In most cases, I assume so. Most Firefox derivatives are essentially “Firefox + a few changes”. So typically things that are available to Firefox are available to derivatives of Firefox unless the derivative takes steps to explicitly exclude something and prevent users from changing it. I know that Firefox’s AI sidebar feature can be used in both Zen and Librewolf, not sure about other browsers.
1 Like
Anonymous AI service
The World isn´t going to stop using AI just because you are against it, you can stop using it if you want but China and other dictatorships aren´t going to stop developing AI for their own advantage regardless of what other countries do.
Big red flag when a service says it is “encrypted” while clearly your requests will be read by the provider.
I’m going to consider this topic completed for our purposes. Additional AI tools (whether local or cloud) should be individually suggested in separate threads going forward.
1 Like