I tested GPT4all on Mac. After you installing the app, you can either download the models from its UI, or you can download them separately and put it into the relevant folder. Then, you can make changes to the settings if you want.
Preferably, check if sharing conversations with model authors and disable it.
Saving chats to disk takes a lot of disk space.
For macs with silicons, you can also use metal, but if you use more ram than you have, it will crash.
I’m not that knowledgeable on this kind of AI. Would a locally run model on GPT4All have lower quality of results than cloud based models like mixtral 8x7b with brave leo?
Ignoring speed differences due to hardware, how is the actual quality and content of responses?
That’s concerning. We wouldn’t want an offline model to send data home, as this nullify the point of running it locally. Is it opt-in with no nudging to enable it ?
Check out arena.lmsys.org. The short answers is most likely it won’t. The long answer is :
The Mixtral 8x7b can be easily run on your computer, but more powerful model like Claude Instant can’t. although it doesn’t matter since they are closed source. Recently though, Meta released Llama 3. Its 8b version is more powerful than any of the model Brave hosts, including Claude Instant (2). The 70B model (can be run on a very high end pc) is only below Google, OpenAI, and Claude. The Llama models have a fairly permissive license.
I think we should definitely lists some resources in the recommendations list. I’m even thinking of making a guide, but I am not sure this is PG role.
About the criteria section, I do not feel comfortable knowing that chats are kept for 30 days.
What are the hardware requirements for the two Llama models? I couldn’t really find a straight-forward answer online.
It was only opt-in last time I checked. have just updated the app. It asks when you open the app first. See screenshot.
It seems that they also polished the UI which I did not like in the past.
The wording is very clear, seems fine to me. Can you do a listing for Gpt4All in the same format I did on my GitHub pull request ?
I have never created a pull request, let me try when at home.
I am becoming concerned that Duck AI only use OpenAI and Anthropic and those keeps your chat for one month. If you consider this category like Search Engines (Providers), then that’s normal, afterall Bing and Google probably keep all queries ever sent to them, and that includes StartPage and DuckDuckGo. So in the Privacy Guides category we warn that there is no “guarantee that those privacy policy are honored”. At the same time, this category is also very close to a Tools category, as you are basically just offloading the work to someone with a more powerful computer. If we consider this a tool, then we must GUARANTEE the effective privacy.
I’m also concerned that despite the warning added against sharing PII, people will do it no matter what, as chatting with an AI feels similar ro doing so with an human, and humans are social creatures that love sharing.
