# A new opt-in AI integration is being tested in Firefox Nightly, with the option to use an offline, on-device, private LLM

**URL:** https://discuss.privacyguides.net/t/a-new-opt-in-ai-integration-is-being-tested-in-firefox-nightly-with-the-option-to-use-an-offline-on-device-private-llm/19113
**Category:** General
**Tags:** article
**Created:** 2024-06-25T23:37:07Z
**Posts:** 7
**Showing post:** 1 of 7

## Post 1 by @xe3 — 2024-06-25T23:37:07Z

_Disclaimer: Firefox Nightly is the alpha (pre-beta) release of Firefox, not all nightly experimental features make it into Firefox and the feature I am going to talk about, is still hidden even in Nightly (but exposed in about:config), so consider it pre-pre-beta, and a rough draft_

_TL;DR for those who don’t want to read my walls of text, opt-in ai integration included in Firefox nightly allows you to integrate your own llm, offline, on-device, and privately._

* * *

[Good news](https://blog.nightly.mozilla.org/2024/06/24/experimenting-with-ai-services-in-nightly/) today for Firefox Nightly testers interested in AI/ML.

Mozilla developers have introduced an _optional and opt-in_ experiment available to Nightly testers which integrates an AI chatbot into the browser sidebar, This wouldn’t be good privacy news if not for this one single sentence mentioned briefly at the end of the announcement:

> Nightly _ **can be configured by advanced testers to use custom prompts and any compatible chatbot** _, such as llamafile, which runs _ **on-device open models, including open-source ones.** _ We are excited for the community to share interesting prompts, chatbots, and models as we make this a better user experience. _ **We are also looking at how we can provide an easy-to-set-up option for a private, fully local chatbot as an alternative to using third-party providers.** _

If the above isn’t clear. Firefox is making it possible to integrate _your own locally hosted, private_ LLM into the browser, running on your own hardware! :clinking_glasses:

This is exciting news to me, and its one example of why I’ve been supportive and defensive of Mozilla’s choice to get involved in the AI space early on, and I hope other browsers will follow suit!

This works with Mozilla’s own format for offline LLMs called `llamafile` as well as whatever other offline (and possibly custom online) llm software you use, you just have to point it at `ip:port`. I’m currently testing it with ollama/open-webui which is the software I use and Llame-3-8B

Screeenshot from my testing (updated to correct my mistake):

 ![Screenshot from 2024-06-25 22-39-58](//forum-uploads.privacyguidesusercontent.com/original/2X/7/7d7e51f9073d5d304088dd1c752c0f8818e2d30c.png)

> **more screenshots of the UI**
>
> ![Screenshot from 2024-06-25 15-05-08](//forum-uploads.privacyguidesusercontent.com/original/2X/b/b9dea85b44f097f3532bd70d6e5ce080138ee7db.png) ![Screenshot from 2024-06-25 15-05-20](//forum-uploads.privacyguidesusercontent.com/original/2X/e/e44b4cc05eaeab4c8fbace2cf7b5c49dbc9ffb48.png)

Because it works with whatever backend you already use or set up, it offers a lot of flexibility and potential, and much more user control and privacy.

There are some other potentially cool things that this approach enables. It [appears](https://docs.openwebui.com/tutorial/web_search/) possible to get this setup with searxng, to integrate _locally hosted search_ into the _locally hosted LLM_ all accessible right from the browser! :sunglasses:

---

_[View the full topic](https://discuss.privacyguides.net/t/a-new-opt-in-ai-integration-is-being-tested-in-firefox-nightly-with-the-option-to-use-an-offline-on-device-private-llm/19113)._