Add a model provider

Fosnie is model-agnostic: it never bundles a model, and it talks to whatever engine you configure over the OpenAI-compatible API. You assign a provider per capability – the chat LLM, embeddings, and the reranker – so you can mix local and hosted as you like. The LLM capability goes further: you can register several LLM providers at once (say a local vLLM, Claude, and GPT) and switch between them per chat.

Where providers live

Sign in as an admin and open Settings → Providers. Embeddings and rerank each have a single provider row (base URL, model, optional API key). The LLM capability is a list: add as many named providers as you like and mark one as the default.

Option A – an external API

Best when you want a hosted frontier model and have ~8 GB RAM.

1. In Settings → Providers → LLM, click Add LLM provider, give it a name (e.g. "Claude"), and set the base URL and model (any OpenAI-compatible API – Anthropic, OpenAI, Gemini, and others).
2. Paste your API key. Keys are encrypted at rest (AES-256-GCM); they never appear in the database or logs in plaintext.
3. Click Test to confirm Fosnie can reach the endpoint, then Save. Add more LLM providers the same way, and star one as the default.
4. Set embeddings (needed for retrieval) and, optionally, the reranker – these stay single.

Option B – a local engine

Best when data must stay fully on your hardware. Fosnie speaks to any OpenAI-compatible local server – Ollama, vLLM, or llama.cpp.

1. Start your engine (see Fully-local models for the turnkey Ollama stack, or GPU tier for vLLM on a GPU host).
2. In Settings → Providers, add an LLM provider pointing at the engine's base URL and model, and set the embeddings/rerank rows the same way. Local engines usually need no key.
3. Test each and Save. Mixing is fine: a local LLM for private work and a hosted one alongside it.

Switching LLM per chat

When more than one LLM provider is available, the chat composer shows a provider dropdown next to the reasoning control. Pick a provider and that conversation uses it; each chat remembers its own choice, so you can run one thread on a local model and another on a hosted one, and switch mid-conversation at any time. Regenerating a message reuses the chat's provider. The reasoning control adapts to whichever model you pick, since reasoning support is per-model. With only one provider configured, the dropdown is hidden and chat looks exactly as it did before.

If a chat's chosen provider is later removed or disabled, Fosnie falls back to the default LLM (and then to the engine's built-in default) rather than erroring – the conversation keeps working.

Bring your own key

Provider API keys are yours: stored encrypted, used only for the calls you configure. Nothing about your prompts or documents is sent anywhere except the provider you point a capability at. When an admin enables per-user BYOK, each person can add their own LLM providers under their profile and pick them in the composer, alongside the ones the deployment offers.

Next step

Upload documents and ask – your first grounded answer.