Fully-local models
Run Fosnie with all inference on your own hardware – no API keys.
The local profile runs every model on the host, so no data or prompt ever leaves the machine and you
need no API keys.
Start it
docker compose --profile local up -d
# or, with the installer:
curl -fsSL https://get.fosnie.dev | sh -s -- --localThis adds two containers – Ollama (chat LLM + embeddings) and a llama.cpp reranker – and the backend seeds them as providers on boot, so chat works immediately with no onboarding step. Sparse (keyword) retrieval uses a model baked into the image, so nothing is downloaded at query time.
The first start pulls the model weights (a few GB). Plan for ~16 GB RAM.
Choosing models
A small instruct model (in the 4B range) is a sensible starting point on a CPU-only box. On a machine with a capable GPU you can pull a larger model (8–14B) for better answers – pull it in Ollama and point the LLM provider row at it under Settings → Providers.
The embedding model is fixed once you've indexed documents – changing it means re-embedding your corpus. Pick it before you index at scale.
Minimum viable host
A plain VPS with 8 vCPU / 16 GB RAM on a current Linux LTS runs the local profile with a ~4B model – the honest minimum for trying Fosnie end to end. For production-quality answers and headroom, move to a GPU host.