AnythingLLM “Model Not Found” with Ollama: Five Plumbing Fixes

Empty dropdowns and tag mismatches are usually HTTP wiring between AnythingLLM and Ollama—not corrupt weights. Work the checklist top-down.

2351 words

AnythingLLM says “model not found” — five plumbing fixes

You pointed AnythingLLM at Ollama, hit send, and got “model not found.” Or the model dropdown stays empty. Or “loading available models” spins forever. Meanwhile ollama list shows the weights sitting happily on disk.

That mismatch is usually not a corrupt download. AnythingLLM talks to Ollama over HTTP. Almost every stubborn case collapses into one of five connectivity or naming problems between two local processes.

The notes below were reproduced against Ollama 0.32.x-class builds: symptom, cause, fix, ordered by how often each is the real culprit.

Key takeaways

By default Ollama listens only on 127.0.0.1:11434, which is why a Dockerized AnythingLLM often cannot see it.
Inside a container, localhost means the container itself — use http://host.docker.internal:11434 (or the host gateway) instead.
Literal “model not found” is frequently a tag mismatch (llama3 vs llama3.1:8b).
Chat can work while uploads fail when the embedding model was never pulled.

Thirty-second diagnosis

Match the symptom first:

Symptom	Likely cause
Empty dropdown / endless loading	Ollama down or Base URL wrong (Docker)
Mid-chat “model not found”	Tag mismatch
Chat OK, document upload fails	Embedder missing
Right names, timeouts	Memory pressure

Ordered checks:

curl http://127.0.0.1:11434 — no response → Cause 1.
ollama list — exact NAME match? else Cause 3.
AnythingLLM in Docker with empty dropdown → Cause 2 (Base URL).
Uploads fail only → Cause 4 (embedder).
Still flaky under load → Cause 5 (RAM/VRAM).

Cause 1 — Ollama is not running (or not listening)

Start the server and verify the API:

# Start the server (foreground; leave it running)
ollama serve
# In another terminal, prove it's alive curl http://127.0.0.1:11434
# -> "Ollama is running"

Confirm the listen address:

lsof -nP -iTCP:11434 -sTCP:LISTEN # ollama ... TCP 127.0.0.1:11434 (LISTEN)

If nothing listens on 11434, AnythingLLM cannot list models. Launch the app/service, wait until the API answers, then refresh the AnythingLLM model dropdown.

Cause 2 — Dockerized AnythingLLM cannot reach host Ollama

Ollama’s default bind is loopback-only. Containers will not see 127.0.0.1 on the host. Expose Ollama on the LAN interface if needed:

# macOS: expose Ollama on the network, then restart the app
launchctl setenv OLLAMA_HOST "0.0.0.0:11434"
# Linux (systemd): add to the service override, then reload + restart
# Environment="OLLAMA_HOST=0.0.0.0:11434"

Point AnythingLLM’s Ollama base URL at the host gateway, for example when running the UI container:

docker run -d -p 3001:3001 \ --add-host=host.docker.internal:host-gateway \ -v ~/anythingllm:/app/server/storage \ mintplexlabs/anythingllm

On Linux, --add-host=host.docker.internal:host-gateway (or the docker bridge gateway IP) is the usual fix. After changing the URL, re-open settings and reload models.

Cause 3 — Tag mismatch

The API is exact-string sensitive. If the UI asks for mistral but you only have mistral:7b-instruct, chat fails mid-flight.

List what you actually have:

ollama list
# NAME ID SIZE MODIFIED
# llama3.1:8b 46e0c10c039e 4.9 GB 2 months ago
# qwen3:8b 500a1f067a9f 5.2 GB 5 months ago

Either pull the tag AnythingLLM expects or change the UI to an exact NAME from that list. Verify with a raw chat call:

curl http://localhost:11434/api/chat -d '{"model":"mistral","messages":[{"role":"user","content":"hi"}]}'
# {"error":"model 'mistral' not found"}
curl http://localhost:11434/api/generate -d '{"model":"llama3","prompt":"hi"}'
# {"error":"model 'llama3' not found"}

When the curl works and the UI fails, the UI is still sending a different model string — fix the workspace model picker.

Cause 4 — Embedder model missing

Chat uses the generative model; document ingest uses an embedding model. If the embedder was never pulled, uploads fail while chat looks fine.

Probe embeddings:

curl http://localhost:11434/api/embeddings -d '{"model":"mxbai-embed-large","prompt":"test"}'
# {"error":"model \"mxbai-embed-large\" not found, try pulling it first"}

Pull the embedder your workspace expects (example):

ollama pull nomic-embed-text

Re-run the upload. Check AnythingLLM embedder settings so the name matches ollama list.

Cause 5 — Not enough memory

Large tags on small machines die with opaque timeouts. Watch Activity Monitor / nvidia-smi / free -h while loading. Remedies: smaller quant, fewer concurrent workspaces, more RAM/VRAM, or close other models (ollama stop).

Prevention checklist

Keep Ollama running as a service.
Document the Base URL for Docker vs native installs.
Pin exact tags in AnythingLLM workspaces.
Always pull both chat and embedder models in bootstrap scripts.
Cap model size to the machine’s memory budget.

Wrap-up

“Model not found” between AnythingLLM and Ollama is almost always wiring: process up?, URL correct across namespaces?, tag exact?, embedder present?, memory enough? Work that list top-down before re-downloading multi-gigabyte weights.

If you change Ollama’s bind address, restart both Ollama and AnythingLLM; stale clients cache the old Base URL in workspace settings more often than people expect.

All articles Send the brief