Pointing Claude Code at GPT and Local Ollama Models with a Proxy

If you pay for a few AI subscriptions, you've probably caught yourself opening another terminal or browser tab just to give a task to a different model, because Claude Code only talks to Claude. The model picker inside it looks promising, but its dropdown lists nothing except Anthropic's own models. Some people have built small gateway plugins that quietly route its traffic to other providers, and it turns out the trick behind them isn't exotic. This is what's needed to set it up, what broke along the way, and the arrangement that stuck.

Why this isn't a hack

Claude Code, like most command-line agents built on the Anthropic SDK, doesn't hardcode api.anthropic.com. It reads ANTHROPIC_BASE_URL, with ANTHROPIC_AUTH_TOKEN for the key, and sends every request to that address. The setting is there for companies that route the tool through an internal gateway, an audit proxy or a mirror in their region. It wasn't meant as a model-swapping switch, yet it acts like one as soon as you notice the service on the other end doesn't need to be Anthropic at all. It only has to accept the Anthropic Messages format and answer in the shape Claude Code expects, including streaming events and tool-call blocks.

So the whole idea is a small HTTP service. It translates Anthropic's request and response schema into whatever the real backend speaks, usually the OpenAI chat completion format, because nearly every provider, local ones included, has settled on that, and then translates the reply back:

Claude Code  --Anthropic Messages format-->  local proxy  --OpenAI format-->  GPT-5 / Grok / Ollama / whatever
Claude Code  <--Anthropic Messages format--  local proxy  <--OpenAI format--  (response, tool calls, streaming deltas)

Seen that way, asking why Claude Code doesn't just support other models stops being interesting. It already does, indirectly, as long as something in the middle handles the translation.

The tools available

There's no single obvious winner. Instead you find a handful of small, mostly one-person projects that solve the same translation problem with different scopes. These are the ones worth comparing:

+---------------------------+--------+------------------------------+------------------------------------------+
| Project                    | Stack  | Backends                      | Notes                                     |
+---------------------------+--------+------------------------------+------------------------------------------+
| claude-code-proxy           | Go     | OpenAI, OpenRouter, Ollama    | Single binary, pattern-based model        |
| (nielspeter)                 |        |                                | mapping (*opus*/*sonnet*/*haiku*),        |
|                              |        |                                | full streaming + tool call support        |
+---------------------------+--------+------------------------------+------------------------------------------+
| claude-code-ollama-proxy    | Python | Ollama, OpenAI (fallback),    | Built specifically for local-first use,   |
| (mattlqx)                    | / uv   | Gemini                        | uses LiteLLM under the hood for format    |
|                              |        |                                | translation                               |
+---------------------------+--------+------------------------------+------------------------------------------+
| claude-code-router          | Node   | OpenAI, Anthropic, Gemini,    | The most feature-complete option, ships   |
| (musistudio)                 |        | DeepSeek, Kimi, custom        | a UI, rule-based routing (background /    |
|                              |        | endpoints                     | think / longContext), retries, failover   |
+---------------------------+--------+------------------------------+------------------------------------------+
| LiteLLM proxy               | Python | 100+ providers                | Not built for Claude Code specifically,   |
| (generic)                    |        |                                | but exposes an Anthropic-compatible route |
|                              |        |                                | so it works the same way                  |
+---------------------------+--------+------------------------------+------------------------------------------+

All are MIT licensed and bind to localhost by default, and they do the same core job. What differs is how much routing logic is stacked on top of the translation layer. Starting with the simplest one is wise, since you can learn the request and response shape before trusting a router's rule engine to decide things for you.

First pass: a paid backend

Before dragging a local model into it, confirm the plumbing works. That's why the first run used OpenAI as the backend through claude-code-proxy, a Go program:

# clone and build (Go toolchain required)
git clone https://github.com/nielspeter/claude-code-proxy.git
cd claude-code-proxy
go build -o claude-code-proxy cmd/claude-code-proxy/main.go

Settings sit in a plain env file, where Claude's model tiers get mapped to whatever you actually pay for:

# ~/.claude/proxy.env
OPENAI_API_KEY=sk-...
OPENAI_BASE_URL=https://api.openai.com/v1

# map Claude's model tiers to whatever you're actually paying for
ANTHROPIC_DEFAULT_OPUS_MODEL=gpt-5
ANTHROPIC_DEFAULT_SONNET_MODEL=gpt-5
ANTHROPIC_DEFAULT_HAIKU_MODEL=gpt-5-mini

Start the proxy, then point Claude Code at it in place of Anthropic's endpoint:

./claude-code-proxy &            # listens on localhost:8082 by default
export ANTHROPIC_BASE_URL=http://localhost:8082
claude

That's all it takes. Claude Code starts up as usual and the interface is unchanged, but each message now goes through GPT-5. Tool calls, file edits and the whole agent loop worked without touching Claude Code's own configuration.

One surprise: this proxy pattern predates the wave of model-routing plugins people write about. It's the same idea behind Anthropic's own enterprise gateway docs, just self-hosted and aimed at an unofficial destination.

The real goal: a free local model

Paying for GPT behind Claude Code defeats half the purpose. The better aim is to hand the dull 80% of a session to something free and local, such as formatting fixes, rote refactors and the obvious test for a function, and keep paid, higher-quality models for work that needs judgment.

Ollama makes that almost too easy, since recent releases expose an OpenAI-compatible endpoint at /v1 with no adapter:

# install Ollama if you don't have it: https://ollama.com/download
ollama pull qwen2.5-coder:14b
ollama serve

Then aim the same proxy at it instead of OpenAI:

# ~/.claude/proxy.env
OPENAI_BASE_URL=http://localhost:11434/v1
# no API key needed for local Ollama, the field just needs to be non-empty
OPENAI_API_KEY=ollama
ANTHROPIC_DEFAULT_SONNET_MODEL=qwen2.5-coder:14b
ANTHROPIC_DEFAULT_HAIKU_MODEL=qwen2.5-coder:7b

Run it exactly as before:

./claude-code-proxy &
export ANTHROPIC_BASE_URL=http://localhost:8082
claude

No API key, no billing dashboard, and nothing leaves the machine. If you don't want Ollama installed natively, a Docker version is the same setup in a container:

# docker-compose.yml
services:
  ollama:
    image: ollama/ollama:latest
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
volumes:
  ollama_data:

Then pull the model inside it:

docker compose up -d
docker exec -it $(docker compose ps -q ollama) ollama pull qwen2.5-coder:14b

OPENAI_BASE_URL stays http://localhost:11434/v1 and nothing else changes.

What went wrong

Not everything worked first time. These failure modes explain why "just point it at a proxy" understates the effort:

+-------------------------------------+--------------------------------------------------------------+
| What broke                            | Why, and what fixed it                                        |
+-------------------------------------+--------------------------------------------------------------+
| Tool calls silently stopped mid-task  | Smaller local models (7B/8B) don't reliably emit well-formed   |
|                                        | tool-call JSON under Claude Code's system prompt, which is     |
|                                        | dense and multi-step. Bumping to a 14B code-tuned model fixed  |
|                                        | most of it, not all of it.                                     |
+-------------------------------------+--------------------------------------------------------------+
| Streaming responses arrived as one     | The proxy has to translate each OpenAI streaming delta into a  |
| big chunk instead of token-by-token    | matching Anthropic streaming event. If it buffers instead of   |
|                                        | forwarding incrementally, you lose the live-typing feel, it    |
|                                        | still works, it just feels laggy.                              |
+-------------------------------------+--------------------------------------------------------------+
| Claude Code's /model picker didn't     | The simple proxies don't hook into the picker UI at all, model |
| show my local model as an option       | selection happens entirely through the env file mapping, not   |
|                                        | inside Claude Code itself. Only the router-style tools with a   |
|                                        | UI (claude-code-router) actually inject entries into /model.    |
+-------------------------------------+--------------------------------------------------------------+
| Context got truncated on long          | Local models running through Ollama default to a much smaller  |
| sessions                               | context window than the model card advertises unless you        |
|                                        | explicitly set `num_ctx` when pulling or running the model.     |
+-------------------------------------+--------------------------------------------------------------+

The context window one is the nastiest. Ollama will happily load a model advertised with 128k tokens and then silently cap the real context at 2048 or 4096 unless you override it. A long Claude Code session then degrades into the model forgetting how the conversation began, with no error at all. If you're doing anything beyond a quick one-off, set it explicitly:

ollama run qwen2.5-coder:14b --keepalive 30m
# or, more durably, in a Modelfile:
# PARAMETER num_ctx 32768

Deciding what goes where

Making the plumbing work is the easy 20%. The harder part is choosing which work goes to which model, and it helps to stop thinking in terms of a local-versus-paid binary and think about the shape of the task:

+----------------------------------+-------------------------------+---------------------------+
| Task type                          | What I route it to             | Why                        |
+----------------------------------+-------------------------------+---------------------------+
| Mechanical edits, renames,          | Local Ollama model              | Free, fast enough, doesn't |
| boilerplate, formatting fixes       | (qwen2.5-coder:14b)             | need real judgment          |
+----------------------------------+-------------------------------+---------------------------+
| Everyday feature work, most         | Mid-tier hosted model           | Good enough reasoning at a  |
| refactors, writing tests            | (gpt-5-mini class)               | fraction of top-tier cost   |
+----------------------------------+-------------------------------+---------------------------+
| Architecture decisions, gnarly      | Whatever your best model is     | This is where model quality |
| bugs, anything security-adjacent    | (Claude Opus, GPT-5, etc.)       | differences actually show    |
+----------------------------------+-------------------------------+---------------------------+

Routing purely on whether something is cheap or free leads to asking a 14B local model for things it can't do reliably, and you burn more time debugging its confidently wrong tool calls than you saved on inference. The proxy makes routing mechanically possible. It doesn't make the judgment for you, and that never got easier across the tools tested. It only got easier after dropping the expectation that a plugin would decide automatically, and swapping the env file by hand based on what was about to be asked.

A security warning

Keep the proxy on localhost. Changing the bind address to 0.0.0.0 to reach it from another machine puts an unauthenticated endpoint on your network that can spend real money on whichever paid backend you've configured, for anyone who can reach it. None of the proxies tried enable authentication by default. If you need remote access, put something in front that actually checks a credential. Don't simply open the port.

What stayed in use

The simplest option won: claude-code-proxy with two env files to switch between. One points at a paid model for anything needing real reasoning, and the other points at local Ollama for mechanical work. A full router with a UI wasn't needed, because the real decision is coarse, two buckets rather than five, and swapping a config file is less friction than learning a rules engine.

The bigger lesson isn't specific to Claude Code. A coding agent and the model behind it are more separate than the interface suggests, and once you've traced one translation proxy end to end, that separation stops feeling like a trick and starts looking like the architecture underneath most of these tools. Claude Code is just the one used daily here, so it was the one worth wiring up.

If you try this, begin with a paid backend. It's much easier to tell whether the proxy setup is broken or the model is simply too small for the task when you aren't debugging both at once.