Local LLMs with Ollama: CLI, API, and WSL networking

Run a local LLM with Ollama: pick models that fit with llmfit, inspect tags via ollama show, and reach a Windows-hosted Ollama API from WSL via the gateway IP.

Publisher: REACTAPP.TOPAdded to this site Sep 26, 20264 min read808 words

This article is published in English.

Sources: Setting up local LLMs with Ollama

Local LLMs got interesting again once open-source quality jumped and laptop GPUs caught up. After moving to an HP OMEN Max with an NVIDIA RTX 5070 Ti (12 GB VRAM), trying models at home stopped feeling like a science project and started feeling like a daily tool.

Plenty of runtimes make the loop painless: Ollama, vLLM, LM Studio, llama.cpp. I picked Ollama for setup speed. Beyond casual chat, I want agent frameworks such as LangChain DeepAgents talking to a local API instead of burning cloud subscriptions on every experiment.

Which model fits this machine?

The catalog grows every week. The recurring question — “which open-source model should I actually run on this GPU?” — now has a helper: llmfit, which sketches which models fit your hardware before you download tens of gigabytes by guesswork. That alone saves evenings of trial-and-error pulls.

Ollama’s chat UI

The interface feels familiar if you have used ChatGPT or Gemini. Pick a cloud-hosted model or a local one (for example qwen3.6) if VRAM allows. A globe toggle turns network access on or off for the model when you want grounded browsing versus air-gapped answers.

CLI commands that reveal the real model

UI labels like qwen3.6 hide architecture details — parameter counts, families, and Modelfile quirks. The CLI fills that gap quickly.

List what is installed locally:

ollama list
NAME              ID              SIZE      MODIFIED
qwen3.6:latest    07d35212591f    23 GB     2 months ago
gemma4:latest     c6eb396dbd59    9.6 GB    3 months ago

Show parameters and architecture for a named tag:

ollama show qwen3.6:latest
  Model
    architecture        qwen35moe
    parameters          36.0B
    context length      262144
    embedding length    2048
    quantization        Q4_K_M

  Capabilities
    completion
    vision
    tools
    thinking

  Parameters
    temperature         1
    top_k               20
    top_p               0.95
    min_p               0
    presence_penalty    1.5
    repeat_penalty      1

  License
    Apache License
    Version 2.0, January 2004
    ...

Dump the Modelfile used to build it:

 ollama show --modelfile qwen3.6:latest
# Modelfile generated by "ollama show"
# To build a new Modelfile based on this, replace FROM with:
# FROM qwen3.6:latest

FROM C:\Users\Guangyuan\.ollama\models\blobs\sha256-f5ee307a2982106a6eb82b62b2c00b575c9072145a759ae4660378acda8dcf2d
TEMPLATE {{ .Prompt }}
RENDERER qwen3.5
PARSER qwen3.5
PARAMETER min_p 0
PARAMETER presence_penalty 1.5
PARAMETER repeat_penalty 1
PARAMETER temperature 1
PARAMETER top_k 20
PARAMETER top_p 0.95
LICENSE """

Those three commands answer “what exactly is running?” without opening docs or guessing from the marketing name.

Talk to models over HTTP

Ollama exposes an HTTP API similar in spirit to OpenAI or Claude endpoints. That is ideal for LangGraph or DeepAgents experiments without paying per token while you iterate on prompts and tools.

Sanity-check the tags endpoint:

curl http://localhost:11434/api/tags

StatusCode        : 200
StatusDescription : OK
Content           : {"models":[{"name":"qwen3.6:latest","model":"qwen3.6:latest","modified_at":"2026-05-17T22:03:48.7536396+01:00","size":23938333577,"digest":"07d35212591fc27746f0a317c975a6d68754fb38e9053d82e25f06057af2...
RawContent        : HTTP/1.1 200 OK
                    Content-Length: 843
                    Content-Type: application/json; charset=utf-8
                    Date: Fri, 14 Aug 2026 21:04:05 GMT

                    {"models":[{"name":"qwen3.6:latest","model":"qwen3.6:latest","modified_at":"...
Forms             : {}
Headers           : {[Content-Length, 843], [Content-Type, application/json; charset=utf-8], [Date, Fri, 14 Aug 2026 21:04:05 GMT]}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 843

WSL twist on Windows

If Ollama runs on Windows and your agent code runs inside WSL, the default localhost is the wrong machine — it points at the Linux VM, not the Windows host where Ollama listens. Enable Expose Ollama to the network in Ollama settings, then discover the Windows host IP from WSL:

wsl

ip route | grep default
default via 172.20.160.1 dev eth0 proto kernel

Hit that gateway IP on port 11434:

curl http://172.20.160.1:11434/api/tags
{"models":[{"name":"qwen3.6:latest","model":"qwen3.6:latest","modified_at":"2026-05-17T22:03:48.7536396+01:00","size":23938333577,"digest":"07d35212591fc27746f0a317c975a6d68754fb38e9053d82e25f06057af28522","details":{"parent_model":"","format":"gguf","family":"qwen35moe","families":["qwen35moe"],"parameter_size":"36.0B","quantization_level":"Q4_K_M","context_length":262144,"embedding_length":2048},"capabilities":["vision","completion","tools","thinking"]},{"name":"gemma4:latest","model":"gemma4:latest","modified_at":"2026-05-08T20:45:49.3801369+01:00","size":9608350718,"digest":"c6eb396dbd5992bbe3f5cdb947e8bbc0ee413d7c17e2beaae69f5d569cf982eb","details":{"parent_model":"","format":"gguf","family":"gemma4","families":["gemma4"],"parameter_size":"8.0B","quantization_level":"Q4_K_M"},"capabilities":["completion","tools","thinking"]}]}

Now Python or Node code in Linux can reach the Windows-hosted models the same way it would hit localhost on a single-OS setup.

How useful local LLMs feel day to day still depends on the tasks you throw at them — set expectations accordingly, then iterate with the CLI and API above until latency and quality match your workflow.

← All articles Send the brief