Run AI on Your Own PC: Ollama, Model Sizes, Tokens and Context Explained

Running a model without an API key

Imagine an AI model that runs on your own machine, with no API key, no cloud server and no monthly fee. That's possible, and Ollama is one of the simplest routes. If words like Llama, Qwen, Gemma, 7B, 32K context or quantization have left you guessing, the sections below give you the overall picture without the theory.

What Ollama does

Ollama is a tool that runs large language models (LLMs) locally. With a typical AI service, your request leaves for a company's servers:

Your Computer
      ↓
   Internet
      ↓
Cloud AI Server
      ↓
     Model

With Ollama the loop stays on your hardware:

Your Computer
      ↓
    Ollama
      ↓
   AI Model

Your own computer does the work, which is the whole appeal. You download a model, run it, and talk to it from a terminal or from your own applications.

The runtime and the model are different things

The first idea to absorb: Ollama is the runtime and the model is the brain. Take this command:

ollama run qwen3:4b

Here ollama is the runtime, qwen3 is the model family, and 4b means roughly 4 billion parameters. You can load another model with the same tool:

ollama run gemma4:e4b

The runtime is unchanged, and only the brain differs. One way to remember it is that Ollama is the music player and the model is the music.

Reading "4B"

Model names carry sizes like these:

3B
4B
7B
14B
32B
70B

The letter B stands for billions of parameters, so a 7B model has about 7 billion of them. You don't need the math behind each one. Just remember that more parameters usually means a larger model with more capacity, though larger doesn't automatically mean better, and a small modern model can be surprisingly good. There's a trade-off too, since bigger models normally need more memory.

Why RAM and VRAM matter

Say your machine has this:

RAM  → 16 GB
VRAM → 8 GB

RAM is ordinary system memory, while VRAM is the memory your GPU can use. Models can draw on both depending on hardware and configuration, and having enough GPU memory often makes inference noticeably faster. That's why grabbing the largest model you can find rarely works well. A beginner is better off starting with something in the smaller size range.

Installing and running your first model

Installation differs by platform. On a Mac, use the Mac download page or the terminal:

curl -fsSL https://ollama.com/install.sh | sh

On Linux, see the Linux instructions, which use the same one-liner:

curl -fsSL https://ollama.com/install.sh | sh

On Windows, open PowerShell (details on the Windows page):

irm https://ollama.com/install.ps1 | iex

With Ollama installed, starting a model can take a single line:

ollama run qwen3:4b

If the model isn't on disk yet, Ollama fetches it and then opens a chat. Try asking something like:

Explain Docker to me like I'm a beginner.

That's a local model answering you, with no key, no frontend and no backend. Only your machine is involved.

The handful of commands you'll use

There's no need to memorize dozens of commands. These cover the basics. First, list what's installed:

# See installed models
ollama list

Then pull, run, remove and inspect models:

# Download a model
ollama pull qwen3:4b# Run a model
ollama run qwen3:4b# Remove a model
ollama rm qwen3:4b# Inspect a model
ollama show qwen3:4b

After a little experimenting, they turn into habit.

Tokens

A model doesn't read text as whole words. It breaks it into smaller units called tokens. A short sentence such as this one gets split up before the model processes it:

Ollama runs AI locally.

Tokens matter because every model has a context window.

Context window

Think of an hour-long conversation with an AI. The model has to keep track of what's been said to follow along, and the amount it can consider at one time is its context window. You'll see sizes like these:

8K
32K
64K
128K

The K means roughly thousands of tokens, so:

32K ≈ 32,000 tokens

A bigger window helps most with long conversations, large documents, codebases, agents and tool calls. It becomes especially important when you move from chatbots to agents.

More than a chat box

Ollama exposes an API, so your own software can talk to the model. A typical setup might look like this:

Next.js Application
        ↓
     Backend
        ↓
      Ollama
        ↓
    Qwen Model

At that point you're building with AI and not only chatting with it. Possible projects include coding assistants, document assistants, private company chatbots, RAG applications, automation systems and agents, and you don't need to construct everything from nothing.

From chatbot to agent

A chatbot mostly answers questions. An agent can use tools to get a task done. Suppose you tell one: "Look at the server and tell me why the app is down." It could plausibly work through steps like these:

Understand the request
       ↓
Check server
       ↓
Read logs
       ↓
Analyze the problem
       ↓
Take an action
       ↓
Report the result

Local models get interesting here. Ollama can serve as the model provider, with an agent framework such as OpenClaw adding tools, memory and the surrounding environment. The layers stack like this:

AI Agent
           ↓
        OpenClaw
           ↓
         Ollama
           ↓
       AI Model

The local model stops being just an answerer and turns into the reasoning engine inside a bigger system.

Local or cloud?

Local isn't always the answer, though it offers real benefits. Your data can stay on your own machine. You aren't necessarily billed for each request. You decide which model and which infrastructure to use. And you can swap models freely without managing a pile of API keys.

Cloud models can still be stronger, particularly on hard reasoning. So the best answer is often neither one alone, and a hybrid can beat both. Simple jobs go to a local model:

Simple tasks
     ↓
Local Ollama model

Complex jobs go to a cloud model:

Complex tasks
     ↓
Cloud AI model

Five ideas to keep

Don't worry about absorbing everything at once. Five ideas are enough to start. Ollama runs the models:

Ollama
  ↓
Runs AI models

The model is the actual AI:

Model
  ↓
The actual AI

Parameters describe the model's capacity:

Parameters
  ↓
Model capacity

Tokens are how text is processed:

Tokens
  ↓
How AI processes text

Context is how much information the model can consider:

Context
  ↓
How much information AI can consider

Once those make sense, the rest of the ecosystem is far easier to follow, and you can try it right now:

ollama run qwen3:4b

One command is all that separates never having run a model locally from having an AI running on your own computer. The step after that is more interesting still: turning the local model into an agent that actually does things.