Running LLMs in the Browser with WebAssembly and WebGPU

Open a web page with no install, no account and no link to any AI server, and start chatting with a bot whose replies are produced entirely on your own laptop. Nothing leaves your machine, there's no API bill, and you can pull the network cable right after the page loads without the chatbot noticing.

That isn't a vision for five years from now. It runs today in the same tab you use for email. Two technologies make it work, and they're usually discussed apart even though they're strongest together: WebAssembly and WebGPU.

If most of your days go to JavaScript, those names may sound like game-engine territory or something for the graphics team. They're actually changing what a browser is, turning a page renderer into a platform for real computation and, more and more, for local AI inference.

Why JavaScript needs help

JavaScript does well at what it was built for: DOM work, event handling and application logic. Give it something truly heavy, like video decoding, image processing, physics simulation or a neural network, and the cracks appear. Garbage collection and dynamic typing make the language pleasant to write, and they also hurt when every millisecond of CPU time counts.

The standard answer used to be pushing the heavy work to a server. Upload the image, process it remotely, download the result. That pattern costs something. Server bills grow with every new user, network latency makes the UI feel slow, and the user's data has to leave their device, which is a privacy problem of its own.

WebAssembly and WebGPU don't replace JavaScript. They give the browser strength it never had.

WebAssembly in brief

Wasm, short for WebAssembly, defines a compact binary format that a stack-based virtual machine executes. The idea is to give high-level languages a portable target to compile to, so web applications can approach native speed.

It was never meant to be web-only. It runs in a memory-safe sandbox and obeys the browser's same-origin policies when embedded in a page, but it can also run in environments that have nothing to do with the web.

In practice the flow looks like this:

• Start from C, C++, Rust, or whichever language has a Wasm-capable compiler.
• It compiles to a .wasm file, which is binary bytecode rather than JavaScript and sits much nearer to machine language.
• The browser runs that bytecode in its own sandbox, at speeds JIT-compiled JavaScript can't match.
• Wasm calls into JavaScript and back, so you don't rewrite a whole app and only move the heavy parts.

Figma uses Wasm for its rendering engine, which is a big reason in-browser design editing feels as responsive as a desktop app. The web version of Google Earth runs on it too, and several AAA-grade game engines can export straight to it.

The point to take away is that Wasm handles computation. It makes heavy logic fast, but it doesn't give you direct GPU access. For massive parallelism and graphics you need another partner.

WebGPU, the successor to WebGL

If you've used WebGL, WebGPU is its successor, and it's more than an incremental upgrade. It was designed from scratch to map onto modern GPU APIs such as Vulkan, Metal and Direct3D 12. WebGL, by contrast, wraps the aging OpenGL ES.

MDN's documentation describes the API as letting web developers use the system's GPU for high-performance computation and for rendering complex graphics shown in the browser. The differences from WebGL go deep:

• WebGL was built around drawing to a canvas and never handled general-purpose GPU compute (GPGPU) well.
• WebGPU makes compute shaders first-class citizens instead of an afterthought.
• It exposes low-level control through concepts like GPUAdapter, GPUDevice and GPUQueue, which is much closer to native graphics APIs.
• Because it maps directly onto modern GPU APIs, its overhead is far smaller than WebGL's, which translates everything through an old OpenGL layer.

The practical effect is that a GPU once reserved for rendering 3D scenes in browser games can serve broader purposes: training small models, running neural-network inference, parallel image processing and large physics simulations, all through compute shaders in the browser.

Two layers that fit together

It's easy to overlook that Wasm and WebGPU aren't rivals. They complete each other.

• Wasm takes orchestration and heavy CPU-side logic. That means model parsing, tokenization, memory management, and all the complex code machine-learning tooling usually writes in C++ or Rust.
• WebGPU takes massively parallel execution. That covers the large matrix operations at the core of every neural network, which would crawl on the CPU alone.

Together they make an LLM inside a browser tab technically feasible. Without WebGPU, the model would be painfully slow on the CPU. Without Wasm, the runtime logic would be hard to optimize in plain JavaScript.

WebLLM as a working example

One of the clearest demonstrations is WebLLM, from the MLC AI team. It's an in-browser LLM inference engine accelerated by WebGPU, with no server at all. It's also fully compatible with the OpenAI API format, so if you already call chat.completions, there's almost nothing new to learn. A simplified usage looks like this:

import { CreateMLCEngine } from "@mlc-ai/web-llm";

// Callback to track model download & loading progress
const initProgressCallback = (progress) => {
  console.log(progress.text);
};

// A small model already compiled into MLC format
const selectedModel = "Llama-3-8B-Instruct-q4f32_1-MLC";
async function main() {
  const engine = await CreateMLCEngine(selectedModel, {
    initProgressCallback,
  });
  const reply = await engine.chat.completions.create({
    messages: [
      { role: "user", content: "Explain WebGPU in one sentence" },
    ],
  });
  console.log(reply.choices[0].message.content);
}

main();

Here's what happens behind the scenes:

• CreateMLCEngine downloads the model weights, already compressed and quantized, plus the Wasm library holding the inference logic.
• The Wasm layer does tokenization, manages conversation context and orchestrates the whole inference process.
• The heavy matrix operations at the heart of each forward pass run as WebGPU compute shaders, using the parallelism of your device's GPU.
• Everything stays inside the browser sandbox, so no prompt or user data leaves the device.

You get a chatbot that's private by default, works offline once the model is cached, and puts no load on a server you'd otherwise have to pay for. Indie developers and small teams can ship an AI feature without a cloud GPU bill that spikes with traffic.

Beyond language models

Wasm plus WebGPU reaches well past LLMs. A few other areas already use it:

• Image and video processing: complex filters, background removal and upscaling that used to need a server round trip now run in real time on the client.
• Computer vision: object detection or face recognition straight from a camera feed, with no video stream leaving the device.
• Interactive data science: heavy NumPy-style numerical work inside browser notebooks.
• 3D games and simulations: Unity and Unreal are both picking up WebGPU export, which brings in-browser rendering far nearer to what native builds deliver.
• Browser-based CAD and 3D design: manipulating complex 3D models with no desktop software.

Limits to plan for

Before you turn every AI backend into a client-side one, keep these constraints in mind:

• Browser support isn't universal. WebGPU is on by default in Chrome, Edge and recent Safari builds, but it's still experimental in Firefox and has to be enabled with a flag.
• The first download can be big. Even a small model has to reach the user's browser first, so design that first-run experience with care, usually with a clear progress indicator.
• Hardware still decides performance. An office laptop with integrated graphics won't approach a high-end discrete GPU, so pick a model that matches your actual audience.
• Debugging changes. You're no longer chasing server logic you control. You're debugging code that runs on a wide, unpredictable range of client devices.

None of that is a reason to avoid the technology. It's a reminder that running on the client still calls for careful architecture.

Why this momentum looks different

Plenty of web trends fade, but this one has nearly every major browser vendor pulling the same way. Google, Apple, Microsoft and Mozilla all help develop these standards through the W3C, so it isn't a single-vendor bet that could quietly be dropped.

The line between web app and native app keeps blurring. A browser that once only displayed documents can now handle serious computation and even AI inference, with no install, no extra server, and noticeably better privacy because user data never has to leave the device.

If you've treated Wasm and WebGPU as niche graphics territory, try a small experiment. Load a compact model through WebLLM in a side project and see how it feels to have AI that runs in your own hands rather than on someone else's server. For next steps, the code lives in the WebLLM repo, the standard itself is at webassembly.org, and for the reference see MDN's WebGPU API page.