Articles for people who
ship the stack
Rewrites on React, Node.js, TypeScript and AI. A translated body is shown when it exists. If it does not, you see the English text and a note that says so.
Tagged: litellm
How a One-Character Bearer Token Opened LiteLLM's MCP Gateway
Honeypots caught attackers abusing a LiteLLM MCP auth bypass and a guardrails code-execution flaw. Here is the attack chain and a six-step checklist to harden it.
5 min readCost-Engineered Agentic Coding: Building a Production-Grade
Cost-engineered agentic coding: a production-grade local LLM setup with LiteLLM routing, prompt caching and OpenAI-compatible endpoints that keep bills down.
17 min readPoint OpenAI Codex at a LiteLLM proxy for routed models and budgets
Point OpenAI Codex at one LiteLLM proxy URL and get routed access to many models, centralized cost logs and per-developer virtual keys with their own budgets.
5 min readCodex CLI through LiteLLM Gateway on Bedrock Mantle
Route Codex CLI through a LiteLLM gateway into Amazon Bedrock Mantle to reach OpenAI models with gateway controls, model aliases and sane operating defaults.
8 min readCut LLM spend with a LiteLLM token router
LiteLLM sits between apps and providers, routing short prompts to cheap models and heavy ones to Opus—about 28% savings in a simulated engineer month.
3 min readWhat LiteLLM cannot see: monitoring GPU serving under the gateway
LiteLLM tracks requests and spend. Queue wait, prefill versus decode, container restarts, and host pressure need Prometheus, vLLM metrics, cAdvisor, and traces.
4 min readSelf-hosted LLM networking: one gateway instead of a host per model
Adding open-weight models should not mean new hostnames and peering for every app. A LiteLLM-style gateway keeps one endpoint while backends change behind it.
5 min readLiteLLM under the hood: multi-tenant AI gateway control plane
Provider abstraction, tenant hierarchy, budgets, model policy, service accounts, routing, observability, agents/MCP, and why the gateway is critical path.
6 min readCompress every LLM call at the LiteLLM gateway with Headroom
Headroom as a LiteLLM pre-call guardrail compresses every LLM call: reversible compression, fail-open design, per-key rollout and the token savings measured.
7 min read
About these articles
Request a 24h estimate
Need the same stack in a production operator layer? Send the brief — estimate within 24 hours.