Articles for people who
ship the stack
Rewrites on React, Node.js, TypeScript and AI. A translated body is shown when it exists. If it does not, you see the English text and a note that says so.
Tagged: gateway
Resumable LLM streaming with Redis Streams as a per-turn rendezvous
Resumable LLM streaming: publish agent events to a turn-scoped Redis Stream so clients survive disconnects and multi-minute tool pauses and pick up again.
9 min readCodex CLI through LiteLLM Gateway on Bedrock Mantle
Route Codex CLI through a LiteLLM gateway into Amazon Bedrock Mantle to reach OpenAI models with gateway controls, model aliases and sane operating defaults.
8 min readLiteLLM under the hood: multi-tenant AI gateway control plane
Provider abstraction, tenant hierarchy, budgets, model policy, service accounts, routing, observability, agents/MCP, and why the gateway is critical path.
6 min readCompress every LLM call at the LiteLLM gateway with Headroom
Headroom as a LiteLLM pre-call guardrail compresses every LLM call: reversible compression, fail-open design, per-key rollout and the token savings measured.
7 min readWire MCP Tools Through LiteLLM: Mock Bank Gateway Walkthrough
MCP tools through a LiteLLM gateway: register a streamable-HTTP mock bank, compare hand-rolled tool loops to one request, prove guardrails run before MCP.
16 min readAgentgateway for LLMs, MCP, and A2A: When a Unified AI Data Plane Helps
How Agentgateway differs from LLM-only proxies—MCP discovery, A2A tracing, routing, security, OpenTelemetry, and cost governance—and when to adopt or skip it.
7 min read
About these articles
Request a 24h estimate
Need the same stack in a production operator layer? Send the brief — estimate within 24 hours.