Cost-Engineered Agentic Coding: Building a Production-Grade

This walkthrough rebuilds the path from raw materials to a working system for: Cost-Engineered Agentic Coding: Building a Production-Grade Local LLM Proxy, Multi-Agent Architecture, and Secure Tooling with Zoo Code, TensorX, and DeepInfra. The focus is operable steps, explicit checks, and code that you can drop into a repo without guessing intent.

Executive Summary

For Executive Summary, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call.

1. Architectural Overview: The Local Proxy Pattern

For 1. Architectural Overview: The Local Proxy Pattern, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call.

Protocol Translation Layer

For Protocol Translation Layer, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call.

2. Master Configuration: Deep-Dive into litellm_config.yaml

For 2. Master Configuration: Deep-Dive into litellm_config.yaml, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call.

# litellm_config.yaml

litellm_settings:
  drop_params: true
  modify_params: true

model_list:
  # =========================================================================
  # WILDCARD CATCH-ALL ROUTES
  # Allows on-the-fly usage of any model published by TensorX or DeepInfra
  # Example: tensorx/qwen/qwen3.8-2.4t-a95b
  # =========================================================================
  - model_name: "tensorx/*"
    litellm_params:
      model: "openai/*"
      api_base: https://api.tensorx.ai/v1
      api_key: "PLACEHOLDER_TENSORX_KEY"
      use_chat_completions_api: true

  - model_name: "deepinfra/*"
    litellm_params:
      model: "openai/*"
      api_base: https://api.deepinfra.com/v1/openai
      api_key: "PLACEHOLDER_DEEPINFRA_KEY"
      use_chat_completions_api: true

  # =========================================================================
  # TENSORX NAMED SHORT ALIASES
  # =========================================================================
  - model_name: kimi-k3
    litellm_params:
      model: openai/moonshotai/kimi-k3
      api_base: https://api.tensorx.ai/v1
      api_key: "PLACEHOLDER_TENSORX_KEY"
      use_chat_completions_api: true

  - model_name: glm-5.2
    litellm_params:
      model: openai/z-ai/glm-5.2
      api_base: https://api.tensorx.ai/v1
      api_key: "PLACEHOLDER_TENSORX_KEY"
      use_chat_completions_api: true

  - model_name: kimi-k2.7-code
    litellm_params:
      model: openai/moonshotai/kimi-k2.7-code
      api_base: https://api.tensorx.ai/v1
      api_key: "PLACEHOLDER_TENSORX_KEY"
      use_chat_completions_api: true

  - model_name: glm-5.1
    litellm_params:
      model: openai/z-ai/glm-5.1
      api_base: https://api.tensorx.ai/v1
      api_key: "PLACEHOLDER_TENSORX_KEY"
      use_chat_completions_api: true

  # =========================================================================
  # DEEPINFRA NAMED SHORT ALIASES
  # =========================================================================
  - model_name: deepseek-v4-pro
    litellm_params:
      model: openai/deepseek-ai/DeepSeek-V4-Pro
      api_base: https://api.deepinfra.com/v1/openai
      api_key: "PLACEHOLDER_DEEPINFRA_KEY"
      use_chat_completions_api: true

  - model_name: qwen3-coder-480b
    litellm_params:
      model: openai/Qwen/Qwen3-Coder-480B-A35B-Instruct
      api_base: https://api.deepinfra.com/v1/openai
      api_key: "PLACEHOLDER_DEEPINFRA_KEY"
      use_chat_completions_api: true

  - model_name: qwen3.6-35b
    litellm_params:
      model: openai/Qwen/Qwen3.6-35B-A3B
      api_base: https://api.deepinfra.com/v1/openai
      api_key: "PLACEHOLDER_DEEPINFRA_KEY"
      use_chat_completions_api: true

  - model_name: glm-4.7-flash
    litellm_params:
      model: openai/zai-org/GLM-4.7-Flash
      api_base: https://api.deepinfra.com/v1/openai
      api_key: "PLACEHOLDER_DEEPINFRA_KEY"
      use_chat_completions_api: true

  - model_name: nemotron-3-ultra
    litellm_params:
      model: openai/nvidia/Nemotron-3-Ultra
      api_base: https://api.deepinfra.com/v1/openai
      api_key: "PLACEHOLDER_DEEPINFRA_KEY"
      use_chat_completions_api: true

# =========================================================================
# PROXY AUTHENTICATION GATEWAY
# =========================================================================
general_settings:
  master_key: "PLACEHOLDER_MASTER_KEY"

Critical Configuration Mechanics

For Critical Configuration Mechanics, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call. For Critical Configuration Mechanics, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph.

3. Threat Modeling & Defense-in-Depth Security

When working through 3. Threat Modeling & Defense-in-Depth Security, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

The Open Relay Vulnerability Pattern

When working through The Open Relay Vulnerability Pattern, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

The Dual-Layer Mitigation Standard

When working through The Dual-Layer Mitigation Standard, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn. When working through The Dual-Layer Mitigation Standard, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph.

# Executable Proxy Invocation Command
LITELLM_USE_CHAT_COMPLETIONS_URL_FOR_ANTHROPIC_MESSAGES=true \
litellm --config ~/.claude/litellm_config.yaml \
        --host 127.0.0.1 \
        --port 4000

# Authentication Enforcement inside litellm_config.yaml
general_settings:
  master_key: "sk-7i9UOL9jkji09löOIhM..."

Separation of Secrets Architecture

Separation of Secrets Architecture works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Budget tokens per turn and per session. Agentic tools expand context aggressively; hard caps keep demos from becoming surprise invoices.

4. Multi-Agent Cost Engineering: Orchestrator, Scout, and Cheap Coder

1. Multi-Agent Cost Engineering: Orchestrator, Scout, and Cheap Coder works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Budget tokens per turn and per session. Agentic tools expand context aggressively; hard caps keep demos from becoming surprise invoices.

The Agentic Context Creep Problem

The Agentic Context Creep Problem works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Budget tokens per turn and per session. Agentic tools expand context aggressively; hard caps keep demos from becoming surprise invoices. The Agentic Context Creep Problem works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph.

The Tri-Agent Solution

For The Tri-Agent Solution, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call.

Context Cost Comparison

For Context Cost Comparison, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call.

Custom Mode Declarations (custom_modes.yaml)

For Custom Mode Declarations (custom_modes.yaml), define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call. For Custom Mode Declarations (custom_modes.yaml), define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph.

customModes:
  - slug: scout
    name: 🔍 Scout
    roleDefinition: >-
      You are a reconnaissance specialist. The orchestrator cannot read files,
      so you read them on its behalf and return the smallest set of facts it
      needs to write a precise implementation brief: exact paths, exact symbol
      locations, and only the excerpts that matter.
    whenToUse: >-
      Use as the first subtask whenever the orchestrator does not already know
      the exact file paths and code shape involved. Always run a scout before
      delegating implementation into a codebase area that has not been mapped.
    groups:
      - read
    customInstructions: >-
      WHY YOU EXIST: Your report is pasted into the orchestrator's context,
      which runs on the most expensive model in this system. Raw file dumps
      read there cost many times what they cost here. Your job is to convert
      a lot of cheap reading into a little expensive context.

      WHAT TO RETURN: Exact repo-relative paths with line numbers. The signature
      or declaration of every symbol that matters. Only excerpts directly relevant
      to the task. Maximum budget: ~40 lines of total quoted code across all files.
      NEVER paste full files. NEVER suggest solutions or fixes—report facts only.

  - slug: cheap-coder
    name: 💸 Cheap Coder
    roleDefinition: >-
      You are a focused implementation worker. You receive a precise,
      self-contained task brief from an orchestrator and apply the smallest
      correct code change that satisfies it. You do not design architecture,
      you do not explore the repository on your own, and you do not invent requirements.
    whenToUse: >-
      Use for delegated single-purpose implementation subtasks whose target files
      and intent are already named in the brief. Do not use for investigation
      or architectural design.
    groups:
      - read
      - edit
      - command
    customInstructions: >-
      SCOPE: Work only from the files and excerpts named in your task brief.
      Do not search or browse the repository to build your own picture; the
      orchestrator already paid for exploration and passed you the result.
      If the brief is missing a file or fact, stop and report exactly what is
      missing instead of hunting for it.

      EDITS: Apply the minimal diff that satisfies the brief. Preserve surrounding
      style, type annotations, and conventions. Run only the single verification
      command specified in the brief. If verification fails twice, stop and report.

Orchestrator Custom Instructions

When working through Orchestrator Custom Instructions, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

You cannot read files. Before delegating any implementation into code you have
not yet mapped, delegate a `scout` subtask first and use its report to write the
implementation brief.

Delegate implementation with new_task to `cheap-coder`, never to `code`. Every
brief must be self-contained—the worker cannot see this conversation. Include:
the exact file paths from the scout report, the excerpts the scout returned, the
precise change wanted, and the single verification command to run.

Never paste a scout report into a brief wholesale; pass only the part that
subtask needs.

After an implementation subtask reports success, do not accept its self-assessment.
Delegate a separate `scout` subtask to verify: have it confirm the changed files now
contain what was asked and report the verification command's output verbatim. Decide
on the scout's facts, not the implementer's summary.

Once the scout's verification confirms the change is correct, use
ask_followup_question to ask the user for permission to commit, and separately ask
for permission to push—approval of one is never approval of the other. If the user
approves committing, delegate to `cheap-coder` with a brief naming only the exact
relative paths of the files that actually changed to stage (never `git add .` or
`git add -A`) and the exact commit message to use: a single short line, imperative
mood, no `Co-Authored-By` trailer, no "Generated with" attribution line. Only include
a push step in that brief if the user separately approved pushing.

5. Model Economics and Prompt Caching Leverage

When working through 5. Model Economics and Prompt Caching Leverage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

Model Spotlight: qwen3.8-2.4t-a95b (TensorX)

When working through Model Spotlight: qwen3.8-2.4t-a95b (TensorX), write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

The Mechanics of Prompt Caching

When working through The Mechanics of Prompt Caching, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

Turn 1: [System Prompt + Tool Definitions + File Map] -> Written to Cache ($2.50 / 1M)
Turn 2: [System Prompt + Tool Definitions + File Map] -> CACHE HIT ($0.63 / 1M) -> 75% Savings
Turn 3: [System Prompt + Tool Definitions + File Map] -> CACHE HIT ($0.63 / 1M) -> 75% Savings

6. Telemetry, Privacy, and Local Execution Auditing

When working through 6. Telemetry, Privacy, and Local Execution Auditing, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

Audit Findings

When working through Audit Findings, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

Hardening Telemetry to Zero

When working through Hardening Telemetry to Zero, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

{
  "telemetry.telemetryLevel": "off"
}

7. Step-by-Step Production Setup Guide

When working through 7. Step-by-Step Production Setup Guide, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

Step 1: Environment Initialization

When working through Step 1: Environment Initialization, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

# Start LiteLLM Proxy in foreground
alias claude-proxy='LITELLM_USE_CHAT_COMPLETIONS_URL_FOR_ANTHROPIC_MESSAGES=true litellm --config ~/.claude/litellm_config.yaml --host 127.0.0.1 --port 4000'

# Start LiteLLM Proxy in background with health check
cx-proxy-up() {
    if ! ss -ltn | grep -q ":4000 "; then
        echo "Starting LiteLLM Proxy on 127.0.0.1:4000..."
        LITELLM_USE_CHAT_COMPLETIONS_URL_FOR_ANTHROPIC_MESSAGES=true \
        nohup litellm --config ~/.claude/litellm_config.yaml --host 127.0.0.1 --port 4000 > /tmp/litellm-proxy.log 2>&1 &

        # Wait for readiness endpoint
        until curl -s http://127.0.0.1:4000/health/readiness | grep -q '"status":"healthy"'; do
            sleep 0.5
        done
        echo "LiteLLM Proxy is online and healthy."
    else
        echo "LiteLLM Proxy is already running on port 4000."
    fi
}

# CLI model catalog search tool
cx-find() {
    python3 ~/.claude/cx-find.py "$@"
}

source ~/.bashrc

Step 2: Initialize the Proxy

When working through Step 2: Initialize the Proxy, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

cx-proxy-up

ss -ltnp | grep :4000
# Expected output: LISTEN 0 128 127.0.0.1:4000 ...

curl -s http://127.0.0.1:4000/health/readiness
# Expected output: {"status":"healthy","db":"Not connected"}

Step 3: Configure Zoo Code Extension Profiles

When working through Step 3: Configure Zoo Code Extension Profiles, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

Profiles

When working through Profiles, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.

Step 4: Mode-to-Profile Bindings

When working through Step 4: Mode-to-Profile Bindings, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn. When working through Step 4: Mode-to-Profile Bindings, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph.

🪃 Orchestrator  --> orchestrator profile
🔍 Scout         --> worker profile
💸 Cheap Coder   --> worker profile
Debug           --> orchestrator profile

8. Verification and Troubleshooting Guide

1. Verification and Troubleshooting Guide works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Budget tokens per turn and per session. Agentic tools expand context aggressively; hard caps keep demos from becoming surprise invoices.

Verifying Active Models

Verifying Active Models works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Budget tokens per turn and per session. Agentic tools expand context aggressively; hard caps keep demos from becoming surprise invoices.

curl -si http://localhost:4000/v1/messages \
  -H "Authorization: Bearer YOUR_MASTER_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model":"kimi-k3","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}' \
  | grep -i x-litellm

Common Error Codes and Resolutions

Common Error Codes and Resolutions works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Budget tokens per turn and per session. Agentic tools expand context aggressively; hard caps keep demos from becoming surprise invoices.

Error: HTTP 400 No connected db. / HTTP 500 Internal Server Error

Error: HTTP 403 Key not allowed to access model

Error: Safe Process Termination during Proxy restarts

OLD_PID=$(ss -ltnp | grep ":4000" | grep -oE "pid=[0-9]+" | cut -d= -f2 | head -1)
if [ -n "$OLD_PID" ]; then
    kill "$OLD_PID"
    sleep 2
    ss -ltn | grep -q ":4000" || echo "Port 4000 cleared successfully."
fi

Conclusion

Operational checklist