Practical notes: What I Learned Shipping a Production AI Agent with Google’s

Use this as an operator-facing rebuild of the ideas in “What I Learned Shipping a Production AI Agent with Google’s Gemini Function Calling”: clear stages, ordered code slots, and recovery notes that survive a handoff. The Overview stage works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion.

1. Function Declarations: Think of Them as an API Contract with the Model

For the 1 Function Declarations Think stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Separate client construction from the message loop so providers can be swapped without rewriting the conversation state machine.

// ❌ Vague — model won’t know when to use it
{ name: ‘get_data’, description: ‘Gets data about a location.’ }

// ✅ Trigger-specific — model knows exactly when this is relevant
{
 name: ‘find_backup_spots’,
 description: ‘Finds nearby indoor/covered alternatives within walking distance ‘
 + ‘if the venue is crowded or weather turns.’,
}

{
 name: ‘calculate_transit_route’,
 description: ‘Calculates precise walking/driving/transit travel times to the venue.’,
 parameters: {
 type: ‘OBJECT’,
 properties: {}, // No parameters — dispatcher injects from context
 },
}

2. The Multi-Turn Tool Loop: A State Machine, Not a Single Call

For the 2 The Multi-Turn Tool stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph. Separate client construction from the message loop so providers can be swapped without rewriting the conversation state machine.

Turn 0: [user prompt + system instructions]
 → Model returns: functionCall { name: “tool_a”, args: {…} }

Turn 1: [user prompt, model’s functionCall, your functionResponse for tool_a]
 → Model returns: text response (synthesized from tool_a’s output)

Turn 0: [user prompt]
 → Model returns: functionCall { name: “tool_a”, args: {…} }

Turn 1: [user prompt, functionCall_a, functionResponse_a]
 → Model returns: functionCall { name: “tool_b”, args: {…} }

Turn 2: [user prompt, functionCall_a, functionResponse_a, functionCall_b, functionResponse_b]
 → Model returns: final text (synthesized from both tools)

3. Model Cascading: Don’t Pin to a Single Model

For the 3 Model Cascading Don stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Separate client construction from the message loop so providers can be swapped without rewriting the conversation state machine. For the 3 Model Cascading Don stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion.

Primary: gemini-3.5-flash-lite (fastest, cheapest, zero-thinking overhead)
Secondary: gemini-3.1-flash-lite (slightly older, same tier)
Tertiary: gemini-2.5-flash (thinking-capable, higher quality)

4. API Key Architecture: The Proxy Pattern

When working through the 4 API Key Architecture stage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Log request id, model id, and latency on every call. Without that trail, intermittent provider errors look like application bugs.

The Solution: A Server-Side Proxy

When working through the The Solution A Server-Side stage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph. Log request id, model id, and latency on every call. Without that trail, intermittent provider errors look like application bugs.

const res = await Promise.race([
 proxyCallable({ model: modelName, body: requestBody }),
 new Promise((_, reject) =>
 setTimeout(() => reject(new Error(‘AI proxy timeout’)), 15000)
 ),
]);

5. Context Compression: What You Feed the Model Matters More Than Which Model You Use

When working through the 5 Context Compression What stage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Log request id, model id, and latency on every call. Without that trail, intermittent provider errors look like application bugs. When working through the 5 Context Compression What stage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion.

[CONTEXT]
Category: Coffee
Date: Saturday, Aug 30, 2026
Time: 10:00 AM — 11:30 AM (90 mins)
Group Size: 4/8 participants
User Role: Attendee
Country: AU
Currency: AUD
[END CONTEXT]

6. Offline-First Agent Design: The Deterministic Fallback

The 6 Offline-First Agent Design stage works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Pin the interpreter and dependency lockfile before teaching the loop. Drift between laptop and CI is the most common silent break for API demos.

7. generationConfig Tuning

The 7 generationConfig Tuning stage works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph. Pin the interpreter and dependency lockfile before teaching the loop. Drift between laptop and CI is the most common silent break for API demos.

{
 “responseMimeType”: “application/json”,
 “temperature”: 0.4,
 “maxOutputTokens”: 600
}

{
 “temperature”: 0.7,
 “maxOutputTokens”: 700
}

8. System Prompt Engineering for Tool Use

The 8 System Prompt Engineering stage works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Pin the interpreter and dependency lockfile before teaching the loop. Drift between laptop and CI is the most common silent break for API demos. The 8 System Prompt Engineering stage works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion.

You have access to autonomous action tools. When users ask for rides,
ALWAYS call request_rideshare. When users ask to book a table,
ALWAYS call reserve_table. When users ask about cultural norms,
ALWAYS call get_cultural_etiquette.

NO RAW MARKDOWN HEADINGS: Never use ‘#’. Use **Bold Text** for headers.
NO ASTERISKS FOR BULLETS: Use clean unicode bullet points (• ).
CONCISE: Jump straight to the answer. No filler introductions.

9. Lessons Learned

For the 9 Lessons Learned stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Separate client construction from the message loop so providers can be swapped without rewriting the conversation state machine.

Operational checklist

When working through the Operational checklist stage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest.

Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline.

Log request id, model id, and latency on every call. Without that trail, intermittent provider errors look like application bugs.

Keep graph state flat and typed. Nested blobs hide which node wrote which field and break resume after interrupts.

Add a smoke test that exercises the critical path in CI with fixtures, not live paid APIs, whenever budgets allow.

Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph.

Before promoting the stack, freeze versions, capture a golden transcript for the critical path, and confirm rollback steps. Shared environments need rate limits, tenancy checks, and a clear owner for secret rotation. Prefer boring reliability over clever one-off demos.

Batch note for 0ead08720324: keep provider keys out of the repo, set a per-session token ceiling, and store transcripts next to the eval fixtures so later model swaps stay comparable.