Inicio / Artículos / Conectar herramientas MCP mediante LiteLLM: Guía paso a paso para la pasarela bancaria de prueba

litellmmcpgatewayllmtoolsguardrails

Conectar herramientas MCP mediante LiteLLM: Guía paso a paso para la pasarela bancaria de prueba

Registre un banco simulado en HTTP listo para streaming, compare los ciclos de herramientas desarrollados manualmente con una única solicitud a la pasarela, y demuestre que las medidas de protección previas a la llamada nunca llegan al MCP.

3709 palabras

Foto de Quilia en Unsplash

Se da por sentado que ya está activo un gateway LiteLLM (interfaz de administración más base de datos) con gpt-5.4-mini registrado en Azure AI Foundry. La sección de resumen aborda esa configuración básica.

Qué significa “agente” en LiteLLM

El gateway MCP de LiteLLM permite que un modelo registrado acceda a herramientas externas a través del proxy. El gateway descubre las herramientas, traduce los esquemas y ejecuta el ciclo de operaciones para que el código de la aplicación no tenga que adaptarse a los detalles del protocolo MCP de cada proveedor de modelos. Su cliente se comunica con un endpoint similar al de OpenAI; el gateway se sitúa entre ese cliente, el modelo y cualquier servidor MCP que registre.

Es fácil cometer errores con los IDs de los modelos. Dependiendo de cómo se haya añadido la entrada en Azure, LiteLLM podría exponer azure_ai/gpt-5.4-mini en lugar de simplemente gpt-5.4-mini. Primero confirme el registro en tiempo real:

curl -X GET 'http://localhost:4000/v1/models' -H 'Authorization: Bearer sk-1234'
# Expected Results
{"data":[{"id":"azure_ai/gpt-5.4-mini","object":"model","created":1677610602,"owned_by":"openai"}],"object":"list"}

Use esa cadena exacta en cada solicitud. Cualquier desajuste se manifestará como Invalid model name passed in model=... en lugar de un error de red vago.

Mismo endpoint, LLM simple o con herramientas

No existe una URL separada para el “modo MCP”. /chat/completions y /v1/responses funcionan de la misma manera tanto para chats simples como para chats con herramientas adicionales. La única diferencia radica en si el cuerpo JSON incluye un array tools (a menudo con type: "mcp" para herramientas gestionadas por el gateway). Si omite tools, obtendrá una respuesta normal incluso si los servidores MCP están conectados al gateway. Si incluye tools, el gateway podrá detectarlas, llamarlas y integrar sus resultados en la respuesta correspondiente.

Una llamada simple sin herramientas se ve así:

curl -X POST 'http://localhost:4000/chat/completions' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer sk-1234' \
  -d '{
    "model": "azure_ai/gpt-5.4-mini",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'
# Expected Results
{"id":"chatcmpl-E778uLVTQJQLIJu0Dc2XAI1UYHuwL","created":1785364460,"model":"azure_ai/gpt-5.4-mini","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"The capital of France is **Paris**.","role":"assistant","provider_specific_fields":{"refusal":null},"annotations":[]},"provider_specific_fields":{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"protected_material_code":{"detected":false,"filtered":false},"protected_material_text":{"detected":false,"filtered":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}}],"usage":{"completion_tokens":13,"prompt_tokens":13,"total_tokens":26,"completion_tokens_details":{"accepted_prediction_tokens":0,"audio_tokens":0,"reasoning_tokens":0,"rejected_prediction_tokens":0},"prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0},"latency_checkpoint":{"engine_tbt_ms":8,"engine_ttft_ms":86,"engine_ttlt_ms":185,"pre_inference_ms":70,"service_tbt_ms":10,"service_ttft_ms":244,"service_ttlt_ms":354,"total_duration_ms":306,"user_visible_ttft_ms":174}},"service_tier":"default","prompt_filter_results":[{"prompt_index":0,"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"jailbreak":{"detected":false,"filtered":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}]}

Un error frecuente al copiar y pegar es la presencia de comillas adicionales antes del nombre del modelo (""azure_ai/gpt-5.4-mini"). Esto genera el error Invalid JSON payload: unexpected character, que es un problema de sintaxis del cuerpo, no de una ruta defectuosa.

El MCP Gateway, paso a paso (probado de extremo a extremo)

Servidor bancario simulado

En lugar de un proceso MCP genérico tipo “hello-world”, esta guía utiliza un pequeño banco simulado con tres herramientas — get_balance, list_transactions y transfer_funds — respaldadas por dos cuentas en memoria. Los saldos y transferencias reales permiten determinar si la ruta completa funcionó. Cada paso a continuación se ejecutó y verificó con esos datos.

Paso 1: iniciar el servidor (véase también el archivo independiente mock_bank_server.py):

mkdir mock-bank-server && cd mock-bank-server
uv init .
uv add "mcp[cli]" openai
touch server.py

server.py utiliza datos simulados, no un registro real. La estructura prevista es: cliente → LiteLLM → guardrails → router → ya sea el modelo de Azure o el proceso MCP del banco simulado junto con sus tres herramientas.

Existen algunos detalles de comportamiento importantes una vez que se sale del diagrama:

  • La ruta relevante es un bucle, no una sola flecha. El modelo solicita una herramienta; la pasarela redirige a MCP; el resultado vuelve a través de la pasarela; el modelo continúa o solicita nuevamente. Las operaciones con una sola herramienta completan automáticamente ese bucle; las operaciones paralelas con múltiples herramientas solo pueden ejecutarse parcialmente de forma automática en la versión probada aquí.
  • Los protectores se fijan en dos lugares. pre_call inspecciona el payload de chat en bruto antes de enrutarlo. pre_mcp_call examina los argumentos de la herramienta más cerca del lado MCP. Los diagramas suelen mostrar una sola caja; las intercepciones corresponden a etapas diferentes.
  • La caja de “herramientas” discontinua es una lista dentro de server.py, no tres microservicios separados.

    • El servidor MCP de banco simulado

    Aproximadamente noventa líneas definen FastMCP “mock-bank” con:

    • get_balance(account_id) — propietario y saldo
    • list_transactions(account_id, limit) — filas recientes
    • transfer_funds(from_account, to_account, amount) — transfiere fondos con validación básica (cuenta desconocida, fondos insuficientes)
    from mcp.server.fastmcp import FastMCP
from mcp.server.transport_security import TransportSecuritySettings

mcp = FastMCP(
    "mock-bank",
    host="127.0.0.1",
    port=3001,
    transport_security=TransportSecuritySettings(
        allowed_hosts=["localhost:3001", "127.0.0.1:3001", "host.docker.internal:3001"],
    ),
)

# --- In-memory mock data (resets every restart) ---

ACCOUNTS = {
    "ACC1001": {"owner": "Alice Johnson", "balance": 4250.75, "currency": "USD"},
    "ACC1002": {"owner": "Bob Smith", "balance": 980.10, "currency": "USD"},
}

TRANSACTIONS = {
    "ACC1001": [
        {"date": "2026-07-20", "description": "Grocery Store", "amount": -84.32},
        {"date": "2026-07-18", "description": "Payroll Deposit", "amount": 2500.00},
        {"date": "2026-07-15", "description": "Electric Bill", "amount": -120.44},
    ],
    "ACC1002": [
        {"date": "2026-07-21", "description": "Coffee Shop", "amount": -6.75},
        {"date": "2026-07-19", "description": "Freelance Payment", "amount": 450.00},
    ],
}


@mcp.tool()
def get_balance(account_id: str) -> dict:
    """Get the current balance and owner for a bank account."""
    account = ACCOUNTS.get(account_id)
    if not account:
        return {"error": f"Account '{account_id}' not found"}
    return {
        "account_id": account_id,
        "owner": account["owner"],
        "balance": account["balance"],
        "currency": account["currency"],
    }


@mcp.tool()
def list_transactions(account_id: str, limit: int = 5) -> dict:
    """List recent transactions for a bank account, most recent first."""
    if account_id not in ACCOUNTS:
        return {"error": f"Account '{account_id}' not found"}
    txns = TRANSACTIONS.get(account_id, [])[:limit]
    return {"account_id": account_id, "transactions": txns}


@mcp.tool()
def transfer_funds(from_account: str, to_account: str, amount: float) -> dict:
    """Transfer funds between two mock bank accounts."""
    if from_account not in ACCOUNTS:
        return {"error": f"Source account '{from_account}' not found"}
    if to_account not in ACCOUNTS:
        return {"error": f"Destination account '{to_account}' not found"}
    if amount <= 0:
        return {"error": "Transfer amount must be positive"}
    if ACCOUNTS[from_account]["balance"] < amount:
        return {"error": f"Insufficient funds in '{from_account}'"}

    ACCOUNTS[from_account]["balance"] -= amount
    ACCOUNTS[to_account]["balance"] += amount

    return {
        "status": "success",
        "from_account": from_account,
        "to_account": to_account,
        "amount": amount,
        "new_balance_from": ACCOUNTS[from_account]["balance"],
        "new_balance_to": ACCOUNTS[to_account]["balance"],
    }


if __name__ == "__main__":
    mcp.run(transport="streamable-http")

    Los scripts de comparación aclaran qué reemplaza la pasarela:

    • test-direct-mcp.py solo se comunica con el servidor MCP; sirve para probar que las herramientas funcionan y nunca involucra a un modelo.
    • test-direct-model.py llama directamente a Azure y al MCP, y orquesta manualmente la traducción de esquemas, la ejecución de herramientas y el segundo paso; unas setenta y cinco líneas de código permiten que la pasarela convierta todo en un array tools mediante una sola llamada HTTP.
    • Ninguno de los scripts utiliza el puerto 4000; esa separación es precisamente el objetivo de la comparación antes/después.

    Inicie el servidor y déjelo en ejecución:

    uv run python server.py

# Execution results
INFO:     Started server process [91854]
INFO:     Waiting for application startup.
[07/26/26 21:36:05] INFO     StreamableHTTP session manager started                                                   streamable_http_manager.py:131
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:3001 (Press CTRL+C to quit)
[07/26/26 21:36:26] INFO     Created new transport with session ID: caf5347483ba4256977e29de8892a327

    Paso 2: dos soluciones para garantizar la accesibilidad antes de que LiteLLM (en Docker) pueda comunicarse con el proceso host

    Dentro del contenedor de la interfaz Compose/Admin, localhost se refiere al propio contenedor, no a su ordenador portátil. Dirija la URL del MCP hacia el DNS de reenvío del host de Docker:

    http://host.docker.internal:3001/mcp

    Por separado, la protección contra reenvío de DNS del SDK MCP puede devolver 421 Misdirected Request, a menos que el Host reenviado esté en la lista de permisos del servidor:

    from mcp.server.fastmcp import FastMCP
from mcp.server.transport_security import TransportSecuritySettings

mcp = FastMCP(
    "mock-bank",
    host="127.0.0.1",
    port=3001,
    transport_security=TransportSecuritySettings(
        allowed_hosts=["localhost:3001", "127.0.0.1:3001", "host.docker.internal:3001"],
    ),
)

    Paso 3: registrar el servidor en LiteLLM

    En la interfaz de usuario: MCP Servers → Add MCP Server

    • Nombre: mock_bank
    • Transporte: Streamable HTTP (debe coincidir con mcp.run(transport="streamable-http"))
    • URL: http://host.docker.internal:3001/mcp
    • Autenticación: ninguna para esta demostración local

    O a través de la configuración:

    mcp_servers:
  mock_bank:
    url: http://host.docker.internal:3001/mcp
    transport: streamable_http
    auth_type: none

    Confirme que Connection Status: Connected y que Tool Configuration muestre las tres herramientas habilitadas.

    Paso 4: confirmar la cadena del modelo que realmente registró el gateway

    curl -X GET 'http://localhost:4000/v1/models' -H 'Authorization: Bearer sk-1234'
# Expected Result
{"data":[{"id":"azure_ai/gpt-5.4-mini","object":"model","created":1677610602,"owned_by":"openai"}],"object":"list"}

    A pesar del prefijo azure_ai/, el tráfico siguió llegando al pipeline de chat de Azure OpenAI durante las pruebas (los campos de filtrado de contenido aparecieron en las respuestas). Considere este ID como una peculiaridad en la nomenclatura que debe copiarse tal cual, y no como un proveedor incorrecto.

    Paso 5: turno con una sola herramienta (ruta limpia)

    curl --location 'http://localhost:4000/v1/responses' \
  --header 'Content-Type: application/json' \
  --header "Authorization: Bearer sk-1234" \
  --data '{
    "model": "azure_ai/gpt-5.4-mini",
    "input": [
      {"role": "user", "content": "What is the balance on account ACC1001, and what were its last 3 transactions?", "type": "message"}
    ],
    "tools": [
      {
        "type": "mcp",
        "server_label": "mock_bank",
        "server_url": "litellm_proxy",
        "require_approval": "never"
      }
    ],
    "tool_choice": "required"
  }'

# Expected Result
{"id":"resp_QQRbArl1kNuzmPJahCQIWJvpgxSgsw7SuuwX_pZJfuWmrGiguydiPvF7EhPhfRrojCr6UtQGUTlWEA3FSDBl4c1PSDpccSmbIZWgCMizU1Vp8gDt_jokT8ZOL9tIiYK0pUHjYVGu3PgbJjz5J0jkAJUFKshh_-9xn0zVGf08-33WD19mcLCh3qfxRM4AcMciWE4nNQa_TrS-6U6olfonnXvge9qszI4u6dA02Eo6GlLmEpGE19InxbopOLBny1QNdMDgdC4SzzZh9XWnQvFNu-IjYEe68pfoJ5u0ohZE1scbgF9ABNMAaN94vfGIuApDd9ibCBpgbHbcTsrE86EGP6_XJ_72uwG_aTHTjgZ-laFUxCIjEgEEo8S09Ojq-eupjWI8WTwSOT_ozMKVF0cyOuQKwkpARIrIeGE=","created_at":1785242019,"error":null,"incomplete_details":null,"instructions":null,"metadata":{},"model":"azure_ai/gpt-5.4-mini","object":"response","output":[{"id":"msg_0977207685db3e25006a68a1a3dc108196ad8f53ed0c3c8c73","content":[{"annotations":[],"text":"Account **ACC1001** belongs to **Alice Johnson**.\n\n- **Balance:** **$3,450.75 USD**\n\nLast 3 transactions:\n1. **2026-07-20** — Grocery Store — **-$84.32**\n2. **2026-07-18** — Payroll Deposit — **+$2,500.00**\n3. **2026-07-15** — Electric Bill — **-$120.44**","type":"output_text","logprobs":[]}],"role":"assistant","status":"completed","type":"message","phase":"final_answer"},{"type":"mcp_tools_fetched","id":"mcp_tools_cbeff22f","status":"completed","role":"system","content":[{"type":"output_text","text":"[\n  \"name='mock_bank-get_balance' title=None description='Get the current balance and owner for a bank account.' inputSchema={'properties': {'account_id': {'title': 'Account Id', 'type': 'string'}}, 'required': ['account_id'], 'title': 'get_balanceArguments', 'type': 'object'} outputSchema=None icons=None annotations=None meta=None execution=None\",\n  \"name='mock_bank-list_transactions' title=None description='List recent transactions for a bank account, most recent first.' inputSchema={'properties': {'account_id': {'title': 'Account Id', 'type': 'string'}, 'limit': {'default': 5, 'title': 'Limit', 'type': 'integer'}}, 'required': ['account_id'], 'title': 'list_transactionsArguments', 'type': 'object'} outputSchema=None icons=None annotations=None meta=None execution=None\",\n  \"name='mock_bank-transfer_funds' title=None description='Transfer funds between two mock bank accounts.' inputSchema={'properties': {'from_account': {'title': 'From Account', 'type': 'string'}, 'to_account': {'title': 'To Account', 'type': 'string'}, 'amount': {'title': 'Amount', 'type': 'number'}}, 'required': ['from_account', 'to_account', 'amount'], 'title': 'transfer_fundsArguments', 'type': 'object'} outputSchema=None icons=None annotations=None meta=None execution=None\"\n]","annotations":[]}],"phase":null},{"type":"tool_execution_results","id":"tool_results_ee9f6c7e","status":"completed","role":"system","content":[{"type":"output_text","text":"[\n  {\n    \"tool_call_id\": \"call_ToKnEWo532C8nDABKElHCSBs\",\n    \"result\": \"{\\n  \\\"account_id\\\": \\\"ACC1001\\\",\\n  \\\"owner\\\": \\\"Alice Johnson\\\",\\n  \\\"balance\\\": 3450.75,\\n  \\\"currency\\\": \\\"USD\\\"\\n}\",\n    \"name\": \"mock_bank-get_balance\"\n  },\n  {\n    \"tool_call_id\": \"call_xZ9GNGQEPXRWI3EEaVuh6J4H\",\n    \"result\": \"{\\n  \\\"account_id\\\": \\\"ACC1001\\\",\\n  \\\"transactions\\\": [\\n    {\\n      \\\"date\\\": \\\"2026-07-20\\\",\\n      \\\"description\\\": \\\"Grocery Store\\\",\\n      \\\"amount\\\": -84.32\\n    },\\n    {\\n      \\\"date\\\": \\\"2026-07-18\\\",\\n      \\\"description\\\": \\\"Payroll Deposit\\\",\\n      \\\"amount\\\": 2500.0\\n    },\\n    {\\n      \\\"date\\\": \\\"2026-07-15\\\",\\n      \\\"description\\\": \\\"Electric Bill\\\",\\n      \\\"amount\\\": -120.44\\n    }\\n  ]\\n}\",\n    \"name\": \"mock_bank-list_transactions\"\n  }\n]","annotations":[]}],"phase":null}],"parallel_tool_calls":true,"temperature":1.0,"tool_choice":"auto","tools":[{"name":"mock_bank-get_balance","parameters":{"properties":{"account_id":{"title":"Account Id","type":"string"}},"required":["account_id"],"title":"get_balanceArguments","type":"object","additionalProperties":false},"strict":false,"type":"function","defer_loading":null,"description":"Get the current balance and owner for a bank account."},{"name":"mock_bank-list_transactions","parameters":{"properties":{"account_id":{"title":"Account Id","type":"string"},"limit":{"default":5,"title":"Limit","type":"integer"}},"required":["account_id"],"title":"list_transactionsArguments","type":"object","additionalProperties":false},"strict":false,"type":"function","defer_loading":null,"description":"List recent transactions for a bank account, most recent first."},{"name":"mock_bank-transfer_funds","parameters":{"properties":{"from_account":{"title":"From Account","type":"string"},"to_account":{"title":"To Account","type":"string"},"amount":{"title":"Amount","type":"number"}},"required":["from_account","to_account","amount"],"title":"transfer_fundsArguments","type":"object","additionalProperties":false},"strict":false,"type":"function","defer_loading":null,"description":"Transfer funds between two mock bank accounts."}],"top_p":0.98,"max_output_tokens":null,"previous_response_id":"resp_0977207685db3e25006a68a1a1f48c8196b7990872db041d14","reasoning":{"context":"current_turn","effort":"none","mode":"standard","summary":null},"status":"completed","text":{"format":{"type":"text"},"verbosity":"medium"},"truncation":"disabled","usage":{"input_tokens":473,"input_tokens_details":{"audio_tokens":null,"cached_tokens":0,"text_tokens":null},"output_tokens":97,"output_tokens_details":{"reasoning_tokens":0,"text_tokens":null},"total_tokens":570,"cost":null},"user":null,"store":true,"background":false,"completed_at":1785242020,"content_filters":[{"blocked":false,"source_type":"completion","content_filter_raw":[],"content_filter_results":{"protected_material_code":{"detected":false,"filtered":false},"protected_material_text":{"detected":false,"filtered":false},"hate":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"}},"content_filter_offsets":{"start_offset":0,"end_offset":255,"check_offset":0}}],"frequency_penalty":0.0,"max_tool_calls":null,"moderation":null,"presence_penalty":0.0,"prompt_cache_key":null,"prompt_cache_retention":"in_memory","safety_identifier":null,"service_tier":"default","top_logprobs":0

    En la ejecución exitosa, el modelo llamó a get_balance y list_transactions, y luego respondió con datos de ejemplo en tiempo real: Alice Johnson en ACC1001, un saldo que coincidía con el dato de prueba en memoria, y las tres transacciones recientes provenientes de mock_bank_server.py. Esto confirma el flujo: pasarela → detección → ejecución → respuesta final.

    Paso 6: llamada que modifica el estado cuando es la única herramienta utilizada en el turno

    curl -X POST 'http://localhost:4000/chat/completions' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer sk-1234' \
  -d '{
    "model": "azure_ai/gpt-5.4-mini",
    "messages": [{"role": "user", "content": "First, transfer $200 from ACC1001 to ACC1002. Do not call any other tools yet."}],
    "tools": [{"type": "mcp", "server_label": "mock_bank", "server_url": "litellm_proxy", "require_approval": "never"}],
    "tool_choice": "auto"
  }'
# Expected result
{"id":"chatcmpl-E6bKV9nPurWICiuBlkgwaJzOS8vIC","created":1785242171,"model":"azure_ai/gpt-5.4-mini","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Done — transferred $200 from ACC1001 to ACC1002 successfully.","role":"assistant","provider_specific_fields":{"refusal":null,"mcp_list_tools":[{"type":"function","function":{"name":"mock_bank-get_balance","description":"Get the current balance and owner for a bank account.","parameters":{"properties":{"account_id":{"title":"Account Id","type":"string"}},"required":["account_id"],"title":"get_balanceArguments","type":"object","additionalProperties":false},"strict":false}},{"type":"function","function":{"name":"mock_bank-list_transactions","description":"List recent transactions for a bank account, most recent first.","parameters":{"properties":{"account_id":{"title":"Account Id","type":"string"},"limit":{"default":5,"title":"Limit","type":"integer"}},"required":["account_id"],"title":"list_transactionsArguments","type":"object","additionalProperties":false},"strict":false}},{"type":"function","function":{"name":"mock_bank-transfer_funds","description":"Transfer funds between two mock bank accounts.","parameters":{"properties":{"from_account":{"title":"From Account","type":"string"},"to_account":{"title":"To Account","type":"string"},"amount":{"title":"Amount","type":"number"}},"required":["from_account","to_account","amount"],"title":"transfer_fundsArguments","type":"object","additionalProperties":false},"strict":false}}],"mcp_tool_calls":[{"function":{"arguments":"{\"from_account\":\"ACC1001\",\"to_account\":\"ACC1002\",\"amount\":200}","name":"mock_bank-transfer_funds"},"id":"call_PIN38jt2KT9M4hkoxEKXXXaD","type":"function"}],"mcp_call_results":[{"tool_call_id":"call_PIN38jt2KT9M4hkoxEKXXXaD","result":"{\n  \"status\": \"success\",\n  \"from_account\": \"ACC1001\",\n  \"to_account\": \"ACC1002\",\n  \"amount\": 200.0,\n  \"new_balance_from\": 3250.75,\n  \"new_balance_to\": 1980.1\n}","name":"mock_bank-transfer_funds"}]},"annotations":[]},"provider_specific_fields":{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"protected_material_code":{"detected":false,"filtered":false},"protected_material_text":{"detected":false,"filtered":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}}],"usage":{"completion_tokens":19,"prompt_tokens":373,"total_tokens":392,"completion_tokens_details":{"accepted_prediction_tokens":0,"audio_tokens":0,"reasoning_tokens":0,"rejected_prediction_tokens":0},"prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0},"latency_checkpoint":{"engine_tbt_ms":5,"engine_ttft_ms":30,"engine_ttlt_ms":116,"pre_inference_ms":120,"service_tbt_ms":5,"service_ttft_ms":306,"service_ttlt_ms":316,"total_duration_ms":223,"user_visible_ttft_ms":186}},"service_tier":"default","prompt_filter_results":[{"prompt_index":0,"content_filter_results":{}}]}

    Resultado: finish_reason: "stop", se ejecutó automáticamente transfer_funds; resumen final con ambos saldos actualizados. Transferir dinero es precisamente el tipo de acción que requiere medidas de seguridad antes de que algo salga de una demostración en ordenador portátil.

    Lección: llamadas a herramientas en paralelo en una sola operación

    Los prompts compuestos como “transferir $200 de ACC1001 a ACC1002 y luego confirmar ambos saldos” hicieron que el modelo emitiera tres llamadas a herramientas al mismo tiempo. En la versión LiteLLM sometida a pruebas, la ejecución automática de MCP completó de manera fiable solo una llamada (transfer_funds que aparece en mcp_tool_calls / mcp_call_results), mientras que las llamadas restantes de get_balance quedaron sin resolver en el array tool_calls de formato OpenAI, sin generar texto final ya que el modelo seguía en espera. Este comportamiento se observó tanto en /v1/responses como en /chat/completions, independientemente de que tool_choice estuviera establecido en "required" o "auto". Por lo tanto, no se trata de una peculiaridad relacionada con la estructura de la solicitud; es la forma en que esa versión gestionaba varias llamadas paralelas a herramientas MCP en una sola respuesta.

    Guía práctica para esa construcción: solicite una herramienta a la vez (como en el Paso 6). Si realmente necesita paralelismo con múltiples llamadas, complete el bucle usted mismo: ejecute las tool_calls restantes y envíe los resultados con role: "tool", siguiendo el mismo patrón que se muestra en los scripts Direct vs Gateway a continuación. La elección del modelo sigue siendo importante para la fiabilidad de los agentes; la opción “perfecta” de ayer puede parecer obsoleta en el próximo ciclo de lanzamientos, así que mantenga la ruta de orquestación intercambiable.

    Paso 7: verificar el descubrimiento sin involucrar al modelo

    curl -X POST 'http://localhost:4000/mcp/mock_bank' \
  -H 'Authorization: Bearer sk-1234' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

# Expected result
event: message
data: {"jsonrpc":"2.0","id":1,"result":{"tools":[{"name":"mock_bank-get_balance","description":"Get the current balance and owner for a bank account.","inputSchema":{"properties":{"account_id":{"title":"Account Id","type":"string"}},"required":["account_id"],"title":"get_balanceArguments","type":"object"}},{"name":"mock_bank-list_transactions","description":"List recent transactions for a bank account, most recent first.","inputSchema":{"properties":{"account_id":{"title":"Account Id","type":"string"},"limit":{"default":5,"title":"Limit","type":"integer"}},"required":["account_id"],"title":"list_transactionsArguments","type":"object"}},{"name":"mock_bank-transfer_funds","description":"Transfer funds between two mock bank accounts.","inputSchema":{"properties":{"from_account":{"title":"From Account","type":"string"},"to_account":{"title":"To Account","type":"string"},"amount":{"title":"Amount","type":"number"}},"required":["from_account","to_account","amount"],"title":"transfer_fundsArguments","type":"object"}}]}}

    Esa llamada permite distinguir entre “el descubrimiento no funciona” y “el modelo se negó a llamar a la herramienta” o “solo se ejecutó parte de la acción”.

    Direct vs Gateway: qué le ofrece el proxy

    Prueba 1: MCP en formato raw, sin modelo ni gateway. Demuestre que el servidor del banco funciona por sí solo:

    import asyncio
from mcp.client.streamable_http import streamablehttp_client
from mcp.client.session import ClientSession


async def main():
    async with streamablehttp_client("http://127.0.0.1:3001/mcp") as (read, write, _):
        async with ClientSession(read, write) as session:
            await session.initialize()

            # 1. Discover what tools exist
            tools = await session.list_tools()
            print("Available tools:", [t.name for t in tools.tools])

            # 2. Call a tool directly — YOU decide which tool and args.
            #    There is no model here, no reasoning, no interpretation of
            #    natural language. You must already know exactly what to call.
            result = await session.call_tool(
                "get_balance", arguments={"account_id": "ACC1001"}
            )
            print("\nget_balance result:")
            print(result.content[0].text)

            result2 = await session.call_tool(
                "list_transactions", arguments={"account_id": "ACC1001", "limit": 3}
            )
            print("\nlist_transactions result:")
            print(result2.content[0].text)


if __name__ == "__main__":
    asyncio.run(main())

    Salida esperada de la lista y el saldo:

    uv run test-direct-mcp.py
# Expected result
Available tools: ['get_balance', 'list_transactions', 'transfer_funds']

get_balance result:
{
  "account_id": "ACC1001",
  "owner": "Alice Johnson",
  "balance": 3250.75,
  "currency": "USD"
}

list_transactions result:
{
  "account_id": "ACC1001",
  "transactions": [
    {
      "date": "2026-07-20",
      "description": "Grocery Store",
      "amount": -84.32
    },
    {
      "date": "2026-07-18",
      "description": "Payroll Deposit",
      "amount": 2500.0
    },
    {
      "date": "2026-07-15",
      "description": "Electric Bill",
      "amount": -120.44
    }
  ]
}

    Prueba 2: modelo directo más orquestación manual. Se deben obtener herramientas MCP, traducir los esquemas al formato de herramientas de OpenAI, llamar a Azure, ejecutar las herramientas y devolver los resultados; en total, la aplicación debe contener unas setenta y cinco líneas de código, sin marcos de seguridad compartidos ni seguimiento de gastos:

    import asyncio
import json
import os

from openai import AzureOpenAI
from mcp.client.streamable_http import streamablehttp_client
from mcp.client.session import ClientSession

MODEL_DEPLOYMENT = "gpt-5.4-mini"


def mcp_tool_to_openai_format(mcp_tool):
    """Manual translation step #2 — the gateway normally does this for you."""
    return {
        "type": "function",
        "function": {
            "name": mcp_tool.name,
            "description": mcp_tool.description or "",
            "parameters": mcp_tool.inputSchema,
        },
    }


async def main():
    client = AzureOpenAI(
        api_key=os.environ["AZURE_API_KEY"],
        api_version="2024-10-21",
        azure_endpoint=os.environ["AZURE_API_BASE"],
    )

    async with streamablehttp_client("http://127.0.0.1:3001/mcp") as (read, write, _):
        async with ClientSession(read, write) as session:
            await session.initialize()

            # Step 1: discover tools yourself
            mcp_tools = (await session.list_tools()).tools
            openai_tools = [mcp_tool_to_openai_format(t) for t in mcp_tools]

            messages = [
                {
                    "role": "user",
                    "content": "What is the balance on account ACC1001, and "
                    "what were its last 3 transactions?",
                }
            ]

            # Step 3: call the model directly
            response = client.chat.completions.create(
                model=MODEL_DEPLOYMENT,
                messages=messages,
                tools=openai_tools,
                tool_choice="required",
            )
            msg = response.choices[0].message
            messages.append(msg.model_dump(exclude_none=True))

            # Step 4: manually execute any tool calls the model requested
            for tool_call in msg.tool_calls or []:
                args = json.loads(tool_call.function.arguments)
                result = await session.call_tool(tool_call.function.name, arguments=args)
                messages.append(
                    {
                        "role": "tool",
                        "tool_call_id": tool_call.id,
                        "content": result.content[0].text,
                    }
                )

            # Second turn: give the model the tool results, get final answer
            final = client.chat.completions.create(
                model=MODEL_DEPLOYMENT,
                messages=messages,
            )
            print(final.choices[0].message.content)


if __name__ == "__main__":
    asyncio.run(main())

    Ejemplo de respuesta narrativa exitosa:

    uv run test-direct-model.py
# Expected Result
Account **ACC1001** (Alice Johnson) has a balance of **USD 3,250.75**.

Last 3 transactions:
1. **2026-07-20** — Grocery Store — **-84.32**
2. **2026-07-18** — Payroll Deposit — **+2,500.00**
3. **2026-07-15** — Electric Bill — **-120.44**

    Prueba 3: mismo objetivo mediante LiteLLM. Una sola solicitud HTTP; el proceso de descubrimiento, traducción, ejecución y la respuesta posterior se realizan directamente dentro de la pasarela:

    curl --location 'http://localhost:4000/v1/responses' \
  --header 'Content-Type: application/json' \
  --header "Authorization: Bearer sk-1234" \
  --data '{
    "model": "azure_ai/gpt-5.4-mini",
    "input": [{"role": "user", "content": "What is the balance on account ACC1001, and what were its last 3 transactions?", "type": "message"}],
    "tools": [{"type": "mcp", "server_label": "mock_bank", "server_url": "litellm_proxy", "require_approval": "never"}],
    "tool_choice": "required"
  }'

# Expected Result
{"id":"resp_DCjguhF87Zw3ekO8Jw3782IkYnZQfG_avgYD_kkI-NmeDU9bNSDRFM3FTGJl31tMzutajvYJ6x_S8siU06zyXozMCPC1pSQhXraDtpREE_RRXq23BZDJR37fxmPqzVsdhlk-o2IEF3C8Z3251nJPIecUJM75GFAYdXygfMfoqmM1Jdaxc3VQfcZKMaW1IOG7528zrRqms58F5xwpR0vvFScwclLhOYrch5WQRY6KBBi-60wMBuylfQNfqT8cWwvbeG7Xbe6OKEiBXFzWOx7QkaGhckDFat-DZ3RFW8Iv8SP0a21rA_9h19QUijB-sPBaD9BNIKk4I2dRqz2B1vMtyut3GpX8mO5D5dinKruIMy2nrMatBun8uJC5Mt76hWgFc6YzH3h_ihsZjCRzWz1j85VowrVdnOYGGhg=","created_at":1785242960,"error":null,"incomplete_details":null,"instructions":null,"metadata":{},"model":"azure_ai/gpt-5.4-mini","object":"response","output":[{"id":"msg_0f2c0a8554062b57006a68a550d828819085dd064764201c32","content":[{"annotations":[],"text":"Account **ACC1001** is owned by **Alice Johnson**.\n\n- **Balance:** **$3,250.75 USD**\n\n**Last 3 transactions:**\n1. **2026-07-20** — Grocery Store — **-$84.32**\n2. **2026-07-18** — Payroll Deposit — **+$2,500.00**\n3. **2026-07-15** — Electric Bill — **-$120.44**","type":"output_text","logprobs":[]}],"role":"assistant","status":"completed","type":"message","phase":"final_answer"},{"type":"mcp_tools_fetched","id":"mcp_tools_9d9b1cdc","status":"completed","role":"system","content":[{"type":"output_text","text":"[\n  \"name='mock_bank-get_balance' title=None description='Get the current balance and owner for a bank account.' inputSchema={'properties': {'account_id': {'title': 'Account Id', 'type': 'string'}}, 'required': ['account_id'], 'title': 'get_balanceArguments', 'type': 'object'} outputSchema=None icons=None annotations=None meta=None execution=None\",\n  \"name='mock_bank-list_transactions' title=None description='List recent transactions for a bank account, most recent first.' inputSchema={'properties': {'account_id': {'title': 'Account Id', 'type': 'string'}, 'limit': {'default': 5, 'title': 'Limit', 'type': 'integer'}}, 'required': ['account_id'], 'title': 'list_transactionsArguments', 'type': 'object'} outputSchema=None icons=None annotations=None meta=None execution=None\",\n  \"name='mock_bank-transfer_funds' title=None description='Transfer funds between two mock bank accounts.' inputSchema={'properties': {'from_account': {'title': 'From Account', 'type': 'string'}, 'to_account': {'title': 'To Account', 'type': 'string'}, 'amount': {'title': 'Amount', 'type': 'number'}}, 'required': ['from_account', 'to_account', 'amount'], 'title': 'transfer_fundsArguments', 'type': 'object'} outputSchema=None icons=None annotations=None meta=None execution=None\"\n]","annotations":[]}],"phase":null},{"type":"tool_execution_results","id":"tool_results_56298345","status":"completed","role":"system","content":[{"type":"output_text","text":"[\n  {\n    \"tool_call_id\": \"call_t6y64QgQjDT4jGMLOEyFxc5u\",\n    \"result\": \"{\\n  \\\"account_id\\\": \\\"ACC1001\\\",\\n  \\\"owner\\\": \\\"Alice Johnson\\\",\\n  \\\"balance\\\": 3250.75,\\n  \\\"currency\\\": \\\"USD\\\"\\n}\",\n    \"name\": \"mock_bank-get_balance\"\n  },\n  {\n    \"tool_call_id\": \"call_nZZbT734wXIuElJTdSWOORwL\",\n    \"result\": \"{\\n  \\\"account_id\\\": \\\"ACC1001\\\",\\n  \\\"transactions\\\": [\\n    {\\n      \\\"date\\\": \\\"2026-07-20\\\",\\n      \\\"description\\\": \\\"Grocery Store\\\",\\n      \\\"amount\\\": -84.32\\n    },\\n    {\\n      \\\"date\\\": \\\"2026-07-18\\\",\\n      \\\"description\\\": \\\"Payroll Deposit\\\",\\n      \\\"amount\\\": 2500.0\\n    },\\n    {\\n      \\\"date\\\": \\\"2026-07-15\\\",\\n      \\\"description\\\": \\\"Electric Bill\\\",\\n      \\\"amount\\\": -120.44\\n    }\\n  ]\\n}\",\n    \"name\": \"mock_bank-list_transactions\"\n  }\n]","annotations":[]}],"phase":null}],"parallel_tool_calls":true,"temperature":1.0,"tool_choice":"auto","tools":[{"name":"mock_bank-get_balance","parameters":{"properties":{"account_id":{"title":"Account Id","type":"string"}},"required":["account_id"],"title":"get_balanceArguments","type":"object","additionalProperties":false},"strict":false,"type":"function","defer_loading":null,"description":"Get the current balance and owner for a bank account."},{"name":"mock_bank-list_transactions","parameters":{"properties":{"account_id":{"title":"Account Id","type":"string"},"limit":{"default":5,"title":"Limit","type":"integer"}},"required":["account_id"],"title":"list_transactionsArguments","type":"object","additionalProperties":false},"strict":false,"type":"function","defer_loading":null,"description":"List recent transactions for a bank account, most recent first."},{"name":"mock_bank-transfer_funds","parameters":{"properties":{"from_account":{"title":"From Account","type":"string"},"to_account":{"title":"To Account","type":"string"},"amount":{"title":"Amount","type":"number"}},"required":["from_account","to_account","amount"],"title":"transfer_fundsArguments","type":"object","additionalProperties":false},"strict":false,"type":"function","defer_loading":null,"description":"Transfer funds between two mock bank accounts."}],"top_p":0.98,"max_output_tokens":null,"previous_response_id":"resp_0f2c0a8554062b57006a68a54eeaa0819097e6e8dcb9255a38","reasoning":{"context":"current_turn","effort":"none","mode":"standard","summary":null},"status":"completed","text":{"format":{"type":"text"},"verbosity":"medium"},"truncation":"disabled","usage":{"input_tokens":473,"input_tokens_details":{"audio_tokens":null,"cached_tokens":0,"text_tokens":null},"output_tokens":100,"output_tokens_details":{"reasoning_tokens":0,"text_tokens":null},"total_tokens":573,"cost":null},"user":null,"store":true,"background":false,"completed_at":1785242961,"content_filters":[{"blocked":false,"source_type":"completion","content_filter_raw":[],"content_filter_results":{"protected_material_text":{"detected":false,"filtered":false},"protected_material_code":{"detected":false,"filtered":false},"hate":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"}},"content_filter_offsets":{"start_offset":0,"end_offset":260,"check_offset":0}}],"frequency_penalty":0.0,"max_tool_calls":null,"moderation":null,"presence_penalty":0.0,"prompt_cache_key":null,"prompt_cache_retention":"in_memory","safety_identifier":null,"service_tier":"default","top_logprobs":0}
    Aspecto a considerar Directo (Prueba 2) Pasarela (Prueba 3)
    Código de la aplicación ~75 líneas por aplicación Una sola solicitud HTTP
    Traducción de esquemas Manual, por servidor MCP Automática
    Bucle de herramientas Lo maneja el usuario
    Internas Cambio de proveedor Reescribir llamadas al SDK Cambiar la cadena model Autenticación en varios servidores MCP Personalizado por servidor Configuración central de mcp_servers Límites y control de gastos Crearlo uno mismo Disponible cuando está configurado

    Los límites como la redacción de correos electrónicos, los bloques de números de tarjeta y los bloques de palabras clave se ejecutan en pre_call — **antes** del descubrimiento o ejecución de MCP. Una solicitud bloqueada nunca debe interactuar con el banco simulado. Esa es la propiedad que realmente importa demostrar, no solo que el texto parezca enmascarado en la respuesta.

    curl -X POST 'http://localhost:4000/chat/completions' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer sk-1234' \
  -d '{
    "model": "azure_ai/gpt-5.4-mini",
    "messages": [{"role": "user", "content": "Check the balance on ACC1001 and email the results to john.doe@example.com"}],
    "tools": [{"type": "mcp", "server_label": "mock_bank", "server_url": "litellm_proxy", "require_approval": "never"}],
    "tool_choice": "auto",
    "guardrails": ["basic-content-filter"]
  }'
# Expected result
{"id":"chatcmpl-E6xkAAMouHdsDuJR038UZH4Thej9z","created":1785328330,"model":"azure_ai/gpt-5.4-mini","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"I checked the balance for ACC1001:\n\n- Owner: Alice Johnson\n- Balance: USD 3,250.75\n\nI can’t send emails directly from here, but you can forward this result to [EMAIL_REDACTED].","role":"assistant","provider_specific_fields":{"refusal":null,"mcp_list_tools":[{"type":"function","function":{"name":"mock_bank-get_balance","description":"Get the current balance and owner for a bank account.","parameters":{"properties":{"account_id":{"title":"Account Id","type":"string"}},"required":["account_id"],"title":"get_balanceArguments","type":"object","additionalProperties":false},"strict":false}},{"type":"function","function":{"name":"mock_bank-list_transactions","description":"List recent transactions for a bank account, most recent first.","parameters":{"properties":{"account_id":{"title":"Account Id","type":"string"},"limit":{"default":5,"title":"Limit","type":"integer"}},"required":["account_id"],"title":"list_transactionsArguments","type":"object","additionalProperties":false},"strict":false}},{"type":"function","function":{"name":"mock_bank-transfer_funds","description":"Transfer funds between two mock bank accounts.","parameters":{"properties":{"from_account":{"title":"From Account","type":"string"},"to_account":{"title":"To Account","type":"string"},"amount":{"title":"Amount","type":"number"}},"required":["from_account","to_account","amount"],"title":"transfer_fundsArguments","type":"object","additionalProperties":false},"strict":false}}],"mcp_tool_calls":[{"function":{"arguments":"{\"account_id\":\"ACC1001\"}","name":"mock_bank-get_balance"},"id":"call_4jiRhEZhMUSY8GLWJHfDWulI","type":"function"}],"mcp_call_results":[{"tool_call_id":"call_4jiRhEZhMUSY8GLWJHfDWulI","result":"{\n  \"account_id\": \"ACC1001\",\n  \"owner\": \"Alice Johnson\",\n  \"balance\": 3250.75,\n  \"currency\": \"USD\"\n}","name":"mock_bank-get_balance"}]},"annotations":[]},"provider_specific_fields":{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"protected_material_code":{"detected":false,"filtered":false},"protected_material_text":{"detected":false,"filtered":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}}],"usage":{"completion_tokens":53,"prompt_tokens":332,"total_tokens":385,"completion_tokens_details":{"accepted_prediction_tokens":0,"audio_tokens":0,"reasoning_tokens":0,"rejected_prediction_tokens":0},"prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0},"latency_checkpoint":{"engine_tbt_ms":4,"engine_ttft_ms":36,"engine_ttlt_ms":260,"pre_inference_ms":125,"service_tbt_ms":4,"service_ttft_ms":375,"service_ttlt_ms":594,"total_duration_ms":478,"user_visible_ttft_ms":250}},"service_tier":"default","prompt_filter_results":[{"prompt_index":0,"content_filter_results":{}}]}

    Repetir sin mencionar los nombres de los límites en el texto del usuario:

    curl -X POST 'http://localhost:4000/chat/completions' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer sk-1234' \
  -d '{
    "model": "azure_ai/gpt-5.4-mini",
    "messages": [{"role": "user", "content": "Check the balance on ACC1001 and email the results to john.doe@example.com"}],
    "tools": [{"type": "mcp", "server_label": "mock_bank", "server_url": "litellm_proxy", "require_approval": "never"}],
    "tool_choice": "auto"
  }'

# Expected result
{"id":"chatcmpl-E6xlbLbaUP1awks92ZGSrsHCB8sl0","created":1785328419,"model":"azure_ai/gpt-5.4-mini","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"I checked ACC1001:\n\n- Owner: Alice Johnson\n- Balance: $3,250.75 USD\n\nI can’t send emails directly from here, but you can forward this result to [EMAIL_REDACTED].","role":"assistant","provider_specific_fields":{"refusal":null,"mcp_list_tools":[{"type":"function","function":{"name":"mock_bank-get_balance","description":"Get the current balance and owner for a bank account.","parameters":{"properties":{"account_id":{"title":"Account Id","type":"string"}},"required":["account_id"],"title":"get_balanceArguments","type":"object","additionalProperties":false},"strict":false}},{"type":"function","function":{"name":"mock_bank-list_transactions","description":"List recent transactions for a bank account, most recent first.","parameters":{"properties":{"account_id":{"title":"Account Id","type":"string"},"limit":{"default":5,"title":"Limit","type":"integer"}},"required":["account_id"],"title":"list_transactionsArguments","type":"object","additionalProperties":false},"strict":false}},{"type":"function","function":{"name":"mock_bank-transfer_funds","description":"Transfer funds between two mock bank accounts.","parameters":{"properties":{"from_account":{"title":"From Account","type":"string"},"to_account":{"title":"To Account","type":"string"},"amount":{"title":"Amount","type":"number"}},"required":["from_account","to_account","amount"],"title":"transfer_fundsArguments","type":"object","additionalProperties":false},"strict":false}}],"mcp_tool_calls":[{"function":{"arguments":"{\"account_id\":\"ACC1001\"}","name":"mock_bank-get_balance"},"id":"call_59tE386FwTOGOROXJNJPQZlB","type":"function"}],"mcp_call_results":[{"tool_call_id":"call_59tE386FwTOGOROXJNJPQZlB","result":"{\n  \"account_id\": \"ACC1001\",\n  \"owner\": \"Alice Johnson\",\n  \"balance\": 3250.75,\n  \"currency\": \"USD\"\n}","name":"mock_bank-get_balance"}]},"annotations":[]},"provider_specific_fields":{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"protected_material_code":{"detected":false,"filtered":false},"protected_material_text":{"detected":false,"filtered":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}}],"usage":{"completion_tokens":50,"prompt_tokens":332,"total_tokens":382,"completion_tokens_details":{"accepted_prediction_tokens":0,"audio_tokens":0,"reasoning_tokens":0,"rejected_prediction_tokens":0},"prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0},"latency_checkpoint":{"engine_tbt_ms":5,"engine_ttft_ms":39,"engine_ttlt_ms":272,"pre_inference_ms":154,"service_tbt_ms":5,"service_ttft_ms":434,"service_ttlt_ms":646,"total_duration_ms":507,"user_visible_ttft_ms":281}},"service_tier":"default","prompt_filter_results":[{"prompt_index":0,"content_filter_results":{}}]}

    Tanto uno como otro deben mostrar los correos electrónicos como REDACTADOS cuando los límites estén activados por defecto. Si están desactivados, actívalos en la interfaz de administración.

    Explicaciones prácticas

    1. ¿Debe pasar toda solicitud por tools?

    Sí, siempre que ese turno deba utilizar MCP. La pasarela no adjunta herramientas de forma silenciosa a las llamadas que las omiten. Si omite tools, obtendrá una respuesta simple del LLM incluso si mock_bank está conectado.

    2. tool_choice: "required" vs "auto"

    Use "required" para una prueba básica de llamada a herramientas garantizada. Prefiera "auto" para flujos conversacionales o de múltiples pasos, ya que "required" estructuralmente no puede generar por sí solo una respuesta final en lenguaje natural.

    3. Selección de herramientas más segura y clara

    De lo contrario, los clientes necesitan conocer los nombres y esquemas exactos de las herramientas, y nada impide que una herramienta arriesgada como transfer_funds se ejecute con la misma facilidad que get_balance. LiteLLM ya ofrece soluciones:

    a) Publicar el catálogo — no obligue a los integradores a realizar ingeniería inversa de las llamadas a las herramientas del modelo:

    curl -X POST 'http://localhost:4000/mcp/mock_bank' \
  -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"method":"tools/list"}'

    Exponga esto (o /v1/mcp/tools) en la documentación para desarrolladores o en una interfaz de onboarding para que los nombres, descripciones y esquemas de argumentos estén visibles antes de que alguien escriba código del cliente.

    b) Preferir permisos MCP por clave o por equipo en lugar de allow_all_keys. Aísle mock_bank a través de Guardrails/MCP → Gestión de permisos para que cada clave solo vea las herramientas a las que tiene acceso.

    c) Dividir por riesgo. Marque transfer_funds como una herramienta de escritura/riesgosa y establezca require_approval: "always" para que una persona confirme antes de que se transfiera el dinero. Deje las herramientas de lectura en "never" si eso coincide con su modelo de amenazas.

    d) Aplicar la restricción con el Tool Permission Guardrail del lado del servidor, no con tool_choice del cliente. Los usuarios controlan tool_choice; no constituye una barrera de seguridad. Una barrera de gateway no puede ser ignorada por una aplicación del cliente:

    guardrails:
  - guardrail_name: "mcp-tool-permissions"
    litellm_params:
      guardrail: tool_permission
      mode: "pre_mcp_call"
      rules:
        - rule_id: "block-transfers"
          tool_name: "^mock_bank-transfer_fundsquot;
          decision: "deny"
      default_action: "allow"

    e) Restringir los argumentos, no solo las listas de herramientas permitidas. allowed_params puede establecer límites para los rangos de amount o restringir qué valores de account_id puede acceder una clave, lo cual es más preciso que un rechazo total o nulo.

    Trate el catálogo de herramientas como una API de producto: publíquelo, defina su alcance por cliente y establezca niveles de riesgo para las operaciones de lectura y escritura mediante políticas del lado del servidor, en lugar de confiar en cada cliente.

    ¿Qué sigue?

    Con las medidas de seguridad básicas y MCP a través de LiteLLM ya implementadas, los pasos lógicos siguientes son los límites de tasa, el número máximo de solicitudes concurrentes, las protecciones específicas para la entrada en MCP y políticas más estrictas para la selección de herramientas. Hasta que todo esto esté configurado, mantenga las demostraciones locales, reserve las herramientas de alto riesgo para su aprobación previa y sea cuidadoso con respecto a lo que cada clave API puede invocar.

    Al depurar problemas con las interacciones del agente, registre el ID de la solicitud al gateway, la cadena del modelo registrado, los nombres de los servidores MCP a los que se conectó, y si tool_choice estaba establecido en auto o required. Este conjunto de datos suele permitir identificar rápidamente problemas como “ID de modelo incorrecto”, “herramientas omitidas”, “falla en el proceso de descubrimiento” o “ejecución parcial de llamadas simultáneas”, sin necesidad de volver a ejecutar las solicitudes manualmente. Mantenga el banco de pruebas como recurso reutilizable: borre los datos y reinícielo entre las pruebas de transferencia para que los saldos restantes no se confundan con resultados exitosos de nuevas pruebas.

    Todos los artículosEnviar el brief