Build a Tool-Cost Preflight for AI Agent Gateways

AI agent budgets often fail before the model starts reasoning. The prompt looks small, the output cap looks reasonable, and the model route looks approved. Then the agent adds tool schemas, retrieval context, web-search calls, code execution traces, retry repair prompts, and reasoning output. The final bill no longer matches the estimate that approved the run.

That is why an agent gateway needs a tool-cost preflight.

A tool-cost preflight is a small check that runs before dispatch. It estimates the token and non-token exposure created by tools, cache state, thinking output, batch eligibility, and fallback policy. It does not try to predict the exact final bill. It answers a narrower production question: is this agent run allowed to start under the current route contract?

The live source checks for this article were refreshed on August 25, 2026. AIWave's public pricing and predictable-pricing pages returned HTTP 200 and still exposed the dated DeepSeek V4 gateway rows checked on 2026-08-19: V4 Flash at $0.638 input, $1.914 output, and $0.0203 cache-hit input per 1M tokens; V4 Pro at $1.914 input, $5.742 output, and $0.0638 cache-hit input per 1M tokens. DeepSeek's official API pricing page returned HTTP 200 and listed deepseek-v4-flash, deepseek-v4-pro, 1M context, and concurrency markers for 2500 and 500. Kimi's K3 page returned HTTP 200 and contained $0.30/MTok cache-hit input, $3.00/MTok cache-miss input, and $15.00/MTok output markers. Z.AI and QwenCloud pricing pages returned HTTP 200 with cached-input, batch, thinking, and tool-fee markers. Recheck the source pages before changing production limits.

The important pattern is not a single price row. The pattern is that modern AI API cost is no longer just input plus output. A production preflight needs to understand every cost class that can appear during an agent run.

What Tool Preflight Catches

Most teams already estimate prompt and completion tokens. Tool preflight adds the missing fields that make agent workloads different from simple chat completions.

A preflight should run before the gateway accepts the job. If the run does not fit, fail clearly or ask the caller to reduce scope. Do not let the worker discover the problem after the first provider call.

The Preflight Contract

Start with a contract that represents the agent run, not only the user prompt.

{"tenant_id":"acme-support","task_class":"support_triage","route":"deepseek-v4-flash","latency_class":"interactive","estimated_user_tokens":1800,"estimated_retrieval_tokens":6200,"estimated_cache_hit_tokens":5000,"max_output_tokens":900,"max_thinking_tokens":600,"tools_requested":["ticket_lookup","policy_search","web_search"],"tool_call_cap":4,"max_tool_result_tokens":7000,"batch_allowed":false,"fallback_allowed":true,"budget_ceiling_usd":0.025,"policy_version":"tool-preflight-2026-08-25"}

This contract should be created by the application layer. The gateway should not infer that a support workflow can use web search, or that a coding workflow can emit long execution traces, simply because the model supports tools. Product policy owns the allowed tool set. The gateway owns enforcement.

Route Rate Cards Need More Than Token Rows

The rate card used by preflight should describe cost classes explicitly. Keep the source URL and checked date next to the numbers.

Do not hide these fields in prose. A preflight job should load the same structure that finance and support can inspect later.

A Minimal Python Preflight

The following Python example estimates a run before it reaches an OpenAI-compatible endpoint. It uses dated AIWave rows for the route math and separates user input, retrieval input, cache-hit input, model output, thinking output, and tool results.

from dataclasses import dataclass, asdict
from datetime import date
from typing import Literal

TODAY = date(2026, 8, 25)


@dataclass(frozen=True)
class RouteRateCard:
    route: str
    source_url: str
    source_checked_at: str
    rate_card_date: str
    input_per_m: float
    cache_hit_input_per_m: float | None
    output_per_m: float
    thinking_billed_as: Literal["output", "separate", "not_supported"]
    tool_schema_billed_as: Literal["input", "separate"]
    tool_result_billed_as: Literal["input", "separate"]
    max_tool_calls: int
    supports_batch: bool
    fallbacks: list[str]


@dataclass(frozen=True)
class AgentRunContract:
    tenant_id: str
    task_class: str
    route: str
    latency_class: Literal["interactive", "async"]
    estimated_user_tokens: int
    estimated_retrieval_tokens: int
    estimated_cache_hit_tokens: int
    max_output_tokens: int
    max_thinking_tokens: int
    tool_schema_tokens: int
    tool_call_cap: int
    max_tool_result_tokens: int
    batch_allowed: bool
    fallback_allowed: bool
    budget_ceiling_usd: float
    policy_version: str


@dataclass(frozen=True)
class PreflightDecision:
    allowed: bool
    route: str
    estimated_usd: float
    reason: str
    policy_version: str
    source_checked_at: str
    estimate_breakdown: dict[str, float]


def assert_fresh(row: RouteRateCard, max_age_days: int = 7) -> None:
    checked = date.fromisoformat(row.source_checked_at)
    age = (TODAY - checked).days
    if age > max_age_days:
        raise ValueError(f"{row.route} pricing source is {age} days old: {row.source_url}")


def usd(tokens: int, per_million: float) -> float:
    return tokens / 1_000_000 * per_million


def estimate(contract: AgentRunContract, row: RouteRateCard) -> PreflightDecision:
    assert_fresh(row)

    if contract.tool_call_cap > row.max_tool_calls:
        return PreflightDecision(
            allowed=False,
            route=row.route,
            estimated_usd=0.0,
            reason="tool_call_cap_exceeds_route_policy",
            policy_version=contract.policy_version,
            source_checked_at=row.source_checked_at,
            estimate_breakdown={},
        )

    if contract.batch_allowed and not row.supports_batch and contract.latency_class == "async":
        return PreflightDecision(
            allowed=False,
            route=row.route,
            estimated_usd=0.0,
            reason="batch_requested_but_route_contract_disallows_batch",
            policy_version=contract.policy_version,
            source_checked_at=row.source_checked_at,
            estimate_breakdown={},
        )

    input_tokens = (
        contract.estimated_user_tokens
        + contract.estimated_retrieval_tokens
        + contract.tool_schema_tokens
        + contract.max_tool_result_tokens
    )
    cache_hit_tokens = min(contract.estimated_cache_hit_tokens, input_tokens)
    fresh_input_tokens = input_tokens - cache_hit_tokens
    cache_rate = row.cache_hit_input_per_m or row.input_per_m

    input_usd = usd(fresh_input_tokens, row.input_per_m)
    cache_usd = usd(cache_hit_tokens, cache_rate)
    output_usd = usd(contract.max_output_tokens, row.output_per_m)
    thinking_usd = usd(contract.max_thinking_tokens, row.output_per_m)

    breakdown = {
        "fresh_input_usd": round(input_usd, 6),
        "cache_hit_input_usd": round(cache_usd, 6),
        "output_cap_usd": round(output_usd, 6),
        "thinking_cap_usd": round(thinking_usd, 6),
    }
    total = round(sum(breakdown.values()), 6)

    if total > contract.budget_ceiling_usd:
        return PreflightDecision(
            allowed=False,
            route=row.route,
            estimated_usd=total,
            reason="estimated_run_exceeds_budget_ceiling",
            policy_version=contract.policy_version,
            source_checked_at=row.source_checked_at,
            estimate_breakdown=breakdown,
        )

    return PreflightDecision(
        allowed=True,
        route=row.route,
        estimated_usd=total,
        reason="within_tool_and_token_budget",
        policy_version=contract.policy_version,
        source_checked_at=row.source_checked_at,
        estimate_breakdown=breakdown,
    )


rate_card = RouteRateCard(
    route="deepseek-v4-flash",
    source_url="https://aiwave.live/pricing",
    source_checked_at="2026-08-25",
    rate_card_date="2026-08-19",
    input_per_m=0.638,
    cache_hit_input_per_m=0.0203,
    output_per_m=1.914,
    thinking_billed_as="output",
    tool_schema_billed_as="input",
    tool_result_billed_as="input",
    max_tool_calls=4,
    supports_batch=False,
    fallbacks=["glm-5.1", "qwen3.8-turbo"],
)

contract = AgentRunContract(
    tenant_id="acme-support",
    task_class="support_triage",
    route="deepseek-v4-flash",
    latency_class="interactive",
    estimated_user_tokens=1800,
    estimated_retrieval_tokens=6200,
    estimated_cache_hit_tokens=5000,
    max_output_tokens=900,
    max_thinking_tokens=600,
    tool_schema_tokens=2400,
    tool_call_cap=4,
    max_tool_result_tokens=7000,
    batch_allowed=False,
    fallback_allowed=True,
    budget_ceiling_usd=0.025,
    policy_version="tool-preflight-2026-08-25",
)

print(asdict(estimate(contract, rate_card)))

This example is intentionally conservative. It treats maximum tool results and thinking tokens as if they could be used. That may overestimate many runs, but overestimation is acceptable at admission time. Underestimation is what causes surprise invoices and incident reviews.

Tool Schemas Are Part of the Prompt

Tool schemas are easy to forget because they are not written by the end user. They still influence the model context. A gateway should count them.

For a small support workflow, the schema tokens may be trivial. For an agent with many MCP tools, long parameter descriptions, enum lists, examples, and policy text, tool definitions can become a material share of the input. If those schemas are sent on every request instead of cached behind a stable context, the estimate should show it.

Use three fields:

The third field is the one that saves teams during incidents. A web-search tool, ticket search tool, or code execution tool can return far more text than the user prompt. Cap it before dispatch.

Thinking Tokens Need a Separate Cap

QwenCloud's docs call out thinking-token billing as output. Other routes may expose reasoning controls differently. Your preflight should not depend on one provider vocabulary. It should store a max_thinking_tokens field and map it to the route's billing model.

If the route bills thinking as output, include it in the output exposure. If the route has a separate reasoning rate, add a separate field to the rate card. If the route does not expose thinking controls, set thinking_billed_as to not_supported and reject task classes that require that control.

This matters for Tier 1/2 production teams because agent quality experiments often increase hidden output before anyone updates the budget. A preflight makes the change visible in code review.

Cache-Hit Input Changes the Shape

Repeated agent runs often reuse the same system prompt, product policy, repository map, or retrieval bundle. If a route supports cache-hit accounting, the preflight should separate cache-hit input from fresh input.

Do not assume every repeated prefix becomes a cache hit. Store the estimate, then compare it with actual usage after the run. If actual cache-hit tokens are consistently below estimate, the problem may be prompt churn, unstable retrieval ordering, route switching, or provider cache behavior.

Useful ledger fields:

Cache economics are only useful when the gateway can prove when they were expected and when they actually happened.

Fallbacks Must Pass Preflight Too

A fallback route can rescue latency or availability. It can also break the budget. Tool support, context limits, cache behavior, and output pricing can change when a run moves to another provider family.

Do not approve fallback by model name alone. Run the same preflight against each allowed fallback route. If the fallback cannot support the requested tool set, reject it. If the fallback estimate exceeds the contract budget, either block fallback or require a higher budget ceiling from the caller.

For incident handling, record both decisions:

This prevents the most common incident shortcut: moving a workload to a route that can answer text but cannot safely run the same agent policy.

Observability After the Run

Preflight is only half of the control. After the response completes, join the estimate with actual usage.

Use these fields to tune the preflight. If estimates are always too high, reduce caps by task class. If estimates are too low, inspect the tool result budget first. Tool output is often the hidden multiplier.

Rollout Plan

Start in shadow mode. Compute preflight decisions for every agent run, but do not block traffic. Compare the estimate with actual usage for one week. Separate the results by task class, model route, tenant, and tool set.

Next, enforce only hard caps: maximum tool calls, maximum tool result tokens, and maximum output tokens. These are straightforward controls because they describe the run shape, not subjective model quality.

Then enforce stale-source checks. If a rate card has not been refreshed within your allowed window, block new route approvals until the pricing source is rechecked. Existing traffic can keep using the last approved policy while the team updates contracts.

After that, enforce budget ceilings for low-risk task classes such as support summaries, classification, extraction, and batch evaluation. Keep high-value reasoning tasks in review until you have enough actual usage data.

Finally, enforce fallback preflight. During an incident, the router should only move traffic to routes that pass the same tool and budget checks as the primary route.

Source Links

For AIWave gateway routes, keep current links to AIWave pricing, predictable pricing, model catalog, and Chat Completions documentation.

For provider calibration, keep dated links to DeepSeek pricing, Kimi K3, Z.AI pricing, and QwenCloud pricing. Treat these as source checks, not permanent constants.

Final Checklist

Before an agent run starts, make sure the preflight can answer these questions:

Tool-cost preflight turns agent routing from a hopeful dispatch into a contract check. The gateway does not need to predict every token perfectly. It needs to know whether the requested tools, thinking budget, cache assumption, output cap, and fallback policy fit the current route before the first provider call starts.