AI agent cost control comes down to one question: what stops a run that keeps spending? An agent that polls a job that never finishes, or calls a paid API once per record, keeps going until something outside the model says no. This guide explains where agent costs come from, which limits catch which kind of spend, and how to build per-run and per-day budgets, token caps and iteration limits with Promptise Foundry, with real output from each one.
How do you control AI agent costs?
Put limits outside the model, in two places. Before each tool call, check a budget that weighs what the call costs and refuse the call that would go over, so the side effect never happens. After each model turn, check a token cap and an iteration limit, so a run that never answers still ends. Add daily limits for agents that run on schedules or webhooks, and decide what happens when a limit is hit: pause, stop or alert.
There are two bills. Model tokens, and the paid APIs your tools call. Most dashboards only show the first.
A tool budget is the only control that prevents spend before it happens and still lets the agent answer with what it has.
A token cap checks after each turn, so every tool call of the last turn has already run.
Iteration and time limits are backstops. They end the run, usually without an answer.
Daily limits need something that resets counters and acts, such as an agent runtime.
[02]
Where does AI agent cost come from?
Every agent run has two costs.
1. Model tokens. Every turn re-sends the conversation, so a loop pays for its growing history again on each turn. Long tool results make each turn more expensive. 2. Tool side effects. These are the paid APIs your tools call: data enrichment, search, SMS, payments, cloud jobs. The model doesn't see these costs, and your LLM bill doesn't show them.
Which one dominates depends on the tools. A polling loop that calls a free status endpoint costs only tokens. An agent that enriches 80 leads at $0.04 each spends $3.20 in vendor fees, and in the example below the model's tokens for a similar run cost about a cent. A token budget alone would leave most of that spend uncapped. A cost-control design has to cover both bills.
Where each limit acts
┌───────────────────────────────┐
│ model turn │
└───────────────────────────────┘
│ asks for tool calls
▼
┌───────────────────────────────────┐
│ BEFORE each tool call: │── over budget ──▶ refused; the model gets
│ tool budget (calls, cost units, │ "stop calling tools, answer"
│ irreversible actions) │
└───────────────────────────────────┘
│ allowed
▼
tool runs (paid side effect happens here)
│
▼
┌───────────────────────────────────┐
│ AFTER each turn: │── over ──▶ run ends with an error
│ token cap, iteration limit, │ (no answer if the model
│ time limit │ hadn't answered yet)
└───────────────────────────────────┘
│
AFTER the run: daily totals, pause / stop / alert[03]
Which cost controls exist and what does each one count?
Each control counts something different, and none of them covers everything. In Promptise they map to four features:
Control | What it counts | When it acts | What the caller gets |
|---|---|---|---|
Tool budget (BudgetConfig: max_tool_calls_per_run, max_cost_per_run, max_cost_per_day) | Tool calls, and cost units: the sum of the weights you give tools. Not dollars, not tokens | Before each tool call. A call that would go over never runs | The model gets BudgetExceededError as the tool result and usually answers with what it has |
Token cap (BudgetHook) | Tokens of the current run, with a rough USD estimate at one flat rate | After each model turn, so after that turn's tool calls ran | GraphBudgetExceededError if the model hadn't answered yet, else the answer |
Iteration limit (max_agent_iterations) | Graph steps, which in a ReAct agent means model calls | After each step | GraphIterationLimitError |
Time limit (max_invocation_time) | Wall-clock seconds | At the timeout | TimeoutError |
Two details from the budget docs change how you use these. Cost units are whatever you define: a tool you don't list weighs 1.0, and model turns weigh 0, so a budget with no weights says nothing about tokens. And max_llm_turns_per_run counts model turns but enforces on tools: once it is over, later tool calls are refused, while model calls continue up to the iteration limit.
[04]
When to use each control (and when not to)
Situation | Control to use |
|---|---|
Tools that cost money per call | A tool budget with cost weights, per run and per day |
Tools that send, charge or delete | irreversible=True and max_irreversible_per_run, plus an approval gate |
A model that may poll or retry | A tool-call cap per run, and a loop guard |
Long transcripts or large tool results | A token cap as a backstop, and context compaction to shrink each turn |
Agents on cron, webhooks or queues | Daily limits in a runtime that resets counters and pauses or alerts |
Every agent | An iteration limit and a time limit as the last line |
Don't use the iteration limit as your main cost control. It counts steps, not money, and it ends the run with an exception, so the user gets nothing for what was spent. And don't rely on alerts alone: an alert that fires after a run is over doesn't save that run's money.
[05]
What you need
Python 3.10 or later and Promptise Foundry 1.4.1 or later.
An OpenAI API key in OPENAI_API_KEY. The examples use openai:gpt-5-mini, and each run costs about a cent. Step 5 needs no API key.
About 20 minutes.
pip install "promptise>=1.4.1"[06]
Step 1: Give every paid tool a cost weight
A budget can only count what you tell it. Pick a unit and give each tool a weight in that unit. Here one cost unit is one US cent of vendor fees, so enrich_lead, which calls a $0.04 enrichment API, weighs 4, and search_leads weighs 1. The budget then caps the run at 100 units, or $1.00 of vendor calls:
from promptise.runtime import BudgetConfig, BudgetState, ToolCostAnnotation
budget = BudgetState(BudgetConfig(
enabled=True,
max_cost_per_run=100, # $1.00 of vendor calls per run
tool_costs={
"enrich_lead": ToolCostAnnotation(cost_weight=4),
"search_leads": ToolCostAnnotation(cost_weight=1),
},
))ToolCostAnnotation also takes irreversible=True for tools that send, charge or delete, which max_irreversible_per_run caps separately. Choose the unit from what the run is worth to you, not from the model's price list. Weights don't need to be exact. They need to be in the right proportion, so that the budget stops a run at about the spend you meant.
[07]
Step 2: Refuse tool calls over a per-run budget
The budget works without the full runtime. RuntimeCallbackHandler(budget=...) turns it into a LangChain callback that runs before every tool call, and you pass it to agent.ainvoke(). The example is a lead-enrichment agent with an autonomous prompt and 80 leads to enrich:
import asyncio
from langchain_core.tools import tool
from promptise import build_agent
from promptise.runtime import BudgetConfig, BudgetState, ToolCostAnnotation
from promptise.runtime.callbacks import RuntimeCallbackHandler
enriched = []
@tool
async def search_leads(query: str, page: int = 1) -> dict:
"""Search CRM leads. Paginated: call again with page+1 while has_more is true."""
return {"page": page, "has_more": page < 8,
"leads": [f"L-{page:03d}-{i}" for i in range(10)] if page <= 8 else []}
@tool
async def enrich_lead(lead_id: str) -> dict:
"""Enrich one lead with firmographics from our paid data vendor ($0.04 per call)."""
enriched.append(lead_id) # the side effect you pay for
n = sum(map(ord, lead_id))
return {"lead_id": lead_id, "industry": "fintech", "employees": 40 + (n * 37) % 900}
async def main():
agent = await build_agent(
servers={},
model="openai:gpt-5-mini",
instructions="You are an autonomous operations agent. Nobody is watching this run, so "
"don't ask questions: keep working with the tools until the task is done.",
extra_tools=[search_leads, enrich_lead],
)
# 1 cost unit = 1 US cent of vendor fees. The budget never sees LLM tokens.
budget = BudgetState(BudgetConfig(
enabled=True,
max_cost_per_run=100, # $1.00 of vendor calls per run
tool_costs={
"enrich_lead": ToolCostAnnotation(cost_weight=4),
"search_leads": ToolCostAnnotation(cost_weight=1),
},
))
handler = RuntimeCallbackHandler(budget=budget)
result = await agent.ainvoke(
{"messages": [{"role": "user", "content":
"Enrich every fintech lead in our CRM and tell me how many have more than 200 employees."}]},
config={"callbacks": handler.callbacks()}, # budget guard runs before every tool call
)
blocked = [m for m in result["messages"] if "Budget limit" in str(m.content)]
print(f"enrich_lead executed: {len(enriched)} (${len(enriched) * 0.04:.2f} of vendor fees)")
print(f"cost units used: {budget.run_cost:g} of 100, tool calls refused: {len(blocked)}")
print(f"violation: {[(v.limit_name, v.tool_name, v.blocked) for v in handler.pending_violations]}")
print(f"what the model got back: {blocked[0].content}")
print(f"final answer: {result['messages'][-1].content}")
await agent.shutdown()
asyncio.run(main())The output from one run:
enrich_lead executed: 23 ($0.92 of vendor fees)
cost units used: 100 of 100, tool calls refused: 57
violation: [('max_cost_per_run', 'enrich_lead', True)]
what the model got back: Error: BudgetExceededError: Budget limit max_cost_per_run reached (limit 100); tool call 'enrich_lead' was not executed. Stop calling tools and give your final answer.
final answer: I started enriching the fintech leads but hit a budget limit before finishing. Here's what I was able to enrich and the results so far.
Summary
- Total fintech leads in CRM: 80 (8 pages × 10 leads)
- Leads enriched so far: 23
- Of those 23, leads with >200 employees: 14
- Leads with ≤200 employees (from enriched set): 9
- Remaining fintech leads not yet enriched: 57
Enriched leads with >200 employees (14)
- L-001-0, L-001-1, L-001-2, L-001-3, L-001-4, L-001-5
- L-002-0, L-002-1, L-002-2, L-002-3, L-002-4
- L-003-0, L-003-1, L-003-2
Reason enrichment stopped
- The enrichment run hit a budget limit (max_cost_per_run). I used the available tool results rather than repeating calls.
If you want me to finish enriching the remaining 57 leads
- Estimated additional cost: 57 × $0.04 = $2.28
- I can proceed and then give the final total of fintech leads with >200 employees across all 80 records.
Next step
- I can continue now to finish the enrichments (will incur the estimated cost), or stop here. Which should I do?What happened:
The guard let calls through until the next one would have passed 100 units. 8 searches at 1 unit plus 23 enrichments at 4 units is exactly 100. The remaining 57 calls were refused before their tool bodies ran, so they cost nothing.
The model got the refusal as each call's result and wrote an honest partial answer: what it enriched, what it found, and what finishing would cost. That is what you want from a budget stop. The run's model tokens cost about a cent, against $0.92 of vendor fees.
The agent handed the decision back to a person, even though the prompt said not to ask. After a budget stop that is usually the right behavior.
LangChain logs one Error in BudgetGuard.on_tool_start callback line to stderr per refused call, 57 here. That is expected logging, not a crash.
[08]
Step 3: Cap tokens per run
The tool budget doesn't see tokens. For tokens, Promptise has the engine's BudgetHook, which counts the tokens of each run and ends the run once it goes over. It isn't a build_agent() argument, so this example builds the engine directly. You can also append it to an agent built with build_agent() through agent.hooks. The tool here is a status check that never finishes, the classic polling loop:
import asyncio
from langchain.chat_models import init_chat_model
from langchain_core.tools import tool
from promptise.engine import BudgetHook, GraphBudgetExceededError, PromptGraph, PromptGraphEngine
polls = []
@tool
async def get_export_status(export_id: str) -> dict:
"""Get the status of an export. When status is 'done' the result has a download_url."""
polls.append(export_id)
return {"export_id": export_id, "status": "processing"} # never finishes
async def main():
hook = BudgetHook(max_tokens=5000) # per run: prompt + completion tokens of every node
engine = PromptGraphEngine(
graph=PromptGraph.react(
tools=[get_export_status],
system_prompt="You are an autonomous operations agent. Nobody is watching this run, "
"so don't ask questions: keep working until the task is done."),
model=init_chat_model("openai:gpt-5-mini"),
hooks=[hook],
)
for run in (1, 2): # the same engine and hook for both runs
polls.clear()
try:
result = await engine.ainvoke({"messages": [{"role": "user", "content":
"Export EX-7731 is running. Send me its download link when it's ready."}]})
print(f"run {run}: answered after {len(polls)} polls: {result['messages'][-1].content[:120]}")
except GraphBudgetExceededError as exc:
print(f"run {run}: {type(exc).__name__} after {len(polls)} polls")
print(f" reason={exc.reason!r} stopped_by={exc.stopped_by} pending_tools={exc.pending_tools}")
asyncio.run(main())run 1: GraphBudgetExceededError after 11 polls
reason='Token budget exceeded: 5349/5000 tokens' stopped_by=BudgetHook pending_tools=['get_export_status']
run 2: GraphBudgetExceededError after 11 polls
reason='Token budget exceeded: 5285/5000 tokens' stopped_by=BudgetHook pending_tools=['get_export_status']Three things show up in that output, and all three follow from where the hook acts:
It overshoots. The hook checks after each node, and in the ReAct graph one node is a model call plus all of its tool calls. Both runs ended a few hundred tokens over 5,000. A turn that asks for twenty paid calls runs all twenty before the cap sees them.
It raises when the model hasn't answered. GraphBudgetExceededError carries reason, stopped_by and pending_tools, the tools whose results the model never saw. If the model answers on the node that crosses the cap, you get the answer instead.
It counts per run. Both runs used the same engine and the same hook, and each got its own 5,000 tokens. One hook can cap every run of an engine, including concurrent ones.
The USD figure (max_cost_usd) uses one flat cost_per_1k_tokens rate (0.002 by default). It ignores the difference between input, cached and output prices, so treat it as a rough ceiling, not billing. For real token spend, read the provider's usage figures or Promptise observability, as AI agent observability shows.
[09]
Step 4: Add daily budgets and pause the agent
A per-run cap doesn't stop a cron trigger from starting 500 runs a day. For that you need daily limits, counters that reset on a schedule, and an action when a limit is hit. That is the job of the agent runtime. Here the same tools run as an MCP server:
from promptise.mcp.server import MCPServer
server = MCPServer("crm")
@server.tool()
async def search_leads(query: str, page: int = 1) -> dict:
"""Search CRM leads. Paginated: call again with page+1 while has_more is true."""
return {"page": page, "has_more": page < 8,
"leads": [f"L-{page:03d}-{i}" for i in range(10)] if page <= 8 else []}
@server.tool()
async def enrich_lead(lead_id: str) -> dict:
"""Enrich one lead with firmographics from our paid data vendor ($0.04 per call)."""
n = sum(map(ord, lead_id))
return {"lead_id": lead_id, "industry": "fintech", "employees": 40 + (n * 37) % 900}
if __name__ == "__main__":
server.run(transport="stdio")The agent runs as an AgentProcess with a 100-unit per-run cap, a 150-unit daily cap and on_exceeded="pause". Two events arrive, one per lead segment:
import asyncio
import sys
from promptise.runtime import (AgentProcess, BudgetConfig, ProcessConfig, ToolCostAnnotation,
TriggerEvent)
async def main():
process = AgentProcess(name="lead-enricher", config=ProcessConfig(
model="openai:gpt-5-mini",
instructions="Each event names a lead segment. Enrich every lead in that segment with "
"search_leads and enrich_lead, then report how many have more than 200 employees.",
servers={"crm": {"type": "stdio", "command": sys.executable, "args": ["crm_server.py"]}},
budget=BudgetConfig(
enabled=True,
max_cost_per_run=100, # cost units = cents of vendor fees
max_cost_per_day=150,
tool_costs={"enrich_lead": ToolCostAnnotation(cost_weight=4),
"search_leads": ToolCostAnnotation(cost_weight=1)},
on_exceeded="pause", # or "stop", or "escalate" (webhook, then pause)
),
))
await process.start()
for segment in ("fintech", "insurtech"):
await process.inject(TriggerEvent(trigger_id="manual", trigger_type="manual",
payload={"segment": segment}))
for _ in range(240): # wait up to 2 minutes for the first run to finish
await asyncio.sleep(0.5)
if process.status()["invocation_count"] >= 1 and process.state.value != "running":
break
status = process.status()
print(f"state={status['state']} runs={status['invocation_count']} queued={status['queue_size']}")
print(f"budget remaining: {status['budget']}")
await process.stop()
asyncio.run(main())state=suspended runs=1 queued=1
budget remaining: {'tool_calls_run': None, 'llm_turns_run': None, 'cost_run': 0.0, 'irreversible_run': None, 'tool_calls_day': None, 'runs_day': None, 'cost_day': 50.0}The runtime logged Budget violation: max_cost_per_run (limit=100.0, current=104.0) — action=pause. 104 is what the refused call would have brought the total to. The first run spent its 100 units, and then the process paused itself (suspended). The second event is waiting in the queue, not running. 50 units are left for the day, and they stay unused until an operator resumes the process.
The on_exceeded action runs after the run that hit the limit. The calls inside that run were already refused before they ran. Besides "pause" you can use "stop", or "escalate", which posts the violation to a webhook and emits it on the event bus before pausing. The process also resets per-run counters before each run, resets daily counters at daily_reset_hour_utc, and adds the remaining budget to the agent's context. See agent processes in the docs.
[10]
Step 5: Reset and test the budget without a model
With RuntimeCallbackHandler alone, nobody does the runtime's bookkeeping for you. This check uses a scripted model that never stops calling a tool, so it needs no API key and shows the mechanics with nothing left to chance:
"""Free, deterministic check of the budget mechanics with a scripted model that polls forever.
No API calls. Verifies: blocked before execution, what the model receives, per-run counters
not reset between ainvoke() calls without AgentProcess, and max_llm_turns_per_run behaviour."""
import asyncio
import itertools
from langchain_core.callbacks import AsyncCallbackHandler
from langchain_core.language_models.fake_chat_models import GenericFakeChatModel
from langchain_core.messages import AIMessage, ToolMessage
from langchain_core.tools import tool
from promptise import build_agent
from promptise.runtime import BudgetConfig, BudgetState
from promptise.runtime.callbacks import RuntimeCallbackHandler
executed = []
@tool
async def get_export_status(export_id: str) -> dict:
"""Check an export's status."""
executed.append(export_id)
return {"export_id": export_id, "status": "processing"}
class Poller(GenericFakeChatModel):
def bind_tools(self, tools, **kwargs):
return self
def script():
for i in itertools.count():
yield AIMessage(content="", tool_calls=[{"name": "get_export_status",
"args": {"export_id": "EX-1"}, "id": f"c{i}"}])
class Turns(AsyncCallbackHandler):
def __init__(self):
self.n = 0
async def on_chat_model_start(self, *a, **k):
self.n += 1
async def run(agent, handler, label):
executed.clear()
turns = Turns()
handler.reset()
try:
out = await agent.ainvoke({"messages": [{"role": "user", "content": "Export Q3."}]},
config={"callbacks": handler.callbacks() + [turns]})
msgs, outcome = out["messages"], "returned"
except Exception as exc:
msgs, outcome = None, f"{type(exc).__name__}"
print(f"[{label}] outcome={outcome} model_turns={turns.n} tool_bodies_executed={len(executed)} "
f"violations={[(v.limit_name, v.blocked) for v in handler.pending_violations]}")
if msgs:
blocked = [m for m in msgs if isinstance(m, ToolMessage) and "Budget limit" in str(m.content)]
print(f" blocked tool messages: {len(blocked)}; first: {blocked[0].content if blocked else None!r}")
return msgs
async def main():
agent = await build_agent(servers={}, model=Poller(messages=script()), extra_tools=[get_export_status])
budget = BudgetState(BudgetConfig(enabled=True, max_tool_calls_per_run=5))
handler = RuntimeCallbackHandler(budget=budget)
await run(agent, handler, "run 1, max_tool_calls_per_run=5")
print(f" counters after run 1: run_tool_calls={budget.run_tool_calls} run_llm_turns={budget.run_llm_turns}")
await run(agent, handler, "run 2, same BudgetState, no reset_run()")
await budget.reset_run()
await run(agent, handler, "run 3, after reset_run()")
budget2 = BudgetState(BudgetConfig(enabled=True, max_llm_turns_per_run=3))
await run(agent, RuntimeCallbackHandler(budget=budget2), "max_llm_turns_per_run=3")
await agent.shutdown()
asyncio.run(main())[run 1, max_tool_calls_per_run=5] outcome=GraphIterationLimitError model_turns=15 tool_bodies_executed=5 violations=[('max_tool_calls_per_run', True)]
counters after run 1: run_tool_calls=5 run_llm_turns=15
[run 2, same BudgetState, no reset_run()] outcome=GraphIterationLimitError model_turns=15 tool_bodies_executed=0 violations=[('max_tool_calls_per_run', True)]
[run 3, after reset_run()] outcome=GraphIterationLimitError model_turns=15 tool_bodies_executed=5 violations=[('max_tool_calls_per_run', True)]
[max_llm_turns_per_run=3] outcome=GraphIterationLimitError model_turns=15 tool_bodies_executed=3 violations=[('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False)]Per-run counters don't reset between `ainvoke()` calls. Run 2 reused the state and executed zero tools. Call await budget.reset_run() (and handler.reset()) before each run, or create a new BudgetState. Daily counters reset only when something calls check_daily_reset().
A model that ignores the refusal keeps paying for model calls. All 15 model turns ran, though only 5 tool calls executed. The budget blocks side effects, not model calls. Real models usually answer after the first refusal, but this is the worst case, and it is why an iteration limit belongs behind the budget.
`max_llm_turns_per_run` refuses tools, not model turns. With a limit of 3, the fourth turn's tool call was refused, and the model kept being called up to the 15-turn node limit. It also records a violation for every turn over the limit.
Keep a check like this in CI. It proves the budget is wired up, and it fails if someone removes the callbacks from the ainvoke() call.
[11]
The complete example
cost-control-demo/
├── guard_demo.py # per-run cost budget that refuses calls (steps 1-2)
├── token_cap_demo.py # BudgetHook token cap on the graph engine (step 3)
├── crm_server.py # the lead tools as an MCP server (step 4)
├── process_demo.py # AgentProcess with per-run and daily limits (step 4)
└── mechanics_check.py # budget mechanics with a scripted model, no API key (step 5)pip install "promptise>=1.4.1"
export OPENAI_API_KEY=sk-...
python guard_demo.py
python token_cap_demo.py
python process_demo.py
python mechanics_check.py # no API key neededTo add the controls to your own agent, combine the three layers:
from promptise import build_agent
from promptise.engine import BudgetHook
from promptise.runtime import BudgetConfig, BudgetState, ToolCostAnnotation
from promptise.runtime.callbacks import RuntimeCallbackHandler
agent = await build_agent(model="openai:gpt-5-mini", servers=servers, instructions=prompt,
max_agent_iterations=25, max_invocation_time=120) # backstops
agent.hooks.append(BudgetHook(max_tokens=200_000)) # token cap
budget = BudgetState(BudgetConfig(enabled=True, max_cost_per_run=100,
tool_costs={"enrich_lead": ToolCostAnnotation(cost_weight=4)}))
handler = RuntimeCallbackHandler(budget=budget)
await budget.reset_run()
handler.reset()
result = await agent.ainvoke({"messages": [...]}, config={"callbacks": handler.callbacks()})[12]
Running it in production
Size limits from real traffic
Look at a week of traces and take the most expensive healthy run, then add a margin. Review every run that hits a limit: it is either a loop or a task that needs a higher limit. A limit nobody reviews drifts until it either blocks good work or stops nothing.
Track real spend per agent and per user
Budgets enforce limits. They don't tell you what you spent. Record tokens and cost per run with observability, tag runs with the agent, the tenant and the user, and compare the totals with your provider's bill. For multi-tenant agents, give each tenant its own daily budget, as multi-tenant AI agents describes.
Make each turn cheaper
Limits cap the worst case. To lower the normal case, send less per turn: smaller tool lists (too many MCP tools), shorter tool results and compacted context (context engineering), and cached answers for repeated questions (semantic caching and fallback).
Fix the loops that burn budget
Most runaway spend is a loop: a status that never changes, an empty search the model keeps rewording. Write tool results that say whether retrying helps, and add a loop guard. AI agent stuck in a loop shows both.
Put a person behind the pause
A paused agent is only useful if someone notices. Use on_exceeded="escalate" to post the violation to your on-call channel, and route expensive or irreversible actions through human approval before they run.
[13]
Common problems and fixes
Symptom | Cause | Fix |
|---|---|---|
The budget never fires on an expensive agent | No tool_costs weights, so every call weighs 1.0 and tokens weigh 0 | Weight every paid tool in a unit you choose |
The second run on an agent refuses every tool call | Per-run counters are not reset between ainvoke() calls | Call budget.reset_run() and handler.reset() before each run, or run in an AgentProcess |
The token cap fires but vendor fees still ran up | BudgetHook checks after a turn, when its tool calls have already run | Cap paid tools with a tool budget, which refuses calls before they run |
GraphBudgetExceededError with no answer | The cap fired before the model answered | Raise the cap, or rely on the tool budget so the model gets a turn to answer |
Many | LangChain logs each refused call | Expected; filter the logger if it is noisy |
A paused process never resumes | on_exceeded="pause" waits for an operator | Alert with "escalate", and resume after review |
The USD estimate doesn't match the bill | BudgetHook uses one flat rate per 1,000 tokens | Use provider usage or observability for spend; use the hook as a ceiling |
[14]
Limitations
The tool budget is not a dollar cap. It counts weights you assign. If they drift from real prices, the budget and your bill disagree. It never counts tokens.
The only token cap is `BudgetHook`, and it is coarse. It overshoots by one turn and lets that turn's tool calls run. Its USD estimate uses one flat price.
Refused calls still cost tokens. Each refusal is a tool result the model reads, and a model that ignores refusals keeps spending until the iteration limit.
Daily limits need the runtime, or your own scheduler. With RuntimeCallbackHandler alone, nothing resets counters or pauses the agent.
Budgets don't judge answers. A run that stays under budget can still be wrong. Pair limits with evaluations.
[15]
Frequently asked questions
How do I stop an AI agent from running up a huge bill?
Put a budget in front of the tools, not only a limit on the model. Give each paid tool a cost weight, set a per-run and a per-day limit, and refuse the call that would go over before it runs. Then add a token cap, an iteration limit and a time limit as backstops, and an action, such as pause or alert, for when a daily limit is hit.
What is the difference between a token budget and a tool budget?
A token budget counts the model's input and output tokens and checks after each turn, so it ends a run but can't stop that turn's tool calls. A tool budget counts tool calls or cost units and checks before each call, so a refused call never runs and the model can still answer. Agents with paid tools need both.
Does max_cost_per_run limit dollars or tokens?
Neither. In Promptise it limits cost units: the sum of the cost_weight values you give tools in tool_costs, with 1.0 for a tool you don't list. Model turns count as zero. If you set one unit to one cent of a tool's real fee, the budget tracks that fee, but only because you defined it that way.
Can I cap tokens per run?
Yes. In Promptise, use the engine's BudgetHook(max_tokens=...) on a PromptGraphEngine, or append it to an agent's agent.hooks. It counts each run separately, so one hook can serve every run. It checks after each node, so it overshoots by one turn, and when it stops a run before the model answered, it raises GraphBudgetExceededError.
What happens when an agent's budget is exceeded?
The tool call that would go over doesn't run, and the model gets an error telling it to stop calling tools and answer. Most models then reply with a partial result. In a Promptise AgentProcess, once that run ends the process applies on_exceeded: pause, which is the default, stop, or escalate to a webhook and then pause.
Do I need an agent runtime to use a budget?
No. In Promptise, pass RuntimeCallbackHandler(budget=BudgetState(config)).callbacks() as the callbacks of any agent.ainvoke(). You then reset the per-run counters yourself, and nothing pauses the agent for you. The runtime adds the resets, the daily schedule and the pause, stop or escalate actions.
[16]
Next steps
Stop the loops that burn budget: AI agent stuck in a loop.
See where limits sit in an agent: what is an agent harness?.
Record real token spend: AI agent observability.
Run agents unattended with budgets and health checks: the autonomous AI agent runtime.
Reference: Autonomy Budget, engine hooks and agent processes in the Promptise docs.