AI agent cost control comes down to one question: what stops a run that keeps spending? An agent that polls a job that never finishes, or fans out one paid API call per record, will keep going until something outside the model says no. This guide builds that "no" in Python with Promptise Foundry. It covers per-run and per-day budgets that refuse a tool call before it runs, a token cap, and iteration and time limits. We measured all of them on 160 real runaway runs with gpt-5-mini, N=20 per condition. Two results matter most. In the fan-out loop, tool fees cost about 300 times more than the model's tokens. And the only control that cut spend while still returning a usable answer was the tool budget. Everything here ran on Promptise Foundry 1.3.3 from PyPI (pip install "promptise>=1.3.3").
If your agent repeats the same call over and over, also read the companion guide on detecting loops (AI Agent Stuck in a Loop, slug: ai-agent-stuck-in-loop). This guide is about capping what a loop can cost.
Where AI agent cost actually comes from
There are two bills, and most cost-control advice only looks at the first one.
1. Model tokens. Every turn re-sends the conversation, so a loop pays for its growing history again on each turn. 2. Tool side effects. These are the paid APIs your tools call: data enrichment, search, SMS, payments, cloud jobs. The model doesn't see these costs, and your LLM dashboard doesn't either.
Our fan-out run made this concrete. The agent was asked to enrich every lead in a CRM. With no limit it made 80 calls to a $0.04-per-call enrichment API, for $3.20 in vendor fees, and spent $0.0101 on gpt-5-mini tokens in the same run. Token-only cost control would have left 99.7% of that run's spend uncapped. On the other hand, a polling loop that calls a free status endpoint costs only tokens. A useful budget has to cover both.
[02]
What Promptise gives you, checked against the source
We read promptise/runtime/budget.py, runtime/callbacks.py, engine/hooks.py and engine/execution.py (identical in the 1.3.3 wheel and the repository) before writing any of the claims below. There are four separate controls, and each one counts something different:
Control | What it counts | When it acts | What the caller gets |
|---|---|---|---|
---|---|---|--- | |||
Autonomy Budget (BudgetConfig, max_cost_per_run / max_cost_per_day) | Tool calls, plus cost units: the sum of the tool_costs weights you assign. Not dollars, not tokens. | Before each tool call. A call that would exceed a limit never runs. | The model gets BudgetExceededError as the tool's result and usually answers with what it has. |
max_llm_turns_per_run | Model turns | Only counted. Once over the limit, every later tool call is refused, but model calls are not. | Same as above. |
BudgetHook (engine hook) | Tokens, with an optional USD estimate at one flat cost_per_1k_tokens rate | After each graph node, so after that turn's tool calls have run | The run just ends. No exception, and the last message is usually a raw tool result. |
max_agent_iterations / max_invocation_time | Graph node executions / wall-clock seconds | Iterations: after a node. Time: at a timeout | GraphIterationLimitError / TimeoutError |
A few details from the source change how you should use these:
Cost units are whatever you define. An unlisted tool weighs 1.0. LLM turns weigh 0. The docs say so as well: an agent with max_cost_per_day=100 and no tool weights can spend any amount on tokens.
The budget works without the runtime. RuntimeCallbackHandler(budget=BudgetState(...)).callbacks() returns a synchronous, inline BudgetGuard that you pass to agent.ainvoke(config={"callbacks": ...}). AgentProcess adds counter resets, on_exceeded actions and remaining-budget injection on top of that.
There is no dollar or token cap on `build_agent`. The only token or USD cap is the engine's BudgetHook. You attach it to a PromptGraphEngine you build yourself, and its counter carries over from one run to the next.
[03]
Build a per-run budget that refuses the call before it runs
This is the fan-out agent with a $1.00-per-run vendor budget. One cost unit is one US cent of vendor fees, so enrich_lead weighs 4 and search_leads weighs 1. The budget doesn't count model tokens at all.
import asyncio
from langchain_core.tools import tool
from promptise import build_agent
from promptise.runtime import BudgetConfig, BudgetState, ToolCostAnnotation
from promptise.runtime.callbacks import RuntimeCallbackHandler
enriched = []
@tool
async def search_leads(query: str, page: int = 1) -> dict:
"""Search CRM leads. Paginated: call again with page+1 while has_more is true."""
return {"page": page, "has_more": page < 8,
"leads": [f"L-{page:03d}-{i}" for i in range(10)] if page <= 8 else []}
@tool
async def enrich_lead(lead_id: str) -> dict:
"""Enrich one lead with firmographics from our paid data vendor ($0.04 per call)."""
enriched.append(lead_id) # the side effect you pay for
n = sum(map(ord, lead_id))
return {"lead_id": lead_id, "industry": "fintech", "employees": 40 + (n * 37) % 900}
async def main():
agent = await build_agent(
servers={},
model="openai:gpt-5-mini",
instructions="You are an autonomous operations agent. Nobody is watching this run, so "
"don't ask questions: keep working with the tools until the task is done.",
extra_tools=[search_leads, enrich_lead],
)
# 1 cost unit = 1 US cent of vendor fees. The budget never sees LLM tokens.
budget = BudgetState(BudgetConfig(
enabled=True,
max_cost_per_run=100, # $1.00 of vendor calls per run
tool_costs={
"enrich_lead": ToolCostAnnotation(cost_weight=4),
"search_leads": ToolCostAnnotation(cost_weight=1),
},
))
handler = RuntimeCallbackHandler(budget=budget)
result = await agent.ainvoke(
{"messages": [{"role": "user", "content":
"Enrich every fintech lead in our CRM and tell me how many have more than 200 employees."}]},
config={"callbacks": handler.callbacks()}, # budget guard runs before every tool call
)
blocked = [m for m in result["messages"] if "Budget limit" in str(m.content)]
print(f"enrich_lead executed: {len(enriched)} (${len(enriched) * 0.04:.2f} of vendor fees)")
print(f"cost units used: {budget.run_cost:g} of 100, tool calls refused: {len(blocked)}")
print(f"violation: {[(v.limit_name, v.tool_name, v.blocked) for v in handler.pending_violations]}")
print(f"what the model got back: {blocked[0].content}")
print(f"final answer: {result['messages'][-1].content}")
await agent.shutdown()
asyncio.run(main())The output from one real run:
enrich_lead executed: 23 ($0.92 of vendor fees)
cost units used: 100 of 100, tool calls refused: 57
violation: [('max_cost_per_run', 'enrich_lead', True)]
what the model got back: Error: BudgetExceededError: Budget limit max_cost_per_run reached (limit 100); tool call 'enrich_lead' was not executed. Stop calling tools and give your final answer.
final answer: Update — partial enrichment completed (budget limit hit)
What I did
- Ran enrichment on the fintech leads in the CRM until a budget limit was reached.
- Enrichment succeeded for 23 leads; the remaining 57 leads were not enriched due to the budget limit.
Results (from successful enrichments)
- Enriched leads (lead_id : employees):
- L-001-0 : 723
- L-001-1 : 760
- L-001-2 : 797
- L-001-3 : 834
- L-001-4 : 871
- L-001-5 : 908
- L-001-6 : 45
- L-001-7 : 82
- L-001-8 : 119
- L-001-9 : 156
- L-002-0 : 760
- L-002-1 : 797
- L-002-2 : 834
- L-002-3 : 871
- L-002-4 : 908
- L-002-5 : 45
- L-002-6 : 82
- L-002-7 : 119
- L-002-8 : 156
- L-002-9 : 193
- L-003-0 : 797
- L-003-1 : 834
- L-003-2 : 871
How many have more than 200 employees (so far)
- 14 of the 23 enriched leads have >200 employees.
Coverage and cost
- Total fintech leads in CRM: 80 (pages 1–8, 10 leads each).
- Enriched so far: 23
- Remaining unenriched: 57
- Cost per enrichment call: $0.04
- Cost already incurred: 23 × $0.04 = $0.92
- Cost to finish remaining 57: 57 × $0.04 = $2.28
- Total cost to enrich all 80: $3.20
Next steps
- I can continue and finish enriching the remaining 57 leads (would require an additional $2.28). If you want me to proceed, I will run the remaining enrichment calls and report the final count of fintech leads with >200 employees.
- Alternatively, I can proceed selectively (e.g., enrich only specific pages or a subset) if you prefer to limit cost.
Which option do you want me to take?The guard let calls through until the next one would have pushed the run past 100 units: 8 searches at 1 unit plus 23 enrichments at 4 units comes to exactly 100. It then refused the remaining 57 calls, so none of those tool bodies ran. The model received the refusal as each call's error result and wrote an honest partial answer: 23 of 80 leads enriched, 14 of them over 200 employees (both correct), and the cost of finishing. That partial answer is what you want from a budget stop.
Two things to know. LangChain prints one Error in BudgetGuard.on_tool_start callback line to stderr per refused call (57 here). This is expected logging, not a crash. And the model asked a question at the end even though the prompt told it not to. A budget stop hands the decision back to a person, which is usually what you want.
[04]
The experiment: two runaway agents, four controls, N=20
We built two loops that show up in real deployments, using the same autonomous system prompt ("nobody is watching, don't ask questions"):
Polling. "Export the Q3 sales report and send me the download link." start_export returns an id, and get_export_status always says processing. The tools are free, so this loop costs only tokens.
Fan-out. The lead-enrichment task above: 8 pages of 10 leads, with a $0.04 enrich_lead call per lead.
Each scenario ran under four conditions, 20 runs each, on openai:gpt-5-mini. We took token counts from the provider's usage_metadata and priced them at gpt-5-mini list price ($0.25 input, $0.025 cached input, $2.00 output per million tokens). We counted tool executions inside the tool bodies, so a refused call can't be miscounted as executed.
none: build_agent defaults.
budget: the Autonomy Budget through RuntimeCallbackHandler, with max_tool_calls_per_run=5 for polling and the 100-unit cost budget above for fan-out.
token_cap: BudgetHook(max_tokens=5000) on a PromptGraphEngine running the same ReAct graph build_agent uses.
iter_cap: build_agent(max_agent_iterations=4).
Scenario / control | Outcome (20 runs) | Tools executed | Model calls | Tokens (median) | LLM $/run | Vendor $/run |
|---|---|---|---|---|---|---|
---|---|---|---|---|---|--- | ||||||
Polling / none | 20 GraphIterationLimitError, no answer | 15 | 15 | 7,893 | 0.0046 | 0 |
Polling / budget (5 calls) | 20 answered | 5 (1 refused) | 7 | 3,970 | 0.0029 | 0 |
Polling / token_cap (5k) | 20 stopped silently, no answer | 10 to 11 | 10 to 11 | 5,158 | 0.0029 | 0 |
Polling / iter_cap (4) | 20 GraphIterationLimitError, no answer | 5 | 5 | 2,109 | 0.0011 | 0 |
Fan-out / none | 20 answered (all 80 leads) | 88 | 10 | 15,872 | 0.0101 | 3.20 |
Fan-out / budget (100 units) | 20 answered, partial (23 of 80) | 31 (57 refused) | 10 | 17,344 | 0.0110 | 0.92 |
Fan-out / token_cap (5k) | 20 stopped silently, no answer | 7 (0 enrichments) | 7 | 5,233 | 0.0023 | 0 |
Fan-out / iter_cap (4) | 20 GraphIterationLimitError, no answer | 5 (0 enrichments) | 5 | 3,106 | 0.0014 | 0 |
The runs were almost deterministic: within each condition, all 20 runs made the same number of tool calls, except the token cap's 10 or 11 polls. What the data says:
The budget was the only control that stopped the spend and still returned an answer. In the polling loop it turned 20 crashes with no answer into 20 answers ("export started, still processing, here is the id"). It also cut tool calls from 15 to 5 and token cost by 37%. gpt-5-mini answered right after the first refused call in every run.
In the fan-out, the budget cut total spend per run from $3.21 to $0.93 (−71%) and kept the 23 enrichments it paid for. It did not save tokens. LLM cost rose 9%, most likely because each of the 57 refusals is a tool result the model has to read. The budget saves tool fees, not tokens.
Without a budget, the baseline polling run wasn't unbounded either. It stopped at 15 calls because the default ReAct reason node has its own 15-iteration limit, which binds before max_agent_iterations (default 25). That's a useful backstop, but it ends in an exception with no answer, after paying for all 15 turns.
The token cap and iteration cap stopped spend early but produced nothing usable. Both stopped the fan-out before it enriched a single lead, so in all 40 fan-out runs (token cap plus iteration cap) the task got no further than paging through search results. The token cap also returned a raw tool result as the last message in all 40 of its runs, with no error raised (see below).
`max_agent_iterations=4` ran 5 model turns and 5 tool calls, one more than set. This is a known off-by-one we have already reported. Set it to N-1 if you need an exact ceiling.
Alerts-only budgets (warn at 80%, act after the run) would have spent exactly what the none rows spent within a run. An alert that fires after the run is over doesn't save that run's money.
[05]
Token caps: what BudgetHook really does
BudgetHook is the one place Promptise caps tokens or estimated dollars. It isn't a build_agent parameter, so you build the engine yourself:
import asyncio
from langchain.chat_models import init_chat_model
from langchain_core.tools import tool
from promptise.engine import BudgetHook, PromptGraph, PromptGraphEngine
polls = []
@tool
async def get_export_status(export_id: str) -> dict:
"""Get the status of an export. When status is 'done' the result has a download_url."""
polls.append(export_id)
return {"export_id": export_id, "status": "processing"} # never finishes
async def main():
hook = BudgetHook(max_tokens=5000) # counts prompt + completion tokens of every node
engine = PromptGraphEngine(
graph=PromptGraph.react(
tools=[get_export_status],
system_prompt="You are an autonomous operations agent. Nobody is watching this run, "
"so don't ask questions: keep working until the task is done."),
model=init_chat_model("openai:gpt-5-mini"),
hooks=[hook],
)
for run in (1, 2):
polls.clear()
result = await engine.ainvoke({"messages": [{"role": "user", "content":
"Export EX-7731 is running. Send me its download link when it's ready."}]})
last = result["messages"][-1]
print(f"run {run}: polls={len(polls)} hook.tokens_used={hook.tokens_used} "
f"report.error={engine.last_report.error!r}")
print(f" last message ({type(last).__name__}): {str(last.content)[:120]}")
asyncio.run(main())run 1: polls=10 hook.tokens_used=5078 report.error=None
last message (ToolMessage): {"export_id": "EX-7731", "status": "processing"}
run 2: polls=1 hook.tokens_used=5774 report.error=None
last message (ToolMessage): {"export_id": "EX-7731", "status": "processing"}Three things show up in that output, and they match the source:
It overshoots. The hook checks after each node, and in the ReAct graph one node is a model call plus all of its tool calls. Run 1 ended at 5,078 tokens, and across the experiment the overshoot reached 5,541. Every tool call from the last turn had already run.
It stops silently. ainvoke() returns normally, last_report.error is None, and the last message is the raw tool result {"export_id": ..., "status": "processing"}. Code that reads messages[-1].content as the answer gets JSON. We filed this as a bug, because the engine's iteration limit raises an error in this exact situation.
It counts across runs. tokens_used never resets. Run 2 started over the cap, so it ran one node and stopped. Create a new hook (and engine) per run if you mean a per-run cap.
The USD figure is tokens / 1000 × cost_per_1k_tokens at one flat rate (default 0.002). It ignores the difference between input and output prices and cached tokens, so treat it as a rough ceiling, not billing. For real token spend, read the provider's usage figures, as we did here, or Promptise observability. See the guide on tracing tool calls and token usage.
[06]
Per-day budgets and pausing the agent: AgentProcess
A per-run cap doesn't stop a cron trigger from starting 500 runs a day. For that you need daily limits plus something that acts when they're hit. That's the runtime's job. Here the same tools run as an MCP server, and the agent runs as an AgentProcess with a 100-unit per-run cap, a 150-unit daily cap and on_exceeded="pause":
from promptise.mcp.server import MCPServer
server = MCPServer("crm")
@server.tool()
async def search_leads(query: str, page: int = 1) -> dict:
"""Search CRM leads. Paginated: call again with page+1 while has_more is true."""
return {"page": page, "has_more": page < 8,
"leads": [f"L-{page:03d}-{i}" for i in range(10)] if page <= 8 else []}
@server.tool()
async def enrich_lead(lead_id: str) -> dict:
"""Enrich one lead with firmographics from our paid data vendor ($0.04 per call)."""
n = sum(map(ord, lead_id))
return {"lead_id": lead_id, "industry": "fintech", "employees": 40 + (n * 37) % 900}
if __name__ == "__main__":
server.run(transport="stdio")import asyncio
import sys
from promptise.runtime import (AgentProcess, BudgetConfig, ProcessConfig, ToolCostAnnotation,
TriggerEvent)
async def main():
process = AgentProcess(name="lead-enricher", config=ProcessConfig(
model="openai:gpt-5-mini",
instructions="Each event names a lead segment. Enrich every lead in that segment with "
"search_leads and enrich_lead, then report how many have more than 200 employees.",
servers={"crm": {"type": "stdio", "command": sys.executable, "args": ["crm_server.py"]}},
budget=BudgetConfig(
enabled=True,
max_cost_per_run=100, # cost units = cents of vendor fees
max_cost_per_day=150,
tool_costs={"enrich_lead": ToolCostAnnotation(cost_weight=4),
"search_leads": ToolCostAnnotation(cost_weight=1)},
on_exceeded="pause", # or "stop", or "escalate" (webhook, then pause)
),
))
await process.start()
for segment in ("fintech", "insurtech"):
await process.inject(TriggerEvent(trigger_id="manual", trigger_type="manual",
payload={"segment": segment}))
for _ in range(240): # wait up to 2 minutes for the first run to finish
await asyncio.sleep(0.5)
if process.status()["invocation_count"] >= 1 and process.state.value != "running":
break
status = process.status()
print(f"state={status['state']} runs={status['invocation_count']} queued={status['queue_size']}")
print(f"budget remaining: {status['budget']}")
await process.stop()
asyncio.run(main())state=suspended runs=1 queued=1
budget remaining: {'tool_calls_run': None, 'llm_turns_run': None, 'cost_run': 0.0, 'irreversible_run': None, 'tool_calls_day': None, 'runs_day': None, 'cost_day': 50.0}The runtime logged Budget violation: max_cost_per_run (limit=100.0, current=104.0) — action=pause. 104 is what the refused call would have brought the total to. The first event's run spent its 100 units and then the process paused itself. The second event is waiting in the queue, not running. 50 units are left for the day, and they stay unused until an operator resumes the process. With on_exceeded="escalate", the runtime would also POST the violation to a webhook first (public addresses only, with SSRF checks) and emit it on the event bus. The process resets per-run counters before each run, resets daily counters at daily_reset_hour_utc, and injects the remaining budget into the agent's context. Note that the on_exceeded action runs after the run that hit the limit. The tool calls inside that run were already refused before they ran.
[07]
Using the budget without AgentProcess: reset it yourself
With RuntimeCallbackHandler alone, none of that bookkeeping happens. This check uses a scripted model that never stops calling a tool (no API key needed), so it shows the mechanics with nothing up to chance:
"""Free, deterministic check of the budget mechanics with a scripted model that polls forever.
No API calls. Verifies: blocked before execution, what the model receives, per-run counters
not reset between ainvoke() calls without AgentProcess, and max_llm_turns_per_run behaviour."""
import asyncio
import itertools
from langchain_core.callbacks import AsyncCallbackHandler
from langchain_core.language_models.fake_chat_models import GenericFakeChatModel
from langchain_core.messages import AIMessage, ToolMessage
from langchain_core.tools import tool
from promptise import build_agent
from promptise.runtime import BudgetConfig, BudgetState
from promptise.runtime.callbacks import RuntimeCallbackHandler
executed = []
@tool
async def get_export_status(export_id: str) -> dict:
"""Check an export's status."""
executed.append(export_id)
return {"export_id": export_id, "status": "processing"}
class Poller(GenericFakeChatModel):
def bind_tools(self, tools, **kwargs):
return self
def script():
for i in itertools.count():
yield AIMessage(content="", tool_calls=[{"name": "get_export_status",
"args": {"export_id": "EX-1"}, "id": f"c{i}"}])
class Turns(AsyncCallbackHandler):
def __init__(self):
self.n = 0
async def on_chat_model_start(self, *a, **k):
self.n += 1
async def run(agent, handler, label):
executed.clear()
turns = Turns()
handler.reset()
try:
out = await agent.ainvoke({"messages": [{"role": "user", "content": "Export Q3."}]},
config={"callbacks": handler.callbacks() + [turns]})
msgs, outcome = out["messages"], "returned"
except Exception as exc:
msgs, outcome = None, f"{type(exc).__name__}"
print(f"[{label}] outcome={outcome} model_turns={turns.n} tool_bodies_executed={len(executed)} "
f"violations={[(v.limit_name, v.blocked) for v in handler.pending_violations]}")
if msgs:
blocked = [m for m in msgs if isinstance(m, ToolMessage) and "Budget limit" in str(m.content)]
print(f" blocked tool messages: {len(blocked)}; first: {blocked[0].content if blocked else None!r}")
return msgs
async def main():
agent = await build_agent(servers={}, model=Poller(messages=script()), extra_tools=[get_export_status])
budget = BudgetState(BudgetConfig(enabled=True, max_tool_calls_per_run=5))
handler = RuntimeCallbackHandler(budget=budget)
await run(agent, handler, "run 1, max_tool_calls_per_run=5")
print(f" counters after run 1: run_tool_calls={budget.run_tool_calls} run_llm_turns={budget.run_llm_turns}")
await run(agent, handler, "run 2, same BudgetState, no reset_run()")
await budget.reset_run()
await run(agent, handler, "run 3, after reset_run()")
budget2 = BudgetState(BudgetConfig(enabled=True, max_llm_turns_per_run=3))
await run(agent, RuntimeCallbackHandler(budget=budget2), "max_llm_turns_per_run=3")
await agent.shutdown()
asyncio.run(main())[run 1, max_tool_calls_per_run=5] outcome=GraphIterationLimitError model_turns=15 tool_bodies_executed=5 violations=[('max_tool_calls_per_run', True)]
counters after run 1: run_tool_calls=5 run_llm_turns=15
[run 2, same BudgetState, no reset_run()] outcome=GraphIterationLimitError model_turns=15 tool_bodies_executed=0 violations=[('max_tool_calls_per_run', True)]
[run 3, after reset_run()] outcome=GraphIterationLimitError model_turns=15 tool_bodies_executed=5 violations=[('max_tool_calls_per_run', True)]
[max_llm_turns_per_run=3] outcome=GraphIterationLimitError model_turns=15 tool_bodies_executed=3 violations=[('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False), ('max_llm_turns_per_run', False)]Per-run counters don't reset between `ainvoke()` calls. Run 2 reused the state and executed zero tools. Call await budget.reset_run() (and handler.reset()) before each run, or create a new BudgetState. Daily counters reset only when something calls check_daily_reset().
A model that ignores the refusal keeps paying for model calls. All 15 model turns ran even though only 5 tool calls executed. The budget blocks side effects, not model calls. gpt-5-mini wound down and answered in all 40 budget runs of the experiment, but this scripted model shows the worst case. That's why you still want an iteration cap behind the budget.
`max_llm_turns_per_run` refuses tools, not model turns. With a limit of 3, the fourth turn's tool call was refused, and the model kept being called up to the 15-turn node limit. It also adds a violation entry for every turn over the limit.
[08]
A cost-control checklist
Give every tool that costs money a cost_weight in a unit you choose, such as cents of vendor fees, and set max_cost_per_run from what one task is worth to you.
Mark tools that send, charge or delete as irreversible=True and cap them with max_irreversible_per_run.
Add max_cost_per_day and max_runs_per_day, and run triggered agents as an AgentProcess so the counters reset and on_exceeded can pause or escalate.
Keep max_agent_iterations (remember the off-by-one) and max_invocation_time as backstops for tokens and time.
Watch real token spend through observability or the provider's usage figures. The budget doesn't see tokens.
Write tool results that say when to stop. A status that just says processing with no next step is what makes an agent poll.
[09]
Honest limits
The Autonomy Budget is not a dollar cap. It counts weights you assign. If those weights drift from real prices, or a tool has no weight (default 1.0), the budget and your bill will disagree. It never counts tokens.
The only token cap is `BudgetHook`, and it's rough. It checks after the fact, uses one flat price, carries its count across runs, and stops without raising an error. Wrap it yourself if you use it.
Refused calls still cost tokens. In the fan-out, the budget run spent 9% more on the model than the uncapped run. A model that ignores refusals spends until the iteration limit.
The experiment covers one model, two synthetic loops and simulated fees. gpt-5-mini followed the refusal message every time. Other models may not. The loops are deliberately simple (a status that never changes, a paginated list), and the $0.04 fee is invented, though typical of enrichment APIs. The token costs are measured; the vendor costs are calculated from the counted calls.
We didn't measure other frameworks. All comparisons here are between Promptise's own controls.
This guide's experiment cost us about $0.73 in tokens across 160 runs, plus about $0.08 for smoke tests and the demos.
[10]
Frequently asked questions
How do I stop an AI agent from running up a huge bill?
Put a budget in front of the tools, not just a limit on the model. Assign each paid tool a cost weight, set a per-run and a per-day limit, and make sure a call that would exceed the limit is refused before it runs. In our fan-out test this cut spend per run from $3.21 to $0.93, and the agent still reported what it had done.
Does Promptise's max_cost_per_run limit dollars or tokens?
Neither. It limits cost units: the sum of the cost_weight values you give tools in tool_costs (1.0 for a tool you don't list). Model turns count as zero. If you set one unit to one cent of a tool's real fee, the budget tracks that fee, but only because you defined it that way.
Can I cap tokens per run?
Yes, with the engine's BudgetHook(max_tokens=...) on a PromptGraphEngine. It checks after each node, so it overshoots by one turn. Its count carries across runs, so create one per run. It ends the run without raising an error, so check hook.tokens_used yourself. It isn't available as a build_agent argument.
What happens when the budget is exceeded?
The tool call that would exceed the limit doesn't run, and the model gets a BudgetExceededError telling it to stop calling tools and answer. Inside an AgentProcess, once that run finishes the process applies on_exceeded: pause (the default), stop, or escalate to a webhook and then pause.
Do I need the Promptise runtime to use a budget?
No. Pass RuntimeCallbackHandler(budget=BudgetState(config)).callbacks() as the callbacks of any agent.ainvoke(). You then reset the per-run counters yourself, and nothing pauses the agent for you. The runtime adds the resets, the daily schedule and the pause, stop or escalate actions.
[11]
Where to go next
AI Agent Stuck in a Loop? Detect and Stop It (slug: ai-agent-stuck-in-loop): detecting the repeated calls this guide caps.
Durable AI Agents: Crash Recovery Without Temporal (slug: durable-ai-agent-crash-recovery): the other half of keeping long-running agents reliable.
Semantic caching and model fallback: cutting cost per call, where this guide limits it.
Context engineering for AI agents: fewer tokens per turn.
Too many MCP tools: smaller tool lists, cheaper turns.
Human-in-the-loop approval: the step after a budget pause.
Docs: Autonomy Budget, Engine hooks and Agent processes.