Idempotent Tool Calls: Make AI Agent Retries Safe

Give every tool that changes the outside world an idempotency key derived from the business action, such as invoice:acme:2026-10, and make the system that performs the action enforce it. The first call reserves the key and does the work. A repeat with the same key returns the first result, or "still in progress", and does nothing. For a call whose outcome is unknown, such as a timeout, give the model a status tool so it checks before it acts again.

• The model is a retry layer. After a timeout or an unclear error, it calls the tool again, with a new tool call ID and sometimes reworded arguments.

• A timeout is not a failure. The work can still be running, and it often finishes.

• Derive the key from what the action means, not from the attempt: no timestamps, no random values per call, no tool call IDs.

• Check the key where the side effect happens: the provider's idempotency header, or a unique constraint in your own database.

• Reads need no key. Creates, sends, payments and anything that counts do.

A classic API client has one retry loop. An agent stack has several, stacked on top of each other, and each one can repeat a tool call without knowing that another layer already did:

• People press send twice, or ask again because the answer was slow.

• Event senders redeliver. Webhook providers and chat services repeat a delivery that didn't get a 2xx answer.

• The agent runtime retries a failed run, redelivers dead-lettered events, or restarts a process after a crash. A run retry repeats the whole turn, including tools that already succeeded.

• The model repeats calls. If a tool times out, returns an error that says "retry later", or returns something the model doesn't trust, it calls the tool again. It also loses track of earlier calls when the context is trimmed.

• The tool wrapper gives up after its timeout while the work underneath keeps going.

• HTTP clients and SDKs resend requests after a dropped connection.

Retries are not the bug. They are how distributed systems recover. The bug is a tool that can't tell a retry from a new request.

A timeout tells you one thing: the caller stopped waiting. It doesn't tell you whether the work happened. Here is an invoicing agent with a tool that gives up after 2 seconds, in front of a billing API that takes 3:

The stand-in billing API is a SQLite ledger with a 3-second delay. It accepts an optional idempotency key, the way Stripe's Idempotency-Key header works:

"""A stand-in for a billing provider: slow, with optional idempotency keys that
work like Stripe's Idempotency-Key header. The ledger is a SQLite file."""
import sqlite3
import time
import uuid

LEDGER = "ledger.db"


class Conflict(Exception):
    """Another request with the same idempotency key is still running (HTTP 409)."""


def _db() -> sqlite3.Connection:
    conn = sqlite3.connect(LEDGER, timeout=10)
    conn.execute(
        "CREATE TABLE IF NOT EXISTS invoices (id TEXT PRIMARY KEY, customer TEXT, amount REAL,"
        " memo TEXT, status TEXT, idempotency_key TEXT UNIQUE)"
    )
    return conn


def create_invoice(customer: str, amount: float, memo: str, idempotency_key: str | None = None) -> dict:
    """Create and send an invoice. Takes about 3 seconds."""
    invoice_id = "inv_" + uuid.uuid4().hex[:8]
    with _db() as conn:
        if idempotency_key:
            row = conn.execute(
                "SELECT id, customer, amount, memo, status FROM invoices WHERE idempotency_key = ?",
                (idempotency_key,),
            ).fetchone()
            if row and row[4] == "processing":
                raise Conflict(f"request {idempotency_key} is still being processed")
            if row:  # a retry of a finished request: return the first result, do nothing
                return {"id": row[0], "customer": row[1], "amount": row[2], "memo": row[3], "replayed": True}
        try:  # reserve the key before doing the work; the UNIQUE constraint settles races
            conn.execute(
                "INSERT INTO invoices VALUES (?, ?, ?, ?, 'processing', ?)",
                (invoice_id, customer, amount, memo, idempotency_key),
            )
        except sqlite3.IntegrityError:
            raise Conflict(f"request {idempotency_key} is already being processed") from None
    time.sleep(3)  # the slow part: PDF, email, accounting sync
    with _db() as conn:
        conn.execute("UPDATE invoices SET status = 'sent' WHERE id = ?", (invoice_id,))
    return {"id": invoice_id, "customer": customer, "amount": amount, "memo": memo}


def find_invoice(idempotency_key: str) -> dict | None:
    with _db() as conn:
        row = conn.execute(
            "SELECT id, amount, status FROM invoices WHERE idempotency_key = ?", (idempotency_key,)
        ).fetchone()
    return dict(zip(("id", "amount", "status"), row)) if row else None


def ledger() -> list[str]:
    with _db() as conn:
        rows = conn.execute("SELECT id, customer, amount, memo, status FROM invoices").fetchall()
    return [f"{r[0]} {r[1]} {r[2]:.2f} EUR {r[3]!r} ({r[4]})" for r in rows]


def reset() -> None:
    with _db() as conn:
        conn.execute("DELETE FROM invoices")

The first version of the agent calls it without a key:

"""An invoicing agent whose tool times out while the invoice is still being sent."""
import asyncio
import json

from promptise import build_agent, tool

import billing


@tool(timeout=2)
def send_invoice(customer: str, amount: float, memo: str) -> dict:
    """Create and send an invoice to a customer.

    Args:
        customer: Customer ID, for example "acme".
        amount: Amount in EUR.
        memo: What the invoice is for.
    """
    return billing.create_invoice(customer, amount, memo)


async def main() -> None:
    billing.reset()
    agent = await build_agent(model="openai:gpt-5-mini", extra_tools=[send_invoice])
    result = await agent.ainvoke(
        {"messages": [{"role": "user", "content": "Invoice acme 120 EUR for October support."}]}
    )
    for message in result["messages"]:
        for call in getattr(message, "tool_calls", None) or []:
            print("tool call:", call["name"], json.dumps(call["args"]))
        if message.type == "tool":
            print("tool result:", message.content.replace("\n", " ")[:160])
    print("agent:", result["messages"][-1].content.split("\n\n")[0], "[...]")
    await asyncio.sleep(4)  # let calls that timed out finish in the background
    print("ledger:", *billing.ledger(), sep="\n  ")
    await agent.shutdown()


asyncio.run(main())

tool call: send_invoice {"customer": "acme", "amount": 120, "memo": "Support for October"}
tool result: {   "error": {     "code": "TIMEOUT",     "message": "Tool 'send_invoice' timed out after 2.0s",     "retryable": true,     "suggestion": "The tool did not fini
tool call: send_invoice {"customer": "acme", "amount": 120, "memo": "Support for October"}
tool result: {   "error": {     "code": "TIMEOUT",     "message": "Tool 'send_invoice' timed out after 2.0s",     "retryable": true,     "suggestion": "The tool did not fini
tool call: send_invoice {"customer": "acme", "amount": 120, "memo": "Support for October"}
tool result: {   "error": {     "code": "TIMEOUT",     "message": "Tool 'send_invoice' timed out after 2.0s",     "retryable": true,     "suggestion": "The tool did not fini
agent: I tried to send the invoice but the invoicing tool timed out (retryable). Details I attempted to send:
- Customer: acme
- Amount: 120 EUR
- Memo: "Support for October" [...]
ledger:
  inv_61040495 acme 120.00 EUR 'Support for October' (sent)
  inv_d9881557 acme 120.00 EUR 'Support for October' (sent)
  inv_3d4356cb acme 120.00 EUR 'Support for October' (sent)

The model called send_invoice three times, got TIMEOUT three times, and told the user it couldn't send the invoice. The ledger has three invoices. Every call timed out from the agent's point of view, and every call succeeded from the customer's.

Two details in the output explain it. The error says "retryable": true and suggests trying again later, which is true for a read and dangerous for a write. And the tool is a plain def, so Promptise runs it in a worker thread that can't be stopped: the tool docs say a timed-out sync tool "finishes in the background and its result is dropped." An async tool is cancelled instead, but cancelling your side doesn't cancel a request the billing API already received.

If the tool's effect stays the same however often it runs, you don't need a key. Most tools that matter aren't like that.

• Python 3.10 or later and Promptise Foundry 1.6.0 for the examples.

• An OpenAI API key for the two agent runs. The retry test needs no model.

• A system of record that can enforce uniqueness: a provider that accepts idempotency keys, or a database table with a unique constraint.

uv venv && uv pip install "promptise==1.6.0"

A good key names one action in your business and is the same on every attempt. For invoices, "one invoice per customer per billing period" is the rule, so the key is invoice:<customer>:<period>:

def invoice_key(customer: str, period: str) -> str:
    return f"invoice:{customer.strip().lower()}:{period}"

Three choices in that line matter:

• Not the tool call ID. Model providers give each tool call an ID, and some guides derive keys from it. When the model re-issues a call after a timeout, it is a new tool call with a new ID, so that key misses exactly the retries you care about.

• Not every argument. The memo is free text the model may word differently on the next try ("October support", then "Support, October 2026"). A hash of all arguments would treat that as a new invoice. Key on the fields that identify the action, and normalize them.

• A parameter that makes the action explicit. The naive tool had no period, so "the October invoice" existed only in the memo. Adding period to the tool's schema gives the key something stable to use, and the model fills it.

When no business identity exists, such as "send this reminder email now", create a request ID once, before the first attempt, and store it with the work item. In an agent runtime, the trigger event's ID is such a value: in Promptise, an event that is retried or redelivered keeps its event_id.

Keep personal data out of keys. Stripe's docs advise against email addresses or personal identifiers as idempotency keys, because keys end up in logs.

The check belongs in the system that performs the action, not in the agent's memory. The agent's journal or conversation is written after the tool returns, so it knows nothing about a call that timed out or crashed halfway.

You have two places to enforce it:

• The provider supports keys. Send yours. Stripe, for example, saves the status code and body of the first request with a key and returns them for later requests with the same key. It compares the parameters and returns an error if they differ, and keys can be pruned once they are at least 24 hours old, so a retry days later is a new request.

• It doesn't. Keep your own table: insert the key with status processing and a unique constraint before you do the work, mark it done afterwards, and return the stored result for a repeat.

billing.py above does the second, and shows two details that are easy to get wrong:

• Reserve the key before the slow work. If the key is written only after the invoice is sent, a retry that arrives during the 3 seconds sees nothing and sends a second invoice. That is exactly the timeout case.

• Let the unique constraint decide races. Two concurrent calls can both read "no such key" before either writes it. The INSERT with a UNIQUE column lets exactly one win; the other gets a conflict. A check-then-insert without the constraint has a race.

The tool now gets one of three answers from the billing API: a new invoice, a replay of the first one, or "still in progress". It turns each into something the model can act on:

"""The same agent with an idempotent tool: a key derived from the business action,
sent to the provider, and a status tool for unknown outcomes."""
import asyncio
import json

from promptise import ToolError, build_agent, tool

import billing


def invoice_key(customer: str, period: str) -> str:
    """One invoice per customer and billing period: the same action always gets the same key."""
    return f"invoice:{customer.strip().lower()}:{period}"


@tool(timeout=2)
def send_invoice(customer: str, period: str, amount: float, memo: str) -> dict:
    """Create and send the invoice for one customer and billing period.

    Safe to retry: a second call for the same customer and period never sends a
    second invoice.

    Args:
        customer: Customer ID, for example "acme".
        period: Billing period as YYYY-MM, for example "2026-10".
        amount: Amount in EUR.
        memo: What the invoice is for.
    """
    key = invoice_key(customer, period)
    try:
        invoice = billing.create_invoice(customer, amount, memo, idempotency_key=key)
    except billing.Conflict:
        raise ToolError(
            f"The invoice for {customer} {period} is still being sent.",
            code="IN_PROGRESS",
            retryable=False,
            suggestion="Don't send it again. Check invoice_status in a few seconds.",
        ) from None
    if invoice.get("replayed") and invoice["amount"] != amount:
        raise ToolError(
            f"{customer} already has an invoice for {period} over {invoice['amount']:.2f} EUR.",
            code="CONFLICT",
            suggestion="Ask the user before changing or adding an invoice.",
        )
    return invoice


@tool
def invoice_status(customer: str, period: str) -> dict:
    """Check whether the invoice for a customer and billing period exists and was sent."""
    found = billing.find_invoice(invoice_key(customer, period))
    return found or {"status": "not found"}


async def main() -> None:
    billing.reset()
    agent = await build_agent(
        model="openai:gpt-5-mini",
        extra_tools=[send_invoice, invoice_status],
        instructions=(
            "You send invoices. If a call that changes something times out, its outcome is "
            "unknown: check invoice_status before you call it again. Reply in one plain sentence."
        ),
    )
    result = await agent.ainvoke(
        {"messages": [{"role": "user", "content": "Invoice acme 120 EUR for October 2026 support."}]}
    )
    for message in result["messages"]:
        for call in getattr(message, "tool_calls", None) or []:
            print("tool call:", call["name"], json.dumps(call["args"]))
        if message.type == "tool":
            print("tool result:", " ".join(message.content.split())[:150])
    print("agent:", result["messages"][-1].content.split("\n\n")[0])
    await asyncio.sleep(4)
    print("ledger:", *billing.ledger(), sep="\n  ")
    await agent.shutdown()


if __name__ == "__main__":
    asyncio.run(main())

• In progress is not retryable. The IN_PROGRESS error says retryable=False and suggests invoice_status. Sending again can't help; checking can.

• A replay with different arguments is a conflict, not a success. If the model asks for 150 EUR and the period already has a 120 EUR invoice, the tool returns CONFLICT and asks the model to check with the user, instead of silently returning the old invoice.

• A status tool resolves unknown outcomes. invoice_status is a read, so the model can call it as often as it likes.

• The description says it is safe to retry. The model reads tool descriptions; "a second call never sends a second invoice" stops it from inventing workarounds.

Each retry path is a plain function call, so you can test them all without an LLM. ainvoke runs the tool the way the agent does, with validation and the 2-second timeout:

"""Retry the keyed tool the ways an agent stack does, without a model, and count invoices."""
import asyncio

import billing
from agent_keyed import send_invoice

ARGS = {"customer": "acme", "period": "2026-10", "amount": 120, "memo": "October support"}


async def call(args: dict) -> str:
    try:
        return await send_invoice.ainvoke(args)  # validated and timed, as the agent runs it
    except Exception as exc:  # a ToolError reaches callers as FunctionToolError
        return f"{type(exc).__name__}: {exc}"


async def main() -> None:
    billing.reset()
    print("1 first call:      ", await call(ARGS))
    print("2 retry at once:   ", await call(ARGS))
    await asyncio.sleep(3.5)
    print("3 retry later:     ", await call({**ARGS, "memo": "Support, October 2026"}))
    print("4 other amount:    ", await call({**ARGS, "amount": 150}))
    print("5 two at once:     ", await asyncio.gather(call({**ARGS, "period": "2026-11"}),
                                                     call({**ARGS, "period": "2026-11"})))
    await asyncio.sleep(3.5)
    print("ledger:", *billing.ledger(), sep="\n  ")


asyncio.run(main())

1 first call:       FunctionToolError: TIMEOUT: Tool 'send_invoice' timed out after 2.0s
2 retry at once:    FunctionToolError: IN_PROGRESS: The invoice for acme 2026-10 is still being sent.
3 retry later:      {"id": "inv_4b0c2526", "customer": "acme", "amount": 120.0, "memo": "October support", "replayed": true}
4 other amount:     FunctionToolError: CONFLICT: acme already has an invoice for 2026-10 over 120.00 EUR.
5 two at once:      ["FunctionToolError: TIMEOUT: Tool 'send_invoice' timed out after 2.0s", 'FunctionToolError: IN_PROGRESS: The invoice for acme 2026-11 is still being sent.']
ledger:
  inv_4b0c2526 acme 120.00 EUR 'October support' (sent)
  inv_1ebc1dd5 acme 120.00 EUR 'October support' (sent)

Five ways to call the same action, two invoices, one per period. Line 1 is the dangerous case from before: a timeout while the invoice is being sent. Line 2 is the model retrying at once and getting IN_PROGRESS. Line 3 retries later with a reworded memo and gets the first invoice back, marked replayed. Line 4 asks for a different amount and gets CONFLICT. Line 5 sends two calls at the same moment for November; one wins, the other is told the invoice is in progress.

Keep tests like these in CI for every tool with side effects: timeout after success, immediate retry, retry after completion, changed arguments and concurrent duplicates. Each should end with exactly one effect.

The same request to the fixed agent:

tool call: send_invoice {"customer": "acme", "period": "2026-10", "amount": 120, "memo": "October 2026 support"}
tool result: { "error": { "code": "TIMEOUT", "message": "Tool 'send_invoice' timed out after 2.0s", "retryable": true, "suggestion": "The tool did not finish withi
tool call: invoice_status {"customer": "acme", "period": "2026-10"}
tool result: {"id": "inv_80152f99", "amount": 120.0, "status": "sent"}
agent: Invoice for acme, period 2026-10, amount €120 has been sent.
ledger:
  inv_80152f99 acme 120.00 EUR 'October 2026 support' (sent)

The call timed out again, because the billing API is just as slow. This time the model followed its instruction, checked invoice_status, found the invoice already sent and told the user so. The ledger has one invoice. Had the model called send_invoice again instead, the key would have answered "in progress" or replayed the first invoice: the instruction makes the agent faster and clearer, and the key makes it correct.

Knowing which layers retry on their own tells you where keys are required. In Foundry 1.6.0:

The MCP spec is explicit about that last row: all tool annotations are hints, and clients should not make decisions based on annotations from untrusted servers. Declaring idempotentHint is useful documentation. The key is what makes it true.

• Keep keys at least as long as retries can happen. Model retries take seconds, webhook redeliveries hours, dead-letter redelivery days. If your provider prunes keys after 24 hours, keep your own record of what was sent for longer.

• Expire stuck reservations carefully. A process that crashes after reserving a key leaves it processing forever, and every retry gets "in progress". Give reservations a lease: after a few minutes, check the provider for the real outcome, then mark the key done or release it. Never release it blindly, or the crash becomes a duplicate.

• Write the key and the effect in one transaction when the effect is in your own database. Two separate writes reopen the gap between them.

• Handle multi-step tools step by step. A tool that creates an invoice and then emails it needs a key per step (invoice:acme:2026-10, invoice-email:acme:2026-10), so a retry skips the finished step and does the missing one.

• Count replays. A rising rate of replayed or in-progress answers is an early sign of timeouts set too short or a model that retries too eagerly. Log them as metrics.

• Set timeouts above the provider's normal latency. A key makes a too-short timeout harmless, but it still wastes calls and confuses the model.

• Gate what keys can't fix. A key stops a repeat; it doesn't stop a wrong first action. Put payments and deletions behind human approval and an irreversible-action budget.

• Idempotency stops repeats, not mistakes. A wrong first invoice is still wrong.

• Some actions can't be checked. A phone call or an SMS through a provider without deduplication has no "did it happen?" query. Keep your own record before sending, and accept that a crash in between leaves you unsure.

• Keys depend on stable arguments. If the model picks a different customer ID or period on a retry, the key changes. Validate identifiers against your data before acting.

• Foundry doesn't generate keys for you. Keys are tool code. Foundry's part is not retrying unsafely on its own and keeping event IDs stable.

What is an idempotent tool call?

An idempotent tool call is one you can repeat without changing the result: the second call with the same intent has no extra effect and returns what the first returned. Reads are idempotent by nature. Tools that create, send or pay become idempotent through an idempotency key that the system performing the action checks.

Why does my AI agent run the same action twice?

Usually because a call timed out or returned an unclear error, and the model, the runtime or a client retried while the first call was still running or had already succeeded. Agent stacks retry at several layers, and the model is one of them. Without an idempotency key, each retry is a new action.

How do I generate an idempotency key for an agent tool?

Derive it from the business action the tool performs, such as invoice:<customer>:<period> or refund:<order_id>, and normalize the parts. If there is no natural identity, create a request ID once before the first attempt and store it with the work. Don't use timestamps, per-attempt random values or the model's tool call ID, which change on every retry.

Is it safe to retry a tool call after a timeout?

Only if the tool is idempotent. A timeout means the caller stopped waiting, not that the work failed; the request may still be running or may have finished. Retry with the same idempotency key, or first check the outcome with a read-only status tool.

Do durable execution frameworks make tool calls idempotent?

No. They record finished steps so they don't run again after a crash, but a step that was running when the crash happened is retried, and its side effect may already have happened. They still need idempotent steps. The key in the call to the outside system is what makes the repeat harmless.

Does the MCP idempotentHint annotation make a tool idempotent?

No. It is a hint a server sends with the tool's description, telling clients that repeated calls with the same arguments have no additional effect. The MCP spec says all annotations are hints and not guaranteed. The tool's own key check is what makes the hint true.

• Durable AI agents covers crash recovery with a journal, the other half of exactly-once.

• AI agent retries and a dead letter queue explains run retries and redelivery in the runtime.

• MCP long-running tasks shows the start-then-poll pattern for slow work.

• Build a personal AI agent uses idempotent reply and reminder tools end to end.

• What is tool calling? explains how the model requests tool calls in the first place.