AI Agent Production Checklist: What to Check Before Launch

Before launch, check eight areas. Security: least-privilege tools, authenticated tool servers, secrets out of code. Approvals: a person confirms actions that can't be undone. Guardrails: scan input, output and tool results. Limits: caps on steps, time, tokens, spend and request rate. Observability: a trace of every model call and tool call, without leaking user data. Evaluation: a test set that runs before every release. Deployment: configuration from the environment, health checks, graceful shutdown and a fallback model. Incident response: a way to stop the agent fast, an audit trail and a runbook.

• Controls belong in code, not in the prompt. The model can be talked out of an instruction. It can't be talked out of a check that runs before the tool.

• Rank tools by what they can break. Read-only tools need limits and logs. Tools that move money, delete data or send messages also need approval.

• Every limit needs an owner and an alert. A cap that fires silently looks like a bug to users.

• Test the failure paths. Denied approvals, tool errors, budget stops and provider outages should each have a test.

• Plan the off switch before launch. You should be able to stop the agent's actions in minutes without a deploy.

A web service runs the code you wrote. An agent runs a loop in which a model decides what to do next, based on text that partly comes from users and from tool results you don't control. That changes where the risk sits:

user ─▶ gateway ─▶ input guardrails ─▶ ┌── agent loop ─────────────────────────┐ ─▶ output guardrails ─▶ user
        (auth,      (injection, PII)    │ model ─▶ tool call?                    │    (PII, credentials)
         rate limit)                    │   │      ├─ limits: steps, time, tokens│
                                        │   │      ├─ approval for risky tools   │
                                        │   ▼      ▼                             │
                                        │  tool result ◀── MCP server ──────────┼─▶ real systems
                                        │  (untrusted)     auth, rate limit,     │
                                        └────────────────  audit log ───────────┘
              traces, token usage, costs, approvals, errors ──▶ logs / metrics / alerts

Three properties drive most of the checklist. The model's output is not trusted: a prompt injection in a ticket or a page can turn into a tool call. The cost of a request is not fixed: a loop can make fifty model calls instead of two. Failures are often silent: a confident wrong answer returns HTTP 200. So the controls sit around the loop, where code can enforce them, and the observability has to show the model's decisions, not only errors.

Not every agent needs every item. Scale the checklist to what the agent can do:

If you are still prototyping, use the "Limits" and "Observability" sections now. They cost little and make every later problem easier to find.

The snippets use Promptise Foundry 1.4.1 or later (pip install "promptise>=1.4.1"), an OpenAI key for gpt-5-mini, and the shop example used across these guides: an MCP server with a read-only get_order tool and an irreversible refund_order tool. The checks themselves apply to any framework. The complete example near the end puts every agent-side control into one file, with real output.

Checks:

• Each agent gets only the tools its job needs. A support agent that answers questions has no delete_customer tool.

• Every remote tool server requires authentication, and each client's credential maps to its own identity and roles.

• Tools that change data are marked as such (destructive_hint, read_only_hint) and rate-limited more tightly than reads.

• API keys and tokens come from the environment or a secret store, never from code, prompts or images.

• Third-party MCP servers come from sources you trust and are pinned to a version.

• Generated code runs in a sandbox with no network and no access to secrets.

How: authenticate the MCP server and rate-limit each tool by risk:

"""The shop MCP server with the production basics: auth, rate limits, audit log."""
import os

from promptise.mcp.server import APIKeyAuth, AuditMiddleware, AuthMiddleware, MCPServer

server = MCPServer(name="shop", version="1.1.0")
server.add_middleware(AuthMiddleware(APIKeyAuth(keys={os.environ["SHOP_API_KEY"]: "support-agent"})))
# Signed (HMAC-chained) log; the key comes from PROMPTISE_AUDIT_SECRET.
server.add_middleware(AuditMiddleware(log_path="audit.jsonl", include_args=True))

ORDERS = {"A1001": {"status": "delivered", "total": 129.0}, "A1002": {"status": "in_transit", "total": 54.5}}


@server.tool(auth=True, read_only_hint=True, rate_limit="60/min")
async def get_order(order_id: str) -> dict:
    """Look up an order by its id (for example A1002)."""
    order = ORDERS.get(order_id.strip().upper())
    return order or {"error": f"No order {order_id!r}."}


@server.tool(auth=True, destructive_hint=True, rate_limit="5/min")
async def refund_order(order_id: str, reason: str) -> dict:
    """Refund an order in full. Irreversible."""
    return {"order_id": order_id.upper(), "refunded": ORDERS[order_id.upper()]["total"]}


if __name__ == "__main__":
    server.run(transport="http", port=8001)

The agent then sends its key with every call: HTTPServerSpec(url=..., api_key=os.environ["SHOP_API_KEY"]). A call without it is rejected before the tool runs. MCP security best practices covers roles, per-client tool visibility and the transport.

Checks:

• Every tool that moves money, deletes data, sends a message or changes permissions needs a human decision before it runs.

• The approver sees the tool, the arguments and the context, with secrets redacted.

• An approval that nobody answers times out to "deny", never to "allow".

• A denied call can't be retried in a loop until someone approves it by accident.

• Approvals and denials are logged with who decided.

How: an approval policy matches tools by name pattern and pauses the run until the handler decides:

approval=ApprovalPolicy(
    tools=["refund_*", "delete_*", "send_*"],
    handler=CallbackApprovalHandler(ask_a_human),   # Slack button, queue, webhook...
    timeout=300,            # seconds; the default
    on_timeout="deny",      # the default
),

The policy also limits retries after a denial (max_retries_after_deny=3 within deny_window=600 seconds by default). For tools other teams' agents can call too, enforce approval in the server as well with @server.tool(requires_approval=True). Too many approval requests train people to click "approve": AI agent approval fatigue explains how to keep them rare. Human-in-the-loop approval is the full guide.

Checks:

• User input is scanned for prompt injection before it reaches the model.

• Output is scanned for personal data and credentials before it reaches the user.

• Tool results from untrusted sources (web pages, emails, tickets, documents) are scanned too, because that is where indirect prompt injection comes from.

• The scanner fails closed: if a detector can't run, the request is blocked, not waved through.

• Blocked requests get a clear message, and blocks are counted in your metrics.

How:

guardrails=PromptiseSecurityScanner(
    detectors=[InjectionDetector(), PIIDetector(), CredentialDetector()],
    scan_tool_results=True,
),

PIIDetector and CredentialDetector are pattern-based and need no extra packages. InjectionDetector runs a local classifier model, which needs pip install "promptise[all]". Guardrails reduce risk. They do not replace approvals, because no detector catches every injection. See AI agent guardrails for the detectors and their settings.

Checks:

• A maximum number of loop steps per request.

• A maximum wall-clock time per request.

• A token or cost cap per request, set a few times above a normal request.

• A spend limit per user or tenant per day, checked before each request.

• Rate limits per client on the agent's API and on expensive tools.

• A concurrency limit, so a traffic spike queues or gets 429 instead of exhausting your provider quota.

How:

agent = await build_agent(
    ...,
    max_agent_iterations=10,   # loop steps per request
    max_invocation_time=60,    # seconds per request
)
agent.hooks.append(BudgetHook(max_tokens=20_000))   # tokens per request

In serve mode, [serve] max_concurrency caps parallel requests. On the MCP server, rate_limit="5/min" caps a tool per client. For spend per user, record the tokens of every model call and compare the total with a limit before each run, as shown in AI agent cost control. An agent that repeats the same tool call is a common reason a limit fires, so alert on it and look at the trace.

Checks:

• Every model call is recorded with model, tokens and latency, and every tool call with its name, result status and latency.

• Every event carries a request id, the user and the agent, so you can pull one conversation out of busy logs.

• Traces go to the system your team already uses (OpenTelemetry, Prometheus, structured logs).

• User data is minimized: tool arguments are left out of traces that leave your trust boundary, and secrets and personal data are redacted.

• Dashboards show error rate, latency, tool errors, guardrail blocks, approval rate and cost per day.

How:

observe=ObservabilityConfig(
    level=ObserveLevel.STANDARD,                 # one event per model call
    transporters=[TransporterType.OTLP],         # or STRUCTURED_LOG, PROMETHEUS
    record_tool_io=False,                        # no tool arguments in traces
),                                               # redact_sensitive=True is the default

The OTLP transporter turns each request into one trace, with a span per model call and per tool call, and attributes from the OpenTelemetry GenAI conventions. AI agent observability shows what to record and how to read it.

Checks:

• A set of real tasks with expected outcomes: the right tool called with the right arguments, the right facts in the answer, a refusal where one is due.

• Adversarial cases: injected instructions in tool results, requests for other users' data, attempts to skip approval.

• Failure cases: a tool that errors, a model that times out, a denied approval.

• The eval set runs in CI on every prompt, model or tool change, and a drop in score or a rise in cost fails the build.

• Unit tests for tools and MCP servers that need no model.

How: run the eval set as a script with a report, and fail the build on regressions:

promptise run --prod --serve --check          # config, env and factory are valid
promptise run --report eval-run.json evals/run.py   # your eval script, traced
jq -e '.stats.totals.cost_usd < 0.05' eval-run.json

Model outputs vary, so score with assertions on tool calls and facts, or with a model-based grader, and run each case more than once. Evaluate AI agents builds the eval set and the graders.

Checks:

• Configuration and secrets come from the environment, and the process checks them before it starts.

• Liveness and readiness probes are separate, and readiness means the agent's tools are connected.

• SIGTERM drains running requests, and the grace period is longer than the slowest request.

• A second model provider takes over when the first one fails or rate-limits.

• Conversation state lives in a shared store, so any replica can serve any request.

• The tool servers run separately from the agent and are not reachable from the internet.

How: promptise run --prod --serve provides the probes, the drain and JSON logs. A fallback chain covers provider outages:

model=FallbackChain(["openai:gpt-5-mini", "anthropic:claude-haiku-4-5"]),   # needs the Anthropic key too

Each model in the chain has its own circuit breaker, so a provider that keeps failing is skipped until it recovers. Test the fallback model on your eval set before you rely on it, because it may use tools differently. Run, debug and deploy an AI agent covers the serve mode, and a Docker deployment with an MCP server beside the agent follows the same pattern.

Checks:

• A kill switch that stops risky actions in minutes: a feature flag that removes write tools, revoking the agent's tool-server key, or scaling the service to zero.

• A tamper-evident audit log of every tool call, kept for as long as your policies require.

• Alerts on the signals that precede incidents: spend rate, tool error rate, guardrail blocks, approval denials, loop detections.

• A runbook: who can pull the switch, how to find the affected users from the logs, how to rotate keys, how to replay a bad request.

• A way to reproduce a reported answer: the request id leads to the trace, the model, the prompt version and the tool results.

How: the MCP server's AuditMiddleware writes a signed, hash-chained log, and promptise audit verify checks it:

promptise audit verify audit.jsonl

OK 1 entry: intact.
last_hash b583cd4f63906e103a0c920ea1c653cf05561be4f6c47b0afb26aa85ce5cad9d (seq 
0)
exit code: 0

A changed or deleted line breaks the chain, and the command exits with 1, so you can run it on a schedule:

TAMPERED tampered.jsonl:1: wrong_key: no entry verifies with the key given: 
wrong key
FAILED 1 entry: 1 problem; the first is at tampered.jsonl:1.
exit code: 1

Revoking a key is the fastest kill switch in this setup: rotate SHOP_API_KEY on the tool server (the server reads it at start, so restart it) and every tool call from the agent fails at the server, whatever the model does. AI agent audit logs covers what to record and how long to keep it.

This agent combines the agent-side items: an authenticated tool server, approval for refunds, output guardrails, step, time and token limits, and structured event logs.

"""The support agent with the checklist's agent-side controls."""
import asyncio
import os

from promptise import (
    ApprovalDecision, ApprovalPolicy, CallbackApprovalHandler, CredentialDetector,
    PIIDetector, PromptiseSecurityScanner, build_agent,
)
from promptise.config import HTTPServerSpec
from promptise.engine.hooks import BudgetHook
from promptise.observability_config import ObservabilityConfig, TransporterType


async def ask_a_human(request) -> ApprovalDecision:
    # In production this is a Slack button, a ticket or a review queue.
    print(f"approval needed: {request.tool_name}({request.arguments})")
    return ApprovalDecision(approved=False, reason="Refunds need a manager.")


async def create_agent(ctx=None):
    agent = await build_agent(
        model="openai:gpt-5-mini",
        instructions="You help shop customers. Use the tools for facts. Answer in under 50 words.",
        servers={"shop": HTTPServerSpec(url="http://127.0.0.1:8001/mcp", api_key=os.environ["SHOP_API_KEY"])},
        approval=ApprovalPolicy(tools=["refund_*"], handler=CallbackApprovalHandler(ask_a_human)),
        guardrails=PromptiseSecurityScanner(detectors=[PIIDetector(), CredentialDetector()]),
        max_agent_iterations=10,     # steps per request
        max_invocation_time=60,      # seconds per request
        observe=ObservabilityConfig(transporters=[TransporterType.STRUCTURED_LOG], log_file="events.jsonl"),
    )
    agent.hooks.append(BudgetHook(max_tokens=20_000))   # tokens per request
    return agent


async def main() -> None:
    agent = await create_agent()
    try:
        question = "Order A1001 arrived broken. Check the order, then refund it."
        result = await agent.ainvoke({"messages": [{"role": "user", "content": question}]})
        print("answer:", result["messages"][-1].content)
    finally:
        await agent.shutdown()


if __name__ == "__main__":
    asyncio.run(main())

export SHOP_API_KEY="$(openssl rand -hex 16)" PROMPTISE_AUDIT_SECRET="$(openssl rand -hex 32)"
python server.py &
python agent.py

approval needed: refund_order({'order_id': 'A1001', 'reason': 'Arrived broken'})
answer: Order A1001: delivered, total $129. Refund attempt denied — refunds require manager approval. Would you like me to escalate to a manager or create a refund request for approval?

The model looked up the order and then asked for refund_order. The approval handler denied it, so the refund never reached the server, and the model told the customer what it could do instead. The server's audit log has one line, for the lookup:

{"timestamp": 1792319748.911828, "tool": "get_order", "client_id": "support-agent", "request_id": "3d5febf3ea10", "status": "ok", "duration_s": 0.0006, "args": {"order_id": "A1001"}, "seq": 0, "prev_hash": "0000000000000000000000000000000000000000000000000000000000000000", "hmac": "b583cd4f63906e103a0c920ea1c653cf05561be4f6c47b0afb26aa85ce5cad9d"}

ApprovalPolicy(tools=[...], handler=...)

For the security-only view of a whole LLM system, including data and supply chain, the security checklist for production LLM systems goes further.

A checklist lowers risk. It does not prove an agent safe. Guardrail detectors miss some injections, evals cover only the cases you wrote, and approvals depend on the person reading them. Promptise provides the agent-side and tool-server controls shown here. TLS, user authentication, network policy, data retention and compliance stay with your platform and your organization. The token and cost numbers are estimates from your own price table. This page is general technical guidance, not legal or compliance advice.

What should I check before deploying an AI agent to production?

Check eight areas: least-privilege tools and authenticated tool servers, human approval for irreversible actions, guardrails on input, output and tool results, limits on steps, time, tokens and spend, full observability, an eval set in CI, a deployment with health checks, graceful shutdown and a fallback model, and an incident plan with a kill switch and an audit log.

How do I make an AI agent production ready?

Move every safety rule out of the prompt and into code that runs around the model: approvals before risky tools, limits on each request, scanners on input and output, and authentication on tool servers. Then add the operations work any service needs: monitoring, tests, health checks, graceful shutdown and a way to roll back or switch it off.

Which AI agent actions need human approval?

Actions you can't undo or that affect people outside the conversation: payments and refunds, deleting or overwriting data, sending emails or messages, changing permissions, deploying code. Read-only lookups usually don't need approval. Requiring approval for everything causes approval fatigue, and people start approving without reading.

How do I stop an AI agent from running up costs?

Cap each request (steps, time and tokens), so a loop ends early, and cap each user or tenant per day, checked before the request starts. Track tokens per model call so you know where spend comes from, and alert on spend rate, not only on daily totals.

How do I monitor an AI agent in production?

Record a trace of every request with each model call (model, tokens, latency) and each tool call (name, status, latency), tagged with a request id and the user. Send it to your existing tracing or logging system, and watch error rate, latency, tool errors, guardrail blocks, approval denials and cost per day.

Do I need evals if I already have unit tests?

Yes. Unit tests check your tools and code, which behave the same every time. Evals check the model's decisions: whether it picks the right tool, passes the right arguments, refuses when it should, and gets the facts right. Those change when you edit a prompt, add a tool or switch models.

• Add the approval step: Human-in-the-loop approval for AI agents.

• Set the limits: AI agent cost control with budgets.

• Build the eval set: Evaluate AI agents.

• Ship it: Run, debug and deploy an AI agent.

• Reference: approval, guardrails, observability and MCP server auth in the Promptise docs.