How to Deploy an AI Agent with Docker (2026 Guide)

Run the agent as a long-lived HTTP service in a slim Python image, with every setting and secret coming from environment variables. Give it a liveness endpoint and a readiness endpoint, start it in exec form so it receives SIGTERM, and let it drain running requests before it exits. Put each MCP tool server in its own container on a private network, and start the agent only once those servers are healthy. With Promptise Foundry, promptise run --prod --serve provides the HTTP API, the probes, JSON logs and the drain.

• Build once, configure at run time. The same image runs in staging and production. Only the environment changes.

• Secrets never enter the image. Exclude .env with .dockerignore, and inject keys from your orchestrator.

• Liveness and readiness are different questions. /healthz means the process is up. /readyz means the agent is built and its tools are connected.

• Shutdown is part of the deploy. A rolling update sends SIGTERM. An agent request can take many seconds, so the grace period must be longer than your slowest request.

• Agents scale out like any stateless web service, as long as conversation state lives in a shared store.

An agent service has the same shape as any API service. The difference is what happens inside a request: several model calls and tool calls that can take from a few seconds to a minute.

                 docker compose network (private)
client ──HTTPS──▶ ┌───────────────────────────────┐        ┌─────────────────────────┐
(app, gateway)    │ agent container               │  MCP   │ shop-mcp container      │
                  │ promptise run --prod --serve  │──HTTP─▶│ python server.py http   │──▶ orders DB
 probes ────────▶ │  GET  /healthz   liveness     │        │  POST /mcp              │
                  │  GET  /readyz    readiness    │        │  GET  /health           │
                  │  POST /v1/invoke, /v1/stream  │        └─────────────────────────┘
                  └──────────────┬────────────────┘
                                 │ HTTPS
                                 ▼
                         model provider API
   env: OPENAI_API_KEY, PROMPTISE_SERVE_TOKEN, SHOP_MCP_URL     (injected, never baked in)

Four things change when an agent moves from a script to a container:

• Configuration comes from the environment. A .env file on your laptop must not end up in the image.

• The process is managed. Docker or Kubernetes starts it, probes it, restarts it when it dies and stops it with SIGTERM.

• Startup has dependencies. The agent connects to its MCP servers when it is built. If a server isn't up yet, the agent fails to start.

• More than one copy may run. Anything kept in process memory, such as a conversation, is lost on restart and not shared between replicas.

Use a container when the agent is a service other systems call: a support bot behind your web app, an internal API, a worker that runs on a schedule. It gives you the same artifact in every environment, and it is the unit Kubernetes, ECS, Cloud Run and Fly.io deploy.

You don't need a separate container when the agent is a library inside an existing web app. Then you deploy the app as usual and call build_agent() inside it. A one-off batch job can also run with plain python main.py in CI. For agents that run on their own for days with triggers and budgets, see deploying long-running AI agents.

• Docker 24 or later with Docker Compose v2. The commands here ran on Docker 27.4.

• Python 3.10 or later and Promptise Foundry 1.4.1 for local runs: pip install "promptise==1.4.1".

• An OpenAI API key in OPENAI_API_KEY. The agent uses gpt-5-mini.

• About 30 minutes, most of it the first image build.

The example is a shop support agent with two tools, get_order and check_stock, served by an MCP server. The layout:

support-bot/
├── compose.yaml
├── agent/          # the agent service
│   ├── Dockerfile  .dockerignore  requirements.txt
│   ├── promptise.toml
│   └── agent.py
└── shop-mcp/       # the MCP tool server
    ├── Dockerfile  .dockerignore
    └── server.py

Build the agent in a function, and read every deployment setting from the environment. The URL of the MCP server is configuration, not code:

"""The support agent. It reaches the shop tools over MCP at SHOP_MCP_URL."""
import os

from promptise import PromptiseAgent, build_agent
from promptise.config import HTTPServerSpec


async def create_agent(ctx=None) -> PromptiseAgent:
    return await build_agent(
        model=os.environ.get("AGENT_MODEL", "openai:gpt-5-mini"),
        instructions="You answer shop customers. Use the tools for facts. Keep answers short.",
        servers={"shop": HTTPServerSpec(url=os.environ["SHOP_MCP_URL"])},
        max_invocation_time=60,      # seconds; a stuck request ends instead of hanging
    )

max_invocation_time=60 ends a request that runs longer than a minute, so one stuck request can't hold a worker forever. The manifest tells promptise run which factory to serve, which variables must be set, and how to behave in production:

name = "support-bot"
version = "1.0.0"
agent = "agent.py:create_agent"

[env]
required = ["OPENAI_API_KEY", "SHOP_MCP_URL", "PROMPTISE_SERVE_TOKEN"]

[prod]
log_format = "json"
observe = "basic"          # one JSON log line per run, with its token totals
shutdown_timeout = 30      # seconds running requests get after SIGTERM

[serve]
port = 8080
max_concurrency = 8        # requests beyond this get 429

[pricing]
"gpt-5-mini*" = { input = 0.25, output = 2.00 }

• `[env] required` lists the variables the container needs. If one is missing, the process stops with exit code 2 before it builds anything.

• `[prod]` turns on JSON logs on stderr, records one log line per agent run with its token totals (observe = "basic"), and gives running requests 30 seconds after SIGTERM.

• `[serve]` sets the port and how many requests run at the same time. Requests beyond max_concurrency get 429 with Retry-After instead of piling up.

• `[pricing]` adds a cost estimate to the final stats log line.

The manifest holds no secrets, so it is safe to commit and to copy into the image.

# syntax=docker/dockerfile:1
FROM python:3.12-slim

ENV PYTHONUNBUFFERED=1 \
    PIP_NO_CACHE_DIR=1 \
    PIP_DISABLE_PIP_VERSION_CHECK=1 \
    # Configuration comes from the container's environment, never from a .env file.
    PROMPTISE_NO_DOTENV=1

# An unprivileged user to run the agent.
RUN useradd --create-home --uid 10001 app
WORKDIR /app

# Dependencies first, so a code change does not reinstall them.
COPY requirements.txt .
RUN pip install -r requirements.txt

# promptise run only trusts a manifest owned by the user that runs it.
COPY --chown=app:app promptise.toml agent.py ./

USER app
EXPOSE 8080
HEALTHCHECK --interval=10s --timeout=3s --start-period=20s --retries=3 \
  CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8080/readyz', timeout=2)"]

# Exec form: promptise is PID 1 and receives the SIGTERM that docker stop sends.
CMD ["promptise", "run", "--prod", "--serve", "--host", "0.0.0.0"]

What each part does:

• `python:3.12-slim` is a small Debian image with Python. The built agent image is about 516 MB, most of it Python packages.

• `PROMPTISE_NO_DOTENV=1` turns off .env loading, so a stray file can never override the container's environment.

• Dependencies before code. requirements.txt pins promptise==1.4.1, and Docker caches that layer, so changing agent.py rebuilds in seconds.

• `COPY --chown=app:app` matters. promptise run refuses a manifest that is not owned by the user running it, because the manifest decides what runs. Without --chown, the files belong to root and the container exits with refusing to use this manifest: owned by uid 0, not by you (uid 10001).

• `USER app` runs the agent without root, which limits what a compromised process can do: it can't install packages or change the Python installation.

• `HEALTHCHECK` uses Python's standard library, because the slim image has no curl. It checks /readyz, so Docker reports the container healthy only once the agent is built.

• Exec form `CMD [...]` makes promptise process 1. With the shell form (CMD promptise run ...), /bin/sh can end up as process 1, and it does not pass the SIGTERM from docker stop on to the agent.

The .dockerignore keeps secrets and local state out of the build context:

.env
.env.*
.venv/
__pycache__/
.promptise/
reports/
*.json
.git/

Every setting the container needs arrives through the environment, and the process checks them before it starts. Run the image without them to see what happens:

docker run --rm -e OPENAI_API_KEY support-bot-agent; echo "exit code: $?"

{"ts": "2026-10-18T10:24:21.054+00:00", "level": "error", "logger": "promptise.run", "msg": "preflight failed", "problems": ["missing required environment variable(s): SHOP_MCP_URL, PROMPTISE_SERVE_TOKEN (set them, or add them to .env)"]}
exit code: 2

One JSON log line lists every missing variable at once, and the exit code is 2, so an orchestrator reports a configuration error instead of a crash loop with a stack trace. The serve token is required because the container listens on 0.0.0.0: promptise run refuses to serve on a network address without a bearer token.

In CI, check the image with the same command the container runs, plus --check:

docker run --rm -e OPENAI_API_KEY -e SHOP_MCP_URL -e PROMPTISE_SERVE_TOKEN \
  support-bot-agent promptise run --prod --serve --check

✓ support-bot: agent.py:create_agent (serve) is ready

--check validates the manifest, the environment and the factory's module, then exits with 0 or 2 without serving anything.

Where secrets come from

Pass secrets as environment variables from the platform's secret store: Kubernetes Secrets, AWS Secrets Manager through ECS task definitions, Docker Swarm secrets, or your CI's protected variables. Never use ENV OPENAI_API_KEY=... or ARG for secrets in a Dockerfile, because both end up in the image metadata. Generate the serve token with openssl rand -hex 32 and rotate it like any API key.

The tool server is a separate service with its own image, so you can deploy, scale and restrict it on its own. It needs one change from the local version: it must listen on all interfaces inside its container, and it should only answer the host names it is reached by.

"""A minimal MCP server for a small shop: two tools and one resource.

Run it over stdio (the default) or over Streamable HTTP:
    python server.py            # stdio: the client starts this process
    python server.py http       # HTTP on port 8000, path /mcp
"""
import os
import sys

from promptise.mcp.server import MCPServer

server = MCPServer(name="shop", version="1.0.0")

ORDERS = {
    "A1001": {"status": "delivered", "carrier": "UPS", "delivered_on": "2026-10-12"},
    "A1002": {"status": "in_transit", "carrier": "DHL", "eta": "2026-10-21"},
}
STOCK = {"TENT-2P": 4, "STOVE-MINI": 0}


@server.tool(read_only_hint=True)
async def get_order(order_id: str) -> dict:
    """Look up an order by its id (for example A1002): status, carrier and dates."""
    order = ORDERS.get(order_id.strip().upper())
    if order is None:
        return {"error": f"No order {order_id!r}. Order ids look like A1001."}
    return {"order_id": order_id.upper(), **order}


@server.tool(read_only_hint=True)
async def check_stock(sku: str) -> dict:
    """How many units of a product (by SKU, for example TENT-2P) are in stock."""
    units = STOCK.get(sku.strip().upper())
    if units is None:
        return {"error": f"Unknown SKU {sku!r}."}
    return {"sku": sku.upper(), "units": units}


@server.resource("shop://policies/returns", mime_type="text/markdown")
async def returns_policy() -> str:
    """The shop's return policy."""
    return "Returns are free within 30 days of delivery. Items must be unused."


if __name__ == "__main__":
    transport = sys.argv[1] if len(sys.argv) > 1 else "stdio"
    server.run(
        transport=transport,
        host=os.environ.get("MCP_HOST", "127.0.0.1"),
        port=8000,
        # Host headers this server answers: its compose service name and localhost.
        allowed_hosts=["shop-mcp:*", "localhost:*", "127.0.0.1:*"],
    )

FROM python:3.12-slim
ENV PYTHONUNBUFFERED=1 PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1
WORKDIR /app
RUN pip install "promptise==1.4.1"
COPY server.py .
RUN useradd --create-home --uid 10001 app
USER app
ENV MCP_HOST=0.0.0.0
EXPOSE 8000
HEALTHCHECK --interval=10s --timeout=3s --start-period=10s --retries=3 \
  CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health', timeout=2)"]
CMD ["python", "server.py", "http"]

MCP_HOST=0.0.0.0 is set in the image, so the same server.py still binds 127.0.0.1 when you run it on your laptop. The Promptise MCP server answers GET /health for probes. allowed_hosts turns on Host header validation for a public bind, which protects against DNS rebinding. This server has no authentication because it is only reachable on the compose network. If anything else can reach it, add API keys or OAuth as shown in MCP server authentication.

services:
  shop-mcp:
    build: ./shop-mcp
    # No ports: the MCP server is reachable only on the compose network.
    restart: unless-stopped

  agent:
    build: ./agent
    ports:
      - "8080:8080"
    environment:
      OPENAI_API_KEY: ${OPENAI_API_KEY:?set OPENAI_API_KEY}
      PROMPTISE_SERVE_TOKEN: ${PROMPTISE_SERVE_TOKEN:?set PROMPTISE_SERVE_TOKEN}
      SHOP_MCP_URL: http://shop-mcp:8000/mcp
    depends_on:
      shop-mcp:
        condition: service_healthy   # the agent connects to MCP when it starts
    stop_grace_period: 40s           # longer than [prod] shutdown_timeout
    restart: unless-stopped

• No `ports` on `shop-mcp`. Only containers on the compose network can reach the tools.

• `${OPENAI_API_KEY:?...}` takes the value from your shell and stops compose with a clear message when it is missing.

• `depends_on` with `service_healthy` waits for the MCP server's health check, because the agent connects to it while it starts. Without it, the agent can start first and fail to connect.

• `stop_grace_period: 40s` gives the agent more time than its 30-second drain before Docker sends SIGKILL. Docker's default is 10 seconds, which is shorter than many agent requests.

export OPENAI_API_KEY=sk-...
export PROMPTISE_SERVE_TOKEN="$(openssl rand -hex 32)"
docker compose up -d --build
docker compose ps

SERVICE    STATUS                    PORTS
agent      Up 20 seconds (healthy)   0.0.0.0:8080->8080/tcp
shop-mcp   Up 26 seconds (healthy)   8000/tcp

curl -s localhost:8080/healthz
curl -s localhost:8080/readyz
curl -s -o /dev/null -w "%{http_code}\n" localhost:8080/v1/info

$ curl -s localhost:8080/healthz
{"status":"ok"}
$ curl -s localhost:8080/readyz
{"status":"ready"}
$ curl -s -o /dev/null -w "%{http_code}\n" localhost:8080/v1/info
401

The probes need no token, so load balancers and Kubernetes can call them. Everything under /v1/ needs the bearer token, so /v1/info without it returns 401. Now ask the agent a question:

curl -s localhost:8080/v1/invoke \
  -H "Authorization: Bearer $PROMPTISE_SERVE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"input": "Where is my order A1002?"}'

{
    "output": "Your order A1002 is in transit with DHL. Estimated delivery: 2026-10-21.\n\nWould you like me to fetch the tracking link/number or do anything else with the shipment?",
    "duration_ms": 4081.4,
    "usage": {
        "llm_calls": 2,
        "prompt_tokens": 468,
        "completion_tokens": 332,
        "total_tokens": 800
    },
    "request_id": "c4038d5aa611418499336fb592e1baa9"
}

The agent called get_order on the MCP container and answered. usage counts the model calls and tokens of this request, which is what per-user cost tracking needs (see serve mode for every field). request_id also appears in the X-Request-ID response header and in every log line of the request, so you can find a bad answer in the logs.

Start a request, and stop the container while it runs:

curl -s localhost:8080/v1/invoke -H "Authorization: Bearer $PROMPTISE_SERVE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"input": "Is TENT-2P in stock, and what about order A1001?"}' &
sleep 1.5; docker compose stop agent; wait

{"output":"TENT-2P — 4 units in stock.\nOrder A1001 — delivered via UPS on 2026-10-12.\n\nAnything else I can help with?","duration_ms":5166.1,"usage":{"llm_calls":2,"prompt_tokens":540,"completion_tokens":297,"total_tokens":837},"request_id":"0501a40d6760458d8b28878d3545a108"}exit=0

The request that was running when SIGTERM arrived still got its full answer. The JSON logs show the order of events:

docker compose logs --no-log-prefix agent | jq -c 'select(.msg == "Received SIGTERM: draining"
  or (.msg == "http" and .path == "/v1/invoke") or .msg == "run.exit" or .msg == "stats")
  | {msg, path, status, duration_ms, code, cost_usd: .totals.cost_usd, total_tokens: .totals.total_tokens}
  | del(..|nulls)'

{"msg":"http","path":"/v1/invoke","status":200,"duration_ms":4082.0}
{"msg":"Received SIGTERM: draining"}
{"msg":"http","path":"/v1/invoke","status":200,"duration_ms":5167.0}
{"msg":"run.exit","code":0}
{"msg":"stats","cost_usd":0.00151,"total_tokens":1637}

On SIGTERM the server starts draining: /readyz and new /v1/* requests answer 503, so the load balancer stops sending traffic, and running requests get up to [prod] shutdown_timeout seconds. Then the agent closes its MCP sessions and exits with code 0. The last line carries the process's totals, including the estimated cost from [pricing].

All files are shown above: agent/agent.py, agent/promptise.toml, agent/Dockerfile, agent/.dockerignore, agent/requirements.txt (one line, promptise==1.4.1), shop-mcp/server.py (the two-tool shop server, with the run() call from Step 4), shop-mcp/Dockerfile and compose.yaml. To run the stack:

cd support-bot
export OPENAI_API_KEY=sk-...
export PROMPTISE_SERVE_TOKEN="$(openssl rand -hex 32)"
docker compose up -d --build
curl -s localhost:8080/v1/invoke -H "Authorization: Bearer $PROMPTISE_SERVE_TOKEN" \
  -H "Content-Type: application/json" -d '{"input": "Where is my order A1002?"}'
docker compose down

Probes and shutdown on Kubernetes

The same image runs on Kubernetes. Map the endpoints to probes and keep the grace period longer than the drain:

spec:
  terminationGracePeriodSeconds: 40
  containers:
    - name: agent
      image: registry.example.com/support-bot-agent:1.0.0
      ports: [{containerPort: 8080}]
      env:
        - name: SHOP_MCP_URL
          value: http://shop-mcp:8000/mcp
        - name: OPENAI_API_KEY
          valueFrom: {secretKeyRef: {name: support-bot, key: openai-api-key}}
        - name: PROMPTISE_SERVE_TOKEN
          valueFrom: {secretKeyRef: {name: support-bot, key: serve-token}}
      livenessProbe:
        httpGet: {path: /healthz, port: 8080}
        periodSeconds: 10
      readinessProbe:
        httpGet: {path: /readyz, port: 8080}
        periodSeconds: 5

Don't point the liveness probe at /readyz: a pod that is draining would be killed instead of finishing its requests.

Scaling

One promptise run --serve process serves one agent and runs up to max_concurrency requests at the same time. Most of a request's time is spent waiting on the model API, so a process handles several requests at once with little CPU. Scale out by adding replicas behind a load balancer, and scale on concurrent requests or latency rather than CPU. Two rules keep replicas interchangeable. Keep conversations in a shared store (RedisConversationStore or PostgresConversationStore with a session_id per conversation), not in memory. And remember that your model provider's rate limit is shared by every replica, so more replicas can mean more 429s from the provider.

MCP servers over Streamable HTTP keep a session per client connection. When an MCP container restarts, the server answers old sessions with 404, and Promptise clients open a new session and retry the call once. For MCP servers that run several replicas, use sticky sessions at the load balancer, or check that your server handles requests without per-session state.

Logs, metrics and traces

--prod writes one JSON object per line on stderr, which Docker's logging drivers, Loki, Datadog and CloudWatch ingest as is. Raise [prod] observe to standard to log every model call and tool call, or use the OpenTelemetry transporter to send each request as a trace. AI agent observability covers what to record and what to keep out of logs.

Image hygiene

Pin the base image and your dependencies, and rebuild regularly for security patches. Scan images in CI (docker scout, Trivy or Grype). Run as a non-root user, as here, and set the root filesystem read-only where your platform allows it. Agents that execute generated code need a separate sandbox, not just a container: see AI agent code execution sandbox.

Put a gateway in front

The serve mode has one bearer token for all callers. It is meant for one application calling the agent, not for end users. Put TLS, user authentication and per-user rate limits in your API gateway or web app, and pass the user's identity on to the agent. To give each user their own identity and permissions inside the agent, embed build_agent() in your own web server instead.

promptise run --serve runs one agent in one process with one bearer token. It has no per-user accounts, TLS or routing, which stay with your gateway and platform. Conversations are only kept across requests when the agent has a conversation store. The image in this guide is a plain single-stage build. A multi-stage build or a distroless base can make it smaller, at the cost of a less convenient debugging shell. Kubernetes manifests are shown as a starting point, not as a tested chart.

How do I run an AI agent in a Docker container?

Install your agent framework in a slim Python image, copy the agent code, and start it as an HTTP service in exec form. With Promptise Foundry, the command is promptise run --prod --serve --host 0.0.0.0, which serves the agent factory named in promptise.toml with health checks, an invoke endpoint and JSON logs.

How should I pass the API key to a Dockerized agent?

As an environment variable set when the container starts, from your platform's secret store. Never write it into the Dockerfile with ENV or ARG, and keep .env files out of the image with .dockerignore. Promptise reads provider keys such as OPENAI_API_KEY from the environment.

What health checks does an AI agent container need?

Two. A liveness check that only confirms the process responds (/healthz), and a readiness check that confirms the agent is built and its tool servers are connected (/readyz). Route traffic on readiness and restart on liveness, so a draining or starting container is not killed.

How do I stop an agent container without losing requests?

Make the agent process 1 with an exec-form CMD so it receives SIGTERM, let it stop accepting new requests and finish running ones, and give it a grace period longer than your slowest request. promptise run --prod --serve drains for [prod] shutdown_timeout seconds and exits with code 0.

Should the MCP server run in the same container as the agent?

Usually not. A separate container lets you deploy, restrict and scale the tools on their own and share them between agents. A stdio MCP server, which the agent starts as a child process, can live in the agent's image when only that agent uses it.

How do I scale an AI agent with Docker?

Run more replicas behind a load balancer. Each replica is stateless if conversations live in a shared store such as Redis or Postgres. Scale on concurrent requests or latency, since agents mostly wait on the model API, and watch the provider's rate limit, which all replicas share.

• Run, debug and serve the same project locally: Run, debug and deploy an AI agent.

• Agents that run on their own with triggers and budgets: Deploy AI agents in production.

• Harden the tool server: Build a production MCP server in Python.

• Survive crashes mid-run: Durable AI agents and crash recovery.

• Reference: promptise run serve mode and Docker and MCP server deployment.