Decision Models for AI Agents: Route, Gate and Classify

A decision model takes an input (the "state") and a set of typed questions, and returns one answer per question: a yes/no probability, one option from a list you supply, or a position on an ordered scale. It never writes free text. In an agent, it handles the many small judgments ("which queue?", "is this call safe?", "how urgent?") so the LLM only runs where reasoning or writing is needed.

answers = await decide("Order A-1002 arrived broken, I want my money back.", [ROUTE])
answers["route"].choice         # 'refund'
answers["route"].probabilities  # order_status=0.01, refund=0.97, other=0.02

Key takeaways:

• Three jobs: route, gate, classify. Pick a handler, approve or stop an action, or attach a label or score to a record.

• The output is a probability, so code decides. Set thresholds: act above one, ask a bigger model or a person below it.

• They are cheap and quick per call. OpenAI, TypeSafe and Cloudflare bill decision calls by input tokens only, because nothing is generated.

• They do not reason or explain. Anything that needs a plan, a written answer or a rationale still goes to an LLM.

• Probabilities are not guarantees. Check them against a labeled set before you let code act on them.

A generative LLM produces an answer one token at a time, and you parse the text afterwards. A decision model is trained or post-trained to score a bounded set of answers directly. You send three things:

1. State: the text or JSON to judge, such as a support message, a tool call's arguments or a retrieved passage.

2. Questions: each has a type, instructions and, for choices and scores, the allowed options or levels.

3. The model name.

You get back one typed answer per question, each with probabilities. Vendors use slightly different names for the same three question types:

Two details matter in practice. First, a score is the probability-weighted average of the level indexes, so on a three-level scale (0, 1, 2) you can get 1.06, between two levels. Second, OpenAI's Decisions API can return a refusal answer for any single question, so your code must handle a question that got no decision.

The category is new. TypeSafe launched Jev on September 15, 2026, and calls it a "System One" model. Cloudflare released Clef and Clef-flash as open weights on October 1 and says they are fully compatible with Jev's API. OpenAI opened the Decisions API as a public beta on October 6, and Microsoft documents Microsoft-Decision-1 in Foundry with a Jev-style request (a state plus typed questions). Strands Labs, the experimental arm of the Strands Agents project, publishes Strands Decider, a small open model you run yourself. The Jev vs Clef vs OpenAI Decisions API comparison covers prices, limits and differences.

Route, gate and classify

• Route. A choice question picks who handles a request: a script with no LLM call, the LLM agent, a specialist agent or a person. This is the same job as an LLM router that picks a small or large model, done with a typed answer instead of parsed text.

• Gate. A predicate question checks one property of an action before it runs: does this reply promise money, does this SQL write data, does this command touch production. A low probability lets the action run, and anything else goes to a person.

• Classify. A choice or score turns a ticket, document or tool output into a stored field, such as urgency, topic or relevance, which code then sorts, filters and counts.

Use a decision model when:

• The answer space is fixed and known in advance: a list of queues, a yes/no policy question, a 3 to 5 level rubric.

• The same judgment runs many times: every inbound message, every tool call, every retrieved passage.

• Latency matters: the decision sits in front of a user or in front of each step of an agent loop.

• You want a number to threshold: "act above 0.8" is easier to tune and audit than a free-text "I think this is a refund".

Don't use one when:

• A rule can decide. A tool name, an amount limit or a permission check belongs in code. In Promptise, deny, ask and allow rules run before any model is asked.

• The task needs reasoning or writing: drafting a reply, planning tool calls, explaining a decision. That is the LLM's job.

• The options are open-ended. If you can't list the answers, it is not a decision-model question.

• The decision is about a person's credit, job, health or legal situation. Microsoft's own documentation says to use its decision model for decision support with human review in those cases, not as the sole decision-maker, and that advice holds for every model here.

• Python 3.10 or later and promptise==1.6.0 (uv pip install "promptise==1.6.0"), which also installs the openai SDK.

• A decision-model API. This guide uses OpenAI's Decisions API (gpt-6-luna) because it shares a key with the agent's model. Jev, Clef and Microsoft-Decision-1 take a state plus a questions dictionary instead; only the helper in Step 1 changes.

• An OPENAI_API_KEY in your environment. The agent uses openai:gpt-5-mini.

Start with one helper that sends the state and questions and returns the answers by name. It turns a refusal into an error, so callers can treat it as "unsure", and it records latency and input tokens.

"""One async helper around a decision-model API (here: OpenAI's Decisions API)."""
import time

from openai import AsyncOpenAI

client = AsyncOpenAI()  # reads OPENAI_API_KEY


async def decide(state: str, questions: list[dict]) -> dict:
    """Ask typed questions about `state`; return {question name: answer}."""
    start = time.perf_counter()
    result = await client.decisions.create(
        model="gpt-6-luna", input=state, questions=questions
    )
    answers = {answer.name: answer for answer in result.answers}
    for name, answer in answers.items():
        if answer.type == "refusal":  # can happen for any question: treat as "unsure"
            raise RuntimeError(f"decision model refused question {name!r}")
    answers["_ms"] = round((time.perf_counter() - start) * 1000)
    answers["_tokens"] = result.usage.input_tokens
    return answers

Now ask three questions about one message in a single call: which queue, whether the customer is angry, and how urgent it is.

import asyncio

from decide import decide

QUESTIONS = [
    {"type": "choice", "name": "route",
     "instructions": "Which queue should handle this customer message?",
     "choices": [
         {"value": "order_status", "description": "Where is my order, delivery date, tracking"},
         {"value": "refund", "description": "Money back, cancel and refund, wrong charge"},
         {"value": "other", "description": "Anything else"},
     ]},
    {"type": "predicate", "name": "angry",
     "instructions": "Is the customer angry or threatening to leave?"},
    {"type": "score", "name": "urgency",
     "instructions": "How urgent is this message?",
     "levels": [{"label": "can wait"}, {"label": "today"}, {"label": "right now"}]},
]


async def main():
    message = "Order A-1002 arrived broken. I want my money back today or I'm done with you."
    a = await decide(message, QUESTIONS)
    route = a["route"]
    print(f"route:   {route.choice}  confidence={route.confidence}")
    print("         " + ", ".join(f"{p.value}={p.probability}" for p in route.probabilities))
    print(f"angry:   p={a['angry'].probability}")
    print(f"urgency: score={a['urgency'].score} (0=can wait, 2=right now)")
    print(f"one call, {a['_tokens']} input tokens, {a['_ms']} ms")


asyncio.run(main())

route:   refund  confidence=0.96
         order_status=0.01, refund=0.97, other=0.02
angry:   p=1.0
urgency: score=1.06 (0=can wait, 2=right now)
one call, 419 input tokens, 346 ms

One request answered all three questions. The route came back as refund with 0.97, the anger check as 1.0, and urgency as 1.06 on a 0 to 2 scale, which means "today", leaning slightly toward "right now". Asking several questions about the same state in one call is cheaper than one call per question, because you pay for the state once. Latency varies with region, load and connection setup: the same script took 1,586 ms in one run and 346 ms in the next.

A router maps each answer to a handler, but only when the decision model is sure enough. Below the threshold, the message cascades to the LLM agent, which is slower but can read the message properly.

import asyncio

from decide import decide

ROUTE = {
    "type": "choice", "name": "route",
    "instructions": "Which queue should handle this customer message?",
    "choices": [
        {"value": "order_status", "description": "Where is my order, delivery date, tracking"},
        {"value": "refund", "description": "Money back, cancel and refund, wrong charge"},
        {"value": "other", "description": "Anything else"},
    ],
}
ACT_AT = 0.80  # below this, the decision model is unsure: hand the message to the LLM agent


async def route(message: str) -> tuple[str, str, float]:
    """Return (handler, label, top probability). Handlers: code, agent, person."""
    answer = (await decide(message, [ROUTE]))["route"]
    top = max(p.probability for p in answer.probabilities)
    if top < ACT_AT:
        return "agent", answer.choice, top  # cascade: unsure, so the bigger model decides
    if answer.choice == "order_status":
        return "code", answer.choice, top   # a database lookup and a template, no LLM call
    if answer.choice == "refund":
        return "agent", answer.choice, top  # needs reasoning and tools
    return "person", answer.choice, top     # nothing automated covers it


async def main():
    messages = [
        "Where is order A-1001? It was due Monday.",
        "Order A-1002 arrived broken, I want my money back.",
        "Do you ship to Iceland?",
        "My order is late and honestly I just want to cancel it.",
        "Hi, it is about the thing I bought last week.",
    ]
    for message in messages:
        handler, label, top = await route(message)
        print(f"{handler:6}  {label:12}  top={top:.2f}  {message}")


if __name__ == "__main__":
    asyncio.run(main())

code    order_status  top=1.00  Where is order A-1001? It was due Monday.
agent   refund        top=0.96  Order A-1002 arrived broken, I want my money back.
person  other         top=0.99  Do you ship to Iceland?
agent   refund        top=0.95  My order is late and honestly I just want to cancel it.
agent   other         top=0.78  Hi, it is about the thing I bought last week.

Read the output from the bottom up. The vague message ("the thing I bought last week") got other with a top probability of 0.78, under the 0.80 threshold, so the agent handles it instead of a person's queue. The order-status question goes to a script that never calls an LLM. The angry "just want to cancel" message lands on refund at 0.95, which is a judgment call your support team should confirm with labeled examples.

The threshold is the part you tune. Start conservative, collect 100 or more labeled messages, and check what share of answers above the threshold were right. The guide on calibrating LLM classifiers with a labeled set shows how. Raise the threshold for routes where a mistake costs more.

Promptise's AutoApprovalClassifier decides each tool call in six ordered layers: deny rules, ask rules, allow rules, a read-only check, an optional classifier function, and finally a human handler. The classifier function (llm_classifier) receives the call's real arguments and returns "allow", "deny" or "escalate". It is the natural place for a decision model, because the rules have already handled everything that can be decided by name or pattern.

The agent below has two tools. get_order is read-only by its name, so layer 4 allows it without asking anyone. send_reply goes to the decision model, which checks one thing: does the reply promise money?

import asyncio

from promptise import (ApprovalDecision, ApprovalPolicy, ApprovalRule,
                       AutoApprovalClassifier, CallbackApprovalHandler, build_agent, tool)

from decide import decide

ORDERS = {"A-1001": "shipped Monday, arrives Thursday", "A-1002": "delivered, reported damaged"}


@tool
def get_order(order_id: str) -> str:
    """Look up an order's status."""
    return ORDERS.get(order_id, "no such order")


@tool
def send_reply(order_id: str, body: str) -> str:
    """Send a reply to the customer who owns the order."""
    return f"sent to the owner of {order_id}"


NEEDS_PERSON = {
    "type": "predicate", "name": "needs_person",
    "instructions": "Does this reply promise money, a refund, a credit or compensation?",
}


async def check_reply(request):
    """Step 5 of the classifier: a decision model judges the fuzzy cases."""
    if request.tool_name != "send_reply":
        return "escalate", "no decision question for this tool"
    answer = await decide(request.arguments["body"], [NEEDS_PERSON])
    p = answer["needs_person"].probability
    if p <= 0.10:
        return "allow", f"needs_person p={p:.2f}"
    return "escalate", f"needs_person p={p:.2f}"


async def person(request):  # stands in for Slack, a queue or a review UI
    print(f"  person asked to review: {request.arguments['body'][:70]}...")
    return ApprovalDecision(approved=False, reviewer_id="sam", reason="I'll handle this refund")


def log(request, decision):
    layer = decision.trace.layer if decision.trace else decision.decided_by
    print(f"  {request.tool_name:10} -> {'allow' if decision.approved else 'deny ':5}"
          f"  [{layer}] {decision.trace.rule_reason if decision.trace else ''}")


classifier = AutoApprovalClassifier(
    deny_rules=[ApprovalRule(tool="send_reply", argument_contains="password")],
    read_only_auto_allow=True,          # get_order is read-only by its name
    llm_classifier=check_reply,
    fallback=CallbackApprovalHandler(person),
)


async def build_support_agent():
    return await build_agent(
        model="openai:gpt-5-mini",
        instructions=("You are a support agent. Look up the order, then answer the customer "
                      "with send_reply. State only facts the tools gave you. If an order arrived damaged, offer a full refund. "
                      "Keep replies under 40 words. Finish with the text you sent. If send_reply "
                      "is not approved, do not try again: answer only 'Handed to a colleague.'"),
        extra_tools=[get_order, send_reply],
        approval=ApprovalPolicy(tools=["*"], handler=classifier, on_decision=log,
                                max_retries_after_deny=1),
    )


async def main():
    agent = await build_support_agent()
    try:
        for message in ["Where is order A-1001?", "Order A-1002 arrived broken, I want my money back."]:
            print(f"> {message}")
            print(await agent.chat(message, session_id=message[:12]))
        s = classifier.stats
        print(f"read_only={s.read_only_allows} llm_allows={s.llm_allows} "
              f"llm_escalations={s.llm_escalations} person_denies={s.fallback_denies}")
    finally:
        await agent.shutdown()


if __name__ == "__main__":
    asyncio.run(main())

> Where is order A-1001?
  get_order  -> allow  [read_only] 
  send_reply -> allow  [llm_allow] needs_person p=0.00
Your order A-1001 shipped Monday and arrives Thursday.
> Order A-1002 arrived broken, I want my money back.
  get_order  -> allow  [read_only] 
  person asked to review: Order A-1002 was delivered and reported damaged. We will issue a full ...
  send_reply -> deny   [llm_escalate_then_fallback] needs_person p=1.00
Handed to a colleague.
read_only=2 llm_allows=1 llm_escalations=1 person_denies=1

The status reply scored 0.00 on "promises money" and ran automatically. The refund reply scored 1.00 and went to the person, who denied it because a colleague handles refunds. The stats line shows which layer did the work: two read-only allows, one decision-model allow, one escalation. In a busy deployment, these counters show whether your rules and thresholds keep people out of routine approvals, which is the problem described in AI agent approval fatigue.

Three design choices in this code are deliberate:

• The decision model never denies on its own. It allows clear cases and escalates the rest. A hard "no" belongs to a deny rule you can read and test, like the password rule here.

• `max_retries_after_deny=1`. Without it, an agent can rephrase a denied reply and call send_reply again; in one run it tried three times, and each attempt reached the person. The limit makes the gate refuse further attempts without asking anyone.

• Every decision is logged with its layer. on_decision receives each decision and its trace, which is what you write to an audit log.

Before adding more questions, write down which decisions stay with rules and which stay with people. A useful split:

The decision model fills the middle row. If you find yourself writing a predicate for something a rule could check, write the rule.

The full support desk combines the router and the gated agent. Order-status questions are answered from the order table with no model call, refunds go to the agent (whose replies pass the money gate), and everything else is queued for a person.

"""The complete example: a decision model routes each message to a script, a gated
LLM agent or a person."""
import asyncio

from gate import ORDERS, build_support_agent
from route import route

INBOX = [
    ("A-1001", "Where is order A-1001? It was due Monday."),
    ("A-1002", "Order A-1002 arrived broken, I want my money back."),
    ("A-1001", "Do you ship to Iceland?"),
]


def answer_in_code(order_id: str) -> str:
    return f"Order {order_id}: {ORDERS[order_id]}."  # no model call at all


async def main():
    agent = await build_support_agent()
    try:
        for order_id, message in INBOX:
            handler, label, top = await route(message)
            print(f"> {message}\n  routed: {label} ({top:.2f}) -> {handler}")
            if handler == "code":
                print(f"  {answer_in_code(order_id)}")
            elif handler == "person":
                print("  queued for the support team")
            else:
                print(f"  {await agent.chat(f'[{order_id}] {message}', session_id=order_id + label)}")
    finally:
        await agent.shutdown()


asyncio.run(main())

> Where is order A-1001? It was due Monday.
  routed: order_status (1.00) -> code
  Order A-1001: shipped Monday, arrives Thursday.
> Order A-1002 arrived broken, I want my money back.
  routed: refund (0.96) -> agent
  get_order  -> allow  [read_only] 
  person asked to review: Order A-1002 shows as delivered and reported damaged. I’ll issue a ful...
  send_reply -> deny   [llm_escalate_then_fallback] needs_person p=1.00
  Handed to a colleague.
> Do you ship to Iceland?
  routed: other (0.99) -> person
  queued for the support team

Three messages, three handlers: a script, the gated agent and a person's queue. Only the refund message used the LLM, and the gate kept a refund promise from reaching the customer without review.

All of them take the same three kinds of question. Prices below are each vendor's published price per million input tokens, checked on October 22, 2026:

If you already run on one cloud, its option is usually the shortest path. If you need to self-host or fine-tune, the open-weight models are the choice. Test two candidates on your own labeled messages before you commit: vendor benchmarks are run by the vendors.

• Cost. You pay for input only, so cost is roughly calls times state size. Keep the state short: send the message, not the whole conversation, and ask several questions per call instead of one call per question.

• Fallback when the API fails. A decision model is one more network dependency. On a timeout, an error or a refusal, fall back to the LLM path or to a person, never to "allow". In the AutoApprovalClassifier, an exception inside your function sends the request to the fallback handler and counts it in stats.errors.

• Pin the model version. Aliases such as jev-latest move. Pin a version (for example jev-1.13.0) and re-run your labeled set before switching, because probabilities shift between releases.

• Wording and option order matter. Microsoft's documentation warns that scores change with how you phrase or order questions and options. Keep question text in version control, and add a "cannot tell" option where a forced choice would be wrong.

• Log the probabilities, not just the decision. Store the question name, model version, top probability and the action taken. That log is your next labeled set: sample it, have someone label it, and recheck thresholds monthly.

• Data handling. The state leaves your system. Redact secrets and personal data you don't need for the decision, the same as for any LLM call. Check each vendor's data terms with your security team.

• Test it. Unit-test the router and gate with a fake decide() that returns fixed probabilities, so you can test the threshold logic without network calls.

400 Missing required parameter: 'questions[1].choices'

• No explanations. A decision model returns numbers, not reasons. If a reviewer needs to know why, an LLM has to write that.

• Bounded questions only. It cannot plan, call tools or write a reply.

• Benchmarks are vendor-run. TypeSafe, Cloudflare and Microsoft all publish their own results. Treat them as a starting point, not as evidence for your task.

• A young category. The OpenAI API is in beta, Jev's image support is a preview, and request formats may change.

• Promptise has no built-in decision-model client. You call the vendor SDK yourself, as in decide.py, and plug the result into AutoApprovalClassifier or your own router. FallbackChain and the reasoning engine's RouterNode work with chat models, not decision APIs.

What is a decision model in AI?

A decision model is a model that answers bounded, typed questions about an input with probabilities instead of generated text. You define the possible answers (yes/no, a list of options or an ordered scale), and it returns a probability for each, which your code uses to decide what happens next.

How is a decision model different from an LLM with structured output?

An LLM with structured output still generates tokens and then fits them into a schema, and it gives you no reliable probability for the choice. A decision model scores the allowed answers directly and returns the probabilities, usually faster and with no output tokens to pay for. Use structured output when you need generated fields; use a decision model when you only need the choice.

Can a decision model approve an AI agent's tool calls?

Yes, as one layer of an approval gate. Let rules decide the clear cases first, let the decision model allow calls it is very sure are safe, and send everything else to a person. In Promptise, pass it as the llm_classifier of AutoApprovalClassifier.

Are decision model probabilities calibrated?

Some vendors, such as TypeSafe, say their models are trained for calibrated probabilities, and Microsoft notes that calibration is strongest on familiar task types. Verify it on your own data. A probability of 0.9 should be right about 90% of the time on your messages; if it isn't, adjust the threshold or recalibrate the scores.

What threshold should I use for routing with a decision model?

There is no universal number. Start high (0.8 to 0.9 for routing, a very low probability for "risky" gates), measure accuracy above the threshold on a labeled set, and move it until the error rate fits what a mistake costs.

Do decision models replace LLM routers?

They replace the routing call, not the LLM. The decision model picks the path cheaply; the LLM still does the work on the paths that need reasoning or writing.

• Compare the vendors in detail: Jev vs Clef vs OpenAI Decisions API compared.

• Measure and fix your thresholds: How to calibrate LLM classifiers with a labeled set.

• Build the approval side in depth: human-in-the-loop approval for AI agents and AI agent approval fatigue.

• Route between models: LLM routing: when to use a small vs large model.

• Why stated confidence is weak: LLM confidence scores and self-check fields.

• Keep costs in check: AI agent cost control with budgets.

• For a non-technical overview to share with your team: decision models explained for leaders.

• Reference: the AutoApprovalClassifier docs.