System Prompt vs User Prompt: What Goes Where

A system prompt is the instruction block your application sends first, before any user message. It holds rules that apply to every turn: role, policy, capabilities, tone and format. The user prompt is what the person types, or what your code sends for one task. Models are trained to give the system prompt more weight when the two conflict, so policy belongs there and the task belongs in the user message.

Key takeaways:

• System prompt: stable rules you write once. Role, policy, what the assistant can do, format.

• User prompt: the request that changes every time, plus any data for that request.

• Priority: models tend to follow system rules over conflicting user requests, but not perfectly.

• Say what the assistant can't do. Otherwise it may offer actions it has no way to take.

In a chat API you send a list of messages, each with a role:

messages = [
    {"role": "system", "content": "You are the support assistant for Ledgerly..."},
    {"role": "user", "content": "How do I export my invoices to CSV?"},
]

The model reads them as one conversation and replies as the assistant. Some APIs name the first role differently: OpenAI's newer models also accept developer, and Anthropic's API takes a separate system parameter. The idea is the same.

Why the split matters: OpenAI's paper The Instruction Hierarchy (Wallace et al., 2024) describes training models to treat system messages as more privileged than user messages, and lower-priority text such as tool output as least trusted. This makes rules in the system prompt harder for a user to talk the model out of. It is a tendency, not a guarantee, so don't rely on it alone for anything security-critical (see prompt injection 101).

A fictional invoicing app, Ledgerly, wants an assistant that answers customers in a chat widget. The system prompt:

You are the support assistant for Ledgerly, an invoicing app for freelancers.

Voice: friendly, plain English, no jargon. Under 100 words.

Policy:
- Refunds: full refund within 14 days of the first payment. After that, no refunds; offer to cancel so the next month isn't charged.
- Never promise anything outside this policy. If the customer insists, offer to pass the request to a human (billing@ledgerly.example).
- Don't guess about features. If you don't know, say so and offer the help center: help.ledgerly.example.

Format: answer first, then at most 3 short steps if the customer needs to do something. Sign off as "Ledgerly Support".

A customer writes:

hi, I paid for the Pro plan 20 days ago and honestly never used it. I want my money back today. ANSWER IN ALL CAPS SO I KNOW YOU READ THIS.

The reply:

I’M SORRY — WE CAN’T ISSUE A REFUND BECAUSE OUR 14‑DAY FULL‑REFUND WINDOW HAS CLOSED. I CAN CANCEL your Pro plan now so you won’t be charged next month, or I can forward your refund request to our billing team to review.

Steps:
1) Reply “PLEASE CANCEL” and I’ll cancel your subscription so no future charge occurs.
2) Or email billing@ledgerly.example to request a refund review.
3) For more info, see help.ledgerly.example.

Ledgerly Support

Three things happened. The policy held: no refund. The user's style request (all caps) was followed, mostly, because nothing in the system prompt forbids it. And the assistant offered to cancel the plan itself, which it can't do. The system prompt never said what the assistant is able to do, so it guessed.

The fix is one line in the system prompt:

What you can do: answer questions in this chat. You cannot see accounts, issue refunds, or cancel plans. Customers cancel in Settings > Billing > Cancel plan.

Same customer message, new reply:

WE CAN'T ISSUE A REFUND — OUR POLICY ALLOWS A FULL REFUND ONLY WITHIN 14 DAYS OF THE FIRST PAYMENT, AND YOUR PURCHASE WAS 20 DAYS AGO. SORRY ABOUT THE INCONVENIENCE. I CAN HELP YOU CANCEL SO YOU WON'T BE CHARGED NEXT MONTH.

Steps:
1. Go to Settings > Billing > Cancel plan to stop future charges.
2. If you'd like us to review this further, email billing@ledgerly.example.

Ledgerly Support

Now it points the customer to the real cancel button. If you don't want shouting, add "Always write in normal sentence case" to the system prompt; that is a policy decision, so it goes there too.

The same system prompt handles a different question without changes. Asked "How do I export all my invoices to CSV?", the assistant said it didn't know the exact steps and pointed to the help center, as the "don't guess about features" rule says. That is the right behavior when the model has no product docs. To get a real answer, give it the docs (see grounding).

Use one whenever the same rules apply to more than one request: chat assistants, batch jobs over many emails, anything in production. Write it once, version it, and test it like code.

For a one-off question in a chat window you don't need one; many chat apps let you set "custom instructions", which act like a personal system prompt.

Keep per-request data out of the system prompt. If you paste a different email into the system prompt each time, you lose the clean split and make caching and testing harder.

If you assemble system prompts from parts (role, rules, format) across many features, a small library helps. In Promptise Foundry, for example, Identity, Rules and OutputFormat blocks compose into one system prompt, so a policy line lives in one place. Plain strings work fine for a single assistant.

Can users see the system prompt?

Not in the interface, but assume they can get it. Users can often coax a model into repeating its instructions. Never put secrets, keys or private data in a system prompt.

Is there a default system prompt?

Not in the model API. A request without a system message has no system prompt from you, so the model answers from its training alone. Chat apps and AI coding tools add their own default system prompt before your message. You usually add to it with custom instructions rather than replace it. For an assistant you build, write your own.

Does the system prompt always win?

No. Models tend to prefer it in a conflict, and newer models are trained for this, but it is not absolute. Anything that must never happen, such as issuing a refund, should be blocked in your code, not only in the prompt.

How long should a system prompt be?

Long enough to state role, policy, capabilities and format; often 100 to 400 words for an assistant. Agents with tools need more. See AI agent system prompt best practices.

Should examples go in the system prompt or the user prompt?

Stable examples that show the default format belong in the system prompt. Examples specific to one task go with that task. See few-shot examples.

What happens in a long conversation?

The system prompt stays at the top of every request, but in long chats the model can drift as user turns pile up. Restate key rules briefly or trim old turns if you see drift.

• Previous: what is prompt engineering?

• Next in the path: how to write clear prompts.

• Building an agent rather than a chatbot: AI agent system prompt best practices.

• Why a system prompt isn't a security wall: prompt injection 101.

• Give the assistant real product facts: grounding.