PromptisePromptise
Docs
GitHub
Promptise - AI Framework LogoPromptise

The foundation layer for agentic intelligence. Build, secure and operate autonomous AI systems with Promptise Foundry.

pip install promptise

[01] Foundry

  • MCPcast
  • The Promptise Agent
  • Reasoning Engine
  • MCP
  • Agent Runtime
  • Prompt Engineering
  • Execution Engine
  • Agent Identity

[02] Resources

  • Documentation
  • GitHub
  • Guides
  • Learning Paths
  • Questions

[03] Company

  • About
  • Terms of Service
  • Privacy Policy
  • Cookie Policy
  • Subprocessors

© 2026 Promptise by Manser Ventures. All rights reserved.

Open source · Python

← Guides> Library> Topic

LLM uncertainty and confidence: knowing when the model is unsure

An answer is only useful if you know how far to trust it. Language models do not report reliable confidence on their own, and a confidence number a model writes in its answer is a rough signal, not a probability.

Better signals come from behaviour: ask the same question several times and measure how much the answers disagree, or group the answers by meaning and compute semantic entropy. Disagreement can also show which inputs need a person or more examples. These guides cover adding self-check and confidence fields to prompts, detecting when a model is unsure with semantic entropy, and active prompting, which uses uncertainty to choose the few-shot examples worth annotating.

[n.01/02]> 3 guides
  1. 01LLM Confidence Scores: Add Self-Check Fields to PromptsGet useful LLM confidence scores: add basis, quote and check fields to the output, define each level and route answers in code. Real example and output.Beginner · 8 minOct 18, 2026
  2. 02Active Prompting: Pick the Few-Shot Examples That MatterActive prompting explained: sample answers, measure disagreement, and annotate only the most uncertain inputs as few-shot examples. Runnable code, output.Advanced · 8 minOct 18, 2026
  3. 03Semantic Entropy: Detect When an LLM Is UnsureSemantic entropy explained: sample answers, group them by meaning, and use the entropy to decide when to answer, retrieve or abstain. Runnable Python code.Advanced · 8 minOct 18, 2026
[n.02/02]> go further

Related topics

  • Hallucinations11
  • Structured Outputs8

By category and level

  • Prompt Engineering guides54
  • Beginner guides36
  • Advanced guides24

Search every guide.

The library lists all guides with filters for topic, level, category and reading time.

All guidesLearning paths