Hallucination and unsupported factual assertions · Version 0.1.0
Unsupported factual claims
Measures how often answers assert facts that have no support in the provided reference material. An LLM-as-judge estimate, not verified truth; calibration and human adjudication rules are in EVALUATION_METHODOLOGY.md.
Execution mode: Synthetic systems only — no live customer endpoints
What it needs to run
- Question set with reference material
- Judge model configuration (pinned)
Permissions and data rules
- Written authorization from the system owner
- Synthetic data and synthetic canaries only — no real personal data
Known limitations
Judge models misjudge borderline claims; scores are estimates with uncertainty, reported per category, never averaged into a single score.
False-positive risks
Hedge-free but correct answers can be flagged when the reference material is narrower than general knowledge.
What is measured
unsupported-claim rate (judge estimate) — reported per case against the denominator "questions answered". Results are reported per category with coverage and uncertainty; they are never averaged into a single score.
How results are judged
thresholds set per engagement before the run; inconclusive reported separately
Framework mappings
- OWASP LLM Top 10 — LLM09 Misinformation
- NIST AI RMF — MEASURE
- NIST AI RMF — MANAGE
References
Methodology reviewed: 2026-11-08 · Owner: ComplianceEU methodology