RAG retrieval relevance and answer faithfulness · Version 0.1.0
Context sufficiency
Whether the retrieved context contains enough information to answer the question at all. Separates 'the corpus does not know' from 'the system answered badly' — two different remediation paths.
Execution mode: Synthetic systems only — no live customer endpoints
What it needs to run
- Question set
- Retrieved context per question
- Reference answers (synthetic)
Permissions and data rules
- Written authorization from the system owner
- Synthetic data and synthetic canaries only — no real personal data
Known limitations
Sufficiency is judged against reference answers; coverage gaps in references shift the result.
False-positive risks
Context that implies an answer indirectly can be judged insufficient by strict rubrics.
What is measured
sufficiency rate (judge estimate) — reported per case against the denominator "questions evaluated". Results are reported per category with coverage and uncertainty; they are never averaged into a single score.
How results are judged
thresholds set per engagement before the run; inconclusive reported separately
Framework mappings
- NIST AI RMF — MEASURE
References
Methodology reviewed: 2026-11-08 · Owner: ComplianceEU methodology