Run artifact · retrieval
retrieval-fixture
System reference-predictions evaluated against synthetic-v1. This page presents only approved public evidence.
Artifact ID
demo-retrieval-reference-v1
Claim scope: Integration Only.
Aggregate values
Metrics
Hit Rate@5
0.8
k preserved in metric name · demo-retrieval-reference-v1
MRR
0.8
k preserved in metric name · demo-retrieval-reference-v1
nDCG@5
0.8
k preserved in metric name · demo-retrieval-reference-v1
Precision@5
0.24
k preserved in metric name · demo-retrieval-reference-v1
Recall@5
0.8
k preserved in metric name · demo-retrieval-reference-v1
Failure summary
1 labeled failures- THQA-002SHOULD_ABSTAIN
Provenance
- Dataset
- retrieval-fixture · synthetic-v1
- System
- reference-predictions
- Evaluation
- retrieval
- Recorded
- 15 Jan 2026
- Top-k
- 5
- Evaluator
- retrieval: deterministic-metrics-v1
Record-level evidence
5 approved records| Record | Status | Metrics | Retrieved IDs | Relevant IDs |
|---|---|---|---|---|
| THQA-001 | Pass | Hit Rate@5 1 · MRR 1 · nDCG@5 1 · Precision@5 0.2 · Recall@5 1 | doc-th-001, doc-th-999 | doc-th-001 |
| THQA-002 | SHOULD_ABSTAIN | Hit Rate@5 0 · MRR 0 · nDCG@5 0 · Precision@5 0 · Recall@5 0 | Not included | Not included |
| THQA-003 | Pass | Hit Rate@5 1 · MRR 1 · nDCG@5 1 · Precision@5 0.4 · Recall@5 1 | doc-th-003, doc-th-002 | doc-th-002, doc-th-003 |
| THQA-004 | Pass | Hit Rate@5 1 · MRR 1 · nDCG@5 1 · Precision@5 0.2 · Recall@5 1 | doc-th-004 | doc-th-004 |
| THQA-005 | Pass | Hit Rate@5 1 · MRR 1 · nDCG@5 1 · Precision@5 0.4 · Recall@5 1 | doc-th-005, doc-th-006 | doc-th-005, doc-th-006 |
Limitations
- • Metrics are deterministic fixture evidence with no external model or provider inference.
- • Synthetic retrieval fixture only; reference evidence for an integration regression demonstration.
Contract notes
Metric names and denominators are inherited from the deterministic evaluator output. No combined quality score, missing denominator, raw response, prompt, corpus text, or hidden reasoning is inferred here.