Comparison artifact · regression policy
Regression detected
Candidate demo-retrieval-fixture-v1 is compared with reference demo-retrieval-reference-v1 using the artifact-provided aggregate policy results.
Artifact ID
demo-retrieval-regression-v1
Claim scope: Integration Only.
Reference run
Run ID: demo-retrieval-reference-v1
Candidate run
Run ID: demo-retrieval-fixture-v1
Population compatibility
Population compatibility: MATCHED
Dataset, protocol, top-k, and evaluator-version metadata are matched before record transitions are derived.
Aggregate policy result
Metric comparison
| Metric | Reference | Candidate | Delta | Direction | Result |
|---|---|---|---|---|---|
| Hit Rate@5 | 0.8 | 0.6 | REGRESSION-0.200 | Higher is better | Regressiondegradation exceeded allowance of 0.1 |
| MRR | 0.8 | 0.6 | REGRESSION-0.200 | Higher is better | Regressiondegradation exceeded allowance of 0.1 |
| nDCG@5 | 0.8 | 0.523 | REGRESSION-0.277 | Higher is better | Regressiondegradation exceeded allowance of 0.1 |
| Precision@5 | 0.24 | 0.16 | PASS-0.080 | Higher is better | Within allowancewithin configured regression allowance |
| Recall@5 | 0.8 | 0.5 | REGRESSION-0.300 | Higher is better | Regressiondegradation exceeded allowance of 0.1 |
Record-change summary
5 matched records2 matched records have a category transition or metric delta. Category transitions remain separate from aggregate regression statuses.
Explore record changesLimitations
- • Regression status uses a fixed aggregate max degradation allowance of 0.10 per configured metric.
- • Synthetic same-population regression demonstration; not a benchmark or model-superiority result.
Interpretation boundary
Synthetic same-population regression demonstration; not a benchmark or model-superiority result.
Status and reason text are sourced from the deterministic comparison artifact. No statistical significance or record-level regression policy is inferred.