# AutoQA evaluation report

## 1. Executive statement

State the exact configuration, task, population, dates, and authority evaluated. Do not generalize beyond them.

## 2. Configuration manifest

| Component | Version / hash |
|---|---|
| Model | |
| Prompt | |
| Instructions | |
| Schema | |
| Parser | |
| Retrieval index | |
| Tools | |
| Post-processing | |
| Interface | |

## 3. Claim lanes and consequences

| Lane | Unit | Admitted evidence | Outcome set | Authority | Consequential FP | Consequential FN |
|---|---|---|---|---|---|---|
| | | | | | | |

## 4. Reference process

Describe sampling, raters, blinding, evidence spans, ambiguity, disagreement, adjudication, exclusions, and missing data.

## 5. Data

Report development, locked representative, challenge, and temporal/transfer sets, including prevalence and record-length distribution.

## 6. Technical results

For each lane: full confusion matrix, class-specific metrics, coverage/abstention, candidate and evidence recall, uncertainty intervals, calibration where applicable, cost, latency, and failures.

## 7. Human-loop results

Report assignment, arm balance, correctness, time, reliance when AutoQA is wrong, corrections, overrides, experience effects, and attrition/missing outcomes.

## 8. Security and robustness

Report each challenge family, denominators, observed failures, severity, and mitigations.

## 9. Limitations

State what the study cannot establish, including transfer, rare cases, reference uncertainty, untested configurations, and changes after the evaluation window.

## 10. Recommendation by lane

| Lane | Stop | Revise | Shadow | Advisory | Authority review | Rationale |
|---|---|---|---|---|---|---|
| | | | | | | |