CSV template for building an adjudicated calibration and test set.
Part of Foundation
- Format
- CSV
- Lines
- 2
- Size
- 0.6 KB
- SHA-256 (v2)
fe10577fe0dcb71ae5aad388468c71b08bdddf1fe9bdb57f9ab0df111eb1df8f
Table preview
| benchmark_item_id | benchmark_version | split | contract_id | contract_version | case_id | criterion_id | applicability | acceptable_verdicts | severity | positive_evidence_spans | negative_evidence_spans | omission_descriptions | acceptable_actions | ambiguity_status | challenge_attributes | slice_tags | adjudication_method | adjudicator_count | domain_expert_count | blind_to_autoqa | blind_to_reviewer_label | initial_agreement_rate | adjudication_notes | allowed_for_prompt_development | held_out_project | weight |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| B-001 | 0.1.0 | calibration | applies | met | none | accept | clear | independent_then_adjudicate | 3 | 1 | true | true | false | false | 1.0 |
Raw CSV
benchmark_item_id,benchmark_version,split,contract_id,contract_version,case_id,criterion_id,applicability,acceptable_verdicts,severity,positive_evidence_spans,negative_evidence_spans,omission_descriptions,acceptable_actions,ambiguity_status,challenge_attributes,slice_tags,adjudication_method,adjudicator_count,domain_expert_count,blind_to_autoqa,blind_to_reviewer_label,initial_agreement_rate,adjudication_notes,allowed_for_prompt_development,held_out_project,weight B-001,0.1.0,calibration,,,,,applies,met,none,,,,accept,clear,,,independent_then_adjudicate,3,1,true,true,,,false,false,1.0 |
On this page
Manifest records
documents/raw/foundation/MANIFEST.json
{
"path": "templates/gold-set-template.csv",
"bytes": 593,
"sha256": "fe10577fe0dcb71ae5aad388468c71b08bdddf1fe9bdb57f9ab0df111eb1df8f"
}site-manifest.json
{
"path": "documents/raw/foundation/templates/gold-set-template.csv",
"bytes": 593,
"sha256": "fe10577fe0dcb71ae5aad388468c71b08bdddf1fe9bdb57f9ab0df111eb1df8f"
}