# Leadership-site v2 - groundedness & decision-use build audit

**Date:** 2026-07-15 (America/New_York)
**Input:** `[local path withheld]` only, plus sources retrievable from URLs named inside it. No other folders were read; the `atlas/` paths referenced inside the corpus were **not** opened.
**Output:** `site/` (deployable), `pipeline/` (full reproducible audit pipeline), `site.zip`.

---

## 1. The inferred goal

The archive is a self-contained leadership site for the AutoQA Foundation 0.1.0 - the case, architecture, pilot design, research corpus (132 sourced claims across 10 domain reports), and a validated contract/benchmark package. What it argued for was already strong; what it could not yet do was (a) let a reader audit any number down to the source quote, and (b) turn the open questions into an actual decision instrument. Both stated concerns - groundedness, and the ability to make decisions on it - map to those two missing layers. Mid-build, a third requirement was added explicitly: verified is not enough - claims must be **usable under current capabilities** (Claude Fable / Opus 4.8 class, GPT-5.5 / 5.6 class), and old-but-true measurements must not masquerade as current signal.

## 2. What was built

**`evidence.html` + `assets/claims.json` - the evidence ledger.** All 132 corpus claims, each with:
- the one primary source the corpus names (title, URL, date, type; peer-review marker where derivable);
- **Layer 1** - the research sweep's own 2026-07-14 verification transcripts (27 claims; preserved verbatim, including its corrections);
- **Layer 2** - an independent re-check run 2026-07-15 for this build: every claim checked against **only** the URL the corpus names (plus served redirects / same-document arXiv variants), verdicts derived exclusively from retrieved text, verbatim supporting quote per claim, per-claim list of asserted specifics *not visible* in retrievable text;
- **Layer 3** - a decision-use grade (see Sec. 4);
- same-source groupings ("repetition across domains is not corroboration"), and bidirectional "used on" links to every page location that cites it.
Plus: the seven "no evidence anywhere" gaps as first-class objects (each assigned to the experiment that closes it), the full per-domain gap and disagreement inventories (preserved, not averaged away), and the package-integrity record.

**`decisions.html` - the decision register.** 5 leadership authorizations (A1-A5), 6 open owner calls (D1-D6, from the philosophy doc's Sec. 8), 5 standing defaults with named falsifiers (C1-C5, including the honestly-flagged single-under-review-study dependency), the 7 experiments that settle them (E1-E7) with a decisions<->gaps<->experiments cross-map, and the non-negotiables section so meetings don't re-litigate encoded constraints. Every evidence row is a graded chip into the ledger; every decision states options, recommendation, reversibility, dependencies, and **what would change the answer**.

**Provenance wiring.** Every quantitative claim on index/brief/research/pilot/architecture now carries a claim-ID chip (with its grade) that resolves to the ledger entry; the pilot risk register gained a Ledger column; the launch gates and claim-ontology table are chipped; nav and footer across all 49 pages gained Decisions + Evidence.

## 3. Verification results (Layer 2, run 2026-07-15)

- **Coverage:** 132/132 claims, 121 distinct sources, checked by 12 independent verifier agents under a hard constraint: no search engines, no substitute sources, no background knowledge - retrieved text only.
- **Verdicts:** 82 confirmed - 43 partially confirmed (core confirmed; some specifics not visible at abstract/landing level, or minor nuances - each listed on the claim) - 7 not retrievable (Cloudflare/paywall: PNAS, two DOI-paywalled classics, Annual Reviews, ScienceDirect, two OpenReview forums) - **0 mismatches, 0 refuted.**
- **Notable nuances surfaced (all disclosed on the ledger, none quoted on leadership pages):**
 - `F26-08` (CoVal): the corpus's 0.58-0.61 concordance figures are the *in-sample* appendix values; out-of-sample validation is 0.75-0.76.
 - `F26-09`: corpus said "~200 labels per class"; the paper's worked regime is ~200 **total** (~100/class).
 - `HS-01`: the paper was retitled on arXiv v3 ("Using Synthetic Data for Model Evaluation"); content confirmed.
 - `CM-08` (RealCritic): several headline numbers in the corpus claim trace to sibling benchmarks rather than the cited paper.
 - Several compound claims bundle 2-3 papers under one URL (AJ-09, CT-08, GF-05, GF-12, HS-09, IP-12, F26-11) - the un-fetched halves are marked "not visible," not assumed.
- **No leadership-page copy needed correction** - every number shown on index/brief/research checked out against its source.

## 4. Decision-use grading (Layer 3) - the capability-currency answer

Verification cannot answer "is this claim usable for decisions under current capabilities?" A verified 2024 JudgeBench score is still false signal if treated as a property of today's frontier. Every claim therefore carries a grade (hand-assigned, all 132, recorded in `pipeline/grade_claims.py` with per-claim rationale):

- **A - durable - 76 claims.** Capability-invariant: law (Platform Work Directive, SCHUFA, ISO 5259), mathematics & statistical method (PPI, HyPAC, the 2x theorem, corrected estimators), human-rater/feedback science (Weigle, G-theory, Kluger & DeNisi, Cochrane), adversarial & economic floors that only *strengthen* as models improve (the $0.05 survey agent, universal adversarial phrases, spam economics), and documented production practice. **Safe to decide on now.**
- **B - current-generation - 40 claims.** Measured in the 2025-26 frontier window (some GPT-5.2-era), but not on the deployment tier and almost never on human evaluative writeups. Strong priors; confirmed at the bakeoff.
- **C - superseded-generation magnitude - 16 claims.** GPT-4o / o1 / flash-tier measurements (JudgeBench, CriticGPT's 24%-vs-6%, the AggreFact ~77 ceiling, WiCE decomposition harm, AbsenceBench, expert-domain 64-68%...). Mechanism informs design; **the number must not set a current threshold in either direction.**

The framing fact, now on the ledger and the register: of 132 claims - 49 measured on model outputs, 26 on human work, 14 in adjacent domains, 43 structural (law/math/standards/practice, tier-independent) - **zero test the deployment tier on claim-level QA of human evaluative work.** Consequences wired into the site: the register's evidence rows are graded; every capability-dependent bar re-measures at the pilot bakeoff on the deployment-tier set; the bakeoff gate on the pilot page now names that set; and the rule is stated symmetrically - yesterday's 55% is not today's ceiling, yesterday's 91% is not today's promise.

## 5. Infrastructure corrections

- **Stale manifest (integrity defect in the shipped archive):** `site-manifest.json` recorded wrong hashes/sizes for `assets/site.css`, `assets/site.js`, `README.md`, and omitted the fonts and favicon entirely. Regenerated over all 108 shipped files; discrepancy disclosed on the ledger's integrity section.
- **Foundation package validated:** `python3 validation/validate_package.py` -> all 8 checks PASS (schemas, referential integrity, canonical hashes, positive-proof invariants). Recorded on the ledger.
- **Observed, deliberately not changed:** `assets/search-index.json` (894 KB) is referenced by no page or script - library search runs on inline attributes. Left in place; flagged here.

## 6. Verification of the build itself

- Link/anchor crawler: 49->50 HTML files, ~2,300 internal links - 0 broken files, 0 broken fragments (after fixing two anchor-case defects it caught).
- All JSON parses; `claims.json` invariants asserted (132 claims, all graded A/B/C, all externally checked).
- Render-consistency spot-check: 12/12 random claims' text, quotes, and grade attributes match the data files; chip grade suffixes consistent across all pages.
- Browser pass (Playwright): zero console errors/warnings; evidence ledger, decision register, chipped research cards verified visually in dark and light themes, desktop and mobile nav; filters and hash-open behavior work.
- Structural HTML parse of both generated pages: zero tag-balance errors.
- Independent review, two passes in fresh contexts: (1) a full seven-category review (HTML correctness, filter script, accessibility, claims.json sanity, diff hygiene, manifest hashes, leadership polish) whose shared checklist completed with no blockers filed - its narrative report had not arrived by ship time and is not claimed as sign-off; (2) a blocker-only critic pass which returned **APPROVED - no blockers** with per-check evidence (270-line decisions.html read; stat-strip counts match block counts 5/6/5/7; claims.json 132/A-B-C/76-40-16 verified; all `claim-ref` fragments resolve against evidence.html's 146 ids; CSS braces balanced 438/438 with every used class defined).
- Every patch operation used exact-match substitution with asserted match counts - nothing silently missed.

## 6b. Mechanisms adopted from the comparison artifact (credited)

After the build, a competing rebuild (`AutoQA-decision-grade.zip`) was analyzed on request (see `COMPARISON.md`). Four of its decision-instrument mechanisms were adopted into v2 - each grounded in the corpus, none touching the evidence base:
1. the precommitted graduation rule (one-sided 95% upper confidence bound below an approved ceiling on an untouched test set; ceilings/sample/multiplicity/stop-rules locked pre-test) - grounded in HS-07, F26-09, HS-03 and the Benchmark Protocol's rare-critical-error sizing;
2. the four named outcomes (stop / narrow / assisted-only / selective graduation) on the pilot and the register;
3. a machine-readable `assets/decision-record.json` (extended beyond its precedent to cover owner decisions, standing defaults, experiments, the never-authorized list, and the capability-currency block, with evidence IDs throughout);
4. bounded absence phrasing ("the reviewed corpus identifies no...", never "none exists") across the gaps section and open-problem cards.

## 7. Reproducibility

`pipeline/` contains the complete audit chain: `parse_corpus.py` (corpus -> 132 structured claims), `batch_*.json` + `results_*.json` (the 12 verifier work orders and their verbatim outputs), `grade_claims.py` (all 132 decision-use grades with rationales), `usage_map.json`, `build_data.py` -> `claims.json`, `build_pages.py` -> the two pages, `patch_pages.py` (asserted edits), `verify_site.py` (crawler + manifest). Re-running the chain end-to-end regenerates the site's evidence layer from the corpus and the recorded verification outputs; re-running the verifier batches re-checks the sources live.

## 8. Round 3 (2026-07-15, later the same day): the philosophical kernel

Direction change on request: no implementation steps; make the foundation airtight first - specifically a **kernel upon which all things are built**.

**`KERNEL.md`** (shipped at delivery root and inside `site/`, linked from the register and ledger): a minimal axiomatic base - one telos, four thin definitions, four mathematical FACTs, four standing domain CONDitions, nine CHOSEN commitments (explicitly chosen, never disguised as theorems), one versioned LAW element - followed by consequences with honest statuses (DERIVED / UNPACKED / DESIGN RESPONSE / governed choice), eight declared interfaces where all perishable capability facts live, five declared choice-points/holes, a dependency table, and a scope declaration with the migration principle (the kernel governs the non-certifiable residue of open-world evaluative work; whatever gains a certificate migrates to the deterministic lane - so model progress moves the boundary, never the sentences).

**Adversarial process, on the record (five reviews, three versions):** v0.1 was attacked by three fresh-context critics - two Claude-family and one GPT-5.6-class (codex, gpt-5.6-sol, xhigh reasoning). Codex round 1: severe - no derivation fully entailed, axioms falsified by formal methods, one legal contradiction, six missing elements, a self-referential discipline defect -> v0.2. The two Claude reports (0 FATALs each; 6 and 9 MAJORs) converged independently on the same deep findings plus new ones: Part III necessity-rhetoric re-smuggling chosen values; the shared-substrate/tooling-parity premise and human-lane independence as contingent empirics posing as walls; the coda's literal falsity. A codex confirmation pass on v0.2 returned **"STILL BROKEN"** - FACT/COND layer still overclaiming, no DERIVED item fully entailed, governed-choice discipline unapplied - with exact repairs per finding. v0.3 implements those repairs and the Claude critics' residuals: the FACT layer shrunk to five limitative facts; substrate parity, independence scarcity, and the "permanent frontier band" demoted from walls to measured presumptions (the permanence claim retracted - what remains permanent is the measurement obligation); the human-decision floor made global by explicit choice (K22); fairness (K23) and exculpatory-only reopening (K24) added as owned commitments; every choice-point given a named owner and version obligation. A bounded codex delta-check verified repair implementation on v0.3 (result recorded in `pipeline/`). All five review reports and both superseded kernel versions ship in `pipeline/` - the kernel's own standing rule applies to itself: re-attacked at every version bump and each model generation.

**Round-2 durable anchors (under the "relevant and certain" bar):** DA-02 Uniform Guidelines **confirmed** verbatim from eCFR; DA-01 NYC Local Law 144 **partially confirmed** from the DCWP rule-adoption document (substance confirmed; admitted with disclosure); DA-03 Holmstrom-Milgrom **partially confirmed** (bibliographic identity verified; proposition paywalled - admitted with that disclosure); DA-04 ANSI/ASQ Z1.4 **not retrievable** (ASQ bot-wall) - quarantined in the visible candidate queue, not admitted. The ledger now carries 132 corpus claims + 3 round-2 anchors, provenance disclosed on each.

**Register tranche:** D1 gained the worked prevalence-arithmetic panel (labeled derivation, not literature); D5 became a two-sided (EU + US) legal gate; A4 and the pilot's human-loop gate carry the honest human-lane expectation sentence; Section D gained the E2-first sequencing note (recommendation only - nothing operational authorized) and the bakeoff callout; `decision-record.json` v1.1.0 carries the kernel reference, standing rule, sequencing note, and two-sided legal gate.

## 9. Round 4 (2026-07-15, evening): presentation rebuilt as a decision document; full ASCII normalization

Direction: the earlier presentation read like marketing - stat tiles, badge chips, taglines. Rebuilt all seven
leadership surfaces as quiet, executive-grade documents: ask-first, prose-led, numbers stated in sentences with
their caveats attached, every assertion carrying a bracketed reference that opens the ledger entry (source,
verification, quote, decision-use grade). Evidence-based decision-making is the organizing principle of the
presentation, not a feature of it. The library and document viewers keep their reference layout.

Also in this round:
- **Full ASCII normalization** across the delivery (pages, ledger data, kernel, pipeline records): typographic
  characters transliterated, brand and UI glyphs replaced with text, zero non-ASCII bytes remaining in shipped
  text files. Exemption: `documents/raw/foundation/**` stays byte-exact so its own hash validators keep passing;
  the research corpus was already ASCII-normalized by its authors and needed only minor cleanup.
- **Robustness check, published on the research page:** striking every LLM experimental result measured on
  pre-2025 models (16 findings outright, 7 reduced to current-model halves) leaves 9 of 11 register decisions
  untouched; the other two keep standing on law, mathematics, or current-generation results. The kernel is
  independent of model experiments entirely - stated as a checkable property in its Part VI.
- **Kernel v0.3.2** (commentary only): the motivating problem - human reviews of human-labeled data not reaching
  consensus - is named inside K0's commentary and answered by existing kernel content: non-consensus has four
  causes needing four different fixes; targeting consensus directly treats all four by erasure and caps validity
  at reviewer reliability; consensus in application rises as a byproduct of removing illegitimate causes. The
  overview and summary pages lead with this framing.
- decision-record.json v1.2.0; reference usage now recorded by the generator itself (site_usage.json), keeping
  ledger back-links exactly synchronized with what the pages actually cite.

## 10. Round 5 (2026-07-15, late): the long-context retrieval question

Question raised: is an LLM incapable of reliable long-context verification - a silent miss of in-record
evidence corrupting approvals? Answer, now encoded: the logical half was already in the kernel (record-level
claims are universals, only as good as the search behind them), but the practical half deserved its own
elements. Kernel v0.3.3 adds P5 (single-pass reading of long records is unreliable retrieval - degrades with
length and semantic distance, and the failure is silent; a presumption with a measurement obligation, not a
wall, because bounded records are exhaustively checkable by procedure), a K14 coverage-accounting clause
(record-level claims extend no further than the coverage achieved), D9 (no single-pass certification beyond
measured reliable length; discovery as a separate auditable step; planted support AND counter-evidence recall
by length, position, semantic distance), and interface I10. Two structural notes: the three-way verdict
semantics already protect workers from the missed-support direction (unsupported triggers an evidence request,
never an auto-fail; only found contradiction fails anyone), so the silent-miss harm concentrates in false
passes over buried counter-evidence - exactly what the confident-pass audit samples, now priced by the
planted-counter-evidence recall number. System page, research page, and the pilot risk register carry the
corresponding sections.

## 11. What was deliberately not done

- The raw corpus documents (`documents/raw/**`) and their rendered views were not edited - they are the record this build audits, not a surface to polish. The two corpus-number nuances (Sec. 3) are disclosed on their ledger entries rather than silently rewritten.
- No new research was imported. Every claim, quote, and citation on the new pages traces to the archive or to the sources it names. The only externally-sourced addition is the deployment-tier model naming (Claude Fable / Opus 4.8, GPT-5.5/5.6), which came from the build instruction itself.
- `library.html` and the search index were not regenerated; the two new pages are navigation-level surfaces, not corpus documents.