Provenance and audit trail

Leadership-site v2 - groundedness & decision-use build audit

Date: 2026-07-15 (America/New_York) Input: a local copy of the archive only, plus sources retrievable from URLs named inside it. No other folders were read; the domain reports paths referenced inside the corpus were not opened. Output: site/ (deployable), pipeline/ (full reproducible audit pipeline), site.zip.

1. The inferred goal

The archive is a self-contained leadership site for the AutoQA Foundation 0.1.0 - the case, architecture, pilot design, research corpus (132 sourced claims across 10 domain reports), and a validated contract/benchmark package. What it argued for was already strong; what it could not yet do was (a) let a reader audit any number down to the source quote, and (b) turn the open questions into an actual decision instrument. Both stated concerns - groundedness, and the ability to make decisions on it - map to those two missing layers. Mid-build, a third requirement was added explicitly: verified is not enough - claims must be usable under current capabilities (Claude Fable / Opus 4.8 class, GPT-5.5 / 5.6 class), and old-but-true measurements must not masquerade as current signal.

2. What was built

evidence.html + assets/claims.json - the evidence ledger. All 132 corpus claims, each with: - the one primary source the corpus names (title, URL, date, type; peer-review marker where derivable); - Layer 1 - the research sweep's own 2026-07-14 verification transcripts (27 claims; preserved verbatim, including its corrections); - Layer 2 - an independent re-check run 2026-07-15 for this build: every claim checked against only the URL the corpus names (plus served redirects / same-document arXiv variants), verdicts derived exclusively from retrieved text, verbatim supporting quote per claim, per-claim list of asserted specifics not visible in retrievable text; - Layer 3 - a decision-use grade (see Sec. 4); - same-source groupings ("repetition across domains is not corroboration"), and bidirectional "used on" links to every page location that cites it. Plus: the seven "no evidence anywhere" gaps as first-class objects (each assigned to the experiment that closes it), the full per-domain gap and disagreement inventories (preserved, not averaged away), and the package-integrity record.

decisions.html - the decision register. 5 leadership authorizations (A1-A5), 6 open owner calls (D1-D6, from the philosophy doc's Sec. 8), 5 standing defaults with named falsifiers (C1-C5, including the honestly-flagged single-under-review-study dependency), the 7 experiments that settle them (E1-E7) with a decisions<->gaps<->experiments cross-map, and the non-negotiables section so meetings don't re-litigate encoded constraints. Every evidence row is a graded chip into the ledger; every decision states options, recommendation, reversibility, dependencies, and what would change the answer.

Provenance wiring. Every quantitative claim on index/brief/research/pilot/architecture now carries a claim-ID chip (with its grade) that resolves to the ledger entry; the pilot risk register gained a Ledger column; the launch gates and claim-ontology table are chipped; nav and footer across all 49 pages gained Decisions + Evidence.

Decision-use grading is summarized on Research, and each claim's grade is in the Evidence ledger.

5. Infrastructure corrections

• Stale manifest (integrity defect in the shipped archive): site-manifest.json recorded wrong hashes/sizes for assets/site.css, assets/site.js, README.md, and omitted the fonts and favicon entirely. Regenerated over all 108 shipped files; discrepancy disclosed on the ledger's integrity section.
• Foundation package validated: python3 validation/validate_package.py -> all 8 checks PASS (schemas, referential integrity, canonical hashes, positive-proof invariants). Recorded on the ledger.
• Observed, deliberately not changed: assets/search-index.json (894 KB) is referenced by no page or script - library search runs on inline attributes. Left in place; flagged here.

6. Verification of the build itself

• Link/anchor crawler: 49->50 HTML files, ~2,300 internal links - 0 broken files, 0 broken fragments (after fixing two anchor-case defects it caught).
• All JSON parses; claims.json invariants asserted (132 claims, all graded A/B/C, all externally checked).
• Render-consistency spot-check: 12/12 random claims' text, quotes, and grade attributes match the data files; chip grade suffixes consistent across all pages.
• Browser pass (Playwright): zero console errors/warnings; evidence ledger, decision register, chipped research cards verified visually in dark and light themes, desktop and mobile nav; filters and hash-open behavior work.
• Structural HTML parse of both generated pages: zero tag-balance errors.
• Independent review, two passes in fresh contexts: (1) a full seven-category review (HTML correctness, filter script, accessibility, claims.json sanity, diff hygiene, manifest hashes, leadership polish) whose shared checklist completed with no blockers filed - its narrative report had not arrived by ship time and is not claimed as sign-off; (2) a blocker-only critic pass which returned APPROVED - no blockers with per-check evidence (270-line decisions.html read; stat-strip counts match block counts 5/6/5/7; claims.json 132/A-B-C/76-40-16 verified; all claim-ref fragments resolve against evidence.html's 146 ids; CSS braces balanced 438/438 with every used class defined).
• Every patch operation used exact-match substitution with asserted match counts - nothing silently missed.

7. Reproducibility

pipeline/ contains the complete audit chain: parse_corpus.py (corpus -> 132 structured claims), batch_*.json + results_*.json (the 12 verifier work orders and their verbatim outputs), grade_claims.py (all 132 decision-use grades with rationales), usage_map.json, build_data.py -> claims.json, build_pages.py -> the two pages, patch_pages.py (asserted edits), verify_site.py (crawler + manifest). Re-running the chain end-to-end regenerates the site's evidence layer from the corpus and the recorded verification outputs; re-running the verifier batches re-checks the sources live.

8. Round 3 (2026-07-15, later the same day): the philosophical kernel

Direction change on request: no implementation steps; make the foundation airtight first - specifically a kernel upon which all things are built.

KERNEL.md (shipped at delivery root and inside site/, linked from the register and ledger): a minimal axiomatic base - one telos, four thin definitions, four mathematical FACTs, four standing domain CONDitions, nine CHOSEN commitments (explicitly chosen, never disguised as theorems), one versioned LAW element - followed by consequences with honest statuses (DERIVED / UNPACKED / DESIGN RESPONSE / governed choice), eight declared interfaces where all perishable capability facts live, five declared choice-points/holes, a dependency table, and a scope declaration with the migration principle (the kernel governs the non-certifiable residue of open-world evaluative work; whatever gains a certificate migrates to the deterministic lane - so model progress moves the boundary, never the sentences).

Adversarial process, on the record (five reviews, three versions): v0.1 was attacked by three fresh-context critics - two Claude-family and one GPT-5.6-class (codex, gpt-5.6-sol, xhigh reasoning). Codex round 1: severe - no derivation fully entailed, axioms falsified by formal methods, one legal contradiction, six missing elements, a self-referential discipline defect -> v0.2. The two Claude reports (0 FATALs each; 6 and 9 MAJORs) converged independently on the same deep findings plus new ones: Part III necessity-rhetoric re-smuggling chosen values; the shared-substrate/tooling-parity premise and human-lane independence as contingent empirics posing as walls; the coda's literal falsity. A codex confirmation pass on v0.2 returned "STILL BROKEN" - FACT/COND layer still overclaiming, no DERIVED item fully entailed, governed-choice discipline unapplied - with exact repairs per finding. v0.3 implements those repairs and the Claude critics' residuals: the FACT layer shrunk to five limitative facts; substrate parity, independence scarcity, and the "permanent frontier band" demoted from walls to measured presumptions (the permanence claim retracted - what remains permanent is the measurement obligation); the human-decision floor made global by explicit choice (K22); fairness (K23) and exculpatory-only reopening (K24) added as owned commitments; every choice-point given a named owner and version obligation. A bounded codex delta-check verified repair implementation on v0.3 (result recorded in pipeline/). All five review reports and both superseded kernel versions ship in pipeline/ - the kernel's own standing rule applies to itself: re-attacked at every version bump and each model generation.

Round-2 durable anchors (under the "relevant and certain" bar): DA-02 Uniform Guidelines confirmed verbatim from eCFR; DA-01 NYC Local Law 144 partially confirmed from the DCWP rule-adoption document (substance confirmed; admitted with disclosure); DA-03 Holmstrom-Milgrom partially confirmed (bibliographic identity verified; proposition paywalled - admitted with that disclosure); DA-04 ANSI/ASQ Z1.4 not retrievable (ASQ bot-wall) - quarantined in the visible candidate queue, not admitted. The ledger now carries 132 corpus claims + 3 round-2 anchors, provenance disclosed on each.

Register tranche: D1 gained the worked prevalence-arithmetic panel (labeled derivation, not literature); D5 became a two-sided (EU + US) legal gate; A4 and the pilot's human-loop gate carry the honest human-lane expectation sentence; Section D gained the E2-first sequencing note (recommendation only - nothing operational authorized) and the bakeoff callout; decision-record.json v1.1.0 carries the kernel reference, standing rule, sequencing note, and two-sided legal gate.

9. Round 4 (2026-07-15, evening): presentation rebuilt as a decision document; full ASCII normalization

Direction: the earlier presentation read like marketing - stat tiles, badge chips, taglines. Rebuilt all seven leadership surfaces as quiet, executive-grade documents: ask-first, prose-led, numbers stated in sentences with their caveats attached, every assertion carrying a bracketed reference that opens the ledger entry (source, verification, quote, decision-use grade). Evidence-based decision-making is the organizing principle of the presentation, not a feature of it. The library and document viewers keep their reference layout.

Also in this round: - Full ASCII normalization across the delivery (pages, ledger data, kernel, pipeline records): typographic characters transliterated, brand and UI glyphs replaced with text, zero non-ASCII bytes remaining in shipped text files. Exemption: documents/raw/foundation/** stays byte-exact so its own hash validators keep passing; the research corpus was already ASCII-normalized by its authors and needed only minor cleanup. - Robustness check, published on the research page: striking every LLM experimental result measured on pre-2025 models (16 findings outright, 7 reduced to current-model halves) leaves 9 of 11 register decisions untouched; the other two keep standing on law, mathematics, or current-generation results. The kernel is independent of model experiments entirely - stated as a checkable property in its Part VI. - Kernel v0.3.2 (commentary only): the motivating problem - human reviews of human-labeled data not reaching consensus - is named inside K0's commentary and answered by existing kernel content: non-consensus has four causes needing four different fixes; targeting consensus directly treats all four by erasure and caps validity at reviewer reliability; consensus in application rises as a byproduct of removing illegitimate causes. The overview and summary pages lead with this framing. - decision-record.json v1.2.0; reference usage now recorded by the generator itself (site_usage.json), keeping ledger back-links exactly synchronized with what the pages actually cite.

10. Round 5 (2026-07-15, late): the long-context retrieval question

Question raised: is an LLM incapable of reliable long-context verification - a silent miss of in-record evidence corrupting approvals? Answer, now encoded: the logical half was already in the kernel (record-level claims are universals, only as good as the search behind them), but the practical half deserved its own elements. Kernel v0.3.3 adds P5 (single-pass reading of long records is unreliable retrieval - degrades with length and semantic distance, and the failure is silent; a presumption with a measurement obligation, not a wall, because bounded records are exhaustively checkable by procedure), a K14 coverage-accounting clause (record-level claims extend no further than the coverage achieved), D9 (no single-pass certification beyond measured reliable length; discovery as a separate auditable step; planted support AND counter-evidence recall by length, position, semantic distance), and interface I10. Two structural notes: the three-way verdict semantics already protect workers from the missed-support direction (unsupported triggers an evidence request, never an auto-fail; only found contradiction fails anyone), so the silent-miss harm concentrates in false passes over buried counter-evidence - exactly what the confident-pass audit samples, now priced by the planted-counter-evidence recall number. System page, research page, and the pilot risk register carry the corresponding sections.

Part VI - Changelog

Meta-analysis; untagged.

v0.1 -> v0.2 (after the first GPT-5.6-class attack): scope + migration principle added; COND tier introduced; construct circularity removed; dial demoted from theorem to unpacking; the platform-work contradiction fixed; evidence integrity, specification-risk allocation, measurement/attribution/sanction separation, symmetric threat model, construct-validity duty, and trust termination added; discipline category (e) created; minimality rhetoric replaced by a dependency table.

v0.2 -> v0.3 (after two Claude-family reviews of v0.1 and the GPT-5.6-class confirmation pass on v0.2, verdict "still broken"): the FACT layer shrunk to five limitative facts, each stated at the width its grounds support (the "cannot be more reliable" ceiling, the universal witness claim, the "only through sampling" clause, and the "jointly necessary" dichotomy all corrected); the shared-substrate premise, the independence-scarcity trend, and the permanent frontier band demoted from walls to measured presumptions (P3, P4, I9) - the permanence claim retracted and replaced by a standing measurement obligation; the human lane's value split into durable grounds (authority, contestability, law) and a measured ground (independence, I9); the underdetermine disposition, global human-decision floor (K22), fairness (K23), and reopening rules (K24: worker-favorable freely; the reverse only through K18 proceedings) made explicit CHOSEN commitments instead of derivation by-products; every Part III item re-statused per the confirmation pass's exact repairs (D3 split, U1->D4 as design response with a holistic-claim residual, U2 tripartite with categorical floors above prices, U3 as governed choice with the derived anchors-are-not-true constraint, U4 conditional-plus-expectation instead of "permanent," U5 admitting transparent recalibration, D7 narrowed, D8 split with K24); every choice-point given a named owner; the dependency table retitled as a rationale map and completed for K1-K4; Part II/VI exempted from the tagging rule explicitly, closing the self-referential violation.

Independence from model experiments, stated as a checkable property: strike every LLM experimental result from the record - any model generation - and Parts I-III do not change. The facts are mathematics and logic; the presumptions are warranted by the domain and carry measurement obligations; the commitments are owned choices; the discipline bars derivations from citing benchmarks. The old-model literature shaped the path of discovery - it suggested which constraints to formalize - but it carries none of the justification. The one deliberate retention: adversarial results of any age remain valid attacker floors, since capability asymmetry only strengthens them.

v0.3.3 -> v0.3.4: P5 rewritten to name the three distinct discovery failures (dilution, recognition, composition) and to state that chunking addresses only the first and that chunk ledgers over the other two are false confidence; D9 rewritten around detector-relative coverage with the substrate-first design order (structured event indexing, citation contracts, state-carrying scans, an honest residue lane); I10 tightened to require distributed and semantically distant plants and to bar lexical-needle scores as coverage evidence. Prompted by the objection that a model cannot reliably match semantically distant but functionally accurate claims against a 500k-token transcript, and that chunking invites false confidence - both points conceded and encoded.

v0.3.2 -> v0.3.3: one presumption, one protocol clause, one consequence, one interface - all in response to the silent long-context retrieval question: P5 (single-pass reading of long records is unreliable retrieval, presumed until measured), the K14 coverage-accounting clause (record-level claims extend no further than coverage achieved), D9 (record-coverage discipline: separate auditable discovery step beyond measured reliable length; planted-evidence recall; the false-pass concentration argument), and I10 (the measurement slot). The dependency map gains the corresponding rows.

v0.3.1 -> v0.3.2: commentary and meta-analysis only; no tagged element changed. The motivating non-consensus problem is named inside K0's commentary and answered by existing kernel content (four causes; erasure versus explanation; consensus in application as a byproduct, never a target), and the model-experiment-independence property above is recorded.

On K25: per that commitment, this version was delta-checked against the confirmation pass's prescriptions (verdict: 21 implemented / 8 partial / 0 unimplemented, 2 new majors - all folded into v0.3.1); the next full hostile pass is owed at the next version bump or model generation, whichever first. v0.3.1 does not claim to have survived a review it has not had.

Earlier editions and working notes

Source files

__pycache__/
*.pyc
.DS_Store
.vercel

.vercel

User-agent: *
Allow: /

{
  "cleanUrls": true,
  "headers": [
    {
      "source": "/assets/(.*)",
      "headers": [
        {
          "key": "Cache-Control",
          "value": "public, max-age=3600"
        }
      ]
    },
    {
      "source": "/documents/raw/(.*)",
      "headers": [
        {
          "key": "X-Content-Type-Options",
          "value": "nosniff"
        }
      ]
    }
  ]
}

Redactions in published copies

Some published copies describe organizations and their work in general terms and replace links with [link withheld]. They also reword one term, a few phrases and local paths. The table counts each file.