# kit-validator: result ## Changed files (all inside the allowed paths) | path | sha256 | lines | |---|---|---| | `validate_kit.py` | `cea233204160f294d66328a058475f99c92b2e70a62f6701289607c744d3c3ba` | 709 | | `tests/test_validate_kit.py` | `ff95d8091f58f71d753608f6cfbe0c905fd09ab4ea4704dd79971ee5a4863411` | 518 | | `docs/validation.md` | `ed8a7e2623055f271f4b1a89da5407e187b1db3a933b1d7b86463440ae17ace7` | 135 | No other tracked file was modified. Every file header credits Taylor (gtmcknight) with amm-sim@4812a6382311624e1506aca131ca35c378f930d9 research/solved, and the report's JSON puts `_credit` first. The three files are staged but **not committed**. `git commit` failed with "Author identity unknown". I did not change git configuration, because that is outside my paths. Fixture and runtime writes went only to `scratch/kit-validator/`, which is git-ignored. Some untracked dotfiles showed up in the checkout (`.bashrc`, `.gitmodules`, `.claude/` and others). I did not create them and left them alone. ## What it is `validate_kit.py` is a standard-library CLI that checks the merged tree's structure and provenance. It works offline, downloads nothing, runs no simulator, and never imports kit code: `ammfast` is read with `ast` and is not executed. It prints deterministic JSON with `status`, per-part status (`pass`/`fail`/`missing`), row coverage, sorted findings and the sha256 of every input read. It exits 0 on PASS, 1 when anything is missing or invalid, and 2 on a usage error. It has six parts: `reference`, `kit`, `ammfast`, `engine`, `ha` and `replica`. Details are in `docs/validation.md`. In summary: - **Strict JSON.** Duplicate keys and NaN, Infinity or overflowing numbers are rejected. `_credit` must come first and must name Taylor, gtmcknight and the full pin. Rows must be exactly the seeds start..end, sorted and unique, with integer (non-bool) seeds and finite `.6f` edge strings. - **reference.** The pinned digest of `sources.json` is checked. All 35 envelopes are decoded and their envelope and content sha256 and size are checked. The qt1 and calib pins are confirmed. - **engine.** The bank is checked against the committed evidence: - the qt1 digest, including bytes decoded from the envelope; - `pins.sh` (engine commit, native .so sha256, wheel version); - every download's sha256 and URL against `bootstrap/manifest.tsv`, including the release wheel; - the `engine_eval.py` and `bootstrap/smoke_engine.py` digests; - `workers == 8`, config seeds in row order, and `configs_sha256` recomputed; - the audit: unrounded float values, `.6f` rounding, nonnegative integer gas counters, all error counters zero, and afterSwap peak gas equal to the bank; - `validation.json`: the bank, audit, validator and inventory digests, and reruns equal to the audit values. - **ha.** The calib file is decoded from its envelope and rows are selected by explicit `kind == "HA"`, unique per seed. Every bank edge must equal `format(source edge, ".6f")`. Provenance must state the selection, edge field, range and counts. - **replica.** Nine banks are required, and the `validation.json` contract is enforced: - `bindings` must cover `sources.json`, all nine banks and every `ammfast/**/*.py`, with digests recomputed. Path escapes, symlinks and self-binding are rejected. - `gates` must hold exactly 384 records and `spots` exactly 576. - `|cpython - ammfast|` is recomputed and must be <= 1e-9. The reported `abs_error` must agree with it within 1e-12, and `.6f` of `ammfast_edge` must equal the bank edge. - Timings must be finite, with positive per-policy rates and a nonnegative cold-JIT time. There is no speed threshold. ## Commands run and what they reported 1. **Benchmark first.** `python3 -I scratch/kit-validator/bench.py` parsed and hashed the engine bank, audit, validation, HA bank and `sources.json`, and decoded the calib envelope. It took **0.038 s** in total. All digests matched the recorded values: bank `f1031aae…`, audit `5256d2bd…`, `sources.json` `62147ff1…`, calib sha256 OK. `engine_eval.py`, `smoke_engine.py` and `validate.py` all equal the digests recorded in the committed provenance. 2. `python3 -B validate_kit.py --timing` on the starting trunk (57ce0df) **exited 1** with status FAIL: - `reference`, `kit`, `engine` and `ha` passed. - `ammfast` was **missing**: `ammfast/` is absent. - `replica` was **missing**: all 9 `baselines/replica/*.json` banks and `baselines/replica/validation.json` are absent (10 missing-file findings). - Coverage was **2560 / 14848 rows verified**: engine 512 and HA 2048. All 12288 replica rows are missing. - Per-part seconds: reference 0.031, kit 0.002, engine 0.014, ha 0.048, ammfast and replica about 0. A full run took 0.12 s of wall time. 3. `python3 -B validate_kit.py --parts reference,kit,engine,ha` **exited 0** (PASS). 4. `timeout 470 python3 -B -m unittest tests.test_validate_kit` ran **57 tests, all OK, in 3.3 s**. - The first run had one failure, in `tests/test_validate_kit.py::test_gate_missing_extra_and_duplicate` (line 314). The fixture's extra gate copied an edge that the validator correctly rejected as `spot-format` before the coverage check ran. I fixed the test to use an in-bank edge, so it now isolates the coverage failure. The validator was not changed for this. - The adversarial cases cover: - missing bank, missing row, missing audit and missing validation; - duplicate, unsorted and out-of-range seeds at 59999, 61024 and 9700512; - bad edge formats, bool seeds, wrong policy or backend; - `_credit` not first, a short pin, duplicate keys, NaN/Infinity/-Infinity/1e999; - stale digests: a bank edited after binding, an ammfast edit, an unbound ammfast file, an `engine_eval.py` edit, the engine validation digest, the wheel sha and the configs sha; - path escapes (`..`, absolute, `./`, backslash, a symlink, empty) and self-binding; - the 1e-9 boundary (2^-30 passes, 2e-9 fails), a mismatched `abs_error`, gate and spot coverage, and record types; - timings: negative, zero, bool, string, missing, plus a tiny rate that passes; - ammfast API signature, re-export, header credit and syntax; - kit file missing or without credit; - engine gas types (-1, 1.5, true, a string, null), error counters, gas mismatch and rounding; - HA mapping, selection kind and missing row; - inventory and envelope tampering; - deterministic output, real-trunk honesty, CLI usage errors and `--out` confined to `scratch/`. 5. The existing suites still pass with these files present: - `tests.test_engine_eval` OK - `tests.test_gatep` OK - `tests.test_ha` OK - `tests.test_score` OK - `tests.test_setup_fast` OK (1 skipped) ## Coverage now | evidence | status on 57ce0df | |---|---| | reference inventory + 35 envelopes | complete, pass | | score.py, setup_fast.sh, gatep.py (exist, credited, parse) | complete, pass | | engine qt1 bank 9700000..9700511 (512 rows) + audit + validation | complete, pass | | HA 10000..12047 (2048 rows) vs calib kind=HA | complete, pass | | ammfast/ API sources | **missing** | | 9 replica banks (12288 rows) + replica validation.json | **missing** | A full PASS needs the merged wave. I make no claim about the ammfast or replica work being written in parallel, and I did not read or import any of it. ## Limitations - The checks are structural and about provenance only. Gate and spot errors are recomputed from values the replica worker reported, not from fresh CPython or ammfast runs, so a PASS is **not** independent parity proof and does not certify code written at the same time. - The `ammfast` API check is static (via `ast`): it confirms the signatures and that the policy ids are present, not their behavior. - Replica bank provenance content is only required to be a nonempty object, with an optional `seed_range` that must match the file name. The contract does not fix its fields. - The real-trunk replica checks have only been exercised on synthetic fixtures, because the replica outputs do not exist yet. - `docs/validation.md` lists future simulator commands (`setup_fast.sh`, `materialize.py`, `gatep.py`, `baselines/engine/validate.py`, and an independent CPython replica2 vs ammfast resample). None were run here. ## Resume There is no long-running state. To resume, rerun `python3 -B validate_kit.py [--parts …]` and `python3 -B -m unittest tests.test_validate_kit`. To land the change, run `git commit` with an author identity configured; the three files are staged.