docs/validation.md

Validation ledger

Setup baseline

See SETUP.md for Nix/direnv, Xet upload/recovery, preservation checks and the original 28 setup regressions. Linux and Intel macOS execution remain unverified.

E01 source mapping — 2026-10-21

See the experiment for commands, scope and limitations. The static scanner adds nine synthetic regressions, bringing the combined suite to 37. Full-corpus checks additionally revalidate five archives/all extracted file hashes, resolve every Core constant, require a matching wrapper signature for all Core interrupt sites, check include candidate closure, and compare generated indexes against report identities.

Code-only tests do not need original assets. The full map_references.py --check command requires hydrated references and checks reproducibility of the Xet report and both generated Markdown indexes.

Host synthetic suite: 37 passed; the code-only Apple Silicon Nix check also built successfully with all 37 tests. The Nix check-reference-map command passed against the hydrated corpus. Full report/index regeneration followed by --check passed. All 560 extracted inputs were hash-verified; 487 include candidates and every Core declaration/wrapper across five builds satisfied the closure checks. A historical Brush-return signature case has its own regression.

The current results establish static mapping only. No runtime API contract, format round trip, editor workflow, executable startup, graphics compatibility or Rust parity is established by E01.

E02 managed converter — 2026-10-21

Experiment: both original binary identities and all generated decompilation/metadata/capture hashes verified. Original executables ran unchanged under Mono 6.14.1 on macOS ARM64: 43 cases for 1.4.0, 58 for 1.5.0 (101 total), with expected exits and semantic/log assertions passing. Fifteen authored network-free tests bring the suite to 52, including every prefix truncation of one representative DAT file. Runtime failures intentionally expected by cases are retained as evidence, including successful corrupt reads. Linux/Windows execution, native editor saves/loads, independent consumer interoperability and Rust parity are still open.

E02 final checks: all 52 tests passed on the host and in the Apple Silicon code-only Nix derivation; converter evidence/semantic checks and E01 source-map reproducibility passed; all five original archives and all 560 extracted files passed preservation verification. Staged storage policy and hydrated asset hashes were checked before committing.

E03 Rust DAT codecs — 2026-10-21

The Rust compatibility profile and experiment document the implementation and its limits. Debug and release each pass 24 unit tests, five integration tests and one doctest. All 58 captured converter DAT outputs round-trip exactly; 11 captured malformed/ambiguous inputs are rejected; cross-version autodetection and property-free migration are covered. Boundary tests include 65,535/65,536 counts/indices and 5,000 bounded deterministic mutations. Formatting and Clippy with warnings denied pass.

Both Apple Silicon Nix derivations built successfully: the asset-free Rust check runs 24 unit tests/one doctest and lint/format checks; the setup check retains all 52 passing Python tests. Full fixture tests ran separately against the SHA-256-verified E02 corpus. Source/asset storage boundaries are preserved; no fixture payloads moved into ordinary Git. Native-editor interoperability, other-platform execution and coverage-guided fuzzing remain open.

E04 TXT and native CLI — 2026-10-21

TXT compatibility, CLI workflow and E04 record the implementation. Full debug and release suites each pass 58 Rust tests: 44 unit tests, 12 fixture/process integration tests and two doctests. All 58 DAT outputs survive exact DAT→TXT→DAT reconstruction through the real executable, seven converter TXT outputs canonicalize stably, 16 direct TXT→DAT pairs match and nine invalid/lossy TXT source fixtures are rejected. The TXT suite adds 3,000 bounded mutations and streaming/array/encoding limits; CLI tests cover publication races, aliases, symlinks, no-clobber and preserving existing destinations on encoder failure.

Formatting/Clippy pass. Both Apple Silicon Nix checks and the asset-free nanobrush package build pass; the Rust check runs 44 unit tests/two doctests, while setup retains all 52 Python regressions. The packaged executable’s help and JSON inspection of the property fixture ran successfully. Converter fixture identity/semantic checks and all 1,082 hydrated asset hashes pass; original captures are unchanged and tests/code remain ordinary Git. Other-platform execution, native-editor interoperability, crash-durability/failure matrices and coverage-guided fuzzing remain open.