docs/experiments/rust-dat-2026-10-21.md

E03 — Safe Rust DAT codecs

Date: 2026-10-21. Scope: implement the two E02 converter DAT layouts as an independent Rust reader/writer, validate fixture parity and deliberate rejection policies, and retain broader editor/format work as open tasks. See F29 for the public API, complete compatibility matrix and resource accounting.

Inputs and identity

No assets were changed or regenerated. The 58 captured converter DAT outputs and selected source inputs come from E02's existing Xet tree. The full E02 probes.json SHA-256 is d69e2a1e1f61af71fa345e2cab3063b35ed4a58979293d89aec0aec5e948c11d; it pins per-file identities and converter binary hashes. analyze-converter check reverified that manifest, original binary identities and semantic assertions before Rust fixture tests.

The implementation is the dependency-free mb3-dat crate in the committed Cargo workspace. Original executables/decompiled code are not Rust dependencies. No original executable was executed by E03. Code and tests are ordinary Git; input fixtures stay in their existing Xet locations. No generated binary fixture bytes were embedded in Rust source; synthetic tests construct small fields or boundary records programmatically.

Commands and observed results

Host: macOS ARM64, pinned Nix Rust/Cargo 1.99.0. Tests run entirely offline after entering the development environment:

cargo fmt --all --check
cargo clippy --workspace --all-targets --locked --offline -- -D warnings
analyze-converter check
cargo test --workspace --locked --offline
cargo test --workspace --release --locked --offline
test-setup
nix build .#checks.aarch64-darwin.rust .#checks.aarch64-darwin.setup --no-link

• Debug and release each pass 24 Rust unit tests, five fixture integration tests and one doctest. All 58 converter output files re-encode byte-for-byte. Eleven malformed/ambiguous captured inputs are rejected; two captured cross-version inputs are accepted through version detection. Property-free computed migration matches the opposite converter's exact output; mixed nonempty brush/patch worlds match independently composed captured record bytes; a downgrade containing properties fails explicitly.
• Synthetic tests cover compact count/index widths through 65,536, shared contours, patch dimensions, properties/UVs, every prefix truncation of both rich documents, count/grid overflow, ASCII/NUL policies, finite bit preservation/non-finites, resource accounting, short/interrupted I/O, write failures and no destination writes on validation failure. Five thousand deterministic mutations under tight budgets produce either an error or a stable canonical round trip, with no panic in these runs.
• Clippy and formatting pass. The separate asset-free Rust Nix derivation runs Clippy/formatting, 24 unit tests and one doctest. The setup Nix derivation passes all 52 Python regressions and source-tree storage checks.

The fixture suite fails when assets are missing; cargo test --workspace --lib --locked --offline and cargo test --workspace --doc --locked --offline are the intentional asset-free subset. Compile-only Clippy checks all targets without opening fixture files.

Interpretation and limits

This proves serializer compatibility for the retained converter corpus, including exact scalar/UV ordering and old/new hierarchy differences. It does not validate native-editor save/load, convex geometry, computed patch evaluation, all legacy malformed inputs, every possible float pattern or every document size. In particular, the old signed-16 writer repair is covered by Rust boundary tests and uses the recovered old reader's existing four-byte branch; direct large-count legacy execution remains open.

The reader deliberately rejects rather than reproduces stale-buffer reads, unterminated-string growth, duplicate slots, high-bit string replacement, reserved flags/selector 3, non-finite numbers and trailing data. It preserves unused property names and original index meaning instead of the converter CLI's usage optimization. These are the fixed compatibility policy in F29, not unexplained regressions.

The four declared Nix systems are evaluation targets; execution in this stage is macOS ARM64 only. Real documents, Linux/Intel execution, minimum-toolchain validation, coverage-guided fuzzing, independent consumers, atomic file workflows and the remaining codecs/CLI remain in P6.15/P6.19/P6.20. P6.21 records the completed DAT implementation without closing those broader tasks.