| # Proposed technical direction |
|
|
| Storage, environment, hydration, and extraction are implemented; see [SETUP.md](SETUP.md). The `mb3-dat` and `mb3-txt` crates plus `nanobrush-cli` now implement native TXT/DAT format handling ([compatibility profile](RUST_DAT.md)); the broader Rust/rendering/conversion boundaries below remain proposed and must be validated through the [TODO phases](TODO.md). |
|
|
| ## Storage and repository boundaries |
|
|
| Follow the active `urban-terror-rs` pattern selected by the user: one Hugging Face Git repository, normal Git source/tests, and Xet-backed assets and references. The initial separate-dataset proposal is superseded. |
|
|
| ```text |
| nanobrush/ # remote hf: git@hf.co:nmarshall/nanobrush |
| .envrc, flake.nix, flake.lock # pinned my-nix environment |
| .gitattributes # code filter; assets/references use LFS + Xet |
| .githooks/ # staged-object/history guards plus LFS pre-push |
| config/references.json # authored preservation metadata; ordinary Git |
| scripts/ # setup, storage, hydration, extraction, checks |
| tests/ # ordinary Git test source |
| docs/ # ordinary Git authored findings/specifications |
| crates/ # native DAT/TXT libraries and nanobrush CLI |
| assets/references/ # every file is Xet-backed, including small text |
| archives/ # unchanged ZIPs |
| provenance/collection/ # original reports/manifest/checksum files |
| provenance/historical/ # historical pages/indexes |
| extracted/<build>/ # immutable raw members, including binaries |
| inventories/<build>.json # generated metadata payloads |
| SHA256SUMS # relocated archive paths |
| ``` |
|
|
| Any additional `references/` directory, including root `references/` or nested `docs/references/`, is also LFS-filtered. Keep authored source/tests/docs outside these payload directories. All pre-existing collection downloads have been relocated beneath `assets/references/` and byte-verified. |
|
|
| There is no nested `.git`, dataset, separate asset branch, or asset lockfile. Each normal Git commit pins asset content through its LFS pointer SHA-256 values. `assets-fetch` hydrates the checked-out revision; `assets-check` verifies working bytes against staged pointers. |
|
|
| Use `filter=lfs` attributes plus Git Xet's custom transfer registration. All reference resources intentionally follow this policy regardless of size. Commit/push guards check staged and historical objects so raw payloads cannot slip into ordinary Git, even when a clean filter is bypassed. The LFS-specified empty-file representation carries no payload bytes. |
|
|
| Build outputs and disposable runtime copies remain ignored. Keep immutable references unchanged; transformations write separate derived assets and provenance. Nix derivation source filesets exclude payloads, while Git-backed flake source imports can still contain hydrated assets; use a skip-smudge clone for code-only CI. |
|
|
| ## Environment shape |
|
|
| Use `flake-parts` and `github:nmrshll/my-nix`, following the current local exports `flakeModuleGroups.essentials` and `flakeModules.rust`. Keep nixpkgs and flake-parts follows consistent. Rust checks and the CLI package use the same pinned toolchain and explicit asset-free source fileset. |
|
|
| Pin remote inputs in `flake.lock`. An optional developer `.envrc` helper may watch local `my-nix` / tools checkouts and pass overrides without rewriting the shared lock; the committed flake remains usable without those checkouts. Separate platform-specific graphics libraries from portable archive/source tooling. |
|
|
| Provide a lightweight preservation/research shell first. Add the Rust workspace, native graphics dependencies, codec tools, and analysis tools as needed. Keep Windows/Wine/.NET oracle requirements in a separate documented environment; native Nanobrush must not inherit them. |
|
|
| ## Planned commands and responsibilities |
|
|
| Setup/storage/hydration/reference commands and a converter-only Mono oracle are implemented. The native CLI supports inspect/validate/convert for TXT and both DAT versions, with a Nix package; other formats and editor/replay interfaces remain future work. The remaining asset-conversion, general oracle/replay and GUI-packaging interfaces below remain proposed. |
|
|
| | Command | Inputs | Output and acceptance contract | |
| | --- | --- | --- | |
| | `setup-repository` | Current checkout and committed policy | Repository-local filters/hooks; detect conflicting setup; no global config edits | |
| | `check-storage` | Index/history or source export | Reject code payloads, invalid pointers, unsafe entry types, or missing policy | |
| | `assets-check` / `assets-fetch` | Same-repo staged pointer identities / checked-out revision | Verified LFS/Xet setup and checksummed hydration; no separate asset checkout | |
| | `verify-references` | Pinned archives and original manifest | Original-byte hash/CRC results with missing/pointer-stub failures | |
| | `unpack-references` | Manifest, selected versions, explicit destination | Traversal-safe deterministic extraction into `assets/references/extracted/`; no execution | |
| | `index-references` | Archives/extractions | Full inventory, hash/version matrix, include/resource graph, missing dependency report | |
| | `diff-references` | Two or more build inventories | Added/removed/renamed/changed candidates with separate semantic review status | |
| | `recode-reference-text` | Raw file plus declared encoding recipe | UTF-8 derivative and source/output hash mapping; raw bytes retained | |
| | `convert-assets` | Named recipes and pinned inputs | Native model/image/font/config derivatives with provenance and validation | |
| | `oracle-run` | Pinned isolated oracle and experiment recipe | Captured commands, logs, exit status, state/file diffs, permitted fixture/capture payloads | |
| | `nanobrush inspect/validate/convert` | Legacy/native document and explicit format flags | Rust format handling with structured diagnostics/loss report and bounded processing | |
| | `nanobrush replay` | Native document and command/input trace | Deterministic semantic result plus state/geometry/render summaries | |
| | `check-environment` / `probe-renderer` | Host/toolchain and synthetic scene | Tool/SDK/native-library and adapter/backend/feature/limit report, no false support claims | |
| | `package` / `verify-package` | Git revision, target, release profile | Native package, checksums/notices, clean-machine launch and asset-lookup checks | |
|
|
| Use Python for archive handling, inventory/provenance, and process orchestration; standard-library ZIP/hash/JSON support is sufficient for initial extraction. Put canonical geometry and document parsing/conversion in Rust and call its CLI from scripts to avoid two competing implementations. Pin any third-party image/font/recode tooling actually selected. |
|
|
| Every conversion recipe identifies its input hashes/version, destination, tool/code revision, options, source/destination encoding or color/coordinate conventions, output hashes, license/disposition, and validation rule. Hash inputs and recipes for incremental rebuilds; generate atomically and report stale/unexpected outputs. A `force` operation must have a bounded owned destination. |
|
|
| ## Rust module boundaries |
|
|
| Suggested crate names are provisional; combine them if separate crates would add overhead without isolation value. |
|
|
| | Boundary | Owns | Must not depend on | |
| | --- | --- | --- | |
| | `nanobrush-geometry` | Planes, brush construction, CSG operations actually specified, Patch9, triangulation, bounds/intersections | UI, GPU, filesystem state | |
| | `nanobrush-document` | Stable world/object IDs, properties, visibility/selection, native schema and migrations | Bevy transient entity identity | |
| | `nanobrush-commands` | Transactions, tool-independent edits, undo/redo, dirty state, deterministic replay | Widgets or render passes | |
| | `nanobrush-formats` | MB3 TXT/DAT, MAP/VMF, OBJ export, IXModel conversion, diagnostics/loss reporting | Original converter executable | |
| | `nanobrush-assets` | Asset manifests, recipes, validation, runtime lookup | Mutable reference originals | |
| | `nanobrush-editor` | Input/tool state machines, views/cameras/grids, UI, settings, native file workflows | Undocumented host-global behavior | |
| | `nanobrush-render` | Bevy meshes/materials/WGSL, diagnostic modes, overlays, Solari integration, capability policy | Authoritative editable document storage | |
| | `nanobrush-cli` | Inspection, conversion, validation, replay and headless integration | A GPU for ordinary document work | |
| | `nanobrush-app` | Native startup, plugin composition, lifecycle, packaging integration | Windows, .NET, IXPL runtime | |
|
|
| The authoritative document lives outside render entities. Commands create transactional changes; background geometry jobs use document revision IDs and discard stale results. Stable object/face identity maps render meshes and pick results back to the document. UI preview is cancellable and creates a coherent undo step only on commit. |
|
|
| Separate document data from user preferences, per-view state, global UI layout, and ephemeral tool state. Determine actual reference ownership before selecting which state is saved in which file. A future plugin interface should expose these deliberate boundaries rather than all host filesystem/process privileges automatically. |
|
|
| ## Rendering and platform strategy |
|
|
| Keep editing diagnostics in a portable Bevy path. Required modes include grid, cursor, face/border/wireframe/normal/see-through displays and selection overlays, with consistent picking regardless of illumination. Solari supplies an additional lighting path over editable brush/patch meshes, rather than becoming the document or tool engine. |
|
|
| The early spike must probe required features and limits, create a real scene, perform mesh edits, and exercise multiple views on macOS Metal and Linux Vulkan. Bevy's [tagged Solari source](https://raw.githubusercontent.com/bevyengine/bevy/v0.20.0/crates/bevy_solari/src/lib.rs) exposes a required-feature function; using it alone still needs runtime testing for limits and backend-specific failures. The [M2 upstream report](https://github.com/bevyengine/bevy/issues/26131) makes macOS adapter qualification a concrete task. |
|
|
| Maintain per-view temporal state. Geometry/material/visibility changes, undo, camera cuts, projection changes, resize, and viewport destruction must invalidate or update relevant lighting resources correctly. Measure BLAS/TLAS rebuild/update costs and memory under interactive edits. Editor grids/icons/gizmos must remain readable and must not accidentally cast lighting into the scene. |
|
|
| Use the upstream example as a spike reference for mesh/material/camera integration, but pin the selected release's API rather than copying moving `main`. DLSS is optional; document quality/performance on other hardware. Shader/material compatibility, orthographic rendering, denoising, and multiview behavior are experiments, not assumptions. |
|
|
| ## Validation layers |
|
|
| 1. **Preservation:** exact hashes, CRCs, original-byte retention, safe extraction, and remote payload recovery. |
| 2. **Behavior:** source/API coverage plus oracle experiments and semantic differential tests. |
| 3. **Core correctness:** geometry invariants, parser fuzzing, round trips, undo/replay, numerical boundary tests. |
| 4. **Interaction:** real input/focus/dialog/multiview workflows on both target operating systems. |
| 5. **Rendering:** analytic visual cases, measured tolerances, named hardware/backend results, dynamic-edit stress. |
| 6. **Distribution:** clean-machine install/launch, native runtime dependency audit, required assets/notices, offline operation, migration/recovery. |
|
|
| Keep synthetic independently authored tests available in code-only CI; run reference-corpus jobs separately when permitted and accessible. Never substitute software-rendered/headless CI for a claim that Solari works on actual target GPUs. |
|
|
