| --- |
| license: mit |
| library_name: pytorch |
| tags: |
| - verified-units |
| - reed-solomon |
| - erasure-coding |
| - deduplication |
| - storage |
| --- |
| |
| # neural-storage — self-healing vault with a verified GF(256) core |
|
|
| **Repositories:** [GitHub](https://github.com/quzi93/neural-storage) · [🤗 HuggingFace](https://huggingface.co/NeuralVerified/neural-storage) |
|
|
| A content-addressed, deduplicating, **self-healing** storage vault. Any *k* of *n* |
| Reed-Solomon shards reconstruct the whole, so pieces can be lost or corrupted and |
| the data survives. The erasure-coding arithmetic — GF(2⁸) multiply — is provided |
| by neural `LOG`/`EXP` units verified **bit-exact over all 65,536 (a,b) pairs**, |
| the same N/N discipline as |
| [neural-aarch64-units](https://huggingface.co/NeuralVerified/neural-aarch64-units). |
|
|
| > **Honest by design:** dedup removes *redundant* data; RS *adds* redundancy for |
| > resilience. Incompressible data never shrinks — you cannot beat entropy. Chunk |
| > hashes are real SHA-256, not a neural emulation. |
|
|
| ## What's verified |
|
|
| - `GF256` (`LOG`/`EXP`): GF(2⁸) multiply — **composed multiply 65536/65536** |
| - Reed-Solomon any-*k*-of-*n* recovery (every loss pattern) |
| - `Vault`: dedup + RS shards on disk, survives deleted/corrupted shards, `heal()`s |
| - `VaultFS` + optional WinFsp drive-letter mount |
|
|
| ## Use |
|
|
| ```bash |
| pip install torch |
| python step1_storage.py # verified GF(256) + dedup chunk store |
| python step2_rs.py # Reed-Solomon recovery |
| python step3_vault.py # self-healing vault |
| |
| python cli.py store <vault> <folder> |
| python cli.py export <vault> <folder> # RS-healed reconstruct |
| python cli.py mount <vault> X: # requires WinFsp |
| |
| # image a whole drive/partition into a self-healing .pt (may need admin): |
| python cli.py image \\.\C: diskC.pt |
| python cli.py image-verify diskC.pt |
| python cli.py image-restore diskC.pt out.img |
| ``` |
|
|
| Weights: `GF256.pt`. |
|
|
| **Create your own verified unit** (template: `storage/gf256.py`): write the exact |
| golden finite function → enumerate the domain (decompose big/linear ones into |
| bit/byte slices, see `storage/rs.py`) → `common.train` → `common.verify` must be |
| bit-exact on 100% of inputs → compose. `step1_storage.py` shows the full loop. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{byrne2026neuralstorage, |
| title = {neural-storage: Self-Healing Erasure-Coded Vault with a Verified GF(256) Core}, |
| author = {Byrne, Dean (Quazim0t0)}, |
| year = {2026}, |
| howpublished = {\url{https://huggingface.co/NeuralVerified/neural-storage}} |
| } |
| ``` |
|
|
| **Dean Byrne (Quazim0t0)** · 2026 |
|
|
|
|
| --- |
|
|
| <!-- neuralverified-relocation-note --> |
| > **Now hosted by [NeuralVerified](https://huggingface.co/NeuralVerified).** |
| > |
| > This repo was moved into the NeuralVerified organization to help organize my profile. |
| > Originally published at [`Quazim0t0/neural-storage`](https://huggingface.co/Quazim0t0/neural-storage). |
|
|
