File size: 2,244 Bytes
dbd8c4c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | # The Net: release notes
This release is a **Safetensors & Xet storage experiment**. It publishes the complete The Net
architecture, with every tensor name and dimension, inside a few hundred megabytes of unique
storage. The weights are constant-initialized and shared; this is not a trained 20T checkpoint,
and the model produces no meaningful output.
## Weights
Every architecture tensor holds a single constant. Linear weights are −1/√fan_in rounded to
float32, RMSNorm weights are one and all other tensors are zero. `initialization.st` stores one
scalar per tensor, and `tensor_shapes.json` gives the full shape it expands to.
## Storage layout
| File | Contents |
| --- | --- |
| `architecture.safetensors` | All 68,464 architecture tensors, stored as reduced F16 blocks |
| `shards/**.safetensors` | 9,400 paths that share one U8 storage bank |
| `initialization.st` | Constant value of each architecture tensor |
| `tensor_shapes.json` | Full architecture shapes |
| `model.safetensors.index.json` | Index in the `the-net-shared-aliases-v2` format |
The shard paths point to the same Xet object, so the bank is stored once however many paths
reference it. The index is not a generic Transformers sharded checkpoint: it marks the bank
aliases explicitly, and `shared_checkpoint.py` resolves them.
## Parameter counts
The architecture defines 20,088,058,521,618 parameters, derived from the modules in
`modeling_omni.py`. The Hub's native counter reports about 20.2T because it counts the stored
blocks plus every repeated bit slot of the shared bank. That figure reflects storage aliasing,
not independently learned weights.
## Reading the checkpoint
```python
import json
from shared_checkpoint import SharedBankCheckpoint
checkpoint = SharedBankCheckpoint(".")
name = "world_model.stages.2.blocks.0.mlp.experts.0.gate_proj.weight"
print(checkpoint.tensor_info(name)) # stored block
print(checkpoint.stored_slice(name, 0, 4))
print(json.load(open("tensor_shapes.json"))[name]) # full shape
print(checkpoint.bit_slice("shared_bank.views.00042.weight", 0, 8))
```
Reads are bounded to one million elements per call. The full architecture is never materialized;
the same classes run end to end at a small configuration.
|