| --- |
| license: mit |
| tags: |
| - joke |
| - large |
| - record |
| --- |
| |
| # leviathan-19t |
|
|
| A 19.0-trillion-parameter model that contains nothing. |
|
|
| ## What it demonstrates |
|
|
| Hugging Face computes a repository's parameter count from safetensors headers |
| alone. It sums `prod(shape)` per tensor and never reads the tensor data, so the |
| count is whatever the headers declare. Here they declare 4,422 |
| tensors of shape `[65536, 65536]` in `F4` (4 bits/param), plus one |
| `[4294967296, 1]` position-embedding tensor, across 444 shards. |
|
|
| That is 15.1% above the previous empty-model record of |
| 16,501,264,351,232 parameters held by |
| [tsfrm/vacuum-16t](https://huggingface.co/tsfrm/vacuum-16t), which is the work |
| this repository builds on. |
|
|
| The headers are honest. `safetensors` requires tensor byte ranges to tile the |
| data section contiguously and requires the file length to equal |
| `8 + header + data` exactly, so overlapping tensors or truncated files are |
| rejected by the format. Every declared byte is really present. The bytes are |
| simply all `0x00`. |
|
|
| ## Real cost |
|
|
| | | | |
| |---|---| |
| | Declared parameters | 18,996,640,350,208 | |
| | Declared bytes | 9,498,320,175,104 (9.50 TB) | |
| | Storage quota consumed | 9.50 TB — quota bills declared bytes | |
| | Bytes actually transferred | a few hundred KB | |
|
|
| Xet content-defined chunking deduplicates the *transfer*: every block of weight |
| data is byte-identical, hashes to one chunk, and crosses the wire once. Storage |
| quota is not deduplicated — it bills the logical size. The saving is in |
| bandwidth only, which is the constraint that decides how large one of these can |
| get. |
|
|
| Locally the shards cost nothing either. They are sparse files: the data section |
| is a filesystem hole created with `ftruncate`, so each shard reports its full |
| size to `stat`, reads back as zeros, and allocates no blocks. |
|
|
| `F4` at 4 bits is the narrowest dtype safetensors supports, which fixes the |
| exchange rate at two parameters per byte. That, and not compute, is what bounds |
| a model like this. |
|
|
| ## Limitations |
|
|
| It has no capabilities. |
|
|
