Update README.md
Browse files
README.md
CHANGED
|
@@ -24,6 +24,8 @@ This repository provides **Model_64_FLOAT (272M)** — an **ablation model** fro
|
|
| 24 |
|
| 25 |
[📚 Paper (Growing Transformers: Modular Composition and Layer-wise Expansion on a Frozen Substrate)](https://huggingface.co/papers/2507.07129) -
|
| 26 |
|
|
|
|
|
|
|
| 27 |
This checkpoint tests whether language modeling and semantic structure can emerge when the **entire input embedding layer is frozen** and contains **no semantic or glyph/visual information**.
|
| 28 |
|
| 29 |
Compared to **Model_64_BIT**, this model uses the same embedding dimensionality (`n_embed=64`) and the same “unique per token” construction, but the embedding vectors are **floating-point** (after a deterministic projection/normalization step), rather than raw binary components.
|
|
|
|
| 24 |
|
| 25 |
[📚 Paper (Growing Transformers: Modular Composition and Layer-wise Expansion on a Frozen Substrate)](https://huggingface.co/papers/2507.07129) -
|
| 26 |
|
| 27 |
+
[📚 Blog Article](https://huggingface.co/blog/Bochkov/emergent-semantics-beyond-token-embeddings)
|
| 28 |
+
|
| 29 |
This checkpoint tests whether language modeling and semantic structure can emerge when the **entire input embedding layer is frozen** and contains **no semantic or glyph/visual information**.
|
| 30 |
|
| 31 |
Compared to **Model_64_BIT**, this model uses the same embedding dimensionality (`n_embed=64`) and the same “unique per token” construction, but the embedding vectors are **floating-point** (after a deterministic projection/normalization step), rather than raw binary components.
|