Upload README.md with huggingface_hub

README.md

@@ -9,39 +9,34 @@ tags:
 - bdh
 - fast-weights
 - polish
-- lang-200m
+- language-model
 base_model: pathwaycom/bdh
 datasets:
 - SlayerLab/polish-dynaword-mix
-
 ---
-# BDH-25M-<span>PL</span> — Polish Looped-Transformer (Fast-Weight) Language Model
+# BDH-25M-PL — Polish Looped-Transformer (Fast-Weight) Language Model
 
-Mały, otwarty model językowy z **architekturą BDH (Fast Weight Layers)**,
-wytrenowany na czystym, polskim mixie **SlayerLab/polish-dynaword-mix**
-(100M tokenów, byte-level).
+A small, open-weight language model built on the **BDH (Fast Weight Layers)** architecture, trained on a clean Polish mix from **SlayerLab/polish-dynaword-mix** (100M tokens, byte-level).
 
-## Architektura
+## Architecture
 
-- `BDH` z [pathwaycom/bdh](https://github.com/pathwaycom/bdh) — looped latent-recurrence / fast weights
+- `BDH` from [pathwaycom/bdh](https://github.com/pathwaycom/bdh) — looped latent recurrence / fast weights
 - `n_layer=8, n_embd=256, n_head=4, mlp_internal_dim_multiplier=128`
-- Vocab: **byte-level (vocab 256)** — brak tokenizera, działa bezpośrednio na UTF-8 bajtach
-- **~25.3M parametrów** · seq 2048
-- Key feature: `Q==K` (fast-weights) — współdzielony stan jako klucz i wartość (locality latent recurrence)
+- Vocab: **byte-level (vocab 256)** — no tokenizer, works directly on UTF-8 bytes
+- **~25.3M parameters** · seq length 2048
+- Key feature: `Q==K` (fast weights) — the shared state acts as both key and value (localized latent recurrence)
 
-## Trening
+## Training
 
-- **Dane**: `SlayerLab/polish-dynaword-mix` (100M tokenów, law cap 7%, dedup, clean)
-- **Kroki**: 10000 · **final val loss ~1.41** · byte-level (lat ≥ 5.6 dla losowych wag)
-- **Zoptymalizowany**: ZClip + protocol B.2 (jak w paperze BDH)
+- **Data**: `SlayerLab/polish-dynaword-mix` (100M tokens, law cap 7%, dedup, cleaned)
+- **Steps**: 10000 · **final val loss ≈ 1.41** · byte-level (random-init baseline ≈ 5.6)
+- **Optimizer**: ZClip + protocol B.2 (as in the BDH paper)
 
-## Uwaga o skali
+## Note on scale
 
-To **model 25M na 100M tokenów** — zgodnie ze scaling laws jest **niedoćwiczony jakościowo**,
-ale stanowi działający baseline: generuje gramatycznie poprawne polskie zdania i
-demonstruje, że architektura BDH trenuje i generuje poprawnie end-to-end.
+This is a **25M model trained on 100M tokens** — per scaling laws it is **undertrained for high quality**, but it serves as a working baseline: it generates grammatically-correct Polish and demonstrates that the BDH architecture trains and generates correctly end-to-end.
 
-## Szybki start
+## Quick start
 
 ```python
 import torch
@@ -55,17 +50,17 @@ model.load_state_dict({k.replace("model.", ""): v
                        for k, v in load_file("model.safetensors").items()}, strict=True)
 model.eval()
 
-# byte-level prompt (bez tokenizera)
+# byte-level prompt (no tokenizer)
 prompt = "Warszawa jest stolicą Polski i "
 ids = torch.tensor([list(prompt.encode("utf-8"))])
-# ... pętla generowania po jednym bajcie z softmax(temp)
+# ... autoregressive loop, one byte at a time with softmax(temp)
 ```
 
-## Pliki
+## Files
 
-- `model.safetensors` — wagi (101MB)
-- `config.json` — hyperparametry i metadane treningu
+- `model.safetensors` — weights (101MB)
+- `config.json` — hyperparameters and training metadata
 
-## Licencja
+## License
 
-Wagi: **CC-BY-4.0**. Architektura: MIT (pathwaycom/bdh).
+Weights: **CC-BY-4.0**. Architecture: MIT (pathwaycom/bdh).