Asilarkness commited on
Commit
d23602a
·
verified ·
1 Parent(s): ae5ddd0

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -20,7 +20,7 @@ Public experimental byte-level causal language model. Training is in progress an
20
  - Approximately 1B stored parameters and approximately one third active per token.
21
  - UTF-8 bytes, vocabulary 256, no external tokenizer.
22
  - Four local causal Transformer layers process every byte.
23
- - Entropy boundaries compress the stream to about one patch per eight bytes, with capacity for 192 patches per 1024-byte sequence.
24
  - Three independent eight-layer global routes are trained cyclically, one full batch per route.
25
  - The previous completed global patch conditions the local prediction, preventing future-token leakage.
26
  - Muon optimizes hidden matrices; AdamW optimizes embeddings and normalization parameters.
 
20
  - Approximately 1B stored parameters and approximately one third active per token.
21
  - UTF-8 bytes, vocabulary 256, no external tokenizer.
22
  - Four local causal Transformer layers process every byte.
23
+ - Entropy boundaries compress the stream to about one patch per eight bytes, with capacity for 192 patches per 1024-byte sequence. Over-cap boundaries are uniformly coalesced across the sequence instead of truncating its tail.
24
  - Three independent eight-layer global routes are trained cyclically, one full batch per route.
25
  - The previous completed global patch conditions the local prediction, preventing future-token leakage.
26
  - Muon optimizes hidden matrices; AdamW optimizes embeddings and normalization parameters.