SykoSLM
/

SykoLLM-v6.9

Text Generation

Model card Files Files and versions

SykoSLM commited on about 1 month ago

Commit

4e29e2d

·

verified ·

1 Parent(s): 0cbb695

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -28,7 +28,7 @@ SykoLLM-V6.9 is a 391M parameter causal language model, trained from scratch on
 | **Context Length** | 1,024 tokens |
 | **Vocabulary Size** | 50,000 |
 | **Hidden Size** | 1,024 |
-| **Intermediate Size** | 2,304 |
 | **Layers** | 24 |
 | **Attention Heads** | 8 (GQA: 2 KV heads) |
 | **Precision** | bfloat16 |

 | **Context Length** | 1,024 tokens |
 | **Vocabulary Size** | 50,000 |
 | **Hidden Size** | 1,024 |
+| **Intermediate Size** | 3,072 |
 | **Layers** | 24 |
 | **Attention Heads** | 8 (GQA: 2 KV heads) |
 | **Precision** | bfloat16 |