Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -150,15 +150,6 @@ for step in range(100): # max tokens
|
|
| 150 |
print(tokenizer.decode(generated_tokens, skip_special_tokens=True))
|
| 151 |
```
|
| 152 |
|
| 153 |
-
## Model Architecture
|
| 154 |
-
|
| 155 |
-
- **Total Parameters**: 8B
|
| 156 |
-
- **Active Parameters**: ~1B per token
|
| 157 |
-
- **Experts**: 32 total, 4 active per token
|
| 158 |
-
- **Hidden Size**: 2048
|
| 159 |
-
- **Layers**: 24 (hybrid conv + attention)
|
| 160 |
-
- **Context Length**: 128K tokens
|
| 161 |
-
|
| 162 |
## License
|
| 163 |
|
| 164 |
This model is released under the [LFM 1.0 License](LICENSE).
|
|
|
|
| 150 |
print(tokenizer.decode(generated_tokens, skip_special_tokens=True))
|
| 151 |
```
|
| 152 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 153 |
## License
|
| 154 |
|
| 155 |
This model is released under the [LFM 1.0 License](LICENSE).
|