Update README.md
Browse files
README.md
CHANGED
|
@@ -21,6 +21,7 @@ Has native thinking in **Hinglish and English.**
|
|
| 21 |
Trained on v4-8 TPU
|
| 22 |
|
| 23 |
* Active Params : 1.7B (Including Embedding Layer)
|
|
|
|
| 24 |
* Tied Embeddings
|
| 25 |
* Torch-XLA (SPMD)
|
| 26 |
* Flash-Attention ( Block-Size = 512 )
|
|
@@ -52,5 +53,4 @@ Evals will be in [Dir](https://huggingface.co/tinycompany/BiBo-Mini-1.7B-SFT-Sta
|
|
| 52 |
|
| 53 |
Compute Provided by [Google](https://huggingface.co/google) ;)
|
| 54 |
|
| 55 |
-
|
| 56 |
❤️ TRC ❤️Google
|
|
|
|
| 21 |
Trained on v4-8 TPU
|
| 22 |
|
| 23 |
* Active Params : 1.7B (Including Embedding Layer)
|
| 24 |
+
* Specialized [Tokenizer (fhai50032/QTK-81K)](https://huggingface.co/fhai50032/QTK-81K) For better Tokenization for Hindi , English, Math & Code
|
| 25 |
* Tied Embeddings
|
| 26 |
* Torch-XLA (SPMD)
|
| 27 |
* Flash-Attention ( Block-Size = 512 )
|
|
|
|
| 53 |
|
| 54 |
Compute Provided by [Google](https://huggingface.co/google) ;)
|
| 55 |
|
|
|
|
| 56 |
❤️ TRC ❤️Google
|