Update README.md
Browse files
README.md
CHANGED
|
@@ -7,6 +7,8 @@ NanoRush Chat is a 283M parameter GPT-style causal language model fine-tuned for
|
|
| 7 |
|
| 8 |
Github- https://github.com/Amogh1221/NanoRush
|
| 9 |
|
|
|
|
|
|
|
| 10 |
## Model Details & Configuration
|
| 11 |
| Detail | Value |
|
| 12 |
| --- | --- |
|
|
@@ -18,10 +20,8 @@ Github- https://github.com/Amogh1221/NanoRush
|
|
| 18 |
| **Embedding Dimension (n_embd)** | 768 |
|
| 19 |
| **Number of Heads (n_head)** | 12 |
|
| 20 |
| **Number of Layers (n_layer)** | 36 |
|
| 21 |
-
| **Base Model** | Custom pre-trained
|
| 22 |
-
| **Fine-tuning Dataset** |
|
| 23 |
-
| **Hardware & Optimizations** | The fine-tuning process was fully optimized for A100/H100 GPUs leveraging TF32 precision, BFloat16 autocast, and `torch.compile` for maximum throughput. |
|
| 24 |
-
| **Training Strategy** | Trained using the AdamW optimizer with a cosine learning rate schedule and linear warmup. It utilizes gradient accumulation and auto-scales the batch size based on available VRAM and sequence length. |
|
| 25 |
|
| 26 |
## Evaluation Results
|
| 27 |
|
|
@@ -42,7 +42,8 @@ The model was evaluated using standard zero-shot accuracy metrics.
|
|
| 42 |
|
| 43 |
## Usage
|
| 44 |
|
| 45 |
-
This model has been exported to be fully compatible with the Hugging Face `transformers` library.
|
|
|
|
| 46 |
|
| 47 |
### Installation
|
| 48 |
Make sure you have the latest version of the `transformers` and `torch` libraries installed:
|
|
@@ -50,9 +51,8 @@ Make sure you have the latest version of the `transformers` and `torch` librarie
|
|
| 50 |
pip install torch transformers
|
| 51 |
```
|
| 52 |
|
| 53 |
-
### Example Code
|
| 54 |
|
| 55 |
-
The following example demonstrates how to run NanoRush Chat efficiently on a CPU using INT8 dynamic quantization and streaming output (as used in the NanoRush web backend).
|
| 56 |
|
| 57 |
```python
|
| 58 |
import torch
|
|
|
|
| 7 |
|
| 8 |
Github- https://github.com/Amogh1221/NanoRush
|
| 9 |
|
| 10 |
+
Live- https://nano-chat-web.vercel.app
|
| 11 |
+
|
| 12 |
## Model Details & Configuration
|
| 13 |
| Detail | Value |
|
| 14 |
| --- | --- |
|
|
|
|
| 20 |
| **Embedding Dimension (n_embd)** | 768 |
|
| 21 |
| **Number of Heads (n_head)** | 12 |
|
| 22 |
| **Number of Layers (n_layer)** | 36 |
|
| 23 |
+
| **Base Model** | Custom pre-trained |
|
| 24 |
+
| **Fine-tuning Dataset** | `HuggingFaceTB/smoltalk` |
|
|
|
|
|
|
|
| 25 |
|
| 26 |
## Evaluation Results
|
| 27 |
|
|
|
|
| 42 |
|
| 43 |
## Usage
|
| 44 |
|
| 45 |
+
This model has been exported to be fully compatible with the Hugging Face `transformers` library.
|
| 46 |
+
You can load it using the standard `AutoModelForCausalLM` pipeline.
|
| 47 |
|
| 48 |
### Installation
|
| 49 |
Make sure you have the latest version of the `transformers` and `torch` libraries installed:
|
|
|
|
| 51 |
pip install torch transformers
|
| 52 |
```
|
| 53 |
|
| 54 |
+
### Example Code
|
| 55 |
|
|
|
|
| 56 |
|
| 57 |
```python
|
| 58 |
import torch
|