updated modelcard
Browse files
README.md
CHANGED
|
@@ -19,7 +19,26 @@ language:
|
|
| 19 |
# Model Card for FinetunedLAMAtoR1-001-3B
|
| 20 |
|
| 21 |
## Model Details
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
### Model Description
|
| 24 |
|
| 25 |
This model is a fine-tuned version of **[unsloth/Llama-3.2-3B-Instruct](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct)** designed to mimic reflective, human-like stream-of-consciousness reasoning. It was trained using **[Unsloth](https://github.com/unslothai/unsloth)** on the **[ServiceNow-AI/R1-Distill-SFT](https://huggingface.co/datasets/ServiceNow-AI/R1-Distill-SFT)** dataset.
|
|
|
|
| 19 |
# Model Card for FinetunedLAMAtoR1-001-3B
|
| 20 |
|
| 21 |
## Model Details
|
| 22 |
+
## Technical Specifications
|
| 23 |
+
|
| 24 |
+
### Model Architecture and Objective
|
| 25 |
+
- **Base Model:** Llama-3.2-3B-Instruct
|
| 26 |
+
- **Architecture:** Causal Decoder-Only Transformer
|
| 27 |
+
- **Hidden Size:** 3072
|
| 28 |
+
- **Layers:** 28
|
| 29 |
+
- **Heads:** 24
|
| 30 |
+
- **Parameters:** ~3.21B (Loaded in 4-bit quantization)
|
| 31 |
+
- **Precision:** Float16 (during inference/training via LoRA)
|
| 32 |
+
|
| 33 |
+
### Compute Infrastructure
|
| 34 |
+
- **Hardware:** Tesla T4 GPU (Google Colab)
|
| 35 |
+
- **VRAM Usage:** ~2.24 GB (Model) + Training Overhead
|
| 36 |
+
- **Quantization:** 4-bit (QLoRA) via `bitsandbytes`
|
| 37 |
+
|
| 38 |
+
### Model Weights
|
| 39 |
+
- **Type:** LoRA Adapter (Peft)
|
| 40 |
+
- **Adapter File Size:** ~92 MB
|
| 41 |
+
- **Total Saved Size:** ~108 MB
|
| 42 |
### Model Description
|
| 43 |
|
| 44 |
This model is a fine-tuned version of **[unsloth/Llama-3.2-3B-Instruct](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct)** designed to mimic reflective, human-like stream-of-consciousness reasoning. It was trained using **[Unsloth](https://github.com/unslothai/unsloth)** on the **[ServiceNow-AI/R1-Distill-SFT](https://huggingface.co/datasets/ServiceNow-AI/R1-Distill-SFT)** dataset.
|