Muhammad-Shaheer commited on
Commit
155ebc2
·
verified ·
1 Parent(s): b86a247

updated modelcard

Browse files
Files changed (1) hide show
  1. README.md +20 -1
README.md CHANGED
@@ -19,7 +19,26 @@ language:
19
  # Model Card for FinetunedLAMAtoR1-001-3B
20
 
21
  ## Model Details
22
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
  ### Model Description
24
 
25
  This model is a fine-tuned version of **[unsloth/Llama-3.2-3B-Instruct](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct)** designed to mimic reflective, human-like stream-of-consciousness reasoning. It was trained using **[Unsloth](https://github.com/unslothai/unsloth)** on the **[ServiceNow-AI/R1-Distill-SFT](https://huggingface.co/datasets/ServiceNow-AI/R1-Distill-SFT)** dataset.
 
19
  # Model Card for FinetunedLAMAtoR1-001-3B
20
 
21
  ## Model Details
22
+ ## Technical Specifications
23
+
24
+ ### Model Architecture and Objective
25
+ - **Base Model:** Llama-3.2-3B-Instruct
26
+ - **Architecture:** Causal Decoder-Only Transformer
27
+ - **Hidden Size:** 3072
28
+ - **Layers:** 28
29
+ - **Heads:** 24
30
+ - **Parameters:** ~3.21B (Loaded in 4-bit quantization)
31
+ - **Precision:** Float16 (during inference/training via LoRA)
32
+
33
+ ### Compute Infrastructure
34
+ - **Hardware:** Tesla T4 GPU (Google Colab)
35
+ - **VRAM Usage:** ~2.24 GB (Model) + Training Overhead
36
+ - **Quantization:** 4-bit (QLoRA) via `bitsandbytes`
37
+
38
+ ### Model Weights
39
+ - **Type:** LoRA Adapter (Peft)
40
+ - **Adapter File Size:** ~92 MB
41
+ - **Total Saved Size:** ~108 MB
42
  ### Model Description
43
 
44
  This model is a fine-tuned version of **[unsloth/Llama-3.2-3B-Instruct](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct)** designed to mimic reflective, human-like stream-of-consciousness reasoning. It was trained using **[Unsloth](https://github.com/unslothai/unsloth)** on the **[ServiceNow-AI/R1-Distill-SFT](https://huggingface.co/datasets/ServiceNow-AI/R1-Distill-SFT)** dataset.