finetuned till 3 epochs

README.md

@@ -1,7 +1,7 @@
 ---
 base_model: unsloth/llama-3.2-1b-instruct-bnb-4bit
 library_name: transformers
-model_name: Llama-3.2-1B-Instruct-HyperMixSub
+model_name: Llama-3.2-1B-HyperMixSub
 tags:
 - generated_from_trainer
 - unsloth
@@ -10,7 +10,7 @@ tags:
 licence: license
 ---
 
-# Model Card for Llama-3.2-1B-Instruct-HyperMixSub
+# Model Card for Llama-3.2-1B-HyperMixSub
 
 This model is a fine-tuned version of [unsloth/llama-3.2-1b-instruct-bnb-4bit](https://huggingface.co/unsloth/llama-3.2-1b-instruct-bnb-4bit).
 It has been trained using [TRL](https://github.com/huggingface/trl).
@@ -21,21 +21,21 @@ It has been trained using [TRL](https://github.com/huggingface/trl).
 from transformers import pipeline
 
 question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="AdityaMayukhSom/Llama-3.2-1B-Instruct-HyperMixSub", device="cuda")
+generator = pipeline("text-generation", model="AdityaMayukhSom/Llama-3.2-1B-HyperMixSub", device="cuda")
 output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
 print(output["generated_text"])
 ```
 
 ## Training procedure
 
- 
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/adityamayukhsom-jadavpur-university/HDP/runs/vdn4flei) 
 
 
 This model was trained with SFT.
 
 ### Framework versions
 
-- TRL: 0.15.2
+- TRL: 0.18.1
 - Transformers: 4.51.3
 - Pytorch: 2.7.0+cu128
 - Datasets: 3.6.0
@@ -50,7 +50,7 @@ Cite TRL as:
 ```bibtex
 @misc{vonwerra2022trl,
 	title        = {{TRL: Transformer Reinforcement Learning}},
-	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
+	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
 	year         = 2020,
 	journal      = {GitHub repository},
 	publisher    = {GitHub},

adapter_config.json

@@ -24,13 +24,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "o_proj",
     "q_proj",
-    "v_proj",
-    "gate_proj",
     "up_proj",
     "k_proj",
-    "down_proj"
+    "v_proj",
+    "gate_proj",
+    "down_proj",
+    "o_proj"
   ],
   "task_type": "CAUSAL_LM",
   "trainable_token_indices": null,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ddd547ceae9741d190f50bbcb2e542d5408a7679662213c7443705f6d467677c
+oid sha256:c5b61acf9865e6b377f901d2a4c4b6a7eb42827ac0cf885990fe42097985e929
 size 45118424

emissions.csv

@@ -1,2 +1,2 @@
 timestamp,project_name,run_id,experiment_id,duration,emissions,emissions_rate,cpu_power,gpu_power,ram_power,cpu_energy,gpu_energy,ram_energy,energy_consumed,country_name,country_iso_code,region,cloud_provider,cloud_region,os,python_version,codecarbon_version,cpu_count,cpu_model,gpu_count,gpu_model,longitude,latitude,ram_total_size,tracking_mode,on_cloud,pue
-2025-05-25T18:47:20,codecarbon,73e076f5-1f3a-42d8-9fbb-7a58de6468df,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,23618.222676728998,0.7737373870994643,3.2760186813795544e-05,42.5,8.162088681307905,11.692362785339355,0.27878866959167214,0.7291384396991739,0.07658777995642749,1.0845148892472738,India,IND,west bengal,,,Linux-6.8.0-60-generic-x86_64-with-glibc2.39,3.12.10,2.7.1,20,12th Gen Intel(R) Core(TM) i7-12700F,1,1 x NVIDIA GeForce RTX 4060 Ti,88.3693,22.5643,31.17963409423828,machine,N,1.0
+2025-06-09T11:41:02,codecarbon,7eec501d-fd41-41a5-a4da-c5d80b70bd51,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,13787.91562495599,0.4645328194170154,3.369130128532377e-05,42.5,13.128350420525347,11.692362785339355,0.16273118284227978,0.443686769671354,0.04469800440083664,0.6511159569144687,India,IND,west bengal,,,Linux-6.8.0-60-generic-x86_64-with-glibc2.39,3.12.10,2.7.1,20,12th Gen Intel(R) Core(TM) i7-12700F,1,1 x NVIDIA GeForce RTX 4060 Ti,88.3693,22.5643,31.17963409423828,machine,N,1.0

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:286cc50ff1f25e7e3a70aed731af1863911b1f5f7636681ea278d31ce06b604d
-size 6033
+oid sha256:38beb4d779204d634a4e1f6096593eb40177155a1f1039ea7ae65f776863ddee
+size 6097