Training completed for llama3-nyc-test

README.md

@@ -1,81 +1,59 @@
 ---
-base_model: unsloth/llama-3-8b
-library_name: peft
-pipeline_tag: text-generation
+base_model: unsloth/llama-3-8b-bnb-4bit
+library_name: transformers
+model_name: llama3-nyc-test
 tags:
-- base_model:adapter:unsloth/llama-3-8b
-- lora
+- generated_from_trainer
+- unsloth
 - sft
-- transformers
 - trl
-- unsloth
+licence: license
 ---
 
-# llama3-nyc-test
-
-This model is a fine-tuned version of [unsloth/llama-3-8b](https://huggingface.co/unsloth/llama-3-8b) using LoRA (Low-Rank Adaptation) and quantization techniques.
-
-## Model Details
-
-- **Base Model:** unsloth/llama-3-8b
-- **Fine-tuned Model:** comp5331poi/llama3-nyc-test
-- **Training Run:** llama3-nyc-test
-- **Device:** cuda
-
-## Training Configuration
+# Model Card for llama3-nyc-test
 
-### Hyperparameters
+This model is a fine-tuned version of [unsloth/llama-3-8b-bnb-4bit](https://huggingface.co/unsloth/llama-3-8b-bnb-4bit).
+It has been trained using [TRL](https://github.com/huggingface/trl).
 
-- **Number of Epochs:** 8
-- **Batch Size:** 4
-- **Gradient Accumulation Steps:** 4
-- **Effective Batch Size:** 16
-- **Learning Rate:** 1e-05
-- **Learning Rate Scheduler:** constant
-- **Warmup Steps:** 20
-- **Max Sequence Length:** 2048
-- **Optimizer:** paged_adamw_8bit
-- **Max Gradient Norm:** 0.3
-- **Random Seed:** 43
+## Quick start
 
-### LoRA Configuration
+```python
+from transformers import pipeline
 
-- **LoRA Rank (r):** 16
-- **LoRA Alpha:** 32
-- **LoRA Dropout:** 0.1
-- **Target Modules:** down_proj, q_proj, v_proj, o_proj, up_proj, gate_proj, k_proj
-- **Task Type:** CAUSAL_LM
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="comp5331poi/llama3-nyc-test", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
 
-### Quantization
+## Training procedure
 
-- **Quantization Bits:** 4-bit
+ 
 
 
-## Usage
+This model was trained with SFT.
 
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-from peft import PeftModel
+### Framework versions
 
-# Load base model
-base_model = AutoModelForCausalLM.from_pretrained("unsloth/llama-3-8b")
+- TRL: 0.23.0
+- Transformers: 4.56.2
+- Pytorch: 2.8.0
+- Datasets: 4.3.0
+- Tokenizers: 0.22.1
 
-# Load LoRA adapter
-model = PeftModel.from_pretrained(base_model, "comp5331poi/llama3-nyc-test")
+## Citations
 
-# Load tokenizer
-tokenizer = AutoTokenizer.from_pretrained("unsloth/llama-3-8b")
-
-# Generate text
-inputs = tokenizer("Your prompt here", return_tensors="pt")
-outputs = model.generate(**inputs, max_length=2048)
-print(tokenizer.decode(outputs[0], skip_special_tokens=True))
-```
 
-## Framework Versions
 
-- Transformers
-- PEFT
-- TRL
-- PyTorch
-- BitsAndBytes
+Cite TRL as:
+    
+```bibtex
+@misc{vonwerra2022trl,
+	title        = {{TRL: Transformer Reinforcement Learning}},
+	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
+	year         = 2020,
+	journal      = {GitHub repository},
+	publisher    = {GitHub},
+	howpublished = {\url{https://github.com/huggingface/trl}}
+}
+```

adapter_config.json

@@ -29,13 +29,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "down_proj",
     "q_proj",
-    "v_proj",
-    "o_proj",
     "up_proj",
+    "k_proj",
+    "v_proj",
     "gate_proj",
-    "k_proj"
+    "o_proj",
+    "down_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:128da5f0fe4cf04ad33cb397ba9d7f54f9738e1b994a2a93198e9961dda92fdf
+oid sha256:5e28406c691e1202c9a1c124f06a52499ae3a1f956671b030f95ac633c86c89f
 size 167832240

logs/events.out.tfevents.1761384274.vislab-4090.675146.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a90eca3e00f3402be292050bf1fc9c774837a3183d916295c0f6c459c5db9751
+size 14664

logs/events.out.tfevents.1761386231.vislab-4090.692860.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b54d165a1eed9396ab44e02deacbafb07292da950e0b3777b29114cad157ee67
+size 83651