Training in progress, epoch 17

README.md

@@ -1,24 +1,17 @@
 ---
 base_model: meta-llama/CodeLlama-7b-Instruct-hf
-library_name: transformers
-model_name: CodeLlama-Instruct-Python-7b
+library_name: peft
+model_name: CodeLlamaInstruct_finetuned_2
 tags:
 - generated_from_trainer
 - trl
 - sft
-- CodeLlama
-- Python
 licence: license
-datasets:
-- cardiffnlp/databench
 ---
 
-# Model Card for CodeLlama-Instruct-Python-7b
+# Model Card for CodeLlamaInstruct_finetuned_2
 
 This model is a fine-tuned version of [meta-llama/CodeLlama-7b-Instruct-hf](https://huggingface.co/meta-llama/CodeLlama-7b-Instruct-hf).
-Finetuned on DataBench [cardiffnlp/databench](https://huggingface.co/datasets/cardiffnlp/databench), which is publicly available on Hugging Face. 
-It is specifically designed to generate a single line of Python code in response to questions from the dataset. 
-The finetuning process ensures it follows instructions for producing the required Python code accurately.
 It has been trained using [TRL](https://github.com/huggingface/trl).
 
 ## Quick start
@@ -27,7 +20,7 @@ It has been trained using [TRL](https://github.com/huggingface/trl).
 from transformers import pipeline
 
 question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="basharatwali/CodeLlama-Instruct-Python-7b", device="cuda")
+generator = pipeline("text-generation", model="basharatwali/CodeLlamaInstruct_finetuned_2", device="cuda")
 output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
 print(output["generated_text"])
 ```
@@ -41,6 +34,7 @@ This model was trained with SFT.
 
 ### Framework versions
 
+- PEFT 0.14.1.dev0
 - TRL: 0.13.0
 - Transformers: 4.48.0.dev0
 - Pytorch: 2.5.1+cu121

adapter_config.json

@@ -24,10 +24,10 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "k_proj",
-    "v_proj",
     "o_proj",
-    "q_proj"
+    "q_proj",
+    "v_proj",
+    "k_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2744cccc9ff911989664ef5b6902ee0294700bd71a3d2b6b015b852f3fb1a645
+oid sha256:5a019e4944279f3fd1b0a1ba23a49cf3aa154ba2b935a9f76a33f3525cfb40c4
 size 67143296

runs/Jan06_06-55-43_2db6e875cc75/events.out.tfevents.1736146549.2db6e875cc75.518.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10e8c89cbb5ae71fc4f7a7cbfa770f27ae0375902ea9e3b8eeb4eaaf22d082d1
+size 7883

special_tokens_map.json

@@ -1,5 +1,9 @@
 {
   "additional_special_tokens": [
+    "▁<PRE>",
+    "▁<MID>",
+    "▁<SUF>",
+    "▁<EOT>",
     "▁<PRE>",
     "▁<MID>",
     "▁<SUF>",

tokenizer_config.json

@@ -60,6 +60,10 @@
     }
   },
   "additional_special_tokens": [
+    "▁<PRE>",
+    "▁<MID>",
+    "▁<SUF>",
+    "▁<EOT>",
     "▁<PRE>",
     "▁<MID>",
     "▁<SUF>",
@@ -73,14 +77,18 @@
   "extra_special_tokens": {},
   "fill_token": "<FILL_ME>",
   "legacy": null,
+  "max_length": 1024,
   "middle_token": "▁<MID>",
   "model_max_length": 1000000000000000019884624838656,
   "pad_token": "</s>",
   "prefix_token": "▁<PRE>",
   "sp_model_kwargs": {},
+  "stride": 0,
   "suffix_token": "▁<SUF>",
   "tokenizer_class": "CodeLlamaTokenizer",
   "torch_dtype": "auto",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
   "unk_token": "<unk>",
   "use_default_system_prompt": false
 }

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1351efb37072ea47d02669d9824da568ea8ad2eaa2e9abe28bb65775bd5909f5
+oid sha256:d76259ed39045eb5635cf118a132e3a778b7de48e2f8b54079ef5d2282da4289
 size 5752