v8: r=96, alpha=192, lr=1e-6, max_len=1024, epoch=2 + upsample (README written by author)

README.md

@@ -2,44 +2,41 @@
 base_model: Qwen/Qwen3-4B-Instruct-2507
 datasets:
 - u-10bei/structured_data_with_cot_dataset_512_v2
-- u-10bei/dpo-dataset-qwen-cot
 language:
 - en
 license: apache-2.0
 library_name: peft
 pipeline_tag: text-generation
 tags:
-- dpo
-- unsloth
-- qwen
-- alignment
+- qlora
+- lora
+- structured-output
 ---
 
-# qwen3-4b-structured-sft-dpo
+#qwen3-4b-structured-output-lora
 
-This model is fine-tuned from **Qwen/Qwen3-4B-Instruct-2507** using SFT, then further optimized with **Direct Preference Optimization (DPO)** via the **Unsloth** library.
+This repository provides a **LoRA adapter** fine-tuned from
+**Qwen/Qwen3-4B-Instruct-2507** using **QLoRA (4-bit, Unsloth)**.
 
-This repository contains **LoRA adapter weights**. The base model must be loaded separately with PeftModel.
+This repository contains **LoRA adapter weights only**.
+The base model must be loaded separately.
 
 ## Training Objective
-This model has been optimized using DPO to align its responses with preferred outputs, focusing on improving reasoning (Chain-of-Thought) and structured response quality based on the provided preference dataset.
+
+This adapter is trained to improve **structured output accuracy**
+(JSON / YAML / XML / TOML / CSV).
+
+Loss is applied only to the final assistant output,
+while intermediate reasoning (Chain-of-Thought) is masked.
 
 ## Training Configuration
-**Stage 1 - SFT:**
-- **Dataset**: u-10bei/structured_data_with_cot_dataset_512_v2
-- **Max sequence length**: 1024
-- **Epochs**: 2
-- **Learning rate**: 1e-06
-- **LoRA**: r=96, alpha=192
 
-**Stage 2 - DPO:**
-- **Dataset**: u-10bei/dpo-dataset-qwen-cot
-- **Method**: DPO (Direct Preference Optimization)
-- **Epochs**: 1
-- **Learning rate**: 1e-07
-- **Beta**: 0.1
-- **Max sequence length**: 1024
-- **LoRA Config**: r=8, alpha=16
+- Base model: Qwen/Qwen3-4B-Instruct-2507
+- Method: QLoRA (4-bit)
+- Max sequence length: 1024
+- Epochs: 2
+- Learning rate: 1e-06
+- LoRA: r=96, alpha=192
 
 ## Usage
 
@@ -55,20 +52,14 @@ tokenizer = AutoTokenizer.from_pretrained(base)
 model = AutoModelForCausalLM.from_pretrained(
     base,
     torch_dtype=torch.float16,
-    device_map="auto"
+    device_map="auto",
 )
 model = PeftModel.from_pretrained(model, adapter)
-
-# Test inference
-prompt = "Your question here"
-inputs = tokenizer.apply_chat_template([{"role": "user", "content": prompt}], tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
-outputs = model.generate(**inputs, max_new_tokens=512)
-print(tokenizer.decode(outputs[0]))
-
 ```
 
-## Sources & License (IMPORTANT)
+## Sources & Terms (IMPORTANT)
+
+Training data: u-10bei/structured_data_with_cot_dataset_512_v2
 
-* **Training Data**: [u-10bei/structured_data_with_cot_dataset_512_v2], [u-10bei/dpo-dataset-qwen-cot]
-* **License**: MIT License. (As per dataset terms).
-* **Compliance**: Users must follow the original base model's license terms.
+Dataset License: MIT License. This dataset is used and distributed under the terms of the MIT License.
+Compliance: Users must comply with the MIT license (including copyright notice) and the base model's original terms of use.

adapter_config.json

@@ -33,13 +33,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "down_proj",
-    "q_proj",
+    "o_proj",
+    "gate_proj",
     "up_proj",
+    "q_proj",
     "k_proj",
-    "v_proj",
-    "gate_proj",
-    "o_proj"
+    "down_proj",
+    "v_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:03e307618d3940106e51aedd1079b3c177b4018ff3748df585c2992b3582a7e0
+oid sha256:d6fb93374d50edd5a452ddc507d0dc809bc7fffef75ce0827d1059a7ffc87906
 size 792791480

tokenizer_config.json

@@ -233,7 +233,7 @@
   "extra_special_tokens": {},
   "model_max_length": 262144,
   "pad_token": "<|vision_pad|>",
-  "padding_side": "left",
+  "padding_side": "right",
   "split_special_tokens": false,
   "tokenizer_class": "Qwen2Tokenizer",
   "unk_token": null