Bonforger commited on
Commit
f548a06
·
verified ·
1 Parent(s): f877b38

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +68 -0
README.md ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # YAML metadata---
2
+ license: apache-2.0
3
+ datasets:
4
+ - u-10bei/structured_data_with_cot_dataset_512_v2
5
+ language:
6
+ - en
7
+ base_model:
8
+ - Qwen/Qwen3-4B-Instruct-2507
9
+ pipeline_tag: text-generation
10
+ library_name: peft
11
+ tags:
12
+ - qlora
13
+ - lora
14
+ - structured-output
15
+ ---
16
+ # qwen3-4b-structured-output-lora
17
+
18
+ This repository provides a **LoRA adapter** fine-tuned from
19
+ **Qwen3-4B-Instruct-2507** using **QLoRA (4-bit, Unsloth)**.
20
+
21
+ This repository contains **LoRA adapter weights only**.
22
+ The base model must be loaded separately.
23
+
24
+ ## Training Objective
25
+
26
+ This adapter is trained to improve **structured output accuracy**
27
+ (JSON / YAML / XML / TOML / CSV).
28
+
29
+ Loss is applied only to the final assistant output,
30
+ while intermediate reasoning (Chain-of-Thought) is masked.
31
+
32
+ ## Training Configuration
33
+
34
+ - Base model: Qwen3-4B-Instruct-2507
35
+ - Method: QLoRA (4-bit)
36
+ - Max sequence length: 512
37
+ - Epochs: 1
38
+ - Learning rate: 1e-6
39
+ - LoRA: r=64, alpha=128
40
+ -
41
+ ## Usage
42
+
43
+ ```python
44
+ from transformers import AutoModelForCausalLM, AutoTokenizer
45
+ from peft import PeftModel
46
+ import torch
47
+
48
+ base = "Qwen/Qwen3-4B-Instruct-2507"
49
+ adapter = "your_id/your-repo"
50
+
51
+ tokenizer = AutoTokenizer.from_pretrained(base)
52
+ model = AutoModelForCausalLM.from_pretrained(
53
+ base,
54
+ torch_dtype=torch.float16,
55
+ device_map="auto",
56
+ )
57
+ model = PeftModel.from_pretrained(model, adapter)
58
+
59
+ ---
60
+
61
+ ##### 6. データセット・ライセンス注意(必須・重要)
62
+ ```md
63
+ ## Sources & Terms (IMPORTANT)
64
+
65
+ Training data: u-10bei/structured_data_with_cot_dataset_512_v2
66
+
67
+ Dataset License: MIT License. This dataset is used and distributed under the terms of the MIT License.
68
+ Compliance: Users must comply with the MIT license (including copyright notice) and the base model's original terms of use.