PEFT
English
qlora
lora
structured-output
Bonforger commited on
Commit
039ebe3
·
verified ·
1 Parent(s): e8e94e3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +52 -0
README.md CHANGED
@@ -12,3 +12,55 @@ tags:
12
  - lora
13
  - structured-output
14
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  - lora
13
  - structured-output
14
  ---
15
+ ## Model Outline
16
+ # qwen3-4b-structured-output-lora
17
+
18
+ This repository provides a **LoRA adapter** fine-tuned from
19
+ **Qwen3-4B-Instruct-2507** using **QLoRA (4-bit, Unsloth)**.
20
+
21
+ This repository contains **LoRA adapter weights only**.
22
+ The base model must be loaded separately.
23
+
24
+ ## Training Objective
25
+
26
+ This adapter is trained to improve **structured output accuracy**
27
+ (JSON / YAML / XML / TOML / CSV).
28
+
29
+ Loss is applied only to the final assistant output,
30
+ while intermediate reasoning (Chain-of-Thought) is masked.
31
+
32
+ ## Training Configuration
33
+
34
+ - Base model: Qwen3-4B-Instruct-2507
35
+ - Method: QLoRA (4-bit)
36
+ - Max sequence length: 512
37
+ - Epochs: 1
38
+ - Learning rate: 5e-6
39
+ - LoRA: r=64, alpha=128
40
+ -
41
+ ## Usage
42
+
43
+ ```python
44
+ from transformers import AutoModelForCausalLM, AutoTokenizer
45
+ from peft import PeftModel
46
+ import torch
47
+
48
+ base = "Qwen/Qwen3-4B-Instruct-2507"
49
+ adapter = "your_id/your-repo"
50
+
51
+ tokenizer = AutoTokenizer.from_pretrained(base)
52
+ model = AutoModelForCausalLM.from_pretrained(
53
+ base,
54
+ torch_dtype=torch.float16,
55
+ device_map="auto",
56
+ )
57
+ model = PeftModel.from_pretrained(model, adapter)
58
+
59
+ ##### 6. データセット・ライセンス注意(必須・重要)
60
+ ```md
61
+ ## Sources & Terms (IMPORTANT)
62
+
63
+ Training data: u-10bei/structured_data_with_cot_dataset_512_v2
64
+
65
+ Dataset License: MIT License. This dataset is used and distributed under the terms of the MIT License.
66
+ Compliance: Users must comply with the MIT license (including copyright notice) and the base model's original terms of use.