nikeda commited on
Commit
539b7a5
·
verified ·
1 Parent(s): fd95b2e

Upload LoRA adapter (README written by author)

Browse files
Files changed (3) hide show
  1. README.md +5 -10
  2. adapter_config.json +4 -4
  3. adapter_model.safetensors +1 -1
README.md CHANGED
@@ -2,7 +2,6 @@
2
  base_model: Qwen/Qwen3-4B-Instruct-2507
3
  datasets:
4
  - u-10bei/structured_data_with_cot_dataset_512_v5
5
- - daichira/structured-hard-sft-4k
6
  language:
7
  - en
8
  license: apache-2.0
@@ -14,7 +13,7 @@ tags:
14
  - structured-output
15
  ---
16
 
17
- sft-lora
18
 
19
  This repository provides a **LoRA adapter** fine-tuned from
20
  **Qwen/Qwen3-4B-Instruct-2507** using **QLoRA (4-bit, Unsloth)**.
@@ -35,8 +34,8 @@ while intermediate reasoning (Chain-of-Thought) is masked.
35
  - Base model: Qwen/Qwen3-4B-Instruct-2507
36
  - Method: QLoRA (4-bit)
37
  - Max sequence length: 512
38
- - Epochs: 1
39
- - Learning rate: 1e-06
40
  - LoRA: r=64, alpha=128
41
 
42
  ## Usage
@@ -47,7 +46,7 @@ from peft import PeftModel
47
  import torch
48
 
49
  base = "Qwen/Qwen3-4B-Instruct-2507"
50
- adapter = "your_id/your-repo"
51
 
52
  tokenizer = AutoTokenizer.from_pretrained(base)
53
  model = AutoModelForCausalLM.from_pretrained(
@@ -60,11 +59,7 @@ model = PeftModel.from_pretrained(model, adapter)
60
 
61
  ## Sources & Terms (IMPORTANT)
62
 
63
- Training data:
64
- - u-10bei/structured_data_with_cot_dataset_512_v5
65
- - daichira/structured-hard-sft-4k
66
-
67
- Dataset License: MIT License. These datasets are used and distributed under the terms of the MIT License.
68
 
69
  Dataset License: MIT License. This dataset is used and distributed under the terms of the MIT License.
70
  Compliance: Users must comply with the MIT license (including copyright notice) and the base model's original terms of use.
 
2
  base_model: Qwen/Qwen3-4B-Instruct-2507
3
  datasets:
4
  - u-10bei/structured_data_with_cot_dataset_512_v5
 
5
  language:
6
  - en
7
  license: apache-2.0
 
13
  - structured-output
14
  ---
15
 
16
+ qwen3-4b-structured-output-lora
17
 
18
  This repository provides a **LoRA adapter** fine-tuned from
19
  **Qwen/Qwen3-4B-Instruct-2507** using **QLoRA (4-bit, Unsloth)**.
 
34
  - Base model: Qwen/Qwen3-4B-Instruct-2507
35
  - Method: QLoRA (4-bit)
36
  - Max sequence length: 512
37
+ - Epochs: 2
38
+ - Learning rate: 2e-05
39
  - LoRA: r=64, alpha=128
40
 
41
  ## Usage
 
46
  import torch
47
 
48
  base = "Qwen/Qwen3-4B-Instruct-2507"
49
+ adapter = "nikeda/sft-lora"
50
 
51
  tokenizer = AutoTokenizer.from_pretrained(base)
52
  model = AutoModelForCausalLM.from_pretrained(
 
59
 
60
  ## Sources & Terms (IMPORTANT)
61
 
62
+ Training data: u-10bei/structured_data_with_cot_dataset_512_v5
 
 
 
 
63
 
64
  Dataset License: MIT License. This dataset is used and distributed under the terms of the MIT License.
65
  Compliance: Users must comply with the MIT license (including copyright notice) and the base model's original terms of use.
adapter_config.json CHANGED
@@ -33,13 +33,13 @@
33
  "rank_pattern": {},
34
  "revision": null,
35
  "target_modules": [
36
- "v_proj",
37
- "up_proj",
38
  "gate_proj",
39
- "q_proj",
40
  "k_proj",
41
  "down_proj",
42
- "o_proj"
 
 
43
  ],
44
  "target_parameters": null,
45
  "task_type": "CAUSAL_LM",
 
33
  "rank_pattern": {},
34
  "revision": null,
35
  "target_modules": [
36
+ "o_proj",
 
37
  "gate_proj",
 
38
  "k_proj",
39
  "down_proj",
40
+ "v_proj",
41
+ "q_proj",
42
+ "up_proj"
43
  ],
44
  "target_parameters": null,
45
  "task_type": "CAUSAL_LM",
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:341c02ca5c2e621e8620d9c6f343b77b126a3236d3284e003e467ef2ffd3c924
3
  size 528550256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0795e818582fa11730ec1bee84e04a0b5c02b241d65ed2455f6ac858091987a9
3
  size 528550256