revana commited on
Commit
99d3fe6
·
verified ·
1 Parent(s): 3630c83

Add model card

Browse files
Files changed (1) hide show
  1. README.md +77 -0
README.md ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ - code
5
+ license: apache-2.0
6
+ base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
7
+ tags:
8
+ - lora
9
+ - code
10
+ - qwen2.5-coder
11
+ - fingpt
12
+ - code-correction
13
+ pipeline_tag: text-generation
14
+ ---
15
+
16
+ # fingpt-coder-1b5
17
+
18
+ LoRA adapter for **[Qwen/Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct)** fine-tuned on
19
+ [m-a-p/Code-Feedback](https://huggingface.co/datasets/m-a-p/Code-Feedback)
20
+ (66K error→fix pairs, 3 epochs).
21
+
22
+ > **Adapter only** — the base model is loaded from the HF Hub automatically.
23
+ > Total download: ~84 MB adapter + ~3 GB base model.
24
+
25
+ ---
26
+
27
+ ## LoRA config
28
+
29
+ | Property | Value |
30
+ |----------|-------|
31
+ | Base model | `Qwen/Qwen2.5-Coder-1.5B-Instruct` |
32
+ | Rank (r) | 16 |
33
+ | Alpha | 32 (scale = 2.0) |
34
+ | Target modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
35
+ | Training step | 48500 |
36
+ | Adapter size | ~84 MB |
37
+
38
+ ---
39
+
40
+ ## Quick start
41
+
42
+ ```bash
43
+ git clone https://huggingface.co/revana/fingpt-coder-1b5
44
+ ```
45
+
46
+ ```python
47
+ import torch, sys
48
+ sys.path.insert(0, "fingpt") # fingpt repo root
49
+ from infer import load_model, generate
50
+
51
+ model, tokenizer = load_model("adapter_final.pt")
52
+
53
+ reply = generate(model, tokenizer, "Fix this bug:\n\ndef fact(n):\n return n * fact(n)")
54
+ print(reply)
55
+ ```
56
+
57
+ Or use the [live demo](https://huggingface.co/spaces/revana/fingpt).
58
+
59
+ ---
60
+
61
+ ## Training
62
+
63
+ | Property | Value |
64
+ |----------|-------|
65
+ | Dataset | [m-a-p/Code-Feedback](https://huggingface.co/datasets/m-a-p/Code-Feedback) |
66
+ | Samples | ~66K error→fix pairs |
67
+ | Epochs | 3 |
68
+ | Batch size | 4 × 4 grad accum = 16 effective |
69
+ | LR | 3e-4, cosine decay, 3% warmup |
70
+ | Precision | bfloat16 |
71
+ | Hardware | A100 80GB |
72
+
73
+ ---
74
+
75
+ ## License
76
+
77
+ Apache 2.0