Bayhakl1 commited on
Commit
935a074
·
verified ·
1 Parent(s): e194537

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -9
README.md CHANGED
@@ -17,9 +17,9 @@ tags:
17
  - llama
18
  ---
19
 
20
- # GrimmCoder (PyEdu Finetuned)
21
 
22
- **GrimmCoder** is a ~107M-parameter LLaMA-style decoder-only transformer trained from scratch exclusively on Python source code for both **left-to-right completion** and **Fill-in-the-Middle (FIM)** code completion.
23
 
24
  This release represents **Phase 2 finetuning** on `Avelina/python-edu-cleaned` (774.1M tokens, 6,000 steps with cosine decay `1e-4 → 1e-5`), initialized from the 8B-token `bigcode/starcoderdata` pretrain base checkpoint.
25
 
@@ -44,15 +44,15 @@ Evaluated on [`openai/openai_humaneval`](https://huggingface.co/datasets/openai/
44
 
45
  | Model Version | Mode & Decoding | `pass@1` | Solved | Syntax Valid |
46
  |---|---|---|---|---|
47
- | **GrimmCoder (PyEdu Finetuned)** | Left-to-Right: greedy (`do_sample=False`) | **9.15%** | **15 / 164** | **137 / 164 (83.5%)** |
48
- | **GrimmCoder (PyEdu Finetuned)** | Left-to-Right: `t=0.2, rep=1.1` | **9.15%** | **15 / 164** | **158 / 164 (96.3%)** |
49
- | **GrimmCoder (PyEdu Finetuned)** | **FIM (PSM): greedy (`do_sample=False`)** | **9.15%** | **15 / 164** | **151 / 164 (92.1%)** |
50
- | **GrimmCoder (PyEdu Finetuned)** | **FIM (PSM): `t=0.2, rep=1.1`** | **8.54%** | **14 / 164** | **153 / 164 (93.3%)** |
51
  | GrimmCoder (Base Pretrain) | Left-to-Right: greedy (`do_sample=False`) | 5.49% | 9 / 164 | 126 / 164 (76.8%) |
52
  | GrimmCoder (Base Pretrain) | Left-to-Right: `t=0.2, rep=1.1` | 4.88% | 8 / 164 | 155 / 164 (94.5%) |
53
  | GrimmCoder (Base Pretrain) | FIM (PSM): greedy (`do_sample=False`) | 4.27% | 7 / 164 | 122 / 164 (74.4%) |
54
 
55
- ### Solved Problems on Exported HF Model
56
  - **L2R Greedy (15)**: `HumanEval/16`, `22`, `23`, `28`, `29`, `30`, `31`, `34`, `45`, `48`, `50`, `52`, `53`, `55`, `60`
57
  - **L2R t0.2_rep1.1 (15)**: `HumanEval/16`, `22`, `23`, `28`, `29`, `30`, `31`, `34`, `45`, `48`, `50`, `53`, `58`, `60`, `61`
58
  - **FIM Greedy (15)**: `HumanEval/16`, `22`, `23`, `28`, `29`, `30`, `31`, `42`, `45`, `48`, `50`, `53`, `55`, `58`, `60`
@@ -77,7 +77,7 @@ Evaluated on [`openai/openai_humaneval`](https://huggingface.co/datasets/openai/
77
  import torch
78
  from transformers import AutoModelForCausalLM, AutoTokenizer
79
 
80
- model_id = "Bayhakl1/GrimmCoder"
81
  tokenizer = AutoTokenizer.from_pretrained(model_id)
82
  model = AutoModelForCausalLM.from_pretrained(
83
  model_id,
@@ -103,7 +103,7 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
103
 
104
  ### 2. Fill-in-the-Middle (FIM)
105
 
106
- GrimmCoder natively supports Fill-in-the-Middle (FIM) to insert code between a **prefix** (code before the cursor) and a **suffix** (code after the cursor). The model was trained 50/50 on **PSM** and **SPM** formats.
107
 
108
  > [!NOTE]
109
  > Special tokens require vertical pipe characters: `<|fim_prefix|>`, `<|fim_suffix|>`, and `<|fim_middle|>`.
 
17
  - llama
18
  ---
19
 
20
+ # GrimmCoder_v2
21
 
22
+ **GrimmCoder_v2** is a ~107M-parameter LLaMA-style decoder-only transformer trained from scratch exclusively on Python source code for both **left-to-right completion** and **Fill-in-the-Middle (FIM)** code completion.
23
 
24
  This release represents **Phase 2 finetuning** on `Avelina/python-edu-cleaned` (774.1M tokens, 6,000 steps with cosine decay `1e-4 → 1e-5`), initialized from the 8B-token `bigcode/starcoderdata` pretrain base checkpoint.
25
 
 
44
 
45
  | Model Version | Mode & Decoding | `pass@1` | Solved | Syntax Valid |
46
  |---|---|---|---|---|
47
+ | **GrimmCoder_v2** | Left-to-Right: greedy (`do_sample=False`) | **9.15%** | **15 / 164** | **137 / 164 (83.5%)** |
48
+ | **GrimmCoder_v2** | Left-to-Right: `t=0.2, rep=1.1` | **9.15%** | **15 / 164** | **158 / 164 (96.3%)** |
49
+ | **GrimmCoder_v2** | **FIM (PSM): greedy (`do_sample=False`)** | **9.15%** | **15 / 164** | **151 / 164 (92.1%)** |
50
+ | **GrimmCoder_v2** | **FIM (PSM): `t=0.2, rep=1.1`** | **8.54%** | **14 / 164** | **153 / 164 (93.3%)** |
51
  | GrimmCoder (Base Pretrain) | Left-to-Right: greedy (`do_sample=False`) | 5.49% | 9 / 164 | 126 / 164 (76.8%) |
52
  | GrimmCoder (Base Pretrain) | Left-to-Right: `t=0.2, rep=1.1` | 4.88% | 8 / 164 | 155 / 164 (94.5%) |
53
  | GrimmCoder (Base Pretrain) | FIM (PSM): greedy (`do_sample=False`) | 4.27% | 7 / 164 | 122 / 164 (74.4%) |
54
 
55
+ ### Solved Problems
56
  - **L2R Greedy (15)**: `HumanEval/16`, `22`, `23`, `28`, `29`, `30`, `31`, `34`, `45`, `48`, `50`, `52`, `53`, `55`, `60`
57
  - **L2R t0.2_rep1.1 (15)**: `HumanEval/16`, `22`, `23`, `28`, `29`, `30`, `31`, `34`, `45`, `48`, `50`, `53`, `58`, `60`, `61`
58
  - **FIM Greedy (15)**: `HumanEval/16`, `22`, `23`, `28`, `29`, `30`, `31`, `42`, `45`, `48`, `50`, `53`, `55`, `58`, `60`
 
77
  import torch
78
  from transformers import AutoModelForCausalLM, AutoTokenizer
79
 
80
+ model_id = "Bayhakl1/GrimmCoder_v2"
81
  tokenizer = AutoTokenizer.from_pretrained(model_id)
82
  model = AutoModelForCausalLM.from_pretrained(
83
  model_id,
 
103
 
104
  ### 2. Fill-in-the-Middle (FIM)
105
 
106
+ GrimmCoder_v2 natively supports Fill-in-the-Middle (FIM) to insert code between a **prefix** (code before the cursor) and a **suffix** (code after the cursor). The model was trained 50/50 on **PSM** and **SPM** formats.
107
 
108
  > [!NOTE]
109
  > Special tokens require vertical pipe characters: `<|fim_prefix|>`, `<|fim_suffix|>`, and `<|fim_middle|>`.