Text Generation
Transformers
Safetensors
qwen2
code
software-engineering
agent
conversational
text-generation-inference
ubowang commited on
Commit
594b765
·
verified ·
1 Parent(s): 0abc0f5

Update model card: official GitHub/dataset links, unified training configs, results, citation

Browse files
Files changed (1) hide show
  1. README.md +46 -12
README.md CHANGED
@@ -4,32 +4,48 @@ library_name: transformers
4
  pipeline_tag: text-generation
5
  base_model:
6
  - Qwen/Qwen2.5-Coder-7B-Instruct
 
 
 
7
  tags:
8
  - code
9
  - software-engineering
10
  - agent
11
  ---
12
 
13
- # FIM-7B Inference on SWE-Bench Verified
14
 
15
- This guide describes how to run the **FIM-7B** checkpoint on SWE-Bench Verified (and Lite) with the R2E-Gym agent scaffold in this repository.
16
 
17
- ## Model
18
 
19
- Local path: `models/FIM-7B/` (checkpoints are gitignored; do not commit them).
20
 
21
- - Base model: `Qwen/Qwen2.5-Coder-7B-Instruct`
22
- - FIM mid-training: `train/FIM_Midtrain_7B.yaml`
23
- - Post-training: SFT on R2E-Gym agent trajectories
24
 
 
25
 
26
- ## 1. Serve the model with vLLM
 
 
 
 
 
 
 
 
 
 
 
 
27
 
28
  ```bash
29
  CUDA_VISIBLE_DEVICES=0 \
30
  VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
31
- .venv-vllm/bin/python -m vllm.entrypoints.openai.api_server \
32
- --model models/FIM-7B \
33
  --served-model-name FIM-7B \
34
  --host 127.0.0.1 \
35
  --port 8400 \
@@ -41,14 +57,15 @@ VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
41
  > vllm_fim7b.log 2>&1 &
42
  ```
43
 
44
-
45
  Wait until the server is up (model load takes ~1 minute):
46
 
47
  ```bash
48
  curl -s http://127.0.0.1:8400/v1/models
49
  ```
50
 
51
- ## 2. Run the agent on SWE-Bench Verified
 
 
52
 
53
  ```bash
54
  export OPENAI_API_KEY=EMPTY
@@ -73,3 +90,20 @@ uv run python src/r2egym/agenthub/run/edit.py runagent_multiple \
73
  --max_tokens 65536 \
74
  --use_existing True
75
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  pipeline_tag: text-generation
5
  base_model:
6
  - Qwen/Qwen2.5-Coder-7B-Instruct
7
+ datasets:
8
+ - TIGER-Lab/FIM-Midtraining-400K
9
+ - R2E-Gym/R2EGym-SFT-Trajectories
10
  tags:
11
  - code
12
  - software-engineering
13
  - agent
14
  ---
15
 
16
+ # FIM-7B
17
 
18
+ [📄 Paper (PDF)](https://github.com/TIGER-AI-Lab/FIM-Midtraining/blob/main/paper.pdf) · [💻 GitHub](https://github.com/TIGER-AI-Lab/FIM-Midtraining) · [🤗 Dataset](https://huggingface.co/datasets/TIGER-Lab/FIM-Midtraining-400K) · [🤗 Collection](https://huggingface.co/collections/TIGER-Lab/fim-midtraining)
19
 
20
+ **FIM-7B** is the 7B coding-agent model of *"Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models"*: `Qwen2.5-Coder-7B-Instruct`, mid-trained on function-aware FIM data, then post-trained on R2E-Gym agent trajectories with the upstream recipe unmodified. The mid-training stage is the only difference from a standard R2E-Gym reproduction — and it is worth **+2.8 points on SWE-Bench-Verified and +3.7 on SWE-Bench-Lite**.
21
 
22
+ ## Training pipeline
23
 
24
+ - **Base model**: [`Qwen/Qwen2.5-Coder-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct)
25
+ - **FIM mid-training**: [`midtraining/configs/fim_midtrain.yaml`](https://github.com/TIGER-AI-Lab/FIM-Midtraining/blob/main/midtraining/configs/fim_midtrain.yaml) on [TIGER-Lab/FIM-Midtraining-400K](https://huggingface.co/datasets/TIGER-Lab/FIM-Midtraining-400K) (as-run copy: [`FIM_Midtrain_7B.yaml`](https://github.com/TIGER-AI-Lab/FIM-Midtraining/blob/main/midtraining/configs/FIM_Midtrain_7B.yaml)) → intermediate checkpoint released as [TIGER-Lab/FIM-Mid-7B](https://huggingface.co/TIGER-Lab/FIM-Mid-7B)
26
+ - **Post-training**: SFT on R2E-Gym agent trajectories — [`posttraining/r2egym/`](https://github.com/TIGER-AI-Lab/FIM-Midtraining/tree/main/posttraining/r2egym) (as-run copy: [`FIM_Posttrain_7B.yaml`](https://github.com/TIGER-AI-Lab/FIM-Midtraining/blob/main/posttraining/r2egym/FIM_Posttrain_7B.yaml))
27
 
28
+ ## Results
29
 
30
+ Means over three evaluation seeds, identical harness for both arms (paper Table 1):
31
+
32
+ | Setting | SWE-Bench-Verified | SWE-Bench-Lite |
33
+ |---|---|---|
34
+ | Qwen2.5-Coder-7B-Instruct + R2E-Gym (reproduced) | 15.00 | 11.33 |
35
+ | **FIM-7B (+ FIM mid-training)** | **17.80** | **15.00** |
36
+ | Δ | +2.80 | +3.67 |
37
+
38
+ ## Evaluate on SWE-Bench Verified
39
+
40
+ FIM-7B is evaluated with the **R2E-Gym agent scaffold** (fixed by its post-training pipeline). The complete pinned walkthrough lives at [`evaluation/swebench/released_checkpoints.md`](https://github.com/TIGER-AI-Lab/FIM-Midtraining/blob/main/evaluation/swebench/released_checkpoints.md).
41
+
42
+ ### 1. Serve the model with vLLM
43
 
44
  ```bash
45
  CUDA_VISIBLE_DEVICES=0 \
46
  VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
47
+ python -m vllm.entrypoints.openai.api_server \
48
+ --model TIGER-Lab/FIM-7B \
49
  --served-model-name FIM-7B \
50
  --host 127.0.0.1 \
51
  --port 8400 \
 
57
  > vllm_fim7b.log 2>&1 &
58
  ```
59
 
 
60
  Wait until the server is up (model load takes ~1 minute):
61
 
62
  ```bash
63
  curl -s http://127.0.0.1:8400/v1/models
64
  ```
65
 
66
+ ### 2. Run the agent on SWE-Bench Verified
67
+
68
+ From an upstream, unmodified [R2E-Gym](https://github.com/R2E-Gym/R2E-Gym) checkout (Docker required):
69
 
70
  ```bash
71
  export OPENAI_API_KEY=EMPTY
 
90
  --max_tokens 65536 \
91
  --use_existing True
92
  ```
93
+
94
+ For SWE-Bench Lite, use `--dataset "R2E-Gym/SWE-Bench-Lite" --k 300`.
95
+
96
+ ### 3. Score with the official SWE-bench harness
97
+
98
+ Convert the trajectories to a submission and score with the official harness — [`evaluation/swebench/score.sh`](https://github.com/TIGER-AI-Lab/FIM-Midtraining/blob/main/evaluation/swebench/score.sh). The reported number is `resolved_instances / total_instances`.
99
+
100
+ ## Citation
101
+
102
+ ```bibtex
103
+ @article{wang2026fim,
104
+ title={Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models},
105
+ author={Wang, Yubo and Liang, Jiarong and Zhang, Yuxuan and Liu, Xuye and Wei, Cong and Zhang, Yuyu and Nie, Ping and Chen, Wenhu},
106
+ journal={arXiv preprint},
107
+ year={2026}
108
+ }
109
+ ```