Upload 4 files
Browse files- .gitattributes +2 -0
- LICENSE +26 -0
- README.md +167 -5
- papers/iquest-coder-v1-logo.png +3 -0
- papers/results.png +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
papers/iquest-coder-v1-logo.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
papers/results.png filter=lfs diff=lfs merge=lfs -text
|
LICENSE
CHANGED
|
@@ -0,0 +1,26 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Modified MIT License
|
| 2 |
+
|
| 3 |
+
Software Copyright© 2025 IQuest Research
|
| 4 |
+
|
| 5 |
+
Our only modification is that, if the Software (or any derivative works
|
| 6 |
+
thereof) is used for any of your commercial products or services, you shall
|
| 7 |
+
prominently display "IQuest Coder" on the user interface of such product or
|
| 8 |
+
service.
|
| 9 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 10 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 11 |
+
in the Software without restriction, including without limitation the rights
|
| 12 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 13 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 14 |
+
furnished to do so, subject to the following conditions:
|
| 15 |
+
|
| 16 |
+
The above copyright notice and this permission notice shall be included in all
|
| 17 |
+
copies or substantial portions of the Software.
|
| 18 |
+
|
| 19 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 20 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 21 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 22 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 23 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 24 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
| 25 |
+
|
| 26 |
+
|
README.md
CHANGED
|
@@ -1,5 +1,167 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+

|
| 2 |
+
|
| 3 |
+
<p align="center">
|
| 4 |
+
📘 <a href="https://iquestlab.github.io">Blog</a >
|
| 5 |
+
•
|
| 6 |
+
📄 <a href="https://github.com/IQuestLab/IQuest-Coder-V1/blob/main/papers/IQuest_Coder_Technical_Report.pdf">Technical Report</a >
|
| 7 |
+
</p >
|
| 8 |
+
|
| 9 |
+
# IQuest-Coder-V1 Model Family
|
| 10 |
+
|
| 11 |
+
| Model | Link |
|
| 12 |
+
|-------|------|
|
| 13 |
+
| IQuest-Coder-V1-40B-Base-Stage1 | [🤗 Hugging Face](https://huggingface.co/IQuestLab/IQuest-Coder-V1-40B-Base-Stage1) |
|
| 14 |
+
| IQuest-Coder-V1-40B-Base | [🤗 Hugging Face](https://huggingface.co/IQuestLab/IQuest-Coder-V1-40B-Base) |
|
| 15 |
+
| IQuest-Coder-V1-40B-Instruct | [🤗 Hugging Face](https://huggingface.co/IQuestLab/IQuest-Coder-V1-40B-Instruct) |
|
| 16 |
+
| IQuest-Coder-V1-40B-Loop-Instruct | [🤗 Hugging Face](https://huggingface.co/IQuestLab/IQuest-Coder-V1-40B-Loop-Instruct) |
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
## IQuest-Coder-V1 Highlights
|
| 21 |
+
|
| 22 |
+
IQuest-Coder-V1 is a new family of code large language models (LLMs) designed to advance autonomous software engineering and code intelligence. Built on the innovative code-flow multi-stage training paradigm, IQuest-Coder-V1 captures the dynamic evolution of software logic, delivering state-of-the-art performance across critical dimensions:
|
| 23 |
+
|
| 24 |
+
- **State-of-the-Art Performance**: Achieves leading results on SWE-Bench Verified (81.4%), BigCodeBench (51.4%), LiveCodeBench v6 (87.0%), and other major coding benchmarks, surpassing competitive models across agentic software engineering, competitive programming, and complex tool use.
|
| 25 |
+
- **Code-Flow Training Paradigm**: Moving beyond static code representations, our models learn from repository evolution patterns, commit transitions, and dynamic code transformations to understand real-world software development processes.
|
| 26 |
+
- **Dual Specialization Paths**: Bifurcated post-training delivers two specialized variants—Thinking models (utilizing reasoning-driven RL for complex problem-solving) and Instruct models (optimized for general coding assistance and instruction-following).
|
| 27 |
+
- **Efficient Architecture**: The IQuest-Coder-V1-Loop variant introduces a recurrent mechanism that optimizes the trade-off between model capacity and deployment footprint.
|
| 28 |
+
- **Native Long Context**: All models natively support up to 128K tokens without requiring additional scaling techniques.
|
| 29 |
+
|
| 30 |
+
## Model Overview
|
| 31 |
+
|
| 32 |
+
The IQuest-Coder-V1 series includes models ranging from 7B to 40B parameters, with both standard and Loop variants:
|
| 33 |
+
|
| 34 |
+
| Model | Parameters | Layers | Hidden Size | Attention Heads (Q/KV) | Context Length |
|
| 35 |
+
|-------|------------|--------|-------------|------------------------|----------------|
|
| 36 |
+
| IQuest-Coder-V1-7B-Instruct | 7B | 14 | 5120 | 40/8 | 128K |
|
| 37 |
+
| IQuest-Coder-V1-7B-Thinking | 7B | 14 | 5120 | 40/8 | 128K |
|
| 38 |
+
| IQuest-Coder-V1-14B-Instruct | 14B | 28 | 5120 | 40/8 | 128K |
|
| 39 |
+
| IQuest-Coder-V1-14B-Thinking | 14B | 28 | 5120 | 40/8 | 128K |
|
| 40 |
+
| IQuest-Coder-V1-40B-Instruct | 40B | 80 | 5120 | 40/8 | 128K |
|
| 41 |
+
| IQuest-Coder-V1-40B-Thinking | 40B | 80 | 5120 | 40/8 | 128K |
|
| 42 |
+
| IQuest-Coder-V1-40B-Loop-Instruct | 40B | 80 (2 iterations) | 5120 | 40/8 | 128K |
|
| 43 |
+
| IQuest-Coder-V1-40B-Loop-Thinking | 40B | 80 (2 iterations) | 5120 | 40/8 | 128K |
|
| 44 |
+
|
| 45 |
+
**Architecture Features:**
|
| 46 |
+
|
| 47 |
+
- Grouped Query Attention (GQA) for efficient inference
|
| 48 |
+
- Native 128K context length support
|
| 49 |
+
- Vocabulary size: 76,800 tokens
|
| 50 |
+
- Loop variants use recurrent transformer design with shared parameters across two iterations
|
| 51 |
+
|
| 52 |
+
For more details, please refer to our Technical Report, GitHub.
|
| 53 |
+
|
| 54 |
+
## Quickstart
|
| 55 |
+
|
| 56 |
+
IQuest-Coder-V1 uses custom modeling code via Hugging Face's auto_map feature. We recommend using transformers>=4.52.4.
|
| 57 |
+
|
| 58 |
+
### Basic Usage with Transformers
|
| 59 |
+
|
| 60 |
+
```python
|
| 61 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 62 |
+
|
| 63 |
+
model_name = "IQuest/IQuest-Coder-V1-40B-Instruct"
|
| 64 |
+
|
| 65 |
+
# Load the tokenizer and model
|
| 66 |
+
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 67 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 68 |
+
model_name,
|
| 69 |
+
torch_dtype="auto",
|
| 70 |
+
device_map="auto"
|
| 71 |
+
)
|
| 72 |
+
|
| 73 |
+
# Prepare the input
|
| 74 |
+
prompt = "Write a Python function to calculate the Fibonacci sequence using dynamic programming."
|
| 75 |
+
messages = [
|
| 76 |
+
{"role": "user", "content": prompt}
|
| 77 |
+
]
|
| 78 |
+
text = tokenizer.apply_chat_template(
|
| 79 |
+
messages,
|
| 80 |
+
tokenize=False,
|
| 81 |
+
add_generation_prompt=True
|
| 82 |
+
)
|
| 83 |
+
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
| 84 |
+
|
| 85 |
+
# Generate response
|
| 86 |
+
generated_ids = model.generate(
|
| 87 |
+
**model_inputs,
|
| 88 |
+
max_new_tokens=8192
|
| 89 |
+
)
|
| 90 |
+
generated_ids = generated_ids[0][len(model_inputs.input_ids[0]):]
|
| 91 |
+
response = tokenizer.decode(generated_ids, skip_special_tokens=True)
|
| 92 |
+
|
| 93 |
+
print(response)
|
| 94 |
+
```
|
| 95 |
+
|
| 96 |
+
### Using Thinking Models
|
| 97 |
+
|
| 98 |
+
For complex reasoning tasks, use the Thinking variant:
|
| 99 |
+
|
| 100 |
+
```python
|
| 101 |
+
model_name = "IQuest/IQuest-Coder-V1-40B-Thinking"
|
| 102 |
+
|
| 103 |
+
# The Thinking model includes explicit reasoning traces
|
| 104 |
+
# Use similar code as above, but expect longer, more detailed responses
|
| 105 |
+
# with step-by-step problem decomposition
|
| 106 |
+
```
|
| 107 |
+
|
| 108 |
+
### Deployment with vLLM
|
| 109 |
+
|
| 110 |
+
For production deployment, you can use vLLM to create an OpenAI-compatible API endpoint. Please refer to the vLLM PR for implementation details.
|
| 111 |
+
|
| 112 |
+
```bash
|
| 113 |
+
vllm serve IQuest/IQuest-Coder-V1-40B-Instruct --tensor-parallel-size 8
|
| 114 |
+
```
|
| 115 |
+
|
| 116 |
+
For Thinking models with reasoning support:
|
| 117 |
+
|
| 118 |
+
```bash
|
| 119 |
+
vllm serve IQuest/IQuest-Coder-V1-40B-Thinking --enable-reasoning --reasoning-parser deepseek_r1 --tensor-parallel-size 8
|
| 120 |
+
```
|
| 121 |
+
|
| 122 |
+
|
| 123 |
+
## Evaluation Results
|
| 124 |
+

|
| 125 |
+
|
| 126 |
+
## Limitations
|
| 127 |
+
|
| 128 |
+
- **Reasoning vs. Efficiency Trade-off**: Thinking models provide superior reasoning but generate longer responses; Instruct models are more efficient for straightforward tasks.
|
| 129 |
+
- **Code Execution**: Models generate code but do not execute it; always validate outputs in sandboxed environments.
|
| 130 |
+
- **Domain Specificity**: While trained on diverse codebases, performance may vary on highly specialized or proprietary frameworks.
|
| 131 |
+
- **Factuality**: Models may generate plausible but incorrect code; verify critical implementations thoroughly.
|
| 132 |
+
|
| 133 |
+
## Citation
|
| 134 |
+
|
| 135 |
+
If you find our work helpful, please cite:
|
| 136 |
+
|
| 137 |
+
```bibtex
|
| 138 |
+
@article{iquest-coder-v1-2025,
|
| 139 |
+
title={IQuest-Coder-V1 Technical Report},
|
| 140 |
+
author={IQuest Coder Team},
|
| 141 |
+
url={https://github.com/IQuestLab/IQuest-Coder-V1/blob/main/papers/IQuest_Coder_Technical_Report.pdf}
|
| 142 |
+
year={2025}
|
| 143 |
+
}
|
| 144 |
+
@article{codescaling,
|
| 145 |
+
title={Scaling Laws for Code: Every Programming Language Matters},
|
| 146 |
+
author={Yang, Jian and Guo, Shawn and Jing, Lin and Zhang, Wei and Liu, Aishan and Hao, Chuan and Li, Zhoujun and Zhao, Wayne Xin and Liu, Xianglong and Lv, Weifeng and others},
|
| 147 |
+
journal={arXiv preprint arXiv:2512.13472},
|
| 148 |
+
year={2025}
|
| 149 |
+
}
|
| 150 |
+
@article{close_the_loop,
|
| 151 |
+
title={Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing},
|
| 152 |
+
author={Yuwen Li, Wei Zhang, Zelong Huang, Mason Yang, Jiajun Wu, Shawn Guo, Huahao Hu, Lingyi Sun, Jian Yang, Mingjie Tang, Byran Dai},
|
| 153 |
+
journal={arXiv preprint arXiv:2512.23611},
|
| 154 |
+
year={2025}
|
| 155 |
+
}
|
| 156 |
+
@article{loopcoder,
|
| 157 |
+
title={LoopCoder: Scaling Code Intelligence via Looped Language Models},
|
| 158 |
+
author={Jian Yang, Wei Zhang, Shawn Guo, Yizhi Li, Lin Jing, Zhengmao Ye, Shark Liu, Yuyang Song, Jiajun Wu, Che Liu, T. Zheng, Siwei Wu, L. Liao, X. Ma, Chuan Hao, Ran Tao, Yan Xing, Jianzhou Wang, Mingjie Tang, Aishan Liu, Zhoujun Li, Xianglong Liu, Weifeng Lv1, Bryan Dai},
|
| 159 |
+
year={2025}
|
| 160 |
+
}
|
| 161 |
+
@article{swe_compress,
|
| 162 |
+
title={Context as a Tool: Context Management for Long-Horizon SWE-Agents},
|
| 163 |
+
author={hukai Liu, Jian Yang, Bo Jiang, Yizhi Li, Jinyang Guo, Xianglong Liu, Bryan Dai},
|
| 164 |
+
journal={arXiv preprint arXiv:2512.22087},
|
| 165 |
+
year={2025}
|
| 166 |
+
}
|
| 167 |
+
```
|
papers/iquest-coder-v1-logo.png
ADDED
|
Git LFS Details
|
papers/results.png
ADDED
|
Git LFS Details
|