|
|
--- |
|
|
license: apache-2.0 |
|
|
base_model: |
|
|
- ByteDance-Seed/Seed-Coder-8B-Base |
|
|
--- |
|
|
|
|
|
# Seed-Coder-8B-Reasoning |
|
|
|
|
|
## Introduction |
|
|
**Seed-Coder-8B-Reasoning** is an 8-billion-parameter model further optimized for **code reasoning**, **problem-solving**, and **algorithmic thinking** tasks. |
|
|
Built upon the strong base of Seed-Coder, it undergoes additional training in sandbox environments to significantly enhance its ability to tackle complex coding problems and competitions. It features: |
|
|
- Trained on a **massively curated corpus**, filtered using an **LLM-based method** to ensure high-quality real-world code, text-code alignment, and synthetic datasets. |
|
|
- **Sandbox fine-tuning** to specifically strengthen **multi-step reasoning**, **algorithm design**, and **competitive programming** capabilities. |
|
|
- Maintains **long-context handling** up to 32K tokens, enabling it to reason over extended problem descriptions and large input-output examples. |
|
|
|
|
|
<p align="center"> |
|
|
<img width="100%" src="imgs/large_model_comparison_split-crop.pdf"> |
|
|
</p> |
|
|
|
|
|
## Model Downloads |
|
|
| Model Name | Type | Length | Download | |
|
|
|---------------------------------------------------------|----------|--------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
|
|
| Seed-Coder-8B-Base | base | 32k | 🤗 [Hugging Face](https://huggingface.co/ByteDance-Seed/Seed-Coder-8B-Base) | |
|
|
| Seed-Coder-8B-Instruct | instruct | 32k | 🤗 [Hugging Face](https://huggingface.co/ByteDance-Seed/Seed-Coder-8B-Instruct) | |
|
|
| **👉Seed-Coder-8B-Reasoning** | reasoning | 32k | 🤗 [Hugging Face](https://huggingface.co/ByteDance-Seed/Seed-Coder-8B-Reasoning) | |
|
|
|
|
|
|
|
|
## Requirements |
|
|
You will need to install the latest versions of `transformers` and `accelerate`: |
|
|
|
|
|
```bash |
|
|
pip install -U transformers accelerate |
|
|
``` |
|
|
|
|
|
## Quickstart |
|
|
|
|
|
Here is a simple example demonstrating how to load the model and perform code generation using the Hugging Face `pipeline` API: |
|
|
|
|
|
```python |
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
import torch |
|
|
|
|
|
model_id = "ByteDance-Seed/Seed-Coder-8B-Reasoning" |
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) |
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True) |
|
|
|
|
|
messages = [ |
|
|
{"role": "user", "content": "Write a quick sort algorithm."}, |
|
|
] |
|
|
|
|
|
input_ids = tokenizer.apply_chat_template( |
|
|
messages, |
|
|
tokenize=True, |
|
|
return_tensors="pt", |
|
|
add_generation_prompt=True, |
|
|
).to(model.device) |
|
|
|
|
|
outputs = model.generate(input_ids, max_new_tokens=16384) |
|
|
response = tokenizer.decode(outputs[0][input_ids.shape[-1]:], skip_special_tokens=True) |
|
|
print(response) |
|
|
``` |
|
|
|
|
|
## Evaluation |
|
|
|
|
|
Seed-Coder-8B-Reasoning has been evaluated extensively on reasoning-intensive code benchmarks, showing: |
|
|
- Significant improvements on **competitive programming** datasets and coding challenges. |
|
|
- Enhanced ability to **break down complex problems**, **design correct algorithms**, and **produce efficient implementations**. |
|
|
- Strong generalization to unseen problems across multiple domains (math, strings, arrays, graphs, DP, etc.). |
|
|
|
|
|
For detailed results, please check our paper. |
|
|
<!-- For detailed results, please check our [📑 paper](https://arxiv.org/pdf/xxx.xxxxx). --> |
|
|
|
