README.md

---
license: apache-2.0
pipeline_tag: text-generation
library_name: transformers
---

# Jacobi Forcing: Fast and Accurate Causal Parallel Decoding

This repository contains the `JacobiForcing_Coder_7B_v1` model, presented in the paper [Fast and Accurate Causal Parallel Decoding using Jacobi Forcing](https://huggingface.co/papers/2512.14681).

Base Model: [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct)

Training Data (Jacobi trajectories): https://huggingface.co/datasets/JacobiForcing/OpenCodeInstruct_training_data_n32

Jacobi Forcing is a novel training technique that converts Large Language Models (LLMs) into native causal parallel decoders. This approach maintains the causal autoregressive backbone and addresses the AR-to-diffusion mismatch by training the model to handle noisy future blocks along its own Jacobi decoding trajectories.

It achieves up to $4.5\times$ higher tokens-per-forward and $4\times$ wall-clock speedup on coding and math tasks, while retaining near-AR generation quality.

You can find more details on the project blog: [Jacobi Forcing Blog](https://hao-ai-lab.github.io/blogs/jacobi-forcing/)
The official code repository is available here: [GitHub Repository](https://github.com/hao-ai-lab/JacobiForcing)

## Usage

You can try the chatbot demo locally or use the provided Python inference code.

### Local Chatbot Demo
```bash
# modify the script to use your local path
streamlit run applications/jacobi_model_chat.py
```

### Inference with Code
You can use our provided `eagenerate` function for speedup generation, similar to using `generate` from Hugging Face. Here is an example:

```python
from eagle.model.ea_model import EaModel
from fastchat.model import get_conversation_template
import torch

# Assuming base_model_path and EAGLE_model_path are defined
# For example:
base_model_path = "Qwen/Qwen2.5-Coder-7B-Instruct"
EAGLE_model_path = "JacobiForcing/JacobiForcing_Coder_7B_v1" # Or your local path to the weights

model = EaModel.from_pretrained(
    base_model_path=base_model_path,
    ea_model_path=EAGLE_model_path,
    torch_dtype=torch.float16,
    low_cpu_mem_usage=True,
    device_map="auto",
    total_token=-1 # Automatically configure draft tokens
)
model.eval()

your_message="Hello"
conv = get_conversation_template("vicuna") # Use appropriate conversation template for your base model
conv.append_message(conv.roles[0], your_message)
conv.append_message(conv.roles[1], None)
prompt = conv.get_prompt()

input_ids = model.tokenizer([prompt]).input_ids
input_ids = torch.as_tensor(input_ids).cuda()

output_ids = model.eagenerate(input_ids, temperature=0.5, max_new_tokens=512)
output = model.tokenizer.decode(output_ids[0])
print(output)
```