ntduc0901/llama3-8b-debugeval-lora

Llama-3-8B DebugEval LoRA (NeuDebugger)

Fine-tuned LoRA adapter for Meta-Llama-3-8B-Instruct on the DebugEval / COAST code-debugging benchmark.

Training Details

Parameter	Value
Base model	meta-llama/Meta-Llama-3-8B-Instruct
Method	LoRA (PEFT)
LoRA rank	8
LoRA alpha	32
LoRA dropout	0.1
Target modules	q_proj, k_proj, v_proj, o_proj
Training data	yangweiqing/DebugEval train split (24,892 samples)
Epochs	1
Batch size	4 × 8 (grad accum) = 32
Learning rate	2e-5 (cosine decay)
Max seq length	2048
Final train loss	0.1765
Hardware	NVIDIA H100 80GB
Training time	~42 minutes

Tasks Covered

• BUG Localization: identify the error code snippet from options
• BUG Identification: classify bug type (Syntax / Reference / Logical / Multiple)
• Code Repair: generate the corrected code
• Code Review: determine which of two code snippets contains a bug

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base = "meta-llama/Meta-Llama-3-8B-Instruct"
adapter = "ntduc0901/llama3-8b-debugeval-lora"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)
model.eval()

prompt = """
You are an AI programming assistant, developed by NEUIR, and you only answer questions related to computer science.
### Instruction:
Given the following buggy code, identify the type of error.
Choose one: (A) Syntax Error (B) Reference Error (C) Logical Error (D) Multiple Errors

def factorial(n):
    if n == 0:
        return 1
    return n * factorial(n - 2)

Final answer format: <Answer>(Option)</Answer>.
### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=256, temperature=0.2, do_sample=True)
print(tokenizer.decode(out[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))

Citation

@misc{yang2025coastenhancingcodedebugging,
  title={COAST: Enhancing the Code Debugging Ability of LLMs through Communicative Agent Based Data Synthesis},
  author={Weiqing Yang and Hanbin Wang and Zhenghao Liu and Xinze Li and Yukun Yan and Shuo Wang and Yu Gu and Minghe Yu and Zhiyuan Liu and Ge Yu},
  year={2025},
  eprint={2408.05006},
  archivePrefix={arXiv},
  primaryClass={cs.SE},
  url={https://arxiv.org/abs/2408.05006},
}