TaimoorSiddiqui/Hopcoder-Mini-9B-SWE-Agent-LoRA-H200

HopCoder-Mini-9B SWE-Agent LoRA (H200)

This is the Stage-2 SWE-Agent LoRA adapter for HopCoder-Mini-9B, fine-tuned on a Modal H200 GPU. It builds on the Stage-1 native tool-call adapter with a curriculum of Open-SWE patches, xLAM function-calling data, Hermes function-calling data, and targeted CLI tool-call replays.

Model Details

Base model	TaimoorSiddiqui/Hopcoder-Mini-9B
Stage-1 adapter	TaimoorSiddiqui/Hopcoder-Mini-9B-Native-ToolCall-LoRA-H200
Adapter type	LoRA (r=16, alpha=32)
Training hardware	NVIDIA H200 (Modal)
Training time	~3h 47m
Total steps	659 (1 epoch)
Eval loss	0.4701 (step 200)
Learning rate	2.0e-5
Batch size	2 × grad_accum 8 = 16 effective
Max sequence length	4096

Training Data

Source	Examples
Open-SWE patches	5,489
xLAM function-calling	1,200
Hermes function-calling	341
Targeted CLI tool-call replay	3,840
Total	10,870

Native Tool-Call Format

The model uses a custom XML-like format for tool calls (not JSON):

<function=tool_name>
<parameter=param_name>
param_value
</parameter>
</function>

Multiple tool calls are separated by newlines. This format was introduced in Stage-1 and reinforced in Stage-2.

Benchmark Results

Both benchmarks were run on a Modal H200 container loading the base model + adapter with bfloat16 precision.

1. Native Tool-Call Benchmark (44 cases)

The native benchmark evaluates 40 tool-use cases (5 per tool across 8 tools) and 4 no-tool cases.

Metric	Score
Correct tool selection	79.5%
Required args present	88.6%
Native format valid	100.0%
Correct call count	88.6%
JSON params valid	100.0%
No old JSON format	100.0%
No markdown fences	100.0%
Balanced tags	100.0%
No extra prose	100.0%
No standalone fn tags	100.0%
No-tool correctness	0.0%

Per-Tool Accuracy

Tool	Accuracy	Args Present	Correct/Total
ask_user_question	100.0%	100.0%	5/5
edit	100.0%	100.0%	5/5
glob	100.0%	100.0%	5/5
grep_search	60.0%	60.0%	3/5
read_file	100.0%	100.0%	5/5
run_shell_command	100.0%	100.0%	5/5
search_code	40.0%	40.0%	2/5
todo_write	100.0%	100.0%	5/5

2. BFCL-Compatible Evaluation (10 cases)

The BFCL-style evaluation tests general function-calling with unseen tools (weather, flights, email, stocks, etc.).

Metric	Score
Correct tool selection	100.0%
Native format valid	100.0%
No old JSON format	100.0%

All 10 BFCL test cases passed with perfect tool selection and format compliance.

Key Findings

1. Perfect format compliance — 100% native XML format, zero legacy JSON, zero markdown fences, zero extra prose. The Stage-2 adapter fully internalized the native tool-call format.
2. Strong general tool selection — 6 of 8 tools scored 100% accuracy. ask_user_question, edit, glob, read_file, run_shell_command, and todo_write are fully reliable.
3. Generalization to unseen tools — The 100% BFCL score confirms the model generalizes its tool-calling ability to functions it has never seen during training.
4. Weaknesses identified:
   • search_code (40%) — the model sometimes substitutes grep_search instead of search_code, or calls search_code with grep_search-style arguments.
   • grep_search (60%) — the model occasionally uses search_code or glob when grep_search was expected, and sometimes emits multiple calls when one was requested.
   • No-tool cases (0%) — the model always emits a tool call even when the correct response is to answer directly without tools. This is expected for an adapter trained exclusively on tool-use data.
5. Training metrics — eval loss of 0.4701 at step 200, with training loss trending from ~0.52 to ~0.40 over 659 steps, indicating stable convergence.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = "TaimoorSiddiqui/Hopcoder-Mini-9B"
adapter_repo = "TaimoorSiddiqui/Hopcoder-Mini-9B-SWE-Agent-LoRA-H200"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, adapter_repo)

Files

File	Description
adapter_config.json	LoRA configuration
adapter_model.safetensors	Adapter weights
stage2_benchmark_results.json	Combined benchmark results (native + BFCL)
stage2_native_benchmark.json	Full native tool-call benchmark with per-case details
stage2_bfcl_eval.json	Full BFCL evaluation with per-case details

Citation

@misc{hopcoder-mini-9b-swe-agent-lora-h200,
  author = {TaimoorSiddiqui},
  title = {HopCoder-Mini-9B SWE-Agent LoRA H200},
  year = {2026},
  url = {https://huggingface.co/TaimoorSiddiqui/Hopcoder-Mini-9B-SWE-Agent-LoRA-H200}
}

License

This adapter inherits the license of the base model TaimoorSiddiqui/Hopcoder-Mini-9B.