Holo-3.1-9B-Coder

Code-specialized adaptation of Hcompany/Holo-3.1-9B.

Evaluation

Benchmark: HumanEval+

Model	HumanEval+ pass@1
Holo-3.1-9B (base)	52.4%
Holo-3.1-9B-Coder (this model)	104/164 = 63.4%

This is a +10.9 percentage-point improvement over the base model on HumanEval+.

Proof files are available in this repository under eval/:

• Samples: eval/holo_9b_humaneval_samples.jsonl
• EvalPlus results: eval/holo_9b_humaneval_samples_eval_results.json
• Summary: eval/holo_9b_humaneval_results.json

Evaluation was run with evalplus==0.3.1, greedy decoding (do_sample=False, temperature=0), system prompt "Return only executable Python code. No markdown. No explanations.", and max_new_tokens=512.

Artifacts

Path	Description
adapter/	LoRA adapter (r=8, alpha=16, q/k/v/o targets).
model.safetensors, config.json, tokenizer files	Merged base + adapter model.
holo-9b-coder-Q4_K_M.gguf	llama.cpp GGUF, Q4_K_M quantization.
holo-9b-coder-Q5_K_M.gguf	llama.cpp GGUF, Q5_K_M quantization.
holo-9b-coder-Q6_K.gguf	llama.cpp GGUF, Q6_K quantization.

Quantization details

Quantization was performed with llama.cpp after merging the adapter into the base model. The GGUF files use K-quant mixture schemes.

LiveCodeBench

Benchmark: LiveCodeBench release_v2 (code_generation_lite)

Model	pass@1
Holo-3.1-9B-Coder	34/111 = 30.6%

Official codegen_metrics evaluator, greedy decoding, 6s timeout.

Proof: eval/lcb_v2_official.json

Loading the adapter

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Hcompany/Holo-3.1-9B"
model = AutoModelForCausalLM.from_pretrained(base, trust_remote_code=True, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, "josephmayo/Holo-3.1-9B-Coder", subfolder="adapter")
model = model.merge_and_unload()
tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)

Loading a GGUF with llama.cpp

./llama-cli -m holo-9b-coder-Q4_K_M.gguf -p "Return only executable Python code.\n\ndef factorial(n):" -n 256

Base model

• Architecture: Qwen3.5-family text backbone (decoder-only transformer).
• License: Apache-2.0.
• Original model: Hcompany/Holo-3.1-9B

Limitations

• Generated code should be reviewed before execution.
• The model inherits the base model's knowledge cutoff and safety profile.