strykes/tiny-giant-2500-q4_k_m

Tiny-Giant 2500 — Q4_K_M (pilot)

Full fine-tune of Qwen2.5-Coder-1.5B-Instruct on the Tiny-Giant pilot dataset (~2,508 execution-verified agentic coding samples), quantized to Q4_K_M for llama.cpp deployment.

This is the pilot (TAG=2500) checkpoint — it validates the train → quant → eval pipeline, not the final production model (target ≥30k samples).

Model file

File	Quant	Size
tiny-giant-2500-Q4_K_M.gguf	Q4_K_M	~940 MB

Training summary

Base	Qwen/Qwen2.5-Coder-1.5B-Instruct
Method	Full fine-tune (bf16), not LoRA
Dataset	Tiny-Giant pilot — 2,508 records → train=2383 / val=125
Epochs	3
Learning rate	7e-6
Seq length	4096
Final train loss	0.5891
Final eval loss	0.5302 (epoch 3)
Chat format	Hermes-style ChatML with <tool_call> blocks (custom renderer — see repo)

Usage (llama.cpp)

Pin the ChatML template explicitly. This model was trained with a custom Hermes/ChatML renderer (train_tiny_giant.render_conversation); do not rely on template auto-detection.

# llama-server
llama-server -m tiny-giant-2500-Q4_K_M.gguf --chat-template chatml --ctx-size 4096

# llama-cpp-python server
python -m llama_cpp.server --model tiny-giant-2500-Q4_K_M.gguf \
  --chat_format chatml --n_ctx 4096 --port 8088

from llama_cpp import Llama

llm = Llama(model_path="tiny-giant-2500-Q4_K_M.gguf", n_ctx=4096, n_gpu_layers=-1)
out = llm.create_completion(
    prompt="<|im_start|>user\nWrite a Python function to merge two sorted lists.\n<|im_start|>assistant\n",
    max_tokens=512,
    temperature=0.0,
    stop=["<|im_start|>"],
)
print(out["choices"][0]["text"])

Evaluation

Benchmark and internal eval results are populated after the Vast GPU eval run completes. See the companion comparison card in the source repository (MODEL_CARD-2500.md).

Benchmark	Score (Q4_K_M)	Notes
HumanEval pass@1	pending	greedy, EvalPlus
HumanEval+ pass@1	pending	greedy, EvalPlus
MBPP pass@1	pending	greedy, EvalPlus
MBPP+ pass@1	pending	greedy, EvalPlus
Tiny-Giant debug pass@1	pending	held-out execution tests
Tool first-action validity	pending	agentic val set

Limitations

• Pilot scale (2.5k samples): format adherence and debugging habits should improve vs base; absolute codegen benchmarks may move little until the full 30k run.
• Q4_K_M quantization: if eval shows >~4pt drop vs bf16, re-quantize the kept f16 GGUF to Q5_K_M — no retraining needed.
• Tool-use eval requires prompts built with the same renderer used in training.

Source

Training pipeline and dataset factory: Tiny-Giant (local repo).

License

Apache-2.0 (inherits from Qwen2.5-Coder-1.5B-Instruct base).