README.md

metadata

language:
  - en
license: apache-2.0
tags:
  - gguf
  - code
  - reasoning
  - agentic
  - qwen2
  - dpo
  - llama.cpp
base_model: Qwen/Qwen2.5-Coder-7B-Instruct
pipeline_tag: text-generation

MalxLabs-Fable5_QwenCoder (Fable-Coder-7B-DPO)

A specialized 4.36 GB Q4_K_M GGUF model optimized for multi-turn agent tool execution and logical programming correctness.

🐙 GitHub Benchmark & Harness • 📊 Findings & Transcripts

---

Overview

Fable-Coder-7B-DPO is an aligned, post-trained derivative of Qwen/Qwen2.5-Coder-7B-Instruct. It was specifically tuned to eliminate arithmetic hallucinations, handle strict constraints, and reliably execute tool paths in autonomous agent workflows.

In a closed comparative benchmark of 33 tasks across 7 domains against its base model, Fable-Coder scored 92/115 (80.0%) versus 88/115 (76.5%) for Qwen2.5-Coder-7B-Instruct, demonstrating a +20.0% improvement in complex reasoning while matching baseline coding capability.

---

Architectural Pipeline Flow

[Base Model: Qwen2.5-Coder-7B] (Natively Loaded in BF16 Precision)
│
▼
[Stage 1: Multi-Turn SFT with LoRA] ──► Adapts agent behavioral trajectories via ChatML
│
▼
[Stage 2: Direct Preference Optimization] ──► Aligns tool call decisions & correct syntax
│
▼
[Stage 3: Full FP16 Parameter Merge] ──► Fuses low-rank adapters back into base model
│
▼
[Stage 4: Tokenizer & GGUF Compilation] ──► Converts PyTorch checkpoint to GGUF format
│
▼
[Stage 5: Q4_K_M CMake Quantization] ──► Compresses FP16 parameters to 4.36 GB binary

Key Training Highlights

1. Supervised Fine-Tuning (SFT) with LoRA: Injected LoRA matrices (r=8, alpha=16) targeting attention projection layers (q_proj, v_proj) using actual agent trajectories from MoreThought/Fable-5.1-Max-Reasoning-Filtered-5000x.
2. Direct Preference Optimization (DPO): Pairwise dataset contrasting clean, efficient tool invocations against redundant loops and lazy stubs (positive reward margin peak: 0.01457).
3. Full FP16 Merge & Tokenizer Patch: Fused weights into native BF16 base and patched tokenizer configuration for clean GGUF compilation.
4. CMake Q4_K_M Quantization: Compressed the 14.5 GB weights to a single-file 4.36 GB binary (~70% storage reduction), fitting within ~5.1 GB VRAM on consumer GPUs.

---

Benchmark Comparison

Category	Fable-Coder-7B-DPO	Qwen2.5-Coder-7B-Instruct	Delta
Overall Score	92 / 115 (80.0%)	88 / 115 (76.5%)	+4 pts (+3.5%) Fable
Reasoning	17 / 20 (85.0%)	13 / 20 (65.0%)	+4 pts (+20.0%) Fable
Coding	25 / 35 (71.4%)	25 / 35 (71.4%)	Tied
Agentic Planning	21 / 25 (84.0%)	21 / 25 (84.0%)	Tied
Security	9 / 11 (81.8%)	9 / 11 (81.8%)	Tied
Knowledge	8 / 8 (100.0%)	8 / 8 (100.0%)	Tied
Context Retention	6 / 6 (100.0%)	6 / 6 (100.0%)	Tied

---

Quickstart & Usage

Using llama.cpp

llama-cli -m fable-coder-7b-dpo.Q4_K_M.gguf -p "Write a Python function to merge intervals without mutating the input." -n 512

Using huggingface-cli

huggingface-cli download MalxTech/MalxLabs-Fable5_QwenCoder fable-coder-7b-dpo.Q4_K_M.gguf --local-dir .

Using Ollama

Create a Modelfile:

FROM ./fable-coder-7b-dpo.Q4_K_M.gguf
TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"

Then build and run:

ollama create fable-coder -f Modelfile
ollama run fable-coder

---

Citation & Acknowledgments

• Base model: Qwen/Qwen2.5-Coder-7B-Instruct
• SFT dataset: MoreThought/Fable-5.1-Max-Reasoning-Filtered-5000x
• Full benchmark repository: YoMosa2009/MalxLabs-Fable5_QwenCoder