| --- |
| language: en |
| library_name: unsloth |
| license: apache-2.0 |
| tags: |
| - spec-forge |
| - command-runway |
| - qwen2.5-coder |
| - lora |
| - code-generation |
| - yaml |
| base_model: Qwen/Qwen2.5-Coder-7B-Instruct |
| datasets: |
| - githeri/spec-forge-training-data |
| pipeline_tag: text-generation |
| --- |
| |
| # qwen2.5-coder-7b-specforge |
|
|
| ## Model Description |
|
|
| Fine-tuned [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) using LoRA adapters on the Spec-Forge training corpus. |
|
|
| The model converts natural-language feature requests into validated YAML specifications that conform to the COMMAND_RUNWAY methodology. Each spec contains: |
| - `task_id`, `summary`, `depends_on`, `local_goals`, `global_goals_refs`, `context` |
| - Every `local_goal` has an `Inspect → Create/Modify → Verify` verification flow |
| - Specs pass a hardened validator (canonical vocabulary, near-duplicate detection, YAML safety) |
| - Specs are scored against runbook-readiness criteria (hard gate: missing Inspect/Create/Verify stages = 0.0) |
|
|
| ## Training Details |
|
|
| | Parameter | Value | |
| |-----------|-------| |
| | Base model | `unsloth/Qwen2.5-Coder-7B-Instruct` | |
| | LoRA rank | 16 | |
| | LoRA alpha | 32 | |
| | LoRA dropout | 0.1 | |
| | Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | |
| | Epochs | 3 | |
| | Learning rate | 2e-4 | |
| | Batch size | 1 (effective: 4 via gradient accumulation) | |
| | Max sequence length | 2048 | |
| | Quantization | 4-bit NF4 | |
| | Optimizer | adamw_8bit | |
| | LR scheduler | cosine | |
| | Warmup ratio | 0.1 | |
|
|
| ## Training Data |
|
|
| - **Source**: 475 seed prompts across 21 feature categories |
| - **Generation**: Ollama (qwen2.5-coder:7b-instruct) at temperature 0.2 |
| - **Validation**: Hardened YAML spec validator (78 test cases) |
| - **Scoring**: Runbook scorer with hard gate (0.75 threshold) |
| - **Format**: Chat format (`system` + `user` + `assistant` turns) |
|
|
| ## Evaluation |
|
|
| See `data/eval_results.json` after running `make eval-model`. |
|
|
| Metrics: |
| - **Validation rate**: percentage of specs that pass the hardened validator |
| - **Score pass rate**: percentage of specs scoring >= 0.75 on the runbook scorer |
| - **Target**: >80% score pass rate (held-out prompts) |
|
|
| ## Usage |
|
|
| ### Ollama (GGUF) |
|
|
| ```bash |
| # Download GGUF from this repo's models/ directory |
| ollama create specforge -f models/qwen2.5-coder-7b-specforge-gguf/Modelfile |
| ollama run specforge |
| ``` |
|
|
| ### HuggingFace Transformers |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| from peft import PeftModel |
| |
| # Load base model |
| base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct", torch_dtype="auto") |
| model = PeftModel.from_pretrained(base, "githeri/qwen2.5-coder-7b-specforge") |
| tokenizer = AutoTokenizer.from_pretrained("githeri/qwen2.5-coder-7b-specforge") |
| |
| messages = [ |
| {"role": "system", "content": "You are a precise specification generator. Output ONLY a YAML document."}, |
| {"role": "user", "content": "Add a POST /health endpoint that returns 200 OK"}, |
| ] |
| text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
| inputs = tokenizer(text, return_tensors="pt").to(model.device) |
| outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2) |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
|
|
| ## Limitations |
|
|
| - Trained on a synthetic corpus generated by the base model itself — quality is bounded by the base model's spec-generation ability |
| - Specs are scoped to a single-file, single-feature granularity (not multi-stage epics) |
| - Context is fixed to TypeScript/Express/Prisma/Vitest stack |
| - GGUF quantization (q4_k_m) introduces minor quality degradation vs the 16-bit merge |
|
|
| ## Ethical Considerations |
|
|
| - This model generates structured specifications, not executable code |
| - All generated specs must pass the hardened validator before use |
| - Human review is required before feeding specs into a COMMAND_RUNWAY executor |
| |
| ## Citation |
| |
| ```bibtex |
| @misc{githeri-specforge, |
| title={Spec-Forge: From Natural Language to Runbook-Ready YAML Specifications}, |
| author={Githeri}, |
| year={2026}, |
| url={https://github.com/nickrotich/githeri} |
| } |
| ``` |
| |
| ## License |
| |
| Apache 2.0 — same as the base Qwen2.5-Coder-7B-Instruct model. |
