continuum-ai
/

qwen2.5-coder-14b-compacted

@@ -1,182 +1,62 @@
 ---
-language:
-- en
-- zh
 license: apache-2.0
-library_name: transformers
-pipeline_tag: text-generation
 tags:
-- qwen2.5
-- experiential-plasticity
-- forged
-- head-pruning
-- neural-plasticity
-- sentinel-ai
-- continuum
-- safetensors
-- compacted
-- expert-pruning
-- moe-pruning
-- code
-- code-generation
-- coding
-- coder
-- programming
-- software-engineering
-- local-inference
-- efficient
-- optimized
-- pruned
-- 14b
-base_model:
-- Qwen/Qwen2.5-Coder-14B
-datasets:
-- m-a-p/CodeFeedback-Filtered-Instruction
 ---
-# qwen2.5-coder-14b-code-forged
-**Optimized through Experiential Plasticity.** Forged from [Qwen/Qwen2.5-Coder-14B](https://huggingface.co/Qwen/Qwen2.5-Coder-14B) for **code** tasks.
-**Not quantized. Not distilled. Structurally reshaped.**
-The architecture co-evolves with training: heads that contribute to the domain specialize, heads that don't are removed. The result is a model architecturally optimized for its task — like biological synaptic pruning during brain development.
-## Expert Compaction (MoE Pruning)
-Mixture-of-Experts models route tokens to specialized sub-networks. Most of these experts are redundant for any given domain. Expert compaction identifies and removes the lowest-importance experts entirely.
-**The gate weights are sliced to match** — no special loading code needed. Load with standard HuggingFace transformers.
-The result: a model that fits on consumer GPUs (RTX 3090/4090/5090) while retaining the specialized knowledge of a much larger model.
-## Results
 | Metric | Value |
 |--------|-------|
-| Base Model | [Qwen/Qwen2.5-Coder-14B](https://huggingface.co/Qwen/Qwen2.5-Coder-14B) |
-| Domain | code |
-| Training Data | wikitext-2 |
-| Strategy | combined |
-| Pruning Level | 30% |
-| Cycles | 3 |
-| Steps/Cycle | 1000 |
-## Runs On
-| Device | Format | Verified |
-|--------|--------|----------|
-| MacBook Pro 16GB | fp16 | Yes |
-| MacBook Pro 32GB | fp16 | Yes |
-These models are designed for **consumer hardware**. No A100s required. Your MacBook, your gaming PC, your home server.
-## Quick Start
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-model = AutoModelForCausalLM.from_pretrained("continuum-ai/qwen2.5-coder-14b-code-forged",
-    torch_dtype="auto", device_map="auto")
-tokenizer = AutoTokenizer.from_pretrained("continuum-ai/qwen2.5-coder-14b-code-forged")
-inputs = tokenizer("Write a Python decorator that caches results:", return_tensors="pt").to(model.device)
-output = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=0.7)
-print(tokenizer.decode(output[0], skip_special_tokens=True))
-```
-## Forge Your Own
-Three commands. Any NVIDIA GPU with 8GB+ VRAM.
 ```bash
-git clone https://github.com/CambrianTech/sentinel-ai && cd sentinel-ai && ./setup.sh
-source .venv/bin/activate
-python scripts/forge_model.py Qwen/Qwen2.5-Coder-14B --domain code
-```
-The forge script auto-detects your GPU, picks the right memory tier (fp16 / 4-bit NF4), trains with LoRA + AMP, prunes attention heads, defrags, and saves. Progress observable via `status.json`.
-## The Science: Experiential Plasticity
-Traditional model compression (quantization, distillation) makes models **smaller but worse**. Experiential Plasticity makes them **smaller AND better**.
-### How It Works
-1. **Train** on domain-specific data (LoRA + AMP mixed precision)
-2. **Measure** each attention head's information contribution (entropy-based importance)
-3. **Prune** the lowest-contributing heads
-4. **Retrain** on the same domain data — surviving heads specialize and compensate
-5. **Defrag** — structurally remove dead heads, free VRAM
-6. **Repeat** — each cycle the model improves on its domain
-### Scaling Law
-Larger models harbor more architectural redundancy. Plasticity exploits this — bigger models benefit more:
-| Model | Params | Domain | Improvement |
-|-------|--------|--------|------------|
-| Qwen2.5-0.5B | 0.5B | General | -3.2% (too small to prune) |
-| Qwen2.5-1.5B | 1.5B | General | +3.0% |
-| Qwen2.5-7B | 7.6B | General | +11.8% |
-| **Qwen3.5-4B** | **3.4B** | **Code** | **+24.0%** |
-| **Qwen3.5-27B** | **23.6B** | **Code** | **+3.5%** (4-bit, runs in 17GB) |
-Domain-specific training amplifies the effect. Qwen3.5-4B on code (+24%) exceeds Qwen2.5-7B on generic text (+11.8%) despite being a smaller model.
-### Transfer Function
-Recovery from iterative pruning follows a measurable exponential decay:
-```
-recovery = 1.45 * exp(-0.18 * cycle) - 0.03
-```
-This connects transformer optimization to classical control theory — the same mathematics used in electrical engineering and robotics for decades. A PID controller can manage the entire forging process with zero human hyperparameters.
-### Continuous Defrag
-Traditional pruning masks heads but doesn't free memory. Continuous defrag structurally removes dead heads between cycles:
 ```
-Cycle 1: train (batch=1, 27B, 17.9GB) -> prune -> defrag -> freed 1.7GB
-Cycle 2: train (batch=2, 24.5B, 16.2GB) -> prune -> defrag -> freed 1.7GB  (2x faster)
-Cycle 3: train (batch=3, 22B, 14.5GB)  -> prune -> defrag                  (2.8x faster)
-```
-40% faster total training and a 33% smaller final model.
-### Head Mitosis
-Pruning frees slots. Mitosis fills them. When a head is overutilized, it gets cloned into a pruned slot — each copy at 50% gate value to maintain output continuity. After continued training, the clones **diverge and specialize**, like cell differentiation after biological mitosis. The model grows new specialized capacity exactly where it's needed.
-**Read the full paper**: [Experiential Plasticity: Transformers That Grow Their Own Architecture From Experience](https://github.com/CambrianTech/continuum/blob/main/docs/papers/EXPERIENTIAL-PLASTICITY.md)
-## Output Samples
-Generated by the forged model immediately after forging — **no cherry-picking, no post-processing**.
-*No generation samples available for this model.*
-## Forging Metadata
-```json
-{
-  "model": "Qwen/Qwen2.5-Coder-14B",
-  "improvement_pct": 0,
-  "baseline_ppl": 0,
-  "final_ppl": 0
-}
-```
-## Research
-- **[Experiential Plasticity](https://github.com/CambrianTech/continuum/blob/main/docs/papers/EXPERIENTIAL-PLASTICITY.md)** — Scaling law, transfer function, self-directed controller, domain forging, continuous defrag
-- **[Neural Plasticity in Transformers](https://github.com/CambrianTech/continuum/blob/main/docs/papers/SENTINEL-AI-NEURAL-PLASTICITY.md)** — Foundation paper with cross-architecture results
-- **[Plasticity Compaction](https://github.com/CambrianTech/continuum/blob/main/docs/papers/PLASTICITY-COMPACTION-MOE.md)** — MoE expert pruning (67GB to 14GB)
-## Links
-- [All published models](https://huggingface.co/continuum-ai)
-- [sentinel-ai](https://github.com/CambrianTech/sentinel-ai) — Open source forge framework
-- [continuum](https://github.com/CambrianTech/continuum) — Distributed AI on consumer hardware

 ---
 license: apache-2.0
 tags:
+  - code
+  - qwen2
+  - compacted
+  - head-pruning
+  - continuum
+  - continuum:compacted
+  - continuum:head-pruning
+language:
+  - en
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+pipeline_tag: text-generation
 ---
+# Qwen2.5-Coder-14B-Instruct — Compacted (25Q/5KV, Q5_K_S)
+A **14-billion parameter** coding model compressed to run on a **16GB MacBook Air**.
+## How It Was Built
+Continuum's adaptive compression pipeline:
+1. **Head Pruning**: 40 Q-heads / 8 KV-heads → 25 Q-heads / 5 KV-heads (37.5% KV cache reduction)
+2. **Quantization**: Q5_K_S (5.1 bits per weight)
+3. **Result**: 27GB BF16 → 8.9GB GGUF
+## Performance
 | Metric | Value |
 |--------|-------|
+| Speed | 9.2 tok/s (M1 Pro 32GB, Metal) |
+| Memory | ~9 GB |
+| Architecture | 48 layers, 25 Q-heads, 5 KV-heads, head_dim=128 |
+| Quantization | Q5_K_S (5.1 BPW) |
+## How to Run
+With [Continuum](https://github.com/cambrian-tech/continuum) — downloads automatically:
 ```bash
+# Model alias "coder" resolves to this model
+./jtag inference/generate --model=coder --prompt="def fibonacci(n):"
 ```
+## Links
+- **[Continuum](https://github.com/cambrian-tech/continuum)** — Local AI runtime
+- **[sentinel-ai](https://github.com/cambrian-tech/sentinel-ai)** — Research project
+- **[continuum-ai](https://huggingface.co/continuum-ai)** — More models
+## License
+Apache 2.0
+## Part of continuum
+[continuum](https://github.com/CambrianTech/continuum) is an open-source AI ecosystem where personas live, work, learn, and evolve on your hardware. Zero API keys required. AGPL-3.0.
+Built on the research foundations of [Synthetic Citizens: AI Personas as Persistent, Evolving Entities](https://github.com/CambrianTech/continuum/blob/main/docs/papers/SYNTHETIC-CITIZENS.md) and [Plasticity Compaction: SOTA-to-COTS via MoE Expert Pruning](https://github.com/CambrianTech/continuum/blob/main/docs/papers/PLASTICITY-COMPACTION-MOE.md). Our core contribution is **utilization-aware model surgery** — runtime profiling determines exactly which components are active for a target domain, how much each contributes, and what precision each requires. MoE experts, attention heads, and weight precision are all targeted independently based on measured activation patterns, not uniform heuristics. The result: SOTA models surgically reduced to fit consumer hardware with reasoning quality preserved.
+[Plasticity Compaction Paper](https://github.com/CambrianTech/continuum/blob/main/docs/papers/PLASTICITY-COMPACTION-MOE.md) | [Get started](https://github.com/CambrianTech/continuum)