| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3.5-4B |
| tags: [lora, federated-distillation, code, golang, fdn] |
| --- |
| |
| # FDN Adapters — Federated Distillation Network (subject: programming/go/concurrency) |
|
|
| LoRA adapters produced by FDN's federated distillation loop: **many independent |
| trainers distill open teachers into one student per subject — and only verified |
| improvement gets accepted.** Every training sample is machine-verified |
| (`go build && go test -race`) before it may train anything; every adapter is |
| graded on held-out exams with contamination structurally excluded |
| (solution-class splits). Teachers are open, distillation-permissive models only |
| (DeepSeek V4 Flash, MIT) — enforced in code, with license texts verified. |
|
|
| Code, methodology and all raw evidence: https://github.com/chainswarm/fdn-subnet |
|
|
| ## Student progression (held-out, contamination-free, bucket/200) |
|
|
| | Round | Student | Score | Notes | |
| |---|---|---|---| |
| | R0 | base Qwen3.5-4B | 36/200 (7 families) | concurrency-bug fixing baseline | |
| | R1 | `r1/` adapter (1017 verified samples) | **200/200** (7 families) | full miner loop, ≈$1.50 | |
| | R2 | `r2/` — 3 independent trainers | every unsolved target 0→200 (3 new harder families) | composed router-student: no regression anywhere | |
|
|
| ## Round R2 — federation in action |
|
|
| Three trainers on separate cloud machines, same public curriculum, independent |
| verified datasets (194/169/197 samples): miner **a** and **b** accepted at full |
| reward; miner **c** deliberately duplicated a's assignment — MinHash manifest |
| overlap (41%) was detected and its reward discounted 30% (novelty economics). |
| Full verdicts: `r2/round-report-v2.json`; per-miner dataset provenance: |
| `r2/miner-*/manifest.json`. |
|
|
| ## Usage |
|
|
| ```python |
| from peft import PeftModel |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B", dtype="bfloat16", device_map="auto") |
| model = PeftModel.from_pretrained(base, "aphex5/fdn-adapters", subfolder="r1") |
| ``` |
|
|
| Adapters compose router-style per subject family; greedy decoding for |
| reproducible evals (prompt format: see `fdn/eval/harness/prompts.py`, go-fix-v1). |
|
|
| ## External anchor: HumanEval (pass@1, greedy) |
|
|
| | variant | pass@1 | |
| |---|---| |
| | base Qwen3.5-4B | 81.1% | |
| | R1 student | 79.9% | |
|
|
| Subject mastery (36→200/200) held general coding ability within binomial noise. |
| A TIES-merged snapshot was built and **rejected by FDN's own Pareto gate** |
| (merge interference destroyed one merged-in skill) — anchor-failing merges |
| don't promote; router composition remains the student. Quality only ratchets up. |
|
|
| ## Round R3 — second subject (`programming/go/errors`), the report-card loop |
|
|
| Base calibration showed 3 of 4 error-handling families already mastered; |
| `nil_map_write` (20%) was THE weakness — so all three independent trainers |
| targeted it, exactly as the weakness→multiplier loop directs. All three took |
| it 60→200 (identical quality). Rewards priced their originality: |
| **289 (first, novel) → 198 (−31%, 40% dataset overlap) → 91 (−69%, 91% overlap |
| near-duplicate)**. Same machinery, new subject, zero code changes. |
|
|
