license: apache-2.0
base_model: Qwen/Qwen3.5-4B
tags:
- lora
- federated-distillation
- code
- golang
- fdn
FDN Adapters — Federated Distillation Network (subject: programming/go/concurrency)
LoRA adapters produced by FDN's federated distillation loop: many independent
trainers distill open teachers into one student per subject — and only verified
improvement gets accepted. Every training sample is machine-verified
(go build && go test -race) before it may train anything; every adapter is
graded on held-out exams with contamination structurally excluded
(solution-class splits). Teachers are open, distillation-permissive models only
(DeepSeek V4 Flash, MIT) — enforced in code, with license texts verified.
Code, methodology and all raw evidence: https://github.com/chainswarm/fdn-subnet
Student progression (held-out, contamination-free, bucket/200)
| Round | Student | Score | Notes |
|---|---|---|---|
| R0 | base Qwen3.5-4B | 36/200 (7 families) | concurrency-bug fixing baseline |
| R1 | r1/ adapter (1017 verified samples) |
200/200 (7 families) | full miner loop, ≈$1.50 |
| R2 | r2/ — 3 independent trainers |
every unsolved target 0→200 (3 new harder families) | composed router-student: no regression anywhere |
Round R2 — federation in action
Three trainers on separate cloud machines, same public curriculum, independent
verified datasets (194/169/197 samples): miner a and b accepted at full
reward; miner c deliberately duplicated a's assignment — MinHash manifest
overlap (41%) was detected and its reward discounted 30% (novelty economics).
Full verdicts: r2/round-report-v2.json; per-miner dataset provenance:
r2/miner-*/manifest.json.
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "aphex5/fdn-adapters", subfolder="r1")
Adapters compose router-style per subject family; greedy decoding for
reproducible evals (prompt format: see fdn/eval/harness/prompts.py, go-fix-v1).
External anchor: HumanEval (pass@1, greedy)
| variant | pass@1 |
|---|---|
| base Qwen3.5-4B | 81.1% |
| R1 student | 79.9% |
Subject mastery (36→200/200) held general coding ability within binomial noise. A TIES-merged snapshot was built and rejected by FDN's own Pareto gate (merge interference destroyed one merged-in skill) — anchor-failing merges don't promote; router composition remains the student. Quality only ratchets up.
Round R3 — second subject (programming/go/errors), the report-card loop
Base calibration showed 3 of 4 error-handling families already mastered;
nil_map_write (20%) was THE weakness — so all three independent trainers
targeted it, exactly as the weakness→multiplier loop directs. All three took
it 60→200 (identical quality). Rewards priced their originality:
289 (first, novel) → 198 (−31%, 40% dataset overlap) → 91 (−69%, 91% overlap
near-duplicate). Same machinery, new subject, zero code changes.