aether-v7.2-unified: first decentralised-trained served adapter (u_r8, root 0x157a01b9), on-chain round 2 4/5 quorum

README.md

@@ -0,0 +1,111 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+tags:
+  - qbc
+  - aethersmind
+  - decentralised-training
+  - diloco
+  - lora
+  - mixture-of-experts
+model-index:
+  - name: aether-v7.2-unified
+    results:
+      - task:
+          type: text-generation
+        dataset:
+          name: MMLU (full, 14042 questions, uncapped)
+          type: mmlu
+        metrics:
+          - type: accuracy
+            value: 0.7109
+      - task:
+          type: text-generation
+        dataset:
+          name: GSM8K (full, 1319 questions, uncapped)
+          type: gsm8k
+        metrics:
+          - type: accuracy
+            value: 0.7498
+---
+
+# aether-v7.2-unified
+
+The Sephirot adapter served by [qbc.network](https://qbc.network), and the first
+adapter produced by our decentralised training loop to pass a full evaluation and
+be promoted into the live served model. It succeeds
+[aether-v7.1-unified](https://huggingface.co/QuantumAI-Blockchain/aether-v7.1-unified).
+
+It is a small adapter over a **frozen Qwen2.5-7B-Instruct base (Q4_K_M)**: ten
+low-rank experts with a top-2 router, about 1.18 million trainable parameters. The
+base model is not redistributed here; load `Qwen/Qwen2.5-7B-Instruct` and apply this
+adapter on top. Serving precision is BF16.
+
+## On-chain attestation
+
+This model's identity is attested on the QBC chain. The on-chain `weight_root` is a
+SHA-256 over (base, adapter, tokenizer, config):
+
+- **weight_root:** `0x3e99b3d2f4e4d5eacbb3fe8d7e2f709dbfc48deb34407035383b3a5263e29e4e`
+- **registry round:** 2, **finalised** by a 4-of-5 validator quorum
+- **submitted at block:** 1698355
+- **content root of the trained adapter:** `0x157a01b9`
+
+## How it was trained
+
+Trained with a DiLoCo-style local-training-plus-aggregation loop across a fleet of
+ordinary 8GB consumer GPUs. A vocab-chunked backward pass keeps a full 7B adapter
+step inside roughly 7.5GB of video memory, so commodity cards can train the served
+model. Training runs as short payable rounds of about 35 minutes: a worker draws a
+fresh shuffled batch from a large corpus, trains the adapter locally, and submits a
+candidate. A candidate is accepted only if it beats the previous accepted candidate
+on a fixed held-out set it never trains on. Ten rounds produced seven accepted
+units, and the best by full held-out evaluation is this release.
+
+## Evaluation
+
+Measured on this adapter over the frozen base, as a snapshot from the promotion
+evaluation (not a live probe):
+
+| Measurement | Base | This adapter | Change |
+| --- | --- | --- | --- |
+| Held-out cross-entropy (clean 1000-example corpus, nats) | 2.9014 | 2.5609 | -0.3405 |
+| MMLU (full 14042, uncapped) | 0.7116 | 0.7109 | -0.07pp (flat) |
+| GSM8K (full 1319, uncapped) | 0.7248 | 0.7498 | +2.50pp |
+
+The BF16 weights served here reproduce the held-out CE of the F32 candidate to four
+decimals, so the -0.3405 nats generalisation gain holds at serving precision.
+Generalisation on unseen text improved, general knowledge held flat, and multi-step
+maths reasoning improved, at no measured cost to base capability. Against the
+previously-served adapter, the promotion gate measured a further held-out CE
+improvement (2.5912 to 2.4811 on the gate's own holdout).
+
+## Scope and honesty
+
+This adapter is the product of decentralised **training** with an owner-gated
+promotion into the served model. This is Stage 1 of a four-stage plan. It is **not**
+yet a permissionless model update: that stage is designed, not built, and is gated
+on robust aggregation that survives a malicious contributor, a stake-and-slash
+economic layer, and honest independent re-derivation of the claimed improvement.
+
+During this work our own evaluation flattered us three times, through memorisation on
+training data, a sampling-window bug, and a mixed-scale gate that false-rejected a
+good round. Each was caught and fixed before any number here was trusted. The full
+account is in the
+[engineering write-up](https://qbc.network/blog/public-gpus-improved-the-served-model).
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
+tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
+model = PeftModel.from_pretrained(base, "QuantumAI-Blockchain/aether-v7.2-unified")
+```
+
+## Licence
+
+Released under Apache-2.0, matching the Qwen2.5-7B-Instruct base licence.

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ad896c1993532cb0ba9d33c079a47dfaa59e40bb6947c87bcaa7a619023b83f
+size 2367499