README.md

---
license: apache-2.0
base_model: Qwen/Qwen2.5-7B-Instruct
library_name: peft
tags:
  - qbc
  - aethersmind
  - decentralised-training
  - diloco
  - lora
  - mixture-of-experts
model-index:
  - name: aether-v7.2-unified
    results:
      - task:
          type: text-generation
        dataset:
          name: MMLU (full, 14042 questions, uncapped)
          type: mmlu
        metrics:
          - type: accuracy
            value: 0.7109
      - task:
          type: text-generation
        dataset:
          name: GSM8K (full, 1319 questions, uncapped)
          type: gsm8k
        metrics:
          - type: accuracy
            value: 0.7498
---

# aether-v7.2-unified

The Sephirot adapter served by [qbc.network](https://qbc.network), and the first
adapter produced by our decentralised training loop to pass a full evaluation and
be promoted into the live served model. It succeeds
[aether-v7.1-unified](https://huggingface.co/QuantumAI-Blockchain/aether-v7.1-unified).

It is a small adapter over a **frozen Qwen2.5-7B-Instruct base (Q4_K_M)**: ten
low-rank experts with a top-2 router, about 1.18 million trainable parameters. The
base model is not redistributed here; load `Qwen/Qwen2.5-7B-Instruct` and apply this
adapter on top. Serving precision is BF16.

## On-chain attestation

This model's identity is attested on the QBC chain. The on-chain `weight_root` is a
SHA-256 over (base, adapter, tokenizer, config):

- **weight_root:** `0x3e99b3d2f4e4d5eacbb3fe8d7e2f709dbfc48deb34407035383b3a5263e29e4e`
- **registry round:** 2, **finalised** by a 4-of-5 validator quorum
- **submitted at block:** 1698355
- **content root of the trained adapter:** `0x157a01b9`

## How it was trained

Trained with a DiLoCo-style local-training-plus-aggregation loop across a fleet of
ordinary 8GB consumer GPUs. A vocab-chunked backward pass keeps a full 7B adapter
step inside roughly 7.5GB of video memory, so commodity cards can train the served
model. Training runs as short payable rounds of about 35 minutes: a worker draws a
fresh shuffled batch from a large corpus, trains the adapter locally, and submits a
candidate. A candidate is accepted only if it beats the previous accepted candidate
on a fixed held-out set it never trains on. Ten rounds produced seven accepted
units, and the best by full held-out evaluation is this release.

## Evaluation

Measured on this adapter over the frozen base, as a snapshot from the promotion
evaluation (not a live probe):

| Measurement | Base | This adapter | Change |
| --- | --- | --- | --- |
| Held-out cross-entropy (clean 1000-example corpus, nats) | 2.9014 | 2.5609 | -0.3405 |
| MMLU (full 14042, uncapped) | 0.7116 | 0.7109 | -0.07pp (flat) |
| GSM8K (full 1319, uncapped) | 0.7248 | 0.7498 | +2.50pp |

The BF16 weights served here reproduce the held-out CE of the F32 candidate to four
decimals, so the -0.3405 nats generalisation gain holds at serving precision.
Generalisation on unseen text improved, general knowledge held flat, and multi-step
maths reasoning improved, at no measured cost to base capability. Against the
previously-served adapter, the promotion gate measured a further held-out CE
improvement (2.5912 to 2.4811 on the gate's own holdout).

## Scope and honesty

This adapter is the product of decentralised **training** with an owner-gated
promotion into the served model. This is Stage 1 of a four-stage plan. It is **not**
yet a permissionless model update: that stage is designed, not built, and is gated
on robust aggregation that survives a malicious contributor, a stake-and-slash
economic layer, and honest independent re-derivation of the claimed improvement.

During this work our own evaluation flattered us three times, through memorisation on
training data, a sampling-window bug, and a mixed-scale gate that false-rejected a
good round. Each was caught and fixed before any number here was trusted. The full
account is in the
[engineering write-up](https://qbc.network/blog/public-gpus-improved-the-served-model).

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = PeftModel.from_pretrained(base, "QuantumAI-Blockchain/aether-v7.2-unified")
```

## Licence

Released under Apache-2.0, matching the Qwen2.5-7B-Instruct base licence.