p2, p3

docs/p2_p3/00-OVERVIEW.md

@@ -0,0 +1,324 @@
+# HearthNet Phase 2 — Spec Set Overview
+
+**Phase 2 scope:** post-hackathon, 1–3 months of work. Hardens the MVP into something other communities can adopt.
+
+**Stance toward Phase 1:** strictly additive. Phase 1 specs are immutable from Phase 2's view. New modules plug into the bus; nothing in Phase 1 needs rewriting. Where a Phase 1 module is "extended", it is *extended* — same public API, new capabilities or backends added behind the existing facade.
+
+---
+
+## 0. What changes vs Phase 1
+
+| Concern | Phase 1 (MVP) | Phase 2 |
+|---------|---------------|---------|
+| Discovery | mDNS + UDP (LAN only) | + DHT (cross-LAN), + relay-assisted NAT traversal |
+| Transport | HTTP/1.1 + SSE long-poll pubsub | + WebSocket upgrade for bidirectional + pubsub |
+| Trust | Per-request signing | + Capability tokens (delegation, federation) |
+| Cross-community | Out of scope | Federation: signed peering, scoped capability access |
+| Encryption | TLS-in-transit + signed-at-rest within community | + E2E (X25519 + ChaCha20-Poly1305) for chat & optionally files |
+| Chat | 1:1 only | + Group chat (`chat.thread.*`), + store-and-forward via anchors |
+| LLM | Text only | + Vision (`llm.chat` with image content), + Tool calls (`tool_call_delta`) |
+| RAG | Digital PDFs | + OCR for scanned PDFs and images, + reranking |
+| Services | LLM, embed, RAG, file, market, chat | + OCR, Translation, STT, TTS, Image generation, Rerank |
+| Mobile | Web view served by anchor | + Native client (Flutter/RN) with push via relay tier |
+| Files | Direct fetch on demand | + Resumable PUT, + Background replication, + At-rest encryption |
+| Relay | None | + Hosted relay tier for NAT traversal, federation discovery, push |
+| Observability | Local Prometheus + ring buffer + optional Trackio | + OTLP export, + Federated metrics aggregation |
+
+---
+
+## 1. Module map (Phase 2 additions)
+
+### New numbered modules
+
+| ID  | Module                       | Spec file                              | Concern                                              |
+|-----|------------------------------|----------------------------------------|-------------------------------------------------------|
+| M14 | Federation                   | `modules/M14-federation.md`            | Cross-community trust, federation manifests, scoped access |
+| M15 | Relay Tier                   | `modules/M15-relay-tier.md`            | Hosted HTTPS relay (NAT traversal, federation discovery, mobile push) |
+| M16 | Capability Tokens            | `modules/M16-tokens.md`                | Short-lived delegation tokens (OAuth-flavoured)       |
+| M17 | OCR Service                  | `modules/M17-ocr.md`                   | `ocr.image`, `ocr.pdf` — Tesseract / TrOCR / multilingual |
+| M18 | Translation Service          | `modules/M18-translation.md`           | `trans.text` — NLLB-backed, DE↔EN↔Plattdeutsch         |
+| M19 | Speech I/O                   | `modules/M19-stt-tts.md`               | `stt.transcribe` (Whisper), `tts.synthesize` (XTTS/Edge) |
+| M20 | Vision Services              | `modules/M20-vision.md`                | `img.describe`, `img.generate`, multimodal LLM input  |
+| M21 | Tool Calls                   | `modules/M21-tool-calls.md`            | `tool_call_delta` frames, OpenAI/Anthropic-compatible |
+| M22 | Mobile Native                | `modules/M22-mobile-native.md`         | Flutter/RN client with push                            |
+| M23 | E2E Encryption               | `modules/M23-e2e-encryption.md`        | X25519 + ChaCha20-Poly1305 for chat (and optional files) |
+| M24 | Reranking                    | `modules/M24-rerank.md`                | `rerank.text` — BGE-reranker, used by RAG and search   |
+| M25 | Group Chat                   | `modules/M25-group-chat.md`            | `chat.thread.*` — multi-party conversations            |
+
+### New cross-cutting modules
+
+| ID  | Module          | Spec file                              | Concern                                                |
+|-----|-----------------|----------------------------------------|---------------------------------------------------------|
+| X05 | DHT             | `cross-cutting/X05-dht.md`             | Kademlia-style cross-LAN peer + content discovery       |
+| X06 | WebSocket       | `cross-cutting/X06-websocket.md`       | Bidirectional upgrade for `/bus/v1/call` and `/pubsub`  |
+| X07 | Federated Metrics | `cross-cutting/X07-federated-metrics.md` | Optional OTLP export + per-community aggregation       |
+
+### Modifications to Phase 1 modules
+
+These do not get new spec files — Phase 1 spec is *extended* in place at next major. The Phase 1 IMPLEMENTATION_REFERENCE will gain entries; flagged in [`IMPLEMENTATION_REFERENCE.md`](IMPLEMENTATION_REFERENCE.md) §0.
+
+| Phase 1 module | Extension |
+|----------------|-----------|
+| M04 LLM        | New backends gain multimodal + tools support; descriptors carry `modalities`, `tools_supported` flags |
+| M05 RAG        | Auto-reindex on embedding model change; hybrid (keyword+dense) search; `rerank.text` integration |
+| M07 File/Blobs | Resumable PUT (server-side partial-transfer index); background replication; at-rest encryption envelope |
+| M10 Chat       | Calls into M23 for encryption; calls into M25 for group threads |
+| M02 Discovery  | Calls into X05 DHT when peers not found via mDNS/UDP |
+| X01 Transport  | Calls into X06 WebSocket on `Upgrade: websocket` header |
+| M09 Emergency  | Phase-2 captive-portal probe |
+
+---
+
+## 2. Dependency graph (Phase 2 additions on top of Phase 1)
+
+```
+                  ┌──────────────────────────────────────────────┐
+                  │              Phase 1 (unchanged)             │
+                  │  X04 X03 X02 X01 M01..M13                    │
+                  └────┬─────────┬───────┬────────┬──────────────┘
+                       │         │       │        │
+                       ▼         ▼       ▼        ▼
+                 ┌─────────┐ ┌─────┐ ┌─────────┐ ┌─────────┐
+                 │  X05    │ │ X06 │ │  X07    │ │  M16    │
+                 │  DHT    │ │ WS  │ │  Fed-M  │ │ Tokens  │
+                 └────┬────┘ └──┬──┘ └─────────┘ └────┬────┘
+                      │         │                     │
+                      └─────┬───┘                     │
+                            ▼                         ▼
+                       ┌────────┐                ┌────────┐
+                       │  M14   │◄───────────────┤  M15   │
+                       │Federat.│                │ Relay  │
+                       └───┬────┘                └────────┘
+                           │
+                           ▼
+                       ┌────────┐
+                       │  M22   │
+                       │ Mobile │
+                       └────────┘
+
+           ┌─── Independent services (each plug into the bus) ───┐
+           │   M17 OCR   M18 Trans   M19 STT/TTS   M20 Vision   │
+           │   M21 Tools (extends M04)             M24 Rerank   │
+           └───────────────────────────────────────────────────┘
+
+       ┌─── Chat extensions ───┐
+       │  M23 E2E              │
+       │  M25 Group            │
+       └───────────────────────┘
+```
+
+Hard rules carried over from Phase 1:
+- No service imports another service. Talk via the bus.
+- No layer below the bus imports anything above it.
+
+---
+
+## 3. File tree additions
+
+```
+hearthnet/
+├── federation/                            # M14
+│   ├── __init__.py
+│   ├── manifest.py
+│   ├── peering.py
+│   └── relay_client.py
+│
+├── relay/                                 # M15  (separate deployable, lives in same repo)
+│   ├── __init__.py
+│   ├── server.py
+│   ├── nat_traversal.py
+│   ├── push.py
+│   └── tier.py
+│
+├── identity/
+│   └── tokens.py                          # M16 (now real, not stub)
+│
+├── dht/                                   # X05
+│   ├── __init__.py
+│   ├── kademlia.py
+│   ├── routing.py
+│   └── storage.py
+│
+├── transport/
+│   └── websocket.py                       # X06
+│
+├── observability/
+│   └── federated.py                       # X07
+│
+├── crypto/                                # M23 (new top-level)
+│   ├── __init__.py
+│   ├── ratchet.py
+│   ├── kem.py
+│   └── envelope.py
+│
+├── services/
+│   ├── ocr/                               # M17
+│   │   ├── __init__.py
+│   │   ├── service.py
+│   │   └── backends/
+│   │       ├── tesseract.py
+│   │       ├── trocr.py
+│   │       └── multilingual.py
+│   ├── translation/                       # M18
+│   │   ├── __init__.py
+│   │   ├── service.py
+│   │   └── backends/
+│   │       ├── nllb.py
+│   │       └── plattdeutsch.py
+│   ├── speech/                            # M19
+│   │   ├── __init__.py
+│   │   ├── stt_service.py
+│   │   ├── tts_service.py
+│   │   └── backends/
+│   │       ├── whisper.py
+│   │       ├── xtts.py
+│   │       └── edge_tts.py
+│   ├── image/                             # M20
+│   │   ├── __init__.py
+│   │   ├── describe_service.py
+│   │   ├── generate_service.py
+│   │   └── backends/
+│   │       ├── florence2.py
+│   │       ├── minicpm_v.py
+│   │       └── flux.py
+│   ├── rerank/                            # M24
+│   │   ├── __init__.py
+│   │   ├── service.py
+│   │   └── backends/
+│   │       └── bge_reranker.py
+│   ├── llm/
+│   │   └── tools.py                       # M21 (extends M04)
+│   ├── chat/
+│   │   ├── encryption.py                  # M23 hook
+│   │   ├── thread_service.py              # M25
+│   │   └── thread_views.py                # M25
+│   └── file/
+│       ├── resume.py                      # P7 extension
+│       └── replication.py                 # P7 extension
+
+mobile-native/                             # M22 — separate codebase, Flutter project
+└── (lives in /mobile-native, not in the Python package)
+```
+
+---
+
+## 4. Canonical conventions (delta from Phase 1)
+
+### 4.1 New type aliases
+
+```python
+# additions to hearthnet/types.py
+
+TokenID         = str        # ULID
+ThreadID        = str        # ULID
+FederationID    = str        # composite: "<community_a>:<community_b>"
+TensorChunkID   = str        # blake3:<hex>, used in M25/X07 phase-3 only
+PushDeviceID    = str        # opaque, assigned by relay tier
+RatchetEpoch    = int        # per-thread monotonic
+EncryptedPayload = bytes     # ciphertext, base64 in JSON
+
+# Extended Literal types:
+TrustLevel      = Literal["unknown","member","trusted","anchor","federated"]
+Stability       = Literal["experimental","beta","stable","deprecated"]
+```
+
+### 4.2 New constants
+
+```python
+# additions to hearthnet/constants.py
+
+TOKEN_DEFAULT_TTL_SECONDS              = 3600
+TOKEN_MAX_TTL_SECONDS                  = 86400
+FEDERATION_MANIFEST_TTL_SECONDS        = 86400
+FEDERATION_HEARTBEAT_SECONDS           = 300
+DHT_REPLICATION_K                      = 8           # bucket size
+DHT_ALPHA                              = 3           # concurrent lookups
+DHT_REFRESH_SECONDS                    = 3600
+DHT_REPUBLISH_SECONDS                  = 86400
+WEBSOCKET_PING_SECONDS                 = 30
+WEBSOCKET_IDLE_CLOSE_SECONDS           = 120
+RELAY_REGISTRATION_TTL_SECONDS         = 7200
+RELAY_PUSH_RETRY_MAX                   = 5
+E2E_RATCHET_MAX_OUT_OF_ORDER           = 32
+E2E_RATCHET_REKEY_AFTER_MESSAGES       = 100
+E2E_PREKEY_BUNDLE_SIZE                 = 20
+OCR_DEFAULT_DPI                        = 300
+OCR_MAX_PAGES_PER_REQUEST              = 50
+TRANSLATION_MAX_CHARS                  = 4000
+STT_MAX_AUDIO_SECONDS                  = 300
+TTS_MAX_TEXT_CHARS                     = 5000
+RERANK_MAX_DOCS                        = 100
+FILE_REPLICATION_DESIRED_COPIES        = 3
+FILE_RESUME_PARTIAL_TTL_SECONDS        = 3600
+```
+
+### 4.3 Capability namespace allocations (Phase 2 promotes from reserved)
+
+| Prefix | Status in Phase 1 | Status in Phase 2 |
+|--------|-------------------|---------------------|
+| `federation.*` | beta (reserved) | stable |
+| `ocr.*` | reserved | stable |
+| `trans.*` | reserved | stable |
+| `stt.*` `tts.*` | reserved | stable |
+| `img.*` | reserved | stable |
+| `rerank.*` | (new) | stable |
+| `chat.thread.*` | reserved | stable |
+| `chat.forward.*` | reserved | stable |
+| `file.put.resume@1.0` | (new) | stable |
+
+---
+
+## 5. Build order (Phase 2)
+
+| Step | Modules / extensions             | What you can demo                              |
+|------|----------------------------------|-------------------------------------------------|
+| P2-1 | M16 Tokens                       | Delegate one capability call via a token        |
+| P2-2 | X05 DHT (basic)                  | Two LANs find each other through a public DHT   |
+| P2-3 | M14 Federation                   | Two communities cross-sign, query each other    |
+| P2-4 | M15 Relay Tier                   | NAT'd peers reach each other via your relay     |
+| P2-5 | X06 WebSocket                    | Lower-latency pubsub                            |
+| P2-6 | M24 Rerank                       | RAG queries get better answers                  |
+| P2-7 | M17 OCR + M05 RAG hook           | Scanned PDFs become searchable                  |
+| P2-8 | M18 Translation                  | DE → EN of a marketplace post                   |
+| P2-9 | M19 STT/TTS                      | "Sprich mit HearthNet" voice query              |
+| P2-10 | M21 Tool calls + M04 ext        | LLM can call `rag.query` as a tool              |
+| P2-11 | M20 Vision                       | "Was siehst du auf diesem Bild?"                |
+| P2-12 | M23 E2E + M10 ext                | Chat is now end-to-end encrypted                |
+| P2-13 | M25 Group chat                   | Three-way conversation                          |
+| P2-14 | M07 ext (resume, replication, encrypt) | Bigger files, more resilient |
+| P2-15 | M22 Mobile native                | iOS / Android app on a real phone               |
+| P2-16 | X07 Federated metrics + observability polish | Real dashboards for operators |
+
+Each step is independently demoable. Each gates on no Phase 1 changes — they all attach via the bus.
+
+---
+
+## 6. Spec versioning
+
+- Capability Contract bumps to **v2.0** (additive within phase 2; major bump only on breaking changes).
+- Contract version in node manifests becomes `"2.0"`; peers running Phase 1 see `contract_version=2.0` and reject the manifest with `schema_mismatch` unless they have a compatibility shim.
+- **Compatibility shim:** a Phase 1 node may negotiate down by serving `/manifest?contract_version=1.0`. Optional. Phase 2 SHOULD include the shim for one minor release window.
+
+---
+
+## 7. What is intentionally NOT in Phase 2
+
+Pushed to Phase 3 (see [`../phase-3/00-OVERVIEW.md`](../phase-3/00-OVERVIEW.md)):
+
+- Distributed-tensor inference (Petals-style)
+- MoE expert routing
+- Federated learning on LoRA layers
+- LoRA long-distance beacons
+- EBKH evidence layer integration
+- Civil-defence pilot
+- Protocol-standardisation work
+- Conformance test suite for multi-implementation interop
+
+---
+
+## 8. Out-of-band documents (Phase 2)
+
+- **THREAT_MODEL_v2.md** — formal security write-up for federation + E2E + tokens
+- **RELAY_OPERATIONS.md** — for whoever runs `relay.hearthnet.de` (likely Christof on Hetzner)
+- **MOBILE_BUILD.md** — Flutter build, code-signing, store-submission notes
+- **MIGRATION_v1_to_v2.md** — for existing Phase-1 communities upgrading

docs/p2_p3/00-OVERVIEW_p3.md

@@ -0,0 +1,288 @@
+# HearthNet Phase 3 — Spec Set Overview
+
+**Phase 3 scope:** research-shaped, 6–12 months. This is where HearthNet stops being a product and starts being a protocol. Each module here is an investment in a long-term capability where the engineering is the easy part — the hard part is establishing trust, governance, and standards.
+
+**Stance:** Phase 3 specs are **roadmaps**, not contracts. Where a Phase-1/2 spec answers "what does this *do*?", a Phase-3 spec answers "what would we *build* if we were ready to commit?". Concrete enough to start, loose enough to be wrong about details without invalidating the direction.
+
+---
+
+## 0. Reading these specs
+
+Phase 3 specs deviate from the Phase 1 / 2 template in three respects:
+
+1. **Stability tag is `experimental` for new capabilities** unless explicitly promoted later. Mesh nodes ignore experimental capabilities unless the operator opts in via `policy.research.enable = true`.
+2. **Each module carries an "Open research questions" section** that is longer than the spec itself, by design. Phase 3 modules answer *some* of their open questions before shipping; the rest stay open.
+3. **Acceptance criteria are described, not enumerated**. The point isn't to grade an implementation against a checklist; it's to say "we'll know this is working when…"
+
+If you read a Phase 3 spec and feel uncertain about how something works, that uncertainty is faithful to the state of the work. The spec is doing its job by being honest about that.
+
+---
+
+## 1. Module map (Phase 3)
+
+### New numbered modules
+
+| ID  | Module                       | Spec file                                       | Concern                                                              |
+|-----|------------------------------|-------------------------------------------------|----------------------------------------------------------------------|
+| M26 | Distributed Inference        | `modules/M26-distributed-inference.md`          | Layer-sharded LLMs across nodes (Petals-style), small models only    |
+| M27 | MoE Expert Routing           | `modules/M27-moe-routing.md`                    | Route queries to the right expert (machine or human) via learned scorer |
+| M28 | Federated Learning           | `modules/M28-fedlearn.md`                       | FedAvg on LoRA layers; per-community fine-tuning without sharing data |
+| M29 | LoRA Long-Distance Beacons   | `modules/M29-lora-beacons.md`                   | 868MHz "community alive" beacons; no AI traffic; emergency-only       |
+| M30 | Evidence / EBKH              | `modules/M30-evidence-ebkh.md`                  | Claim graph alongside the event log; provenance + verifiability       |
+| M31 | Civil Defence Pilot          | `modules/M31-civil-defense.md`                  | THW / DRK / KatS bridge; compliance profile; audit trail              |
+| M32 | Protocol Standardisation     | `modules/M32-protocol-standard.md`              | Reference implementation, conformance suite, governance for the spec |
+
+### New cross-cutting modules
+
+| ID  | Module                | Spec file                                         | Concern                                              |
+|-----|-----------------------|---------------------------------------------------|------------------------------------------------------|
+| X08 | Tensor Transport      | `cross-cutting/X08-tensor-transport.md`           | High-throughput chunked tensor passing for M26       |
+| X09 | Conformance Suite     | `cross-cutting/X09-conformance-suite.md`          | Black-box tests defining what "HearthNet-compliant" means |
+
+### Modifications to earlier modules
+
+| Phase 1/2 module | Phase 3 extension |
+|------------------|-------------------|
+| M03 Bus          | Optional MoE routing layer between dispatcher and handler (M27) |
+| M04 LLM          | Optional `experimental.distributed_llm.chat@1.0` backend (M26) |
+| X02 Event log    | Optional `evidence.*` claim records side-by-side with events (M30) |
+| M14 Federation   | Federated learning rounds use federation as the trust substrate (M28) |
+| X03 Observability | Per-call expert-routing trace; per-shard tensor-transport metrics (M27, X08) |
+
+---
+
+## 2. Dependency graph (Phase 3 additions on top of Phases 1–2)
+
+```
+                ┌─────────────────────────────────────────────┐
+                │       Phase 1 + Phase 2 (unchanged)         │
+                └────┬────────────────┬──────────────┬────────┘
+                     │                │              │
+                     ▼                ▼              ▼
+              ┌──────────┐      ┌──────────┐    ┌──────────┐
+              │  X08     │      │  M27     │    │  M30     │
+              │  Tensor  │      │  MoE     │    │  EBKH    │
+              │  Transp. │      │  Routing │    │  Evidence│
+              └─────┬────┘      └─────┬────┘    └────┬─────┘
+                    ▼                 │              │
+              ┌──────────┐            │              │
+              │  M26     │            │              │
+              │  Distrib.│            │              │
+              │  Infer.  │            │              │
+              └──────────┘            │              │
+                                      ▼              ▼
+                                ┌──────────┐   ┌──────────┐
+                                │  M28     │   │  M31     │
+                                │  FedLearn│   │  CivDef. │
+                                └──────────┘   └──────────┘
+
+       Standalone (no software deps, governance / hardware):
+                                ┌──────────┐
+                                │  M29     │   (hardware)
+                                │  LoRa    │
+                                │  Beacons │
+                                └──────────┘
+                                ┌──────────┐
+                                │  X09     │   (process)
+                                │  Conform.│
+                                └──────────┘
+                                ┌──────────┐
+                                │  M32     │   (governance)
+                                │  Standard│
+                                └──────────┘
+```
+
+Most Phase 3 modules are independent of each other. The exceptions:
+- M26 depends on X08
+- M27 informs M26 (MoE routing picks which expert/shard)
+- M28 reuses M14 federation for cross-community rounds
+- M31 reuses M30 for evidence-grade emergency claims
+
+---
+
+## 3. File tree additions
+
+```
+hearthnet/
+├── distributed_inference/       # M26
+│   ├── __init__.py
+│   ├── shard.py
+│   ├── pipeline.py
+│   ├── routing.py
+│   └── backends/
+│       ├── petals_like.py
+│       └── small_model_layered.py
+│
+├── moe/                         # M27
+│   ├── __init__.py
+│   ├── router.py
+│   ├── scorer.py
+│   └── human_in_the_loop.py
+│
+├── fedlearn/                    # M28
+│   ├── __init__.py
+│   ├── coordinator.py
+│   ├── round.py
+│   ├── lora_diff.py
+│   └── aggregation.py
+│
+├── lora_beacons/                # M29 — hardware integration; tiny Python surface
+│   ├── __init__.py
+│   ├── beacon_bridge.py         # serial protocol to a LoRa USB stick
+│   └── policy.py
+│
+├── evidence/                    # M30
+│   ├── __init__.py
+│   ├── claim.py
+│   ├── claim_graph.py
+│   ├── provenance.py
+│   └── ebkh_bridge.py           # bridge to Christof's EBKH v3+
+│
+├── civil_defense/               # M31
+│   ├── __init__.py
+│   ├── profile.py               # THW / DRK / KatS member types
+│   ├── audit.py
+│   └── nrw_katastrophenschutz.py
+│
+├── transport/
+│   └── tensor.py                # X08
+│
+└── conformance/                 # X09
+    ├── __init__.py
+    ├── runner.py
+    ├── suites/
+    │   ├── identity.py
+    │   ├── transport.py
+    │   ├── bus.py
+    │   ├── services.py
+    │   └── federation.py
+    └── report.py
+
+protocol/                        # M32 — separate top-level dir at repo root
+├── README.md
+├── spec/                        # the protocol spec, decoupled from the impl
+│   ├── 00-overview.md           # mirror of CAPABILITY_CONTRACT but
+│   ├── 01-identity.md           # implementation-agnostic
+│   └── ...
+└── governance/
+    ├── CHANGELOG.md
+    ├── CONTRIBUTING.md
+    └── ROADMAP.md
+```
+
+---
+
+## 4. Conventions delta from Phase 2
+
+### 4.1 New `experimental` namespace
+
+A Phase-3 capability MAY be advertised as `experimental.<name>@<ver>`. Mesh nodes default to **not registering** experimental capabilities; the operator must opt in via:
+
+```toml
+[policy.research]
+enable                = true
+enabled_capabilities  = ["experimental.distributed_llm.chat@1.0", "experimental.fedlearn.round.*"]
+```
+
+Once a capability is sufficiently proven, it is promoted out of the `experimental.` prefix in a contract bump.
+
+### 4.2 New type aliases
+
+```python
+# additions to hearthnet/types.py
+
+ShardID         = str        # "<model_id>:<layer_range>"
+ExpertID        = str        # opaque, refers to a routable subsystem
+ClaimID         = str        # ULID
+RoundID         = str        # fedlearn round identifier (ULID)
+LoraBeaconID    = str        # 8-byte hex, hardware-issued
+EvidenceLevel   = Literal["unverified","cited","cross_referenced","attested","disputed"]
+ExpertKind      = Literal["model","human","service","external"]
+```
+
+### 4.3 New constants
+
+```python
+# additions to hearthnet/constants.py — Phase 3
+
+# Distributed inference (M26)
+DISTRIBUTED_MAX_SHARDS_PER_REQUEST  = 16
+DISTRIBUTED_SHARD_HEALTH_TIMEOUT_S  = 30
+DISTRIBUTED_FALLBACK_TO_LOCAL_AFTER_FAILURES = 2
+
+# MoE routing (M27)
+MOE_ROUTER_TOP_K                    = 3
+MOE_ROUTER_TRAIN_MIN_EXAMPLES       = 200
+MOE_ROUTER_RETRAIN_EVERY_HOURS      = 24
+
+# Federated learning (M28)
+FEDLEARN_MAX_ROUND_MINUTES          = 120
+FEDLEARN_MIN_PARTICIPANTS           = 3
+FEDLEARN_MAX_LORA_RANK              = 64
+FEDLEARN_GRAD_CLIP                  = 1.0
+FEDLEARN_DP_NOISE_SCALE_DEFAULT     = 0.0    # off by default; off-by-default differential privacy
+
+# Evidence (M30)
+EVIDENCE_CLAIM_TTL_DAYS_DEFAULT     = 365
+EVIDENCE_MAX_PROVENANCE_DEPTH       = 16
+
+# Civil defence (M31)
+CIVDEF_AUDIT_RETENTION_YEARS        = 10
+CIVDEF_HEARTBEAT_SECONDS            = 60
+
+# Tensor transport (X08)
+TENSOR_CHUNK_BYTES                  = 1_048_576   # 1 MB
+TENSOR_FLOW_CONTROL_WINDOW          = 16          # chunks
+TENSOR_COMPRESSION_THRESHOLD_BYTES  = 65_536
+
+# LoRa beacons (M29)
+LORA_BEACON_PERIOD_SECONDS_DEFAULT  = 600          # 10 minutes
+LORA_BEACON_MAX_PAYLOAD_BYTES       = 32
+```
+
+---
+
+## 5. Build order (Phase 3)
+
+Phase 3 is not a release; it is a set of long-running tracks. Suggested ordering by independence + value:
+
+| Track | Modules                          | Outcome                                                                       |
+|-------|----------------------------------|-------------------------------------------------------------------------------|
+| A     | X09 Conformance + M32 Standard   | Other people can build HearthNet-compliant nodes                              |
+| B     | M30 Evidence / EBKH              | Marketplace claims and emergency posts carry provenance                        |
+| C     | M27 MoE Routing (machines only)  | Better answers for free; routes RAG queries to best-suited backend             |
+| D     | M27 + M28 (human routing)        | Neighbour gets pinged when their expertise matches                            |
+| E     | M28 FedLearn                     | Communities co-train a small LoRA without sharing source data                  |
+| F     | X08 + M26 Distributed Inference  | Two anchors jointly serve a 7B model; large models become feasible LAN-wide   |
+| G     | M29 LoRa Beacons                 | Resilient "I am alive" pings during regional internet outages                  |
+| H     | M31 Civil Defence Pilot          | A real Niederrhein THW Ortsverband uses HearthNet for an exercise              |
+
+Tracks can run in parallel. None of them block the existing Phase-2 system.
+
+---
+
+## 6. Spec versioning
+
+- Capability Contract bumps to **v3.0** but the bump is *additive*. v2 nodes coexist with v3 nodes; experimental capabilities simply aren't seen by v2 nodes.
+- The first concrete deliverable of Track A (M32) is to **decouple** the protocol spec from the implementation. After that, the contract has its own version track separate from the Python implementation's version.
+
+---
+
+## 7. Out-of-band documents (Phase 3)
+
+- **RESEARCH_AGENDA.md** — the deeper "why" for each module; intended audience: PhD students and grant reviewers
+- **GOVERNANCE.md** — how spec changes are proposed, reviewed, and accepted; ties into M32
+- **ETHICS_REVIEW.md** — the framework for evaluating MoE-driven routing-to-humans (M27) and fedlearn-on-personal-data (M28)
+- **CIVDEF_AGREEMENT_TEMPLATE.md** — the MoU template for a civil-defence pilot
+
+---
+
+## 8. What is NOT in Phase 3
+
+Even with all of Phase 3 done, the following remain explicit non-goals:
+
+- A central directory of communities. There is no "HearthNet.com" listing all communities. Discovery is via word of mouth + DHT + federation. Pushed indefinitely.
+- An app store for capabilities. Capabilities are code in the source tree, reviewed by maintainers. Not pluggable at runtime by untrusted code.
+- A consensus protocol (Paxos, Raft). Communities do not vote on shared state beyond event-log gossip. Federation does not imply consensus.
+- A cryptocurrency / token economy. Not even for fedlearn incentives. Reputational signals only.
+- AGI. Even the distributed inference module targets at-most-mid-sized models (7B-class). The thesis is "small models close to people are more useful than large models far away", and Phase 3 doesn't change that.

docs/p2_p3/CAPABILITY_CONTRACT_v2.md

@@ -0,0 +1,899 @@
+# HearthNet Capability Contract — Phase 2 additions (v2.0)
+
+**Spec version:** v2.0
+**Last touched:** 2026-06-09
+**Builds on:** [`../CAPABILITY_CONTRACT.md`](../CAPABILITY_CONTRACT.md) (v1.0)
+
+This document is **additive** to v1.0. Everything in v1.0 still holds unless explicitly overridden here. Bumping a node's `contract_version` to `"2.0"` means: "I implement all of v1 plus the additions below."
+
+---
+
+## 1. Conventions delta
+
+### 1.1 New encoded forms
+
+- **Token format:** `hntoken://v1/<base64-url-nopad of canonical-JSON of token body + signature>`. See [M16 §3](modules/M16-tokens.md).
+- **Federation peering blob:** `hnfed://v1/<base64>` — analogous to invite blob, signed by both community roots (cross-sig).
+- **Encrypted payload header:** when a chat body is E2E-encrypted, the event's `data.body` becomes a `{"e2e": true, "header": {...}, "ciphertext": "<base64>"}` object. See [M23 §4](modules/M23-e2e-encryption.md).
+
+### 1.2 Sign-over-method choice
+
+Phase 2 capability tokens use the **JWS-flavoured** envelope, not the canonical-JSON envelope. Rationale: tokens are short-lived and frequently passed through HTTP intermediaries; JWS is the lingua franca.
+
+```
+hntoken_envelope = base64url(header) + "." + base64url(payload) + "." + base64url(signature)
+```
+
+Both forms continue to use Ed25519.
+
+### 1.3 New error codes (additive)
+
+| Code | Meaning |
+|------|---------|
+| `federation_forbidden` | The caller's community is not federated with ours for this capability |
+| `token_invalid` | Token signature failed |
+| `token_expired` | Token past `exp` |
+| `token_scope_insufficient` | Token does not grant this capability |
+| `relay_unreachable` | Configured relay tier is down |
+| `e2e_session_missing` | Caller did not establish an X3DH session before sending encrypted message |
+| `e2e_decrypt_failed` | Ciphertext could not be decrypted (key mismatch, ratchet drift) |
+| `dht_lookup_failed` | DHT lookup timed out before finding sources |
+| `not_federated` | Federation manifest does not exist between these communities |
+
+---
+
+## 2. Capability namespace — Phase 2 stable set
+
+Promoted from "reserved" in v1.0:
+
+| Prefix | Now | Defined |
+|--------|-----|---------|
+| `federation.*` | stable | [M14](modules/M14-federation.md) |
+| `ocr.*` | stable | [M17](modules/M17-ocr.md) |
+| `trans.*` | stable | [M18](modules/M18-translation.md) |
+| `stt.*` `tts.*` | stable | [M19](modules/M19-stt-tts.md) |
+| `img.*` | stable | [M20](modules/M20-vision.md) |
+| `rerank.*` | stable | [M24](modules/M24-rerank.md) |
+| `chat.thread.*` | stable | [M25](modules/M25-group-chat.md) |
+| `chat.forward.*` | stable | [M14](modules/M14-federation.md) (via relay) |
+| `auth.*` | stable (new) | [M16](modules/M16-tokens.md) |
+
+---
+
+## 3. Complete new capabilities list
+
+| Name | Stability | Stream? | Trust required | Section |
+|------|-----------|---------|----------------|---------|
+| `federation.peer.add@1.0` | stable | no | anchor (with co-sig) | §4.1 |
+| `federation.peer.remove@1.0` | stable | no | anchor (with co-sig) | §4.2 |
+| `federation.peer.list@1.0` | stable | no | member | §4.3 |
+| `federation.proxy@1.0` | stable | yes | federated | §4.4 |
+| `auth.token.issue@1.0` | stable | no | member | §4.5 |
+| `auth.token.revoke@1.0` | stable | no | issuer or trusted | §4.6 |
+| `auth.token.introspect@1.0` | stable | no | self | §4.7 |
+| `ocr.image@1.0` | stable | no | member | §4.8 |
+| `ocr.pdf@1.0` | stable | yes (progress) | trusted | §4.9 |
+| `trans.text@1.0` | stable | no | member | §4.10 |
+| `stt.transcribe@1.0` | stable | yes (segments) | member | §4.11 |
+| `tts.synthesize@1.0` | stable | yes (audio chunks) | member | §4.12 |
+| `img.describe@1.0` | stable | no | member | §4.13 |
+| `img.generate@1.0` | stable | yes (progress) | trusted | §4.14 |
+| `rerank.text@1.0` | stable | no | member | §4.15 |
+| `chat.thread.create@1.0` | stable | no | member | §4.16 |
+| `chat.thread.send@1.0` | stable | no | thread member | §4.17 |
+| `chat.thread.history@1.0` | stable | no | thread member | §4.18 |
+| `chat.thread.leave@1.0` | stable | no | thread member | §4.19 |
+| `chat.forward.put@1.0` | stable | yes | anchor with forward | §4.20 |
+| `chat.forward.fetch@1.0` | stable | yes | self | §4.21 |
+| `file.put.resume@1.0` | stable | yes | trusted | §4.22 |
+| `llm.chat@2.0` (UPDATE) | stable | yes | member | §4.23 |
+| `llm.tools.call@1.0` (NEW, used by `llm.chat` tool flow) | stable | no | member | §4.24 |
+
+---
+
+## 4. Per-capability specifications
+
+### 4.1 `federation.peer.add@1.0`
+
+Establish a federation link with another community.
+
+**Request:**
+```json
+{
+  "params": {},
+  "input": {
+    "client_id":         "01HXR...",
+    "peer_community_id": "ed25519:<other community root pubkey>",
+    "peer_endpoints":    [{"transport":"https","host":"...","port":7080}],
+    "co_signers":        [{"node_id":"...","signature":"..."}, "...", "..."],
+    "scope":             {
+      "capabilities":   ["rag.query","market.list"],
+      "data_visibility":"public_corpora_only"
+    },
+    "expires_at":        "2027-06-09T00:00:00Z"
+  }
+}
+```
+
+`co_signers` requires `policy.min_signatures_to_federate` (new policy field; default 3, see [M14 §5](modules/M14-federation.md)). The remote community must also have us in their federation manifest before federated calls work.
+
+**Response:**
+```json
+{"output": {"event_id": "01HXS...", "federation_id": "ed25519:A:ed25519:B"}, "meta": {"ms": 14}}
+```
+
+Emits `federation.peer.added` event.
+
+**Errors:** `unauthorized`, `bad_request`, `not_found` (peer endpoints unreachable).
+
+### 4.2 `federation.peer.remove@1.0`
+
+Terminate a federation link.
+
+**Request:**
+```json
+{
+  "params": {},
+  "input": {
+    "client_id":         "01HXR...",
+    "peer_community_id": "ed25519:...",
+    "reason":            "policy_violation|unused|mutual",
+    "co_signers":        [...]
+  }
+}
+```
+
+Emits `federation.peer.removed`.
+
+### 4.3 `federation.peer.list@1.0`
+
+List active federations.
+
+**Response:**
+```json
+{
+  "output": {
+    "peers": [
+      {
+        "community_id": "ed25519:...",
+        "name":         "Geldern Demo",
+        "scope":        {"capabilities":["rag.query"]},
+        "established_at": "...",
+        "expires_at":   "...",
+        "last_heartbeat": "..."
+      }
+    ]
+  },
+  "meta": {"ms": 2}
+}
+```
+
+### 4.4 `federation.proxy@1.0`
+
+A federated peer asks *our* community to forward a capability call to one of *our* members. This is how cross-community RAG query works: peer's anchor calls `federation.proxy` on our anchor, which then internally routes to `rag.query` on whichever local node has the corpus.
+
+**Request:**
+```json
+{
+  "params": {"target_capability": "rag.query@1.0"},
+  "input": {
+    "client_id":   "01HXR...",
+    "token":       "hntoken://v1/...",
+    "body":        { /* the body of the underlying capability */ }
+  }
+}
+```
+
+**Response:** Whatever the target capability returns. Streams pass through transparently.
+
+The proxy verifies the token's scope includes `target_capability`. Returns `federation_forbidden` otherwise.
+
+### 4.5 `auth.token.issue@1.0`
+
+Issue a capability token.
+
+**Request:**
+```json
+{
+  "params": {},
+  "input": {
+    "client_id":         "01HXR...",
+    "subject":           "ed25519:<recipient NodeID>",
+    "scope": {
+      "capabilities":   ["rag.query@1.0", "embed.text@1.0"],
+      "corpora":        ["niederrhein-emergency"],
+      "rate_limit_per_minute": 60
+    },
+    "ttl_seconds":       3600,
+    "audience":          "ed25519:<community_id where token is presented, optional>"
+  }
+}
+```
+
+**Response:**
+```json
+{
+  "output": {"token": "hntoken://v1/eyJhbGc...", "token_id": "01HXS..."},
+  "meta": {"ms": 4}
+}
+```
+
+See [M16](modules/M16-tokens.md) for token body schema.
+
+### 4.6 `auth.token.revoke@1.0`
+
+Revoke a previously-issued token.
+
+**Request:**
+```json
+{"params": {}, "input": {"client_id":"01HXR...","token_id":"01HXR..."}}
+```
+
+Emits `auth.token.revoked` event.
+
+### 4.7 `auth.token.introspect@1.0`
+
+Self-only: check whether a token is still valid.
+
+**Request:** `{"params":{},"input":{"token":"hntoken://v1/..."}}`
+
+**Response:** `{"output":{"active":bool,"scope":{...},"expires_at":"..."},"meta":{...}}`
+
+### 4.8 `ocr.image@1.0`
+
+Extract text from a single image.
+
+**Request:**
+```json
+{
+  "params": {"backend": "tesseract", "languages": ["deu","eng"]},
+  "input":  {
+    "image_cid": "blake3:...",
+    "preprocess": {"deskew": true, "denoise": false}
+  }
+}
+```
+
+**Response:**
+```json
+{
+  "output": {
+    "text":   "Trinkwasser ohne Strom ...",
+    "blocks": [
+      {"text":"Trinkwasser ohne Strom","bbox":[10,20,300,40],"confidence":0.94}
+    ],
+    "language": "de"
+  },
+  "meta": {"backend":"tesseract","ms":820}
+}
+```
+
+### 4.9 `ocr.pdf@1.0`
+
+Extract text from a (scanned) PDF. Streams per-page progress.
+
+**Request:**
+```json
+{
+  "params": {"backend":"multilingual","languages":["deu","lat"]},
+  "input":  {
+    "doc_cid":     "blake3:...",
+    "page_range":  [1, 50],
+    "preprocess":  {"deskew": true},
+    "store_text":  true
+  }
+}
+```
+
+**Stream frames:**
+```
+event: progress
+data: {"current": 3, "total": 12, "stage": "OCRing page 3"}
+
+event: page
+data: {"page": 3, "text": "...", "confidence_mean": 0.91}
+
+event: done
+data: {"pages": 12, "stored_cid": "blake3:...", "ms": 18342}
+```
+
+If `store_text:true`, the extracted text is stored as a new blob and its CID returned. Useful for piping into `rag.ingest`.
+
+### 4.10 `trans.text@1.0`
+
+Translate between languages.
+
+**Request:**
+```json
+{
+  "params": {"backend":"nllb"},
+  "input":  {
+    "text":     "Brauche Wasserkanister",
+    "from":     "de",
+    "to":       "en",
+    "domain":   "everyday"
+  }
+}
+```
+
+**Response:**
+```json
+{
+  "output": {"text":"Need water canister", "confidence": 0.97},
+  "meta":   {"backend":"nllb","model":"nllb-200-distilled-600M","ms":312}
+}
+```
+
+Plattdeutsch supported as `nds`. Marketplace UI offers one-click translate on a foreign-language post.
+
+### 4.11 `stt.transcribe@1.0`
+
+Transcribe an audio blob.
+
+**Request:**
+```json
+{
+  "params": {"backend":"whisper","model":"large-v3"},
+  "input":  {
+    "audio_cid": "blake3:...",
+    "language":  "auto",
+    "diarize":   false,
+    "translate_to_en": false
+  }
+}
+```
+
+**Stream frames:**
+```
+event: segment
+data: {"start": 0.0, "end": 4.2, "text": "Hallo, ich brauche...", "language":"de"}
+
+event: segment
+data: {"start": 4.2, "end": 8.1, "text": "Hilfe mit dem Generator."}
+
+event: done
+data: {"language":"de","ms":2100,"duration_seconds":18.4}
+```
+
+### 4.12 `tts.synthesize@1.0`
+
+Synthesize speech from text.
+
+**Request:**
+```json
+{
+  "params": {"backend":"xtts","voice":"hannes_v1","language":"de"},
+  "input":  {
+    "text":   "Das Regenwasser muss zuerst gefiltert werden.",
+    "speed":  1.0,
+    "format": "ogg_vorbis"
+  }
+}
+```
+
+**Stream frames:**
+```
+event: chunk
+data: {"i":0,"size_bytes":16384,"data_b64":"..."}
+
+event: done
+data: {"total_bytes":91247,"duration_seconds":4.2,"format":"ogg_vorbis","ms":1832}
+```
+
+### 4.13 `img.describe@1.0`
+
+Describe what's in an image.
+
+**Request:**
+```json
+{
+  "params": {"backend":"florence2"},
+  "input":  {
+    "image_cid": "blake3:...",
+    "task":      "detailed_caption",
+    "language":  "de"
+  }
+}
+```
+
+`task` ∈ `{"caption","detailed_caption","ocr","objects","tags"}`.
+
+**Response:**
+```json
+{
+  "output": {
+    "caption": "Ein Schaltplan einer einfachen Wasserfilteranlage mit ...",
+    "tags":    ["schaltplan","wasserfilter","skizze"],
+    "objects": [{"label":"pipe","bbox":[10,20,80,90]}]
+  },
+  "meta": {"backend":"florence2","ms":640}
+}
+```
+
+### 4.14 `img.generate@1.0`
+
+Generate an image from a text prompt.
+
+**Request:**
+```json
+{
+  "params": {"backend":"flux","model":"flux.1-dev","lora":"local-style-v1"},
+  "input":  {
+    "prompt":       "ein einfacher schaltplan einer wasserfilteranlage, schwarz auf weiss",
+    "negative_prompt": "color, photorealistic",
+    "width":        1024,
+    "height":       1024,
+    "steps":        20,
+    "seed":         12345
+  }
+}
+```
+
+**Stream frames:**
+```
+event: progress
+data: {"step":5,"total":20}
+
+event: done
+data: {"image_cid":"blake3:...","width":1024,"height":1024,"ms":12800}
+```
+
+### 4.15 `rerank.text@1.0`
+
+Rerank a list of documents against a query.
+
+**Request:**
+```json
+{
+  "params": {"model":"BAAI/bge-reranker-v2-m3"},
+  "input":  {
+    "query":      "Wie reinige ich Regenwasser ohne Strom?",
+    "documents":  [
+      {"id":"doc1","text":"..."},
+      {"id":"doc2","text":"..."}
+    ],
+    "top_k":      10
+  }
+}
+```
+
+**Response:**
+```json
+{
+  "output": {
+    "ranked": [
+      {"id":"doc2","score":0.91},
+      {"id":"doc1","score":0.42}
+    ]
+  },
+  "meta": {"model":"BAAI/bge-reranker-v2-m3","ms":42}
+}
+```
+
+### 4.16 `chat.thread.create@1.0`
+
+Create a multi-party thread.
+
+**Request:**
+```json
+{
+  "params": {},
+  "input": {
+    "client_id":    "01HXR...",
+    "name":         "Nachbarschaftshilfe Mai",
+    "members":      ["ed25519:...","ed25519:...","ed25519:..."],
+    "e2e_enabled":  true
+  }
+}
+```
+
+**Response:** `{"output":{"thread_id":"01HXR...","event_id":"01HXR..."},"meta":{...}}`
+
+### 4.17 `chat.thread.send@1.0`
+
+Send to a thread. Body is E2E-encrypted when `e2e_enabled`.
+
+**Request:**
+```json
+{
+  "params": {"thread_id":"01HXR..."},
+  "input": {
+    "client_id": "01HXR...",
+    "body":      "...",                       // cleartext or {"e2e":true,...} envelope
+    "attachments": [{"cid":"blake3:...","name":"..."}]
+  }
+}
+```
+
+### 4.18 `chat.thread.history@1.0`
+
+Self-only history retrieval for a thread.
+
+**Request:** `{"params":{"thread_id":"01HXR..."},"input":{"since_lamport":4000,"limit":200}}`
+
+### 4.19 `chat.thread.leave@1.0`
+
+Leave a thread.
+
+### 4.20 `chat.forward.put@1.0`
+
+Store-and-forward: leave a chat message with an anchor for later delivery.
+
+**Stream initiator pattern** identical to `file.put`. Anchors that opt into the role register this capability.
+
+### 4.21 `chat.forward.fetch@1.0`
+
+Self-only: collect queued messages from an anchor.
+
+### 4.22 `file.put.resume@1.0`
+
+Resume a partial PUT.
+
+**Request:**
+```json
+{"params":{},"input":{"manifest_cid":"blake3:...","client_id":"01HXR..."}}
+```
+
+**Response (server tells client which chunks are missing):**
+```
+event: ready
+data: {"missing":[3,4,5,8]}
+
+(client sends only those chunks)
+
+event: done
+data: {"received":4}
+```
+
+Server keeps partial transfer state for `FILE_RESUME_PARTIAL_TTL_SECONDS` (1 hour). After that, partial transfers are discarded and client must restart.
+
+### 4.23 `llm.chat@2.0` (update)
+
+Backward-compatible **minor bump** (still `name="llm.chat"`, callers can still ask for `@>=1.0` and be matched). New optional fields:
+
+```json
+{
+  "params": {"model":"...","modalities":["text","vision"]},
+  "input": {
+    "messages": [
+      {
+        "role": "user",
+        "content": [
+          {"type": "text", "text": "Was siehst du?"},
+          {"type": "image", "image_cid": "blake3:..."}
+        ]
+      }
+    ],
+    "tools": [
+      {
+        "name":        "rag.query",
+        "description": "Search the niederrhein-emergency corpus",
+        "parameters_schema": { /* JSON Schema for tool args */ }
+      }
+    ],
+    "tool_choice": "auto"
+  }
+}
+```
+
+### 4.24 `llm.tools.call@1.0`
+
+When an LLM emits a `tool_call_delta` stream frame followed by `tool_call` end, the **caller** is responsible for executing the tool. To make this composable, the LLM service offers `llm.tools.call` as a convenience that wraps "execute one bus call, return its output as a tool message". Callers MAY use it; the more general flow is to have the orchestrator (UI / agent) handle it.
+
+**Request:**
+```json
+{
+  "params": {},
+  "input": {
+    "tool_call_id":     "tc_01HXR...",
+    "target_capability":"rag.query@1.0",
+    "target_body":      { /* the tool's args, validated against the tool's parameters_schema */ }
+  }
+}
+```
+
+**Response:** mirrors target capability's response.
+
+---
+
+## 5. Wire format changes
+
+### 5.1 WebSocket upgrade
+
+For `/bus/v1/call`, clients MAY include:
+
+```
+Connection: Upgrade
+Upgrade: websocket
+Sec-WebSocket-Protocol: hearthnet-bus.v2
+```
+
+Server responds with a 101 if it supports WebSocket (Phase 2 nodes do). Once upgraded, the connection is bidirectional and persistent for the life of the request — useful for tool-call loops and streaming RAG.
+
+Frames over WebSocket are the same JSON event-name + data envelope as SSE, just delivered as binary or text WebSocket frames instead of `data:` lines.
+
+See [X06](cross-cutting/X06-websocket.md).
+
+### 5.2 Token-bearer requests
+
+When a caller carries a capability token instead of (or in addition to) a per-request signature:
+
+```
+X-HearthNet-Token: hntoken://v1/<base64>
+```
+
+The server validates the token (signature, expiry, scope) and uses the token's `subject` as the effective caller for the trust check. The token's `issuer` must be a member of a federated community.
+
+If both `X-HearthNet-Signature` and `X-HearthNet-Token` are present, signature is checked first; token is used to widen scope (e.g. "the caller is a federated peer, but for this single call they presented a token granting access").
+
+### 5.3 Federation routing
+
+When a node receives a call where `X-HearthNet-Community` ≠ our community ID:
+
+1. Look up federation manifest for the calling community.
+2. If absent → `not_federated` (404).
+3. If present but scope does not include the requested capability → `federation_forbidden` (403).
+4. Else, dispatch normally; record federation usage in metrics.
+
+---
+
+## 6. Manifests
+
+### 6.1 Federation manifest (new)
+
+```json
+{
+  "schema_version": 1,
+  "federation_id":  "<community_a>:<community_b>",
+  "community_a":    "ed25519:...",
+  "community_b":    "ed25519:...",
+  "established_at": "2026-06-09T10:00:00Z",
+  "expires_at":     "2027-06-09T10:00:00Z",
+  "scope": {
+    "a_grants_b":   {"capabilities":["rag.query"], "corpora":["public-emergency"]},
+    "b_grants_a":   {"capabilities":["rag.query"]}
+  },
+  "bootstrap_endpoints_a": [{"transport":"https","host":"...","port":7080}],
+  "bootstrap_endpoints_b": [{"transport":"https","host":"...","port":7080}],
+  "signatures":     {
+    "a": {"signed_by":"ed25519:<anchor of A>","signature":"...","co_signers":[{...},{...}]},
+    "b": {"signed_by":"ed25519:<anchor of B>","signature":"...","co_signers":[{...},{...}]}
+  }
+}
+```
+
+Both sides must sign with their `min_signatures_to_federate` threshold. The federation manifest lives in **both** communities' event logs.
+
+### 6.2 Token body (new)
+
+JWS-style. Header:
+
+```json
+{"alg":"EdDSA","typ":"hntoken","v":1}
+```
+
+Payload:
+
+```json
+{
+  "iss":          "ed25519:<issuer NodeID>",
+  "sub":          "ed25519:<subject NodeID>",
+  "aud":          "ed25519:<audience community, optional>",
+  "iat":          1717939200,
+  "exp":          1717942800,
+  "jti":          "01HXR...",
+  "scope": {
+    "capabilities":         ["rag.query@1.0"],
+    "params_constraints":   {"corpus":["niederrhein-emergency"]},
+    "rate_limit_per_minute": 60
+  }
+}
+```
+
+Signature: Ed25519 over `base64url(header) + "." + base64url(payload)`.
+
+### 6.3 Node manifest delta
+
+Phase 2 nodes set `contract_version: "2.0"`. Additional fields in `capabilities[].params`:
+
+```json
+{
+  "name": "llm.chat",
+  "version": "2.0",
+  "params": {
+    "model": "...",
+    "modalities": ["text","vision"],
+    "tools_supported": true,
+    "max_tools_per_call": 16,
+    "requires_internet": false
+  }
+}
+```
+
+---
+
+## 7. Events (additive to v1.0 §7.2)
+
+### 7.1 New event types
+
+```
+federation.peer.added
+federation.peer.removed
+federation.heartbeat
+auth.token.issued
+auth.token.revoked
+chat.thread.created
+chat.thread.member.added
+chat.thread.member.removed
+chat.thread.message.sent
+chat.thread.message.delivered
+chat.thread.archived
+e2e.prekeys.published
+e2e.session.established
+e2e.session.broken
+file.replication.scheduled
+file.replication.completed
+ocr.document.indexed
+```
+
+### 7.2 Selected schemas
+
+#### `federation.peer.added`
+
+```json
+{
+  "peer_community_id":  "ed25519:...",
+  "federation_id":      "...",
+  "scope":              {...},
+  "co_signers":         [{...},{...},{...}]
+}
+```
+
+#### `auth.token.issued`
+
+Stored without the signature payload (just metadata for audit):
+
+```json
+{
+  "token_id":  "01HXR...",
+  "subject":   "ed25519:...",
+  "scope":     {...},
+  "expires_at":"...",
+  "audience":  "ed25519:..."
+}
+```
+
+#### `auth.token.revoked`
+
+```json
+{"token_id":"01HXR...","reason":"manual|policy|compromise"}
+```
+
+#### `chat.thread.created`
+
+```json
+{
+  "thread_id":   "01HXR...",
+  "client_id":   "01HXR...",
+  "name":        "Nachbarschaftshilfe Mai",
+  "members":     ["ed25519:...","ed25519:..."],
+  "e2e_enabled": true,
+  "ratchet_root_pubkey": "x25519:..."
+}
+```
+
+#### `chat.thread.message.sent`
+
+```json
+{
+  "thread_id": "01HXR...",
+  "client_id": "01HXR...",
+  "body":      {"e2e":true,"header":{...},"ciphertext":"..."} | "<cleartext>",
+  "attachments": [...]
+}
+```
+
+#### `e2e.prekeys.published`
+
+```json
+{
+  "node_id":   "ed25519:...",
+  "identity_pubkey": "x25519:...",
+  "signed_prekey": {"pubkey":"x25519:...","signature":"ed25519:..."},
+  "one_time_prekeys": ["x25519:...","x25519:...","..."]
+}
+```
+
+#### `file.replication.scheduled`
+
+```json
+{
+  "cid":               "blake3:...",
+  "desired_copies":    3,
+  "current_copies":    1,
+  "candidate_holders": ["ed25519:...","ed25519:..."]
+}
+```
+
+#### `ocr.document.indexed`
+
+```json
+{
+  "doc_cid":     "blake3:...",
+  "text_cid":    "blake3:...",
+  "pages":       12,
+  "languages":   ["de","la"],
+  "ocr_backend": "multilingual"
+}
+```
+
+### 7.3 Federation events propagate cross-community
+
+Events with `event_type ∈ {federation.*, auth.token.issued, auth.token.revoked}` MAY be cross-published into a federated community's event log. The community receiving such an event records the originating community in `data._source_community`. This is the only case where an event's `community_id` does not equal the log it lives in.
+
+---
+
+## 8. Pub-sub topics (additive)
+
+| Topic | Producer | Subscriber |
+|-------|----------|------------|
+| `federation.peer.added` | member adding | all members |
+| `federation.peer.heartbeat.<peer_community>` | federation client loop | UI |
+| `auth.token.issued` | issuer | issuer + subject |
+| `chat.thread.message.<thread_id>` | sender | thread members |
+| `e2e.prekey.request.<our_short_id>` | sender wanting session | recipient |
+| `e2e.session.handshake.<our_short_id>` | initiator | responder |
+| `file.replication.request.<cid_prefix>` | replication scheduler | all anchors |
+| `mobile.push.<device_id>` | sender | push relay tier (M15) |
+
+---
+
+## 9. Errors — complete delta (additive to v1.0 §9)
+
+| Code | When | Retry? |
+|------|------|--------|
+| `federation_forbidden` | Caller's community not federated for this capability | no |
+| `not_federated` | No federation manifest with caller's community | no |
+| `token_invalid` | Token signature bad | no |
+| `token_expired` | Token past `exp` | no, request a new token |
+| `token_scope_insufficient` | Token does not include this capability | no |
+| `token_revoked` | Token id in revoked list | no |
+| `relay_unreachable` | Configured relay tier down | yes, exp backoff |
+| `e2e_session_missing` | No active X3DH session | yes, after key exchange |
+| `e2e_decrypt_failed` | Ciphertext can't be decrypted | no, request rekey |
+| `dht_lookup_failed` | DHT did not find sources in time | yes |
+| `ratchet_out_of_order` | Message too far out of order; sender must rewind | maybe |
+
+---
+
+## 10. Versioning and migration
+
+### 10.1 Mixed-version mesh
+
+A v1.0 node and a v2.0 node may coexist on the same LAN, but:
+
+- A v2.0 node calling a v1.0 node for a Phase 2 capability gets `not_found` (v1 didn't register it).
+- A v1.0 node calling a v2.0 node for a v1 capability works fine (additive contract).
+- v2.0 routes around v1.0 nodes for any capability that requires v2 features.
+
+### 10.2 Migration of an existing community
+
+When the founder upgrades to v2.0:
+
+1. New `policy.min_signatures_to_federate` field added with default 3
+2. New event types unlock; old log still replays cleanly
+3. Existing nodes prompted to upgrade via `community.policy.updated` event
+4. After 30 days, federation capabilities won't dispatch to non-upgraded nodes
+
+See `MIGRATION_v1_to_v2.md` (out of band).
+
+---
+
+## 11. Out of scope still (deferred to Phase 3)
+
+- Distributed-tensor inference capabilities (`experimental.distributed_llm.chat`)
+- MoE-style expert routing (lives inside the bus as a learned scorer)
+- Federated learning capabilities (`fedlearn.*`)
+- LoRA long-distance beacons (no capability, hardware-only)
+- Evidence-layer integration (`evidence.*` namespace reserved here, defined in Phase 3)
+- Conformance test suite as a protocol surface

docs/p2_p3/CAPABILITY_CONTRACT_v3.md

@@ -0,0 +1,651 @@
+# HearthNet Capability Contract — Phase 3 additions (v3.0)
+
+**Spec version:** v3.0
+**Last touched:** 2026-06-09
+**Builds on:** [`../phase-2/CAPABILITY_CONTRACT_v2.md`](../phase-2/CAPABILITY_CONTRACT_v2.md) (v2.0)
+
+This document is **additive** to v2.0. Phase 3 capabilities are mostly under the `experimental.` namespace; nodes default to ignoring them unless `policy.research.enable = true`.
+
+---
+
+## 1. Conventions delta
+
+### 1.1 The `experimental.` namespace
+
+A capability of the form `experimental.<name>@<ver>` is treated specially:
+
+- Nodes do not register experimental capabilities by default.
+- The bus discovery layer (M02 / X05) excludes experimental capabilities from outbound advertisements unless the local policy opts in.
+- A capability in `experimental.` MAY change its contract between minor versions without a contract bump. Callers MUST tolerate breakage.
+- Once a capability is sufficiently proven and stable, it is **promoted** out of `experimental.` in a future minor contract bump. The prior `experimental.` form is then deprecated; both forms work for one minor-release window.
+
+Promotion criteria (a soft checklist; the protocol working group, see M32, decides):
+- ≥ 2 independent implementations.
+- Used in production by ≥ 3 communities for ≥ 90 days.
+- Open security review with no unresolved high-severity findings.
+- Per-call cost (compute, latency, bytes) within a 2× factor of the budget in the capability spec.
+
+### 1.2 Claim records (new top-level concept)
+
+Phase 3 introduces a second persistent surface alongside the event log: the **claim graph**. Where the event log records "X did Y at T", the claim graph records "X asserts P, citing E".
+
+```json
+{
+  "schema_version": 1,
+  "claim_id":       "01HXR...",
+  "claim_type":     "factual|preference|policy|sighting|...",
+  "predicate":      {"subject":"...","verb":"...","object":"...","modifiers":{...}},
+  "evidence":       [{"kind":"event_ref","value":"01HXR..."}, {"kind":"document_cid","value":"blake3:..."}],
+  "asserted_by":    "ed25519:...",
+  "asserted_at":    "...",
+  "evidence_level": "unverified|cited|cross_referenced|attested|disputed",
+  "supersedes":     ["01HXR..."],
+  "signature":      "..."
+}
+```
+
+Claims live in a separate Merkle-DAG store ([M30 §4](modules/M30-evidence-ebkh.md)). They are not events. Events describe what *happened*; claims describe what is *believed*.
+
+### 1.3 New error codes (additive)
+
+| Code | Meaning |
+|------|---------|
+| `experimental_disabled` | Caller asked for an experimental capability that the node has not opted into |
+| `shard_unavailable` | Distributed inference: one shard host failed mid-stream |
+| `pipeline_stalled` | Distributed inference: no progress within timeout |
+| `fedlearn_round_quorum` | Federated learning: too few participants for this round |
+| `fedlearn_diff_invalid` | Submitted LoRA diff failed schema or bounds check |
+| `evidence_contradiction` | A new claim directly contradicts a previously-attested claim |
+| `civdef_audit_required` | Operation rejected because civil-defence audit policy is active and the call is unsigned by an authorised role |
+
+---
+
+## 2. Capability namespace allocations
+
+Promoted from "reserved" in v2.0 or introduced new:
+
+| Prefix | Status | Defined |
+|--------|--------|---------|
+| `experimental.distributed_llm.*` | experimental | [M26](modules/M26-distributed-inference.md) |
+| `experimental.moe.*` | experimental | [M27](modules/M27-moe-routing.md) |
+| `experimental.fedlearn.*` | experimental | [M28](modules/M28-fedlearn.md) |
+| `evidence.*` | stable | [M30](modules/M30-evidence-ebkh.md) |
+| `civdef.*` | stable (when civdef profile active) | [M31](modules/M31-civil-defense.md) |
+| `protocol.*` | stable | [M32](modules/M32-protocol-standard.md) (conformance reporting) |
+
+---
+
+## 3. Phase 3 capabilities
+
+| Name | Stability | Stream? | Trust required | Section |
+|------|-----------|---------|----------------|---------|
+| `experimental.distributed_llm.chat@1.0` | experimental | yes | member + research opt-in | §4.1 |
+| `experimental.distributed_llm.shard.advertise@1.0` | experimental | no | trusted + research opt-in | §4.2 |
+| `experimental.distributed_llm.shard.serve@1.0` | experimental | yes | trusted + research opt-in | §4.3 |
+| `experimental.moe.route@1.0` | experimental | no | member + research opt-in | §4.4 |
+| `experimental.moe.expert.register@1.0` | experimental | no | self + research opt-in | §4.5 |
+| `experimental.moe.expert.handoff@1.0` | experimental | yes | as configured | §4.6 |
+| `experimental.fedlearn.round.start@1.0` | experimental | no | anchor + research opt-in | §4.7 |
+| `experimental.fedlearn.round.participate@1.0` | experimental | yes | member + research opt-in | §4.8 |
+| `experimental.fedlearn.round.aggregate@1.0` | experimental | no | round coordinator | §4.9 |
+| `experimental.fedlearn.lora.publish@1.0` | experimental | no | anchor | §4.10 |
+| `evidence.claim.assert@1.0` | stable | no | member | §4.11 |
+| `evidence.claim.dispute@1.0` | stable | no | member | §4.12 |
+| `evidence.claim.attest@1.0` | stable | no | trusted | §4.13 |
+| `evidence.claim.query@1.0` | stable | no | member | §4.14 |
+| `evidence.provenance.trace@1.0` | stable | no | member | §4.15 |
+| `civdef.alert.publish@1.0` | stable | no | authorised KatS role | §4.16 |
+| `civdef.role.register@1.0` | stable | no | anchor (with role-cert) | §4.17 |
+| `civdef.audit.export@1.0` | stable | yes | authorised auditor | §4.18 |
+| `protocol.conformance.report@1.0` | stable | no | self | §4.19 |
+| `protocol.version.list@1.0` | stable | no | unknown | §4.20 |
+
+---
+
+## 4. Per-capability specifications
+
+### 4.1 `experimental.distributed_llm.chat@1.0`
+
+Like `llm.chat@2.0` but the inference is sharded across multiple shard-server nodes. The caller's node acts as the orchestrator and streams tokens back to the user.
+
+**Request:**
+```json
+{
+  "params": {
+    "model": "Qwen2.5-7B-Instruct",
+    "sharding": "auto",
+    "fallback_to_local": true
+  },
+  "input": {
+    "messages": [...]
+  }
+}
+```
+
+**Stream frames:** same as `llm.chat@2.0` (`token_delta`, `done`), plus diagnostic frames:
+
+```
+event: shard_status
+data: {"shards":[
+  {"shard_id":"Qwen2.5-7B:0-7","host":"ed25519:...","status":"online","latency_ms":4},
+  {"shard_id":"Qwen2.5-7B:8-15","host":"ed25519:...","status":"online","latency_ms":7}
+]}
+
+event: shard_failover
+data: {"failed_shard":"Qwen2.5-7B:8-15","replacement":"Qwen2.5-7B:8-15@other_host"}
+```
+
+**Errors:** `shard_unavailable`, `pipeline_stalled`, `experimental_disabled`.
+
+### 4.2 `experimental.distributed_llm.shard.advertise@1.0`
+
+A node informs the bus that it is willing to serve a specific shard range.
+
+**Request:**
+```json
+{
+  "params": {},
+  "input": {
+    "shard_id":      "Qwen2.5-7B:0-7",
+    "model_id":      "Qwen2.5-7B-Instruct",
+    "layer_range":   [0, 7],
+    "max_concurrent_streams": 2,
+    "vram_required_mb": 6800
+  }
+}
+```
+
+Emits `experimental.shard.advertised` into the event log. Other nodes can then call `experimental.distributed_llm.shard.serve` on us to use the shard.
+
+### 4.3 `experimental.distributed_llm.shard.serve@1.0`
+
+Tensor-passing inner call. Not normally invoked by user code — used by orchestrators only. See [X08](cross-cutting/X08-tensor-transport.md) for the wire format.
+
+### 4.4 `experimental.moe.route@1.0`
+
+Decide which expert (model, human, service) to route a request to.
+
+**Request:**
+```json
+{
+  "params": {},
+  "input": {
+    "request_summary": "User asks about Sankt Martins parade route in Issum, 2026.",
+    "tags":            ["local_knowledge","event_planning"],
+    "top_k":           3
+  }
+}
+```
+
+**Response:**
+```json
+{
+  "output": {
+    "routes": [
+      {"expert_id":"human:ed25519:...", "kind":"human", "score":0.91, "name":"Maria K."},
+      {"expert_id":"corpus:niederrhein-events", "kind":"service", "score":0.74, "endpoint":"rag.query@1.0"},
+      {"expert_id":"model:llama3-70b-instruct", "kind":"model", "score":0.41}
+    ],
+    "rationale": "Sankt Martins is a local cultural event; humans with annotated knowledge of Issum specifically score highest."
+  },
+  "meta": {"ms": 28}
+}
+```
+
+### 4.5 `experimental.moe.expert.register@1.0`
+
+A node (or a human via their node) declares itself an expert on some topics.
+
+```json
+{
+  "params": {},
+  "input": {
+    "expert_kind":      "human",
+    "topics":           ["sankt_martins","niederrhein_local_history"],
+    "availability":     {"weekdays_19_21_local": true},
+    "consent_to_route": true
+  }
+}
+```
+
+### 4.6 `experimental.moe.expert.handoff@1.0`
+
+When a route to a human expert is chosen, this capability hands the conversation off to the expert's UI (typically: chat thread invite, optional E2E).
+
+```json
+{
+  "params": {},
+  "input": {
+    "expert_id":         "human:ed25519:...",
+    "context_summary":   "...",
+    "permitted_replies": ["text","attachment"],
+    "deadline_minutes":  60
+  }
+}
+```
+
+### 4.7 `experimental.fedlearn.round.start@1.0`
+
+Anchor opens a federated learning round.
+
+```json
+{
+  "params": {},
+  "input": {
+    "round_id":         "01HXR...",
+    "base_model":       "Qwen2.5-3B-Instruct",
+    "lora_config":      {"r":16,"alpha":32,"target_modules":["q_proj","v_proj"]},
+    "training_corpus":  "niederrhein-emergency",
+    "min_participants": 3,
+    "max_minutes":      120,
+    "objective":        "next_token_loss",
+    "dp_noise_scale":   0.0
+  }
+}
+```
+
+### 4.8 `experimental.fedlearn.round.participate@1.0`
+
+A node opts into a round and streams its computed LoRA diff back.
+
+**Stream frames:**
+```
+event: phase
+data: {"phase":"training","step":0,"total":200}
+
+event: phase
+data: {"phase":"training","step":200,"total":200}
+
+event: diff
+data: {"lora_diff_cid":"blake3:...","examples_seen":4321,"loss_end":0.84}
+```
+
+### 4.9 `experimental.fedlearn.round.aggregate@1.0`
+
+Coordinator aggregates submitted diffs (FedAvg, weighted by `examples_seen`).
+
+```json
+{
+  "params": {},
+  "input": {
+    "round_id":  "01HXR...",
+    "diff_cids": ["blake3:...","blake3:...","blake3:..."]
+  }
+}
+```
+
+**Response:** `{"output":{"aggregated_lora_cid":"blake3:...","participants_used":3,"dropped":[]},"meta":{...}}`
+
+### 4.10 `experimental.fedlearn.lora.publish@1.0`
+
+After aggregation, the anchor publishes the new LoRA to the community.
+
+```json
+{
+  "params": {},
+  "input": {
+    "round_id":            "01HXR...",
+    "aggregated_lora_cid": "blake3:...",
+    "base_model":          "Qwen2.5-3B-Instruct",
+    "version":             "niederrhein-emergency-v3"
+  }
+}
+```
+
+Emits `experimental.fedlearn.lora.published` event. Nodes that have opted into the corpus' LoRA can pull the new version.
+
+### 4.11 `evidence.claim.assert@1.0`
+
+Assert a claim into the claim graph.
+
+**Request:**
+```json
+{
+  "params": {},
+  "input": {
+    "claim_type":   "factual",
+    "predicate":    {"subject":"<Brunnen 12 Issum>","verb":"yields","object":"<200L/h drinkable water>"},
+    "evidence":     [
+      {"kind":"event_ref","value":"01HXR..."},
+      {"kind":"document_cid","value":"blake3:..."}
+    ],
+    "ttl_days":     365
+  }
+}
+```
+
+**Response:** `{"output":{"claim_id":"01HXR...","evidence_level":"cited"},"meta":{...}}`
+
+### 4.12 `evidence.claim.dispute@1.0`
+
+```json
+{"params":{},"input":{"claim_id":"01HXR...","reason":"...","counter_evidence":[...]}}
+```
+
+### 4.13 `evidence.claim.attest@1.0`
+
+Trusted member adds an attestation, raising the claim's evidence level.
+
+```json
+{"params":{},"input":{"claim_id":"01HXR...","attestation":"I confirmed personally on 2026-06-08"}}
+```
+
+A claim becomes `attested` after `policy.evidence.attestations_required_for_attested` distinct trusted attestations (default 3).
+
+### 4.14 `evidence.claim.query@1.0`
+
+```json
+{
+  "params": {},
+  "input": {
+    "predicate_pattern": {"subject":"<Brunnen 12 Issum>","verb":"*"},
+    "min_evidence_level":"cited",
+    "limit": 20
+  }
+}
+```
+
+### 4.15 `evidence.provenance.trace@1.0`
+
+Walk the evidence chain backwards.
+
+```json
+{
+  "params": {},
+  "input": {"claim_id":"01HXR...","max_depth":8}
+}
+```
+
+**Response:** a DAG of `{claim_id, predicate, evidence_summary, asserted_by, asserted_at, evidence_level}` nodes.
+
+### 4.16 `civdef.alert.publish@1.0`
+
+Civil-defence-grade alert. Differs from `emergency.publish@1.0` (Phase 1) in that the caller must hold a `civdef.role` credential and the alert is signed for legal-evidence retention.
+
+```json
+{
+  "params": {},
+  "input": {
+    "client_id":       "01HXR...",
+    "severity":        "warning|alert|emergency|extreme",
+    "category":        "weather|fire|chemical|flood|infrastructure|other",
+    "title":           "Stromausfall Issum Mitte",
+    "body":            "...",
+    "areas":           [{"polygon":"<geojson>"}],
+    "issued_by_role":  "thw_ortsverband_geldern",
+    "audit_evidence":  [{"kind":"role_certificate","value":"..."}]
+  }
+}
+```
+
+### 4.17 `civdef.role.register@1.0`
+
+Anchor registers a community member as holding an authorised KatS role.
+
+```json
+{
+  "params": {},
+  "input": {
+    "subject":         "ed25519:...",
+    "role":            "thw_helfer|drk_sanitaeter|feuerwehr|katastrophenschutzbeauftragter",
+    "role_certificate_cid": "blake3:...",
+    "expires_at":      "2027-01-01T00:00:00Z"
+  }
+}
+```
+
+### 4.18 `civdef.audit.export@1.0`
+
+Stream the audit trail for a time range — used by KatS auditors.
+
+```json
+{
+  "params": {},
+  "input": {"from":"2026-04-01T00:00:00Z","to":"2026-06-01T00:00:00Z"}
+}
+```
+
+**Stream frames:** one frame per audit record; signed batches every 1000 records for tamper evidence.
+
+### 4.19 `protocol.conformance.report@1.0`
+
+Generate a conformance report against the [X09](cross-cutting/X09-conformance-suite.md) suite.
+
+```json
+{
+  "params": {},
+  "input": {"suite_version":"3.0"}
+}
+```
+
+**Response:** `{"output":{"report_cid":"blake3:...","passed":214,"failed":3,"skipped":17},"meta":{...}}`
+
+### 4.20 `protocol.version.list@1.0`
+
+Returns the contract versions this node supports and the conformance suite versions it has passed.
+
+```json
+{
+  "output": {
+    "contract_versions": ["1.0","2.0","3.0"],
+    "conformance_passed": [{"suite":"3.0","report_cid":"blake3:..."}],
+    "implementation": {"name":"hearthnet-py","version":"0.7.2","commit":"abc123"}
+  }
+}
+```
+
+---
+
+## 5. Wire format additions
+
+### 5.1 Tensor transport (binary)
+
+For `experimental.distributed_llm.shard.serve@1.0` only. WebSocket frames carry **binary payloads** with a 16-byte header:
+
+```
++---------------+---------------+---------------+
+|  4B chunk_id  |  4B chunk_seq |  4B total_seq |
+|  2B flags     |  2B reserved  |
++---------------+---------------+---------------+
+|             tensor chunk (≤ 1 MB)             |
++-----------------------------------------------+
+```
+
+`flags`:
+- `0x0001` LAST (this is the final chunk of the message)
+- `0x0002` COMPRESSED (zstd-compressed; only when payload ≥ `TENSOR_COMPRESSION_THRESHOLD_BYTES`)
+- `0x0004` FP16 (else FP32)
+
+See [X08](cross-cutting/X08-tensor-transport.md) for the protocol.
+
+### 5.2 Claim records
+
+Claims are stored in their own Merkle-DAG; they reference events but are not events. A claim record header:
+
+```
+X-HearthNet-Claim: 01HXR...
+X-HearthNet-Claim-Asserted-By: ed25519:...
+```
+
+Claim records flow through the same transport but with `Content-Type: application/vnd.hearthnet.claim+json`.
+
+### 5.3 Civil-defence audit signatures
+
+`civdef.*` capability calls always require both a per-call signature (per X01) **and** a `civdef.role` credential reference in headers:
+
+```
+X-HearthNet-CivDef-Role: thw_helfer
+X-HearthNet-CivDef-Role-Cert: blake3:...
+```
+
+Without these, the call returns `civdef_audit_required`.
+
+---
+
+## 6. Manifests
+
+### 6.1 Node manifest delta
+
+```json
+{
+  "contract_version": "3.0",
+  "experimental_capabilities_enabled": false,
+  "civdef_profile": {
+    "active": false,
+    "authority": "",
+    "audit_endpoint": ""
+  },
+  "research_opt_in": {
+    "fedlearn": false,
+    "distributed_inference": false,
+    "moe_human_routing": false
+  }
+}
+```
+
+A node with `experimental_capabilities_enabled=false` does not advertise any `experimental.*` capabilities.
+
+### 6.2 Community policy delta
+
+```yaml
+research:
+  enable: false
+  enabled_capabilities: []
+
+fedlearn:
+  participate: false
+  share_compute_with_federated: false
+  dp_noise_scale_min: 0.0
+
+civdef:
+  active: false
+  authority: ""   # e.g. "Kreis Kleve Bevölkerungsschutz"
+  audit_export_to: ""
+
+evidence:
+  attestations_required_for_attested: 3
+  default_claim_ttl_days: 365
+  retain_disputed_claims_days: 1825
+```
+
+---
+
+## 7. Events (additive to v2.0 §7.1)
+
+```
+experimental.shard.advertised
+experimental.shard.retired
+experimental.fedlearn.round.opened
+experimental.fedlearn.round.closed
+experimental.fedlearn.lora.published
+experimental.moe.expert.registered
+experimental.moe.expert.unregistered
+evidence.claim.asserted
+evidence.claim.disputed
+evidence.claim.attested
+evidence.claim.superseded
+civdef.alert.published
+civdef.role.registered
+civdef.role.revoked
+civdef.audit.exported
+protocol.conformance.reported
+```
+
+### Selected schemas
+
+#### `evidence.claim.asserted`
+
+```json
+{
+  "claim_id":     "01HXR...",
+  "claim_type":   "factual",
+  "predicate":    {...},
+  "asserted_by":  "ed25519:...",
+  "evidence_level": "cited",
+  "claim_payload_cid": "blake3:..."
+}
+```
+
+The event references the full claim record by CID; the claim itself lives in the claim store (M30 §4).
+
+#### `civdef.alert.published`
+
+```json
+{
+  "client_id":     "01HXR...",
+  "severity":      "alert",
+  "category":      "infrastructure",
+  "title":         "Stromausfall Issum Mitte",
+  "issued_by":     "ed25519:...",
+  "issued_by_role":"thw_ortsverband_geldern",
+  "areas":         [{...}]
+}
+```
+
+#### `experimental.fedlearn.lora.published`
+
+```json
+{
+  "round_id":            "01HXR...",
+  "base_model":          "Qwen2.5-3B-Instruct",
+  "aggregated_lora_cid": "blake3:...",
+  "version":             "niederrhein-emergency-v3",
+  "participants":        5,
+  "objective":           "next_token_loss",
+  "dp_noise_scale":      0.0
+}
+```
+
+---
+
+## 8. Pub-sub topics (additive)
+
+| Topic | Producer | Subscriber |
+|-------|----------|------------|
+| `experimental.shard.advertised` | shard host | orchestrators |
+| `experimental.fedlearn.round.opened.<round_id>` | coordinator | members |
+| `experimental.moe.expert.registered` | expert | router |
+| `evidence.claim.<claim_id>.changed` | asserter / disputer | watchers |
+| `civdef.alert.<area_hash>` | civdef caller | members in area |
+
+---
+
+## 9. Errors — complete Phase 3 set
+
+(additive to v2.0 §9)
+
+| Code | When |
+|------|------|
+| `experimental_disabled` | Caller asked for an experimental capability the node has not opted into |
+| `shard_unavailable` | A required shard host failed mid-pipeline |
+| `pipeline_stalled` | No progress within `DISTRIBUTED_SHARD_HEALTH_TIMEOUT_S` |
+| `fedlearn_round_quorum` | Round closed for lack of participants |
+| `fedlearn_diff_invalid` | Submitted diff failed schema or norm bounds |
+| `evidence_contradiction` | A new claim directly contradicts an attested claim; explicit override needed |
+| `civdef_audit_required` | Operation rejected because civdef profile is active and call is not properly signed |
+| `civdef_role_invalid` | Civdef role credential is missing, expired, or revoked |
+| `conformance_failed` | A `protocol.*` operation depends on a passed conformance suite that this node has not passed |
+
+---
+
+## 10. Compatibility
+
+- v3 contract nodes are backward-compatible with v2 nodes: v2 nodes simply do not see `experimental.*` capabilities and do not understand claim records (the relevant event types are skipped on replay).
+- A v3 community must include at least one anchor that has passed the `protocol.conformance.report@1.0` suite at level 3.0, otherwise capabilities under `civdef.*` and the claim graph features remain inert (the spec calls this **degraded v3**; everything else still works).
+- Promotion of an `experimental.X` to `X` is a normal v3 minor contract bump; the experimental form remains registered for one minor cycle.
+
+---
+
+## 11. Glossary additions
+
+| Term | Meaning |
+|------|---------|
+| Shard | One contiguous range of transformer layers, served by one node |
+| Pipeline | The chain of shards that produces a single LLM forward-pass |
+| Expert | A routable subsystem (model, service, or human) that can answer a class of requests |
+| Round | A federated-learning training session bounded in time |
+| Diff | A LoRA delta tensor produced by one round participant |
+| Claim | A signed assertion of a predicate, with evidence, in the claim graph |
+| Attestation | A signed endorsement by a trusted member that a claim is correct |
+| KatS | Katastrophenschutz — German civil protection |
+| Conformance | The property of an implementation passing the X09 suite at a stated level |

docs/p2_p3/IMPLEMENTATION_REFERENCE.md

@@ -0,0 +1,288 @@
+# HearthNet Phase 3 — Spec Set Overview
+
+**Phase 3 scope:** research-shaped, 6–12 months. This is where HearthNet stops being a product and starts being a protocol. Each module here is an investment in a long-term capability where the engineering is the easy part — the hard part is establishing trust, governance, and standards.
+
+**Stance:** Phase 3 specs are **roadmaps**, not contracts. Where a Phase-1/2 spec answers "what does this *do*?", a Phase-3 spec answers "what would we *build* if we were ready to commit?". Concrete enough to start, loose enough to be wrong about details without invalidating the direction.
+
+---
+
+## 0. Reading these specs
+
+Phase 3 specs deviate from the Phase 1 / 2 template in three respects:
+
+1. **Stability tag is `experimental` for new capabilities** unless explicitly promoted later. Mesh nodes ignore experimental capabilities unless the operator opts in via `policy.research.enable = true`.
+2. **Each module carries an "Open research questions" section** that is longer than the spec itself, by design. Phase 3 modules answer *some* of their open questions before shipping; the rest stay open.
+3. **Acceptance criteria are described, not enumerated**. The point isn't to grade an implementation against a checklist; it's to say "we'll know this is working when…"
+
+If you read a Phase 3 spec and feel uncertain about how something works, that uncertainty is faithful to the state of the work. The spec is doing its job by being honest about that.
+
+---
+
+## 1. Module map (Phase 3)
+
+### New numbered modules
+
+| ID  | Module                       | Spec file                                       | Concern                                                              |
+|-----|------------------------------|-------------------------------------------------|----------------------------------------------------------------------|
+| M26 | Distributed Inference        | `modules/M26-distributed-inference.md`          | Layer-sharded LLMs across nodes (Petals-style), small models only    |
+| M27 | MoE Expert Routing           | `modules/M27-moe-routing.md`                    | Route queries to the right expert (machine or human) via learned scorer |
+| M28 | Federated Learning           | `modules/M28-fedlearn.md`                       | FedAvg on LoRA layers; per-community fine-tuning without sharing data |
+| M29 | LoRA Long-Distance Beacons   | `modules/M29-lora-beacons.md`                   | 868MHz "community alive" beacons; no AI traffic; emergency-only       |
+| M30 | Evidence / EBKH              | `modules/M30-evidence-ebkh.md`                  | Claim graph alongside the event log; provenance + verifiability       |
+| M31 | Civil Defence Pilot          | `modules/M31-civil-defense.md`                  | THW / DRK / KatS bridge; compliance profile; audit trail              |
+| M32 | Protocol Standardisation     | `modules/M32-protocol-standard.md`              | Reference implementation, conformance suite, governance for the spec |
+
+### New cross-cutting modules
+
+| ID  | Module                | Spec file                                         | Concern                                              |
+|-----|-----------------------|---------------------------------------------------|------------------------------------------------------|
+| X08 | Tensor Transport      | `cross-cutting/X08-tensor-transport.md`           | High-throughput chunked tensor passing for M26       |
+| X09 | Conformance Suite     | `cross-cutting/X09-conformance-suite.md`          | Black-box tests defining what "HearthNet-compliant" means |
+
+### Modifications to earlier modules
+
+| Phase 1/2 module | Phase 3 extension |
+|------------------|-------------------|
+| M03 Bus          | Optional MoE routing layer between dispatcher and handler (M27) |
+| M04 LLM          | Optional `experimental.distributed_llm.chat@1.0` backend (M26) |
+| X02 Event log    | Optional `evidence.*` claim records side-by-side with events (M30) |
+| M14 Federation   | Federated learning rounds use federation as the trust substrate (M28) |
+| X03 Observability | Per-call expert-routing trace; per-shard tensor-transport metrics (M27, X08) |
+
+---
+
+## 2. Dependency graph (Phase 3 additions on top of Phases 1–2)
+
+```
+                ┌─────────────────────────────────────────────┐
+                │       Phase 1 + Phase 2 (unchanged)         │
+                └────┬────────────────┬──────────────┬────────┘
+                     │                │              │
+                     ▼                ▼              ▼
+              ┌──────────┐      ┌──────────┐    ┌──────────┐
+              │  X08     │      │  M27     │    │  M30     │
+              │  Tensor  │      │  MoE     │    │  EBKH    │
+              │  Transp. │      │  Routing │    │  Evidence│
+              └─────┬────┘      └─────┬────┘    └────┬─────┘
+                    ▼                 │              │
+              ┌──────────┐            │              │
+              │  M26     │            │              │
+              │  Distrib.│            │              │
+              │  Infer.  │            │              │
+              └──────────┘            │              │
+                                      ▼              ▼
+                                ┌──────────┐   ┌──────────┐
+                                │  M28     │   │  M31     │
+                                │  FedLearn│   │  CivDef. │
+                                └──────────┘   └──────────┘
+
+       Standalone (no software deps, governance / hardware):
+                                ┌──────────┐
+                                │  M29     │   (hardware)
+                                │  LoRa    │
+                                │  Beacons │
+                                └──────────┘
+                                ┌──────────┐
+                                │  X09     │   (process)
+                                │  Conform.│
+                                └──────────┘
+                                ┌──────────┐
+                                │  M32     │   (governance)
+                                │  Standard│
+                                └──────────┘
+```
+
+Most Phase 3 modules are independent of each other. The exceptions:
+- M26 depends on X08
+- M27 informs M26 (MoE routing picks which expert/shard)
+- M28 reuses M14 federation for cross-community rounds
+- M31 reuses M30 for evidence-grade emergency claims
+
+---
+
+## 3. File tree additions
+
+```
+hearthnet/
+├── distributed_inference/       # M26
+│   ├── __init__.py
+│   ├── shard.py
+│   ├── pipeline.py
+│   ├── routing.py
+│   └── backends/
+│       ├── petals_like.py
+│       └── small_model_layered.py
+│
+├── moe/                         # M27
+│   ├── __init__.py
+│   ├── router.py
+│   ├── scorer.py
+│   └── human_in_the_loop.py
+│
+├── fedlearn/                    # M28
+│   ├── __init__.py
+│   ├── coordinator.py
+│   ├── round.py
+│   ├── lora_diff.py
+│   └── aggregation.py
+│
+├── lora_beacons/                # M29 — hardware integration; tiny Python surface
+│   ├── __init__.py
+│   ├── beacon_bridge.py         # serial protocol to a LoRa USB stick
+│   └── policy.py
+│
+├── evidence/                    # M30
+│   ├── __init__.py
+│   ├── claim.py
+│   ├── claim_graph.py
+│   ├── provenance.py
+│   └── ebkh_bridge.py           # bridge to Christof's EBKH v3+
+│
+├── civil_defense/               # M31
+│   ├── __init__.py
+│   ├── profile.py               # THW / DRK / KatS member types
+│   ├── audit.py
+│   └── nrw_katastrophenschutz.py
+│
+├── transport/
+│   └── tensor.py                # X08
+│
+└── conformance/                 # X09
+    ├── __init__.py
+    ├── runner.py
+    ├── suites/
+    │   ├── identity.py
+    │   ├── transport.py
+    │   ├── bus.py
+    │   ├── services.py
+    │   └── federation.py
+    └── report.py
+
+protocol/                        # M32 — separate top-level dir at repo root
+├── README.md
+├── spec/                        # the protocol spec, decoupled from the impl
+│   ├── 00-overview.md           # mirror of CAPABILITY_CONTRACT but
+│   ├── 01-identity.md           # implementation-agnostic
+│   └── ...
+└── governance/
+    ├── CHANGELOG.md
+    ├── CONTRIBUTING.md
+    └── ROADMAP.md
+```
+
+---
+
+## 4. Conventions delta from Phase 2
+
+### 4.1 New `experimental` namespace
+
+A Phase-3 capability MAY be advertised as `experimental.<name>@<ver>`. Mesh nodes default to **not registering** experimental capabilities; the operator must opt in via:
+
+```toml
+[policy.research]
+enable                = true
+enabled_capabilities  = ["experimental.distributed_llm.chat@1.0", "experimental.fedlearn.round.*"]
+```
+
+Once a capability is sufficiently proven, it is promoted out of the `experimental.` prefix in a contract bump.
+
+### 4.2 New type aliases
+
+```python
+# additions to hearthnet/types.py
+
+ShardID         = str        # "<model_id>:<layer_range>"
+ExpertID        = str        # opaque, refers to a routable subsystem
+ClaimID         = str        # ULID
+RoundID         = str        # fedlearn round identifier (ULID)
+LoraBeaconID    = str        # 8-byte hex, hardware-issued
+EvidenceLevel   = Literal["unverified","cited","cross_referenced","attested","disputed"]
+ExpertKind      = Literal["model","human","service","external"]
+```
+
+### 4.3 New constants
+
+```python
+# additions to hearthnet/constants.py — Phase 3
+
+# Distributed inference (M26)
+DISTRIBUTED_MAX_SHARDS_PER_REQUEST  = 16
+DISTRIBUTED_SHARD_HEALTH_TIMEOUT_S  = 30
+DISTRIBUTED_FALLBACK_TO_LOCAL_AFTER_FAILURES = 2
+
+# MoE routing (M27)
+MOE_ROUTER_TOP_K                    = 3
+MOE_ROUTER_TRAIN_MIN_EXAMPLES       = 200
+MOE_ROUTER_RETRAIN_EVERY_HOURS      = 24
+
+# Federated learning (M28)
+FEDLEARN_MAX_ROUND_MINUTES          = 120
+FEDLEARN_MIN_PARTICIPANTS           = 3
+FEDLEARN_MAX_LORA_RANK              = 64
+FEDLEARN_GRAD_CLIP                  = 1.0
+FEDLEARN_DP_NOISE_SCALE_DEFAULT     = 0.0    # off by default; off-by-default differential privacy
+
+# Evidence (M30)
+EVIDENCE_CLAIM_TTL_DAYS_DEFAULT     = 365
+EVIDENCE_MAX_PROVENANCE_DEPTH       = 16
+
+# Civil defence (M31)
+CIVDEF_AUDIT_RETENTION_YEARS        = 10
+CIVDEF_HEARTBEAT_SECONDS            = 60
+
+# Tensor transport (X08)
+TENSOR_CHUNK_BYTES                  = 1_048_576   # 1 MB
+TENSOR_FLOW_CONTROL_WINDOW          = 16          # chunks
+TENSOR_COMPRESSION_THRESHOLD_BYTES  = 65_536
+
+# LoRa beacons (M29)
+LORA_BEACON_PERIOD_SECONDS_DEFAULT  = 600          # 10 minutes
+LORA_BEACON_MAX_PAYLOAD_BYTES       = 32
+```
+
+---
+
+## 5. Build order (Phase 3)
+
+Phase 3 is not a release; it is a set of long-running tracks. Suggested ordering by independence + value:
+
+| Track | Modules                          | Outcome                                                                       |
+|-------|----------------------------------|-------------------------------------------------------------------------------|
+| A     | X09 Conformance + M32 Standard   | Other people can build HearthNet-compliant nodes                              |
+| B     | M30 Evidence / EBKH              | Marketplace claims and emergency posts carry provenance                        |
+| C     | M27 MoE Routing (machines only)  | Better answers for free; routes RAG queries to best-suited backend             |
+| D     | M27 + M28 (human routing)        | Neighbour gets pinged when their expertise matches                            |
+| E     | M28 FedLearn                     | Communities co-train a small LoRA without sharing source data                  |
+| F     | X08 + M26 Distributed Inference  | Two anchors jointly serve a 7B model; large models become feasible LAN-wide   |
+| G     | M29 LoRa Beacons                 | Resilient "I am alive" pings during regional internet outages                  |
+| H     | M31 Civil Defence Pilot          | A real Niederrhein THW Ortsverband uses HearthNet for an exercise              |
+
+Tracks can run in parallel. None of them block the existing Phase-2 system.
+
+---
+
+## 6. Spec versioning
+
+- Capability Contract bumps to **v3.0** but the bump is *additive*. v2 nodes coexist with v3 nodes; experimental capabilities simply aren't seen by v2 nodes.
+- The first concrete deliverable of Track A (M32) is to **decouple** the protocol spec from the implementation. After that, the contract has its own version track separate from the Python implementation's version.
+
+---
+
+## 7. Out-of-band documents (Phase 3)
+
+- **RESEARCH_AGENDA.md** — the deeper "why" for each module; intended audience: PhD students and grant reviewers
+- **GOVERNANCE.md** — how spec changes are proposed, reviewed, and accepted; ties into M32
+- **ETHICS_REVIEW.md** — the framework for evaluating MoE-driven routing-to-humans (M27) and fedlearn-on-personal-data (M28)
+- **CIVDEF_AGREEMENT_TEMPLATE.md** — the MoU template for a civil-defence pilot
+
+---
+
+## 8. What is NOT in Phase 3
+
+Even with all of Phase 3 done, the following remain explicit non-goals:
+
+- A central directory of communities. There is no "HearthNet.com" listing all communities. Discovery is via word of mouth + DHT + federation. Pushed indefinitely.
+- An app store for capabilities. Capabilities are code in the source tree, reviewed by maintainers. Not pluggable at runtime by untrusted code.
+- A consensus protocol (Paxos, Raft). Communities do not vote on shared state beyond event-log gossip. Federation does not imply consensus.
+- A cryptocurrency / token economy. Not even for fedlearn incentives. Reputational signals only.
+- AGI. Even the distributed inference module targets at-most-mid-sized models (7B-class). The thesis is "small models close to people are more useful than large models far away", and Phase 3 doesn't change that.

docs/p2_p3/IMPLEMENTATION_REFERENCE_p3.md

@@ -0,0 +1,493 @@
+# HearthNet Phase 3 — Implementation Reference
+
+**Spec set:** v3.0 — *experimental*
+**Status:** Research-shaped. Names and contracts may shift before promotion to stable.
+
+This document is the symbol index and quick-reference for everything introduced in Phase 3. It mirrors the structure of Phase 1's and Phase 2's `IMPLEMENTATION_REFERENCE.md`, so a maintainer can find the relevant spec section, file, class, capability, event, or constant by name.
+
+Read this with the Phase 1 + Phase 2 references already in hand. Phase 3 is purely additive; it does not redefine anything from earlier phases.
+
+---
+
+## 1. Module index
+
+| ID | Name | Status | Spec file |
+|----|------|--------|-----------|
+| M26 | Distributed Inference (layer sharding) | experimental | [phase-3/modules/M26-distributed-inference.md](modules/M26-distributed-inference.md) |
+| M27 | MoE Expert Routing | experimental | [phase-3/modules/M27-moe-routing.md](modules/M27-moe-routing.md) |
+| M28 | Federated Learning (LoRA aggregation) | experimental | [phase-3/modules/M28-fedlearn.md](modules/M28-fedlearn.md) |
+| M29 | LoRa Hardware Beacons | experimental | [phase-3/modules/M29-lora-beacons.md](modules/M29-lora-beacons.md) |
+| M30 | Evidence Graph & EBKH Integration | experimental | [phase-3/modules/M30-evidence-ebkh.md](modules/M30-evidence-ebkh.md) |
+| M31 | Civil Defense (NRW Bevölkerungsschutz) | experimental | [phase-3/modules/M31-civil-defense.md](modules/M31-civil-defense.md) |
+| M32 | Protocol Standardisation & Conformance | provisional | [phase-3/modules/M32-protocol-standard.md](modules/M32-protocol-standard.md) |
+| X08 | Tensor Transport | experimental | [phase-3/cross-cutting/X08-tensor-transport.md](cross-cutting/X08-tensor-transport.md) |
+| X09 | Conformance Suite | provisional | [phase-3/cross-cutting/X09-conformance-suite.md](cross-cutting/X09-conformance-suite.md) |
+
+All `experimental` modules are gated by per-module feature flags in `hearthnet/config.py` and default to `False`.
+
+---
+
+## 2. Capability index
+
+All Phase 3 capabilities (except `protocol.*`) live in the `experimental.*` namespace. The bus refuses to register them unless the corresponding feature flag is on.
+
+### 2.1 Distributed Inference (M26)
+
+| Capability | Module | Notes |
+|------------|--------|-------|
+| `experimental.distributed_llm.shard.list` | M26 | Local advertisement of shards we host |
+| `experimental.distributed_llm.shard.connect` | M26 | Negotiate an X08 tensor session |
+| `experimental.distributed_llm.shard.forward` | M26 | Run forward through a hosted shard |
+| `experimental.distributed_llm.pipeline.plan` | M26 | Construct a layer pipeline for a model |
+| `experimental.distributed_llm.pipeline.run` | M26 | Execute a planned pipeline |
+| `experimental.distributed_llm.pipeline.status` | M26 | Pipeline state and stats |
+
+### 2.2 MoE Routing (M27)
+
+| Capability | Module | Notes |
+|------------|--------|-------|
+| `experimental.moe.expert.register` | M27 | Register self as an expert for some topics |
+| `experimental.moe.expert.list` | M27 | List registered experts |
+| `experimental.moe.expert.unregister` | M27 | Withdraw expert registration |
+| `experimental.moe.route.query` | M27 | Get top-K experts for a query |
+| `experimental.moe.route.handoff` | M27 | Initiate human-in-the-loop handoff |
+| `experimental.moe.feedback.record` | M27 | Record outcome for scorer training |
+
+### 2.3 Federated Learning (M28)
+
+| Capability | Module | Notes |
+|------------|--------|-------|
+| `experimental.fedlearn.round.announce` | M28 | Coordinator announces a round |
+| `experimental.fedlearn.round.list` | M28 | List open rounds |
+| `experimental.fedlearn.round.join` | M28 | Participant joins a round |
+| `experimental.fedlearn.round.submit` | M28 | Submit gradient delta |
+| `experimental.fedlearn.round.status` | M28 | Get round state |
+| `experimental.fedlearn.round.finalize` | M28 | Coordinator finalises and aggregates |
+| `experimental.fedlearn.adapter.fetch` | M28 | Fetch an aggregated adapter by SHA |
+| `experimental.fedlearn.adapter.apply` | M28 | Apply adapter to session or node |
+
+### 2.4 LoRa Beacons (M29)
+
+| Capability | Module | Notes |
+|------------|--------|-------|
+| `experimental.lora.status` | M29 | Hardware and link status |
+| `experimental.lora.beacon.send` | M29 | Send a normal beacon |
+| `experimental.lora.panic.send` | M29 | Send a panic burst |
+| `experimental.lora.peer.list` | M29 | Known LoRa peers |
+| `experimental.lora.peer.verify` | M29 | TOFU-confirm a peer's NodeID binding |
+| `experimental.lora.recent_beacons` | M29 | Recent RX'd beacons |
+| `experimental.lora.duty_cycle` | M29 | Current duty-cycle budget status |
+
+### 2.5 Evidence Graph (M30)
+
+| Capability | Module | Notes |
+|------------|--------|-------|
+| `experimental.evidence.claim.assert` | M30 | Assert a new claim |
+| `experimental.evidence.claim.dispute` | M30 | Dispute an existing claim |
+| `experimental.evidence.claim.attest` | M30 | Attest to an existing claim |
+| `experimental.evidence.claim.get` | M30 | Fetch a single claim by ID |
+| `experimental.evidence.claim.query` | M30 | Query claims by triple |
+| `experimental.evidence.provenance.trace` | M30 | Walk the derivation graph |
+| `experimental.evidence.subject.summary` | M30 | Multi-claim summary for a subject |
+| `experimental.evidence.ebkh.sync` | M30 | Sync with external EBKH endpoint |
+
+### 2.6 Civil Defense (M31)
+
+| Capability | Module | Notes |
+|------------|--------|-------|
+| `experimental.civdef.alert.publish` | M31 | Role-cert-gated alert publication |
+| `experimental.civdef.alert.cancel` | M31 | Cancel an active alert |
+| `experimental.civdef.alert.list` | M31 | List active alerts (filtered) |
+| `experimental.civdef.alert.get` | M31 | Fetch an alert envelope |
+| `experimental.civdef.alert.subscribe` | M31 | Subscribe to alerts matching a filter |
+| `experimental.civdef.alert.ack` | M31 | Acknowledge an alert |
+| `experimental.civdef.alert.acks` | M31 | List acks for an alert |
+| `experimental.civdef.role.register` | M31 | Register a role certificate |
+| `experimental.civdef.role.list` | M31 | List registered certificates |
+| `experimental.civdef.role.revoke` | M31 | Revoke a certificate |
+| `experimental.civdef.audit.export` | M31 | Export tamper-evident audit chain |
+
+### 2.7 Protocol (M32) — stable
+
+| Capability | Module | Notes |
+|------------|--------|-------|
+| `protocol.version.list` | M32 | Versions supported |
+| `protocol.self.describe` | M32 | Implementation descriptor |
+| `protocol.conformance.report` | M32 | Run / fetch conformance report |
+| `protocol.registry.list` | M32 | Known implementations |
+| `protocol.registry.announce` | M32 | Announce self to registry |
+
+---
+
+## 3. Event types
+
+All Phase 3 event types follow the convention `<area>.<entity>.<verb>` and are recorded in the X02 event log.
+
+### 3.1 Distributed inference
+
+```
+distributed_llm.shard.advertised
+distributed_llm.shard.withdrawn
+distributed_llm.pipeline.planned
+distributed_llm.pipeline.started
+distributed_llm.pipeline.shard_failed
+distributed_llm.pipeline.failover
+distributed_llm.pipeline.completed
+distributed_llm.pipeline.cancelled
+```
+
+### 3.2 MoE
+
+```
+moe.expert.registered
+moe.expert.unregistered
+moe.route.computed
+moe.handoff.initiated
+moe.handoff.accepted
+moe.handoff.declined
+moe.handoff.timed_out
+moe.feedback.recorded
+```
+
+### 3.3 Federated learning
+
+```
+fedlearn.round.announced
+fedlearn.round.joined
+fedlearn.round.consent.granted
+fedlearn.round.submitted
+fedlearn.round.aggregated
+fedlearn.round.completed
+fedlearn.round.aborted
+fedlearn.round.takeover
+fedlearn.adapter.published
+fedlearn.adapter.applied
+```
+
+### 3.4 LoRa
+
+```
+lora.beacon.sent
+lora.beacon.received
+lora.panic.sent
+lora.panic.received
+lora.peer.unknown
+lora.peer.verified
+lora.peer.conflict
+lora.duty_cycle.exhausted
+lora.duty_cycle.overridden
+lora.rx.dropped
+```
+
+### 3.5 Evidence
+
+```
+evidence.claim.asserted
+evidence.claim.attested
+evidence.dispute.opened
+evidence.dispute.retracted
+evidence.ebkh.synced
+evidence.ebkh.sync_partial
+```
+
+### 3.6 Civil defense
+
+```
+civdef.alert.published
+civdef.alert.forwarded
+civdef.alert.acked
+civdef.alert.cancelled
+civdef.alert.dropped.revoked
+civdef.alert.foreign_role
+civdef.role.registered
+civdef.role.revoked
+civdef.audit.checkpointed
+civdef.audit.broken
+```
+
+### 3.7 Protocol
+
+```
+protocol.descriptor.announced
+protocol.registry.updated
+protocol.conformance.ran
+```
+
+---
+
+## 4. Type aliases (added to `hearthnet/types.py`)
+
+```python
+ShardID         = NewType("ShardID", str)               # "model:layer_range[:tier]"
+ExpertID        = NewType("ExpertID", str)              # "human:..." | "model:..." | "service:..." | "external:..."
+ExpertKind      = Literal["human","model","service","external"]
+ClaimID         = NewType("ClaimID", str)               # base32 of SHA-256 canonical claim
+SourceID        = NewType("SourceID", str)
+EvidenceLevel   = Literal["unverified","cited","cross_referenced","attested","disputed"]
+RoundID         = NewType("RoundID", str)               # ULID
+LoraBeaconID    = NewType("LoraBeaconID", str)
+LoraDeviceID    = NewType("LoraDeviceID", str)
+AlertID         = NewType("AlertID", str)               # ULID
+AlertSeverity   = Literal["info","advisory","warning","emergency","extreme"]
+AckStatus       = Literal["received","acting","need_help","standing_down","mistaken"]
+
+@dataclass(frozen=True)
+class ProtocolVersion: major: int; minor: int; patch: int; suffix: str = ""
+
+# Reused from earlier phases but referenced here for completeness:
+NodeID          # M01
+EventID         # X02
+AuthToken       # M16
+Bbox            # M07 spatial extensions
+Tensor          # local LLM tensor type, dtype-tagged
+```
+
+---
+
+## 5. Centralised constants (`hearthnet/constants.py`, Phase 3 additions)
+
+```python
+# --- Distributed inference (M26) ---
+DISTRIBUTED_LLM_MAX_SHARDS_PER_PIPELINE              = 16
+DISTRIBUTED_LLM_SHARD_HEARTBEAT_SECONDS              = 5
+DISTRIBUTED_LLM_FAILOVER_TIMEOUT_SECONDS             = 10
+DISTRIBUTED_LLM_MAX_PIPELINE_LATENCY_TOKENS_PER_S    = 2.0     # advisory floor
+DISTRIBUTED_LLM_DEFAULT_DTYPE                        = "fp16"
+
+# --- MoE routing (M27) ---
+MOE_TOP_K_DEFAULT                                    = 3
+MOE_LEARNED_SCORER_MIN_FEEDBACK_SAMPLES              = 200
+MOE_HUMAN_HANDOFF_DEFAULT_TIMEOUT_HOURS              = 24
+MOE_HUMAN_HANDOFF_COOLDOWN_HOURS                     = 2
+MOE_HUMAN_RATE_LIMIT_PER_DAY                         = 5
+
+# --- Federated learning (M28) ---
+FEDLEARN_MAX_LORA_RANK                               = 64
+FEDLEARN_MAX_LORA_TARGET_MODULES                     = 8
+FEDLEARN_MAX_TRAIN_STEPS                             = 1000
+FEDLEARN_MAX_PARTICIPANTS                            = 32
+FEDLEARN_MIN_PARTICIPANTS                            = 3
+FEDLEARN_DP_NOISE_SCALE_DEFAULT                      = 0.0     # off
+FEDLEARN_CLIP_NORM_DEFAULT                           = 1.0
+FEDLEARN_SUBMISSION_MAX_BYTES                        = 64 * 1024 * 1024
+
+# --- LoRa beacons (M29) ---
+LORA_BEACON_PERIOD_SECONDS_DEFAULT                   = 600     # 10 min
+LORA_BEACON_MAX_PAYLOAD_BYTES                        = 32
+LORA_RX_QUEUE_MAX                                    = 256
+LORA_PEER_RX_MAX_PER_MINUTE                          = 20
+LORA_PANIC_BURST_COUNT                               = 3
+LORA_PANIC_BURST_GAP_MS                              = 800
+
+# --- Evidence (M30) ---
+EVIDENCE_CLAIM_TTL_DAYS_DEFAULT                      = 365
+EVIDENCE_DISPUTE_MIN_TRUST                           = 0.3
+EVIDENCE_MAX_PROVENANCE_DEPTH                        = 8
+
+# --- Civil defense (M31) ---
+CIVDEF_AUDIT_RETENTION_YEARS                         = 10      # operator must validate against current law
+CIVDEF_ACK_MAX_PER_MINUTE_PER_NODE                   = 5
+CIVDEF_ALERT_TITLE_MAX_CHARS                         = 80
+CIVDEF_ALERT_BODY_MAX_CHARS                          = 1000
+
+# --- Tensor transport (X08) ---
+TENSOR_CHUNK_BYTES                                   = 1 * 1024 * 1024     # 1 MiB
+TENSOR_FLOW_CONTROL_WINDOW                           = 16
+TENSOR_COMPRESSION_THRESHOLD_BYTES                   = 64 * 1024
+TENSOR_KEEPALIVE_SECONDS                             = 30
+TENSOR_MAX_SESSION_LIFETIME_SECONDS                  = 3600
+
+# --- Conformance suite (X09) ---
+CONFORMANCE_DEFAULT_SEED                             = 0xC0FFEE
+CONFORMANCE_DEFAULT_OUTPUT_DIR                       = "./conformance-report"
+```
+
+---
+
+## 6. Error codes (Phase 3 additions)
+
+| Code | Module | When |
+|------|--------|------|
+| `experimental_disabled` | shared | Capability called with the feature flag off |
+| `shard_unavailable` | M26 | No replica for the required layer range |
+| `shard_unreachable` | M26 | All replicas connectivity-failed |
+| `pipeline_failed` | M26 | Aggregate failure of an in-flight pipeline |
+| `pipeline_cancelled` | M26 | Pipeline cancelled by caller |
+| `tensor_too_large` | X08 | Tensor exceeds rx_buffer_bytes_max |
+| `unknown_frame_type` | X08 | Frame type outside the defined set |
+| `expert_unknown` | M27 | Referenced expert is not registered |
+| `expert_unavailable` | M27 | Expert known but currently outside availability window |
+| `human_handoff_declined` | M27 | Human expert explicitly declined |
+| `human_handoff_timed_out` | M27 | Handoff exceeded timeout without ack |
+| `consent_required` | M28 | join() without explicit operator consent |
+| `base_model_mismatch` | M28 | Local base model SHA differs from manifest |
+| `insufficient_resources` | M28 | Estimated VRAM/disk exceeds budget |
+| `delta_invalid` | M28 | Submitted state-dict fails structural validation |
+| `fedlearn_aggregation_failed` | M28 | Aggregation produced NaN/Inf |
+| `fedlearn_min_participants_unmet` | M28 | Round closed below quorum |
+| `fedlearn_aggregator_unreachable` | M28 | Finalize attempted while coordinator offline |
+| `adapter_not_found` | M28 | Fetch for unknown adapter SHA |
+| `lora_hardware_unavailable` | M29 | No stick present |
+| `lora_hardware_unsupported` | M29 | Adapter init failed |
+| `lora_duty_cycle_exhausted` | M29 | Non-panic send with empty budget |
+| `lora_peer_unknown` | M29 | Verify against unseen sender_hash |
+| `lora_peer_conflict` | M29 | Verify would create conflicting binding |
+| `lora_frame_malformed` | M29 | RX frame structurally invalid |
+| `claim_not_found` | M30 | Reference to unknown ClaimID |
+| `claim_signature_invalid` | M30 | Signature doesn't verify |
+| `evidence_cycle_detected` | M30 | Derivation chain forms a cycle |
+| `evidence_contradiction` | M30 | (advisory) conflicting claims on same triple |
+| `ebkh_unavailable` | M30 | EBKH endpoint unreachable |
+| `civdef_cert_not_owned` | M31 | Cert holder ≠ caller identity |
+| `civdef_cert_invalid` | M31 | Cert expired, revoked, or signature broken |
+| `civdef_cert_unrecognised` | M31 | Issuer chain doesn't reach a trust root |
+| `civdef_cert_out_of_scope` | M31 | Cert lacks the requested role/region |
+| `civdef_alert_not_found` | M31 | Operation on unknown AlertID |
+| `civdef_alert_target_invalid` | M31 | Malformed target or outside scope |
+| `civdef_audit_chain_broken` | M31 | Audit chain hash/signature mismatch |
+| `civdef_role_revoked` | M31 | Op with revoked cert |
+| `civdef_region_unsupported` | M31 | No region adapter loaded |
+| `civdef_ack_rate_limited` | M31 | Ack rate exceeded |
+| `protocol_version_unknown` | M32 | Reference to unknown protocol version |
+| `protocol_suite_not_installed` | M32 | Conformance report requested without X09 |
+| `protocol_descriptor_invalid` | M32 | Malformed descriptor announcement |
+| `protocol_unsupported_capability` | M32 | Federation negotiates no compatible major |
+
+---
+
+## 7. File map (top-level)
+
+```
+hearthnet/
+├── distributed_inference/       # M26
+│   ├── shard.py
+│   ├── orchestrator.py
+│   ├── router.py
+│   ├── plan.py
+│   └── failover.py
+├── moe/                         # M27
+│   ├── router.py
+│   ├── scorer.py
+│   ├── expert_registry.py
+│   ├── human_in_the_loop.py
+│   └── feedback.py
+├── fedlearn/                    # M28
+│   ├── coordinator.py
+│   ├── participant.py
+│   ├── trainer.py
+│   ├── aggregator.py
+│   ├── delta.py
+│   ├── privacy.py
+│   └── manifest.py
+├── lora/                        # M29
+│   ├── service.py
+│   ├── serial_bridge.py
+│   ├── frame.py
+│   ├── duty_cycle.py
+│   ├── peer_map.py
+│   └── adapters/{meshtastic,rfm95w,sx126x}.py
+├── evidence/                    # M30
+│   ├── service.py
+│   ├── claim.py
+│   ├── store.py
+│   ├── query.py
+│   ├── extractor.py
+│   ├── ebkh_adapter.py
+│   └── trust.py
+├── civdef/                      # M31
+│   ├── service.py
+│   ├── alert.py
+│   ├── role.py
+│   ├── audit.py
+│   ├── target.py
+│   ├── ack.py
+│   └── regions/nrw.py
+├── protocol/                    # M32 runtime
+│   ├── service.py
+│   ├── registry.py
+│   └── report.py
+└── transport/
+    └── tensor/                  # X08
+        ├── session.py
+        ├── frame.py
+        ├── flow.py
+        └── compress.py
+
+protocol/                        # M32 spec artefacts (repo root)
+├── VERSION
+├── README.md
+├── CHANGELOG.md
+├── governance.md
+├── versioning.md
+├── reference-implementations.md
+├── core/...
+└── experimental/...
+
+conformance/                     # X09 suite (repo root)
+├── VERSION
+├── runner.py
+├── report.py
+├── harness/...
+├── suites/...
+└── vectors/...
+```
+
+---
+
+## 8. Configuration index
+
+Each module defines its own `*Config` dataclass; all are surfaced through the global `HearthnetConfig` and read from `~/.config/hearthnet/config.toml`. Phase 3 additions:
+
+```python
+@dataclass(frozen=True)
+class HearthnetConfig:
+    # ... (Phase 1, Phase 2 fields) ...
+    distributed_llm:   DistributedLlmConfig
+    moe:               MoeConfig
+    fedlearn:          FedLearnConfig
+    lora:              LoraConfig
+    evidence:          EvidenceConfig
+    civdef:            CivDefConfig
+    tensor_transport:  TensorTransportConfig
+    protocol:          ProtocolConfig
+```
+
+Every Phase 3 config has `enabled: bool = False` except `protocol` (default `True`). The bus dispatcher refuses to register Phase 3 capabilities when their module's enabled flag is False.
+
+---
+
+## 9. Build order (recap from `00-OVERVIEW.md`)
+
+Phase 3 has eight independent tracks A-H that can be parallelised:
+
+```
+Track A:  X09 conformance suite scaffolding  →  M32 protocol service
+Track B:  M30 evidence + EBKH adapter
+Track C:  M27 MoE machine experts (router, registry, scorer)
+Track D:  M27 human-in-the-loop coordinator (depends on Track C base)
+Track E:  M28 federated LoRA aggregation
+Track F:  X08 tensor transport  →  M26 distributed inference
+Track G:  M29 LoRa beacons (hardware-gated)
+Track H:  M31 civil defense (depends on M30 evidence)
+```
+
+Tracks A and F unlock the most downstream work (M32 needs X09; M26 needs X08). Tracks G and H are most easily deferred if Phase 3 needs to ship a minimal cut.
+
+---
+
+## 10. Open-question summary
+
+Each module spec has its own §10 with detailed open questions. The recurring themes across Phase 3:
+
+1. **Real-world identity binding** (M28 sybil defence, M31 institutional keys, M30 EBKH trust roots, M27 human verification) — the cryptographic story is solid; the social/institutional story is the work.
+2. **Adversarial robustness** (M26 byzantine shards, M28 poisoning, M30 disputed claims, M31 forged certs) — all have stub defences and known harder problems.
+3. **Second implementation** (M32) — until a non-reference impl exists, conformance is performative. This is the single most important next step.
+4. **Cross-Land / cross-border generalisation** (M31 regional adapter, M30 EBKH OSINT scope, M29 regulatory regions) — designed for NRW first; structures admit other regions but they're unbuilt.
+5. **Resource tiers** (M26 phone-class participants, M28 hardware fairness) — heterogeneous hardware aggregation is largely unsolved.
+6. **Privacy / DP calibration** (M28 noise scale, M30 sensitive claims, M29 sender hash) — defaults are conservative; tuning is operator-by-operator.
+
+Each module also lists module-specific items. Read them.
+
+---
+
+*Last updated: spec set v3.0. Phase 3 specs were authored with the intent that any of M26–M31 could be cut from a shipping release without affecting Phase 1 + Phase 2 functionality. M32 + X09 are the long-term durability investment and should ship even when other Phase 3 modules don't.*

docs/p2_p3/M14-federation.md

@@ -0,0 +1,434 @@
+# M14 — Federation
+
+**Spec version:** v1.0 (Phase 2)
+**Depends on:** M01 (identity), M16 (tokens), X02 (events), X05 (DHT, for cross-LAN bootstrap), X01 (transport), M03 (bus), M15 (relay, for NAT traversal)
+**Depended on by:** M22 (mobile uses federation for cross-community discovery), all services indirectly (federated calls route through M14)
+
+---
+
+## 1. Responsibility
+
+Let two **communities** establish a trust link and grant each other scoped access to specific capabilities. Federation is the bridge from "your house" to "the neighbourhood".
+
+- Establish a federation between two communities (mutual cross-signature)
+- Enforce scope on cross-community capability calls
+- Route federated calls through a designated **Bridge** node profile
+- Heartbeat between federated communities so liveness is visible
+- Issue federation tokens (via M16) for individual calls
+
+Out of scope:
+- Many-community mesh federation — Phase 3 (this module handles bilateral only)
+- Anonymous federation — never
+
+---
+
+## 2. File layout
+
+```
+hearthnet/federation/
+├── __init__.py
+├── manifest.py         # FederationManifest builder + verifier
+├── peering.py          # the cross-sig handshake
+├── relay_client.py     # outbound calls into peer community
+└── service.py          # FederationService — registers federation.* capabilities
+```
+
+---
+
+## 3. Federation manifest
+
+Format defined in [CAP2 §6.1](../CAPABILITY_CONTRACT_v2.md). Key properties:
+
+- Lives in **both** communities' event logs
+- Signed by anchors of both, with per-community `min_signatures_to_federate` co-signatures
+- Carries the **scope** each side grants the other
+- Has an expiry (`expires_at`) — typically 1 year, with auto-renewal via heartbeats
+
+A node may belong to **one** community (Phase 1) and that community may be federated with **N** other communities. Federation is a community-level relationship, not per-node.
+
+---
+
+## 4. Public API
+
+### 4.1 `manifest.py`
+
+```python
+# hearthnet/federation/manifest.py
+from dataclasses import dataclass
+
+@dataclass(frozen=True)
+class FederationScope:
+    """What one community grants the other."""
+    capabilities:        list[str]                   # ["rag.query@1.0"]
+    params_constraints:  dict[str, list[str]]         # {"corpus":["public-emergency"]}
+    rate_limit_per_minute: int                        # cross-community budget
+    data_visibility:     str                          # "public_corpora_only"|"members_only"|"open"
+
+@dataclass(frozen=True)
+class FederationManifest:
+    schema_version:  int
+    federation_id:   str
+    community_a:     str
+    community_b:     str
+    established_at:  str
+    expires_at:      str
+    scope_a_to_b:    FederationScope                  # what A grants B
+    scope_b_to_a:    FederationScope                  # what B grants A
+    bootstrap_endpoints_a: list[Endpoint]
+    bootstrap_endpoints_b: list[Endpoint]
+    signature_a:     dict                              # {signed_by, signature, co_signers}
+    signature_b:     dict
+
+    def grants_to(self, calling_community_id: str) -> FederationScope | None:
+        """Returns the scope grant *to* the calling community (if federated, else None)."""
+
+    def is_expired(self, now: datetime | None = None) -> bool: ...
+
+def build_federation_proposal(
+    our_community_manifest: CommunityManifest,
+    peer_community_manifest_url: str,
+    proposed_scope_to_grant: FederationScope,
+    proposed_scope_to_receive: FederationScope,
+    bootstrap_endpoints: list[Endpoint],
+) -> 'FederationProposal':
+    """Step 1: prepare a proposal. Not yet a manifest — just a draft for the other side."""
+
+@dataclass(frozen=True)
+class FederationProposal:
+    community_a:           str
+    community_b:           str
+    scope_a_to_b:          FederationScope
+    scope_b_to_a:          FederationScope
+    bootstrap_endpoints_a: list[Endpoint]
+    bootstrap_endpoints_b: list[Endpoint]
+    proposer_signature:    str
+
+def co_sign_federation(
+    proposal: FederationProposal,
+    signing_kp: KeyPair,
+    role: str,                 # "a" or "b"
+) -> dict:
+    """Returns {signed_by, signature, co_signers[]} payload."""
+
+def finalize_federation_manifest(
+    proposal: FederationProposal,
+    sig_a: dict,
+    sig_b: dict,
+) -> FederationManifest:
+    """Assemble fully-signed manifest after both sides have signed."""
+
+def parse_federation_manifest(blob: bytes | dict) -> FederationManifest: ...
+def verify_federation_manifest(
+    m: FederationManifest,
+    community_a_manifest: CommunityManifest,
+    community_b_manifest: CommunityManifest,
+) -> None:
+    """Verify both sides signed, anchors are valid in their communities,
+    co-signer counts meet policy, expiry is in the future."""
+```
+
+### 4.2 `peering.py`
+
+```python
+# hearthnet/federation/peering.py
+class FederationHandshake:
+    """Conducts the multi-step cross-signing handshake.
+       Stateful; one instance per active proposal."""
+
+    def __init__(
+        self,
+        our_community_manifest: CommunityManifest,
+        our_kp: KeyPair,
+        transport_client: HttpClient,
+        event_log: EventLog,
+    ):
+        ...
+
+    async def initiate(
+        self,
+        peer_endpoints: list[Endpoint],
+        scope_to_grant: FederationScope,
+        scope_to_receive: FederationScope,
+    ) -> FederationProposal:
+        """1. Fetch peer's community manifest.
+        2. Build proposal.
+        3. Sign as community A's anchor.
+        4. POST to peer.
+        5. Receive peer's signed proposal back.
+        6. Verify both signatures and gather more local co-signers if policy requires.
+        Returns the fully-signed proposal ready to finalize."""
+
+    async def accept(self, proposal: FederationProposal) -> FederationManifest:
+        """The other side accepting an incoming proposal.
+        Returns the finalized manifest (publishable to event log)."""
+
+    async def publish(self, manifest: FederationManifest) -> None:
+        """Append federation.peer.added event to local log.
+        Push the manifest to peer so they can do the same."""
+```
+
+### 4.3 `relay_client.py`
+
+```python
+# hearthnet/federation/relay_client.py
+class FederationCaller:
+    """Outbound side: makes calls into federated communities.
+       Used by services when their request triggers a federated lookup
+       (e.g. rag.query across federated corpora)."""
+
+    def __init__(
+        self,
+        bus: CapabilityBus,
+        our_kp: KeyPair,
+        our_community_id: str,
+        federation_manifests_provider: Callable[[], list[FederationManifest]],
+    ):
+        ...
+
+    async def call_in_peer(
+        self,
+        peer_community_id: str,
+        capability: str,
+        version: Version,
+        body: dict,
+        *,
+        timeout_seconds: float | None = None,
+    ) -> dict:
+        """1. Look up federation manifest for peer_community_id.
+        2. Verify scope includes (capability, params).
+        3. Issue an auth.token via local M16 with capability scope.
+        4. Pick a peer Bridge endpoint (from manifest.bootstrap_endpoints_b).
+        5. POST /bus/v1/call to peer's federation.proxy@1.0 with token + body.
+        Returns the result. Raises FederationError on scope/auth issues."""
+
+    async def stream_in_peer(...) -> AsyncIterator[Frame]:
+        """Streaming variant."""
+```
+
+### 4.4 `service.py`
+
+```python
+# hearthnet/federation/service.py
+class FederationService:
+    name    = "federation"
+    version = "1.0"
+
+    def __init__(
+        self,
+        bus: CapabilityBus,
+        event_log: EventLog,
+        replay_engine: ReplayEngine,
+        author_kp: KeyPair,
+        community_manifest_provider: Callable[[], CommunityManifest],
+        revocation_cache: RevocationCache,
+    ):
+        ...
+
+    def capabilities(self) -> list[tuple[CapabilityDescriptor, Callable, ParamsPredicate]]:
+        """Registers: federation.peer.add, federation.peer.remove,
+        federation.peer.list, federation.proxy (all @1.0)."""
+
+    async def start(self) -> None: ...
+    async def stop(self) -> None: ...
+    def health(self) -> dict: ...
+
+    # --- handlers ---
+
+    async def handle_peer_add(self, req: RouteRequest) -> dict:
+        """CAP2 §4.1. Run handshake; check co-signatures; emit event."""
+
+    async def handle_peer_remove(self, req: RouteRequest) -> dict:
+        """CAP2 §4.2."""
+
+    async def handle_peer_list(self, req: RouteRequest) -> dict:
+        """CAP2 §4.3."""
+
+    async def handle_proxy(self, req: RouteRequest) -> dict | AsyncIterator[dict]:
+        """CAP2 §4.4. Forward a federated call to the local bus.
+        1. Verify token attached to request.
+        2. Look up federation manifest, check scope.
+        3. Call bus.call(target_capability, version, body) internally.
+        4. Return result (or stream frames)."""
+
+    # --- maintenance ---
+
+    async def heartbeat_loop(self) -> None:
+        """Per FEDERATION_HEARTBEAT_SECONDS, ping each federated peer's
+        federation.peer.list. Update last_heartbeat. Emit
+        federation.heartbeat event."""
+```
+
+---
+
+## 5. Behaviour
+
+### 5.1 Two-phase handshake
+
+```
+Community A's anchor decides to federate with B.
+  ↓
+A: peering.initiate(B_endpoints, scope_a_to_b, scope_b_to_a)
+   → fetch B's community manifest
+   → build proposal; sign as A
+   → POST /federation/proposal to B's bridge
+   ↓
+B receives proposal, presents to a trusted member.
+  human decision (or auto-policy)
+   ↓
+B: peering.accept(proposal)
+   → sign as B
+   → return signed proposal
+   ↓
+A: gather more local co-signers if our policy.min_signatures_to_federate > 1
+   ↓
+A: finalize_federation_manifest(proposal, sig_a, sig_b)
+   ↓
+A: publish federation.peer.added event locally
+   ↓
+A: POST manifest to B so they publish too
+   ↓
+both communities heartbeat each other periodically
+```
+
+### 5.2 Bridge node profile
+
+A community designates one or more nodes with `profile: "bridge"`. Bridge nodes:
+
+- Always-on (best-effort)
+- Have a publicly-reachable endpoint or a relay-tier (M15) registration
+- Run `FederationService` and act as the proxy for inbound federated calls
+- Hold the bandwidth budget for cross-community traffic
+
+Non-bridge nodes can still **call into** federated communities (via M14 `FederationCaller`); they just don't *serve* cross-community calls.
+
+### 5.3 Scope enforcement (inbound)
+
+When `federation.proxy` is invoked:
+
+1. Caller signature verified (Phase 1 §1.3)
+2. Caller's community is parsed from token's `iss`
+3. Federation manifest lookup; absent → `not_federated`
+4. Scope check: `(capability, version)` ∈ scope and params allowed → else `federation_forbidden`
+5. Token's signature verified against issuer's community anchors
+6. Token's `aud` must match our community
+7. Token's `scope` ⊆ federation manifest's scope (caller's community can't grant themselves more than they were granted)
+8. Dispatch internally via bus
+9. Record metrics: `hearthnet_federation_calls_total{peer_community, capability, result}`
+
+### 5.4 Heartbeats and expiry
+
+- Every `FEDERATION_HEARTBEAT_SECONDS` (300), each bridge calls `federation.peer.list` on each federated peer
+- If a heartbeat fails 3 times in a row, the peer is marked `degraded` in the local view
+- Federation manifests have `expires_at`. 30 days before expiry, a renewal handshake is auto-initiated. If renewal fails by expiry, the federation lapses; calls return `not_federated`.
+
+### 5.5 Revocation
+
+To break a federation:
+
+- Either side may call `federation.peer.remove`
+- Co-signature requirements: same as creation (`policy.min_signatures_to_federate`)
+- Event `federation.peer.removed` is published locally; peer is notified and publishes their own
+- All outstanding tokens issued under this federation are implicitly revoked (M16 verifies federation still exists)
+
+### 5.6 Identity import (Phase 2.5 hook)
+
+A federated user with NodeID in community A wishing to access community B's services *as themselves* (not via their community A anchor's token) can use `federation.identity.attest` (reserved capability, Phase 2.5). Out of scope for first cut.
+
+### 5.7 Trust transitivity
+
+**Not transitive.** A↔B and B↔C do not imply A↔C. Each pair establishes its own manifest. This is intentional — explicit consent.
+
+### 5.8 Conflict: federation with revoked member's community
+
+If community A has federated with B, and later A's anchor (the signer) is revoked from A:
+
+- The federation manifest's signature *was* valid at sign time
+- Going forward, A's community may renew with a new anchor signature
+- B verifies federation against A's current anchor set on every call — if no current anchor co-signs, the federation is invalid
+
+---
+
+## 6. Discovery integration
+
+A community wishing to find a federated peer they haven't talked to in a while:
+
+1. Look up `bootstrap_endpoints` in their stored federation manifest
+2. Try each; if all fail, fall back to [X05 DHT](../cross-cutting/X05-dht.md): `find_value(blake3(peer_community_id))`
+3. If DHT also returns nothing, try [M15 relay](M15-relay-tier.md): `relay.lookup_community(peer_community_id)`
+4. Only after all three fail → mark federation as `unreachable`; UI shows offline indicator
+
+---
+
+## 7. Errors
+
+`FederationError`:
+
+- `not_federated` — no manifest for this peer
+- `federation_expired` — manifest past expires_at
+- `scope_violation` — request outside granted scope
+- `bridge_unreachable` — couldn't reach any of peer's bridges
+- `co_signer_insufficient` — proposal lacks required signature count
+- `peer_community_invalid` — peer's manifest failed verification
+
+Wire mapping per [CAP2 §9](../CAPABILITY_CONTRACT_v2.md).
+
+---
+
+## 8. Configuration
+
+```python
+config.federation.enabled              = False           # opt-in
+config.federation.bridge_node          = False           # we serve cross-community calls
+config.federation.relay_url            = None            # M15 hosted relay for NAT
+config.federation.auto_renew_days_before = 30
+config.federation.max_peer_communities = 16
+config.federation.heartbeat_seconds    = FEDERATION_HEARTBEAT_SECONDS
+config.federation.scope_default_rate_limit_per_minute = 60
+```
+
+Constants: `FEDERATION_MANIFEST_TTL_SECONDS`, `FEDERATION_HEARTBEAT_SECONDS`.
+
+---
+
+## 9. Tests
+
+### Unit
+- `test_federation_proposal_builds_correctly`
+- `test_co_sign_signature_verifies`
+- `test_finalize_requires_min_signers`
+- `test_grants_to_returns_scope_for_correct_direction`
+- `test_expired_federation_rejects_calls`
+
+### Integration
+- `test_two_community_federation_round_trip` — A and B in different processes federate, then A queries B's RAG via proxy
+- `test_scope_violation_returns_403`
+- `test_heartbeat_marks_degraded_after_3_failures`
+- `test_revocation_breaks_existing_tokens`
+- `test_renewal_30_days_before_expiry`
+
+### Chaos
+- `test_partition_during_federation_handshake_resumable`
+
+---
+
+## 10. Cross-references
+
+| What | Where |
+|------|-------|
+| Federation manifest schema | [CAP2 §6.1](../CAPABILITY_CONTRACT_v2.md) |
+| `federation.*` capabilities | [CAP2 §4.1–4.4](../CAPABILITY_CONTRACT_v2.md) |
+| Token issuance for cross-community | [M16 §5.1](M16-tokens.md) |
+| DHT bootstrap | [X05 §4.3](../cross-cutting/X05-dht.md) |
+| Relay tier NAT traversal | [M15](M15-relay-tier.md) |
+| Bridge node profile | [Phase 1 PRD §5.4 + this module §5.2](../../HEARTHNET_PRD_v2.md) |
+| Phase 3 transitive federation | TBD |
+
+---
+
+## 11. Open questions
+
+1. **Multi-party federation (mesh of N>2 communities)** — currently bilateral only. Phase 3 candidate.
+2. **Federated marketplace** — should `market.list` cross federations? Reserved scope param; default off.
+3. **Federated identity** — single-sign-on across federated communities. Phase 2.5; design depends on token-on-token.
+4. **Federation revocation event propagation** — if A↔B and A↔C, and B unilaterally revokes A, should C see this? MVP: no, each pair is independent.
+5. **Audit log for federation activity** — should there be a separate "federation_audit" log so cross-community activity is easy to surface to operators? Yes, Phase 2.5.

docs/p2_p3/M15-relay-tier.md

@@ -0,0 +1,389 @@
+# M15 — Relay Tier
+
+**Spec version:** v1.0 (Phase 2)
+**Depends on:** M01 (identity), M16 (tokens, for relay registration auth), X01 (transport), X05 (DHT), X04 (config)
+**Depended on by:** M14 (federation, when bridges are NAT'd), M22 (mobile push delivery), X05 (DHT bootstrap)
+
+---
+
+## 1. Responsibility
+
+A **relay node** is a public-internet-reachable service that helps HearthNet nodes that cannot directly reach each other (NAT, mobile networks, dynamic IPs). It provides:
+
+- **NAT traversal**: registered nodes receive forwarded traffic from peers
+- **Federation discovery**: a public lookup of "which IP currently runs community X's bridge"
+- **Mobile push**: delivers chat/marketplace notifications to mobile devices via APNs/FCM
+- **DHT bootstrap**: serves as a stable initial DHT endpoint
+
+A relay is **infrastructure** — typically one or a few well-known servers per region. Christof's planned `relay.hearthnet.de` (Hetzner) is the reference deployment.
+
+Relays do **not**:
+- Store any community state long-term
+- See cleartext of E2E-encrypted chat
+- Make any trust decisions — they are credential-free transport for already-authenticated traffic
+- Replace community anchors
+
+---
+
+## 2. File layout
+
+```
+hearthnet/relay/                # client-side helpers, ships with normal Hearthnet
+├── __init__.py
+├── client.py                   # RelayClient — registration, forwarding fetch
+├── push_subscriber.py          # iOS APNs / Android FCM token registration
+
+relay-server/                   # separate deployable, lives in /relay-server in the repo
+├── pyproject.toml
+├── README.md
+├── relay_server/
+│   ├── __init__.py
+│   ├── app.py                  # FastAPI app
+│   ├── registration.py         # /relay/v1/register, /heartbeat
+│   ├── forward.py              # /relay/v1/forward
+│   ├── lookup.py               # /relay/v1/community/<id>
+│   ├── push.py                 # outbound APNs/FCM
+│   ├── billing.py              # optional, for paid tier
+│   ├── storage.py              # SQLite for registrations + push device tokens
+│   └── observability.py        # logging, metrics
+├── deploy/
+│   ├── docker-compose.yml
+│   ├── caddy.Caddyfile         # TLS termination
+│   └── systemd/relay.service
+└── tests/
+```
+
+The relay server is a thin FastAPI app deployable to any VPS. ~1500 LOC.
+
+---
+
+## 3. Wire surface (relay server endpoints)
+
+| Endpoint | Method | Purpose |
+|----------|--------|---------|
+| `/relay/v1/register` | POST | A node registers itself for forwarding |
+| `/relay/v1/heartbeat` | POST | Keep registration alive |
+| `/relay/v1/deregister` | POST | Cleanly remove a registration |
+| `/relay/v1/forward/<node_short>` | POST | A peer wanting to reach a registered node sends the encapsulated request here |
+| `/relay/v1/community/<community_id>` | GET | Look up current bridge endpoints for a community |
+| `/relay/v1/push/register` | POST | Register an APNs/FCM device token for push delivery |
+| `/relay/v1/push/<device_id>` | POST | Send a push notification (typically from another node) |
+| `/relay/v1/dht/bootstrap` | GET | Return a list of known-good DHT contacts |
+| `/relay/v1/health` | GET | Operator endpoint |
+
+Auth: every endpoint except `/health` requires a HearthNet signature OR a relay-issued token (registered nodes get a session token for performance).
+
+---
+
+## 4. Public API (client-side)
+
+### 4.1 `client.py`
+
+```python
+# hearthnet/relay/client.py
+@dataclass(frozen=True)
+class RelayRegistration:
+    relay_url:        str
+    node_id_full:     str
+    expires_at:       int          # unix seconds
+    session_token:    str          # short-lived bearer for subsequent calls
+
+class RelayClient:
+    """Used by federation bridges and mobile clients to be reachable through a relay."""
+
+    def __init__(
+        self,
+        relay_url: str,
+        kp: KeyPair,
+        community_id: str,
+    ):
+        ...
+
+    async def register(
+        self,
+        *,
+        capabilities_offered: list[str] | None = None,
+        external_endpoint_hint: Endpoint | None = None,
+    ) -> RelayRegistration:
+        """Register us with this relay. Relay will forward inbound /relay/v1/forward/<our_short>
+        calls to our actual endpoint via reverse-WebSocket (we hold a persistent WS to the relay).
+        Returns registration; client should hold an open WS until deregister."""
+
+    async def heartbeat(self) -> None:
+        """Refresh registration. Should be called every RELAY_REGISTRATION_TTL_SECONDS / 2."""
+
+    async def deregister(self) -> None: ...
+
+    async def maintain(self) -> None:
+        """Long-running task: keeps registration alive, reconnects on failure."""
+
+    # --- lookups (no registration required) ---
+
+    async def lookup_community(self, community_id: str) -> list[Endpoint]:
+        """Find current bridge endpoints for a community."""
+
+    async def dht_bootstrap_endpoints(self) -> list[Endpoint]: ...
+
+    async def send_push(
+        self,
+        device_id: str,
+        payload: dict,
+        *,
+        push_token: str,         # token from M16 with relay.push scope
+    ) -> None:
+        """Send a push notification via this relay."""
+```
+
+### 4.2 `push_subscriber.py`
+
+```python
+# hearthnet/relay/push_subscriber.py
+class PushSubscriber:
+    """Mobile-side: registers an APNs / FCM device token with the relay
+       so the relay can deliver push notifications for chat / marketplace events."""
+
+    def __init__(
+        self,
+        relay_url: str,
+        kp: KeyPair,
+        community_id: str,
+        platform: str,            # "ios" | "android" | "web"
+    ):
+        ...
+
+    async def register(self, device_token: str) -> str:
+        """Returns our PushDeviceID. Stored locally for later push send authorization."""
+
+    async def unregister(self) -> None: ...
+```
+
+### 4.3 Reverse-WebSocket pattern for forwarding
+
+NAT'd nodes can't accept inbound connections. So:
+
+1. Node POSTs `/relay/v1/register`
+2. Server returns 101 Switching Protocols, upgrading to WebSocket
+3. Node holds the WS open; relay sends forwarded calls down it
+4. Node processes and responds back through the same WS
+
+The relay server's `forward.py` proxies between the inbound HTTP caller and the registered node's WS. The inbound caller sees a normal HTTP/SSE response.
+
+---
+
+## 5. Server-side internals (sketch)
+
+### 5.1 Registration table
+
+```sql
+CREATE TABLE registrations (
+  node_id_full TEXT PRIMARY KEY,
+  community_id TEXT NOT NULL,
+  external_ip TEXT,
+  ws_session_id TEXT,            -- in-memory WS connection id
+  capabilities_offered TEXT,     -- JSON
+  registered_at INTEGER,
+  expires_at INTEGER,
+  last_heartbeat INTEGER
+);
+CREATE INDEX idx_reg_community ON registrations(community_id);
+```
+
+### 5.2 Push table
+
+```sql
+CREATE TABLE push_devices (
+  device_id TEXT PRIMARY KEY,    -- ULID, assigned by relay
+  node_id_full TEXT NOT NULL,
+  community_id TEXT NOT NULL,
+  platform TEXT NOT NULL,
+  device_token TEXT NOT NULL,    -- APNs / FCM token (kept secret on relay)
+  registered_at INTEGER,
+  last_active INTEGER
+);
+CREATE INDEX idx_push_node ON push_devices(node_id_full);
+```
+
+### 5.3 Forwarding flow
+
+```
+peer P wants to call NAT'd node N (only knows N's NodeID and relay URL)
+  ↓
+P → POST https://relay.hearthnet.de/relay/v1/forward/<N_short>
+   headers: standard X-HearthNet-* (signed by P)
+   body: original capability call
+  ↓
+relay looks up N's WS session
+  if absent → 503 relay_unreachable
+  if present → wraps request as a WS message and sends to N's WS
+  ↓
+N processes → sends response frames back through WS
+  ↓
+relay streams those frames as HTTP/SSE response to P
+  ↓
+relay never inspects body (E2E content); relay does check signatures
+   are valid (against peer manifests it caches) to prevent abuse
+```
+
+### 5.4 Push flow
+
+```
+sender wants to push to mobile user U
+  ↓
+sender → bus.call("auth.token.issue", scope={"capabilities":["relay.push"],"audience":"<device_id>"})
+  ↓
+sender → POST relay/v1/push/<device_id> with token + payload
+  ↓
+relay verifies token; resolves device_id → APNs/FCM token
+  ↓
+relay sends via APNs/FCM
+  ↓
+Apple/Google delivers to the device
+  ↓
+mobile app opens, calls bus to fetch new chat / event
+```
+
+The payload itself is opaque to the relay (`{"event_type":"chat.message.sent","sender_short":"7H4G-..."}`). The mobile app fetches the actual content via the bus when it opens.
+
+### 5.5 Federation lookup flow
+
+A community publishes its bridge endpoints to the relay via `/relay/v1/community/<id>` (POST, signed by an anchor). The relay caches `{community_id → [endpoints, last_updated]}` for 24 hours. GETs are free, signed (anti-spam) but lightweight.
+
+---
+
+## 6. Behaviour
+
+### 6.1 Trust model
+
+The relay is **untrusted-but-honest**. It:
+
+- Sees who is talking to whom (NodeID-level)
+- Sees signature envelopes (but not E2E ciphertext)
+- Can deny service (DoS), refuse to forward, or rate-limit
+- Can NOT impersonate anyone (no private keys)
+- Can NOT decrypt E2E content (no DH secrets)
+- Can NOT modify forwarded bytes without breaking signatures
+
+Operators of relays are accountable through public reputation: the relay's URL is in plain sight in community configs. A misbehaving relay gets blackballed by communities.
+
+### 6.2 Rate limiting
+
+| Endpoint | Limit |
+|----------|-------|
+| `/register` | 10 per hour per node |
+| `/forward` | 10 RPS per (peer, target_node) |
+| `/community/<id>` GET | 100 RPS total |
+| `/push/<device>` | 60 per hour per (sender, device) |
+
+Exceeded → 429 + `retry_after_ms`.
+
+### 6.3 Tier policy (Christof's hosted instance)
+
+| Tier | Communities | Push notifications | Cost |
+|------|-------------|--------------------|------|
+| Free | ≤ 5 nodes per community | 100/day | €0 |
+| Hearth | ≤ 50 nodes | 5000/day | €5/month |
+| Anchor | unlimited | 50000/day | ��25/month |
+| Self-hosted | unlimited | unlimited | infrastructure |
+
+The relay is open-source; any community can run their own. Hosted tier is a convenience layer.
+
+### 6.4 Privacy guarantees
+
+- Per-call signature verification (the relay checks them, but signatures contain only public NodeID — not user identity in a deeper sense)
+- Sender hides destination by sending to `forward/<short>`; the relay sees both
+- For traffic-pattern privacy (who talks to whom), no protection — outside scope
+- Logs retain registration + forwarding metadata for 30 days for abuse handling, then purged
+
+### 6.5 Failure modes
+
+- Relay down → mobile push delivery delayed; federated lookups fall back to DHT or stored endpoints; direct LAN calls unaffected
+- Relay overloaded → 429s; clients exponential-backoff
+- Relay key rotation → relay publishes new pubkey signed by previous key; clients update via standard manifest refresh
+
+---
+
+## 7. Configuration (client side)
+
+```python
+config.relay.enabled              = False
+config.relay.urls                 = ["https://relay.hearthnet.de"]
+config.relay.tier                 = "free"           # informational
+config.relay.register_as_bridge   = False            # if True, holds persistent WS to relay
+config.relay.push_enabled         = False
+config.relay.push_platform        = "web"
+```
+
+Constants: `RELAY_REGISTRATION_TTL_SECONDS=7200`, `RELAY_PUSH_RETRY_MAX=5`.
+
+### Relay server config (`relay-server/relay_server/config.py`)
+
+```python
+config.bind                       = "0.0.0.0:443"
+config.tls_cert_file              = "/etc/relay/cert.pem"
+config.tls_key_file               = "/etc/relay/key.pem"
+config.database                   = "/var/lib/relay/relay.db"
+config.apns_cert                  = "/etc/relay/apns.pem"
+config.fcm_key_file               = "/etc/relay/fcm.json"
+config.tier                       = "free|hearth|anchor"
+config.stripe_secret              = None             # for paid tiers
+config.admin_token                = "<random>"       # for operator endpoints
+```
+
+---
+
+## 8. Errors
+
+`RelayError` (client domain):
+
+- `relay_unreachable` — TCP fails or 5xx
+- `registration_expired` — call requires re-register
+- `forward_target_offline` — target node not currently registered with this relay
+- `push_token_invalid` — APNs/FCM rejected the device token
+- `tier_limit_exceeded` — quota for this tier reached
+
+Wire mapping: `relay_unreachable` is its own code in [CAP2 §9](../CAPABILITY_CONTRACT_v2.md).
+
+---
+
+## 9. Tests
+
+### Client-side unit
+- `test_register_includes_signature`
+- `test_heartbeat_refreshes_expires_at`
+- `test_lookup_returns_endpoints`
+
+### Server-side unit
+- `test_forward_requires_target_registered`
+- `test_signature_required_on_register`
+- `test_rate_limit_per_peer_target`
+- `test_push_dispatch_apns_mock`
+
+### Integration
+- `test_two_nat_peers_communicate_through_relay`
+- `test_federation_bridge_via_relay`
+- `test_push_delivered_to_real_test_device` (manual, with APNs sandbox)
+
+### Operational
+- Smoke tests on the deployed `relay.hearthnet.de` instance run hourly
+
+---
+
+## 10. Cross-references
+
+| What | Where |
+|------|-------|
+| Token use for push auth | [M16 §5.5](M16-tokens.md) |
+| Federation routes through relay | [M14 §6](M14-federation.md) |
+| DHT bootstrap endpoint | [X05 §4.4](../cross-cutting/X05-dht.md) |
+| Mobile push subscriber | [M22 §6](M22-mobile-native.md) |
+| Wire `relay_unreachable` | [CAP2 §9](../CAPABILITY_CONTRACT_v2.md) |
+
+---
+
+## 11. Open questions
+
+1. **TURN-style relay vs message relay** — current spec is message-level (peer sends entire capability call). Could also do session-level TCP relay (more efficient for streams). Phase 2.5 candidate.
+2. **STUN integration** — clients could try direct connection via STUN before falling back to relay. Phase 3.
+3. **Multi-relay redundancy** — a node could register with two relays for HA. MVP picks one; multi is Phase 2.5.
+4. **Payment integration** — Stripe webhooks → tier upgrade. Implementation detail, not specced here.
+5. **Self-hosting documentation quality** — for the "appliance" go-to-market path, the relay needs a one-command install. Defer to `RELAY_OPERATIONS.md` doc.

docs/p2_p3/M16-tokens.md

@@ -0,0 +1,391 @@
+# M16 — Capability Tokens
+
+**Spec version:** v1.0 (Phase 2)
+**Depends on:** M01 (identity), X02 (events, for `auth.token.*`), X04 (config), X03 (observability)
+**Depended on by:** M14 (federation), M15 (relay), M22 (mobile), M23 (optionally, for session credentials)
+
+---
+
+## 1. Responsibility
+
+Issue, verify, and revoke short-lived **capability tokens** for delegation. A token says: "the holder of this token may invoke capability X (with these constraints) on behalf of issuer Y, until time Z."
+
+Tokens are the mechanism Phase 2 uses for:
+- Federation calls (a federated peer presents a token issued by an anchor of the peer community)
+- Mobile clients (the mobile app presents a token issued during onboarding)
+- Limited-scope sharing (e.g. "let this neighbour query our emergency corpus for the next hour")
+
+Per-request Ed25519 signatures (Phase 1 §1.3) remain the default authentication; tokens are an *additional* mechanism.
+
+---
+
+## 2. File layout
+
+```
+hearthnet/identity/
+└── tokens.py         # CapabilityToken, encode/decode, verify, revocation cache
+
+hearthnet/services/auth/
+├── __init__.py
+└── service.py        # AuthService — registers auth.token.* capabilities
+```
+
+The token primitives live under `identity/` (low-level crypto). The capability handlers live as a normal service so they go through the bus.
+
+---
+
+## 3. Token envelope
+
+Compact JWS-style. Compatible with off-the-shelf JWS decoders that accept `EdDSA`. Length budget: ≤ 800 bytes (fits a QR at error correction M).
+
+### 3.1 Header
+
+```json
+{"alg": "EdDSA", "typ": "hntoken", "v": 1}
+```
+
+### 3.2 Payload
+
+```json
+{
+  "iss":  "ed25519:<issuer NodeID full form>",
+  "sub":  "ed25519:<subject NodeID full form>",
+  "aud":  "ed25519:<audience community_id, optional>",
+  "iat":  1717939200,
+  "exp":  1717942800,
+  "nbf":  1717939200,
+  "jti":  "01HXR...",
+  "scope": {
+    "capabilities": ["rag.query@1.0", "embed.text@1.0"],
+    "params_constraints": {
+      "corpus": ["niederrhein-emergency"],
+      "model":  ["bge-small-en-v1.5"]
+    },
+    "rate_limit_per_minute": 60,
+    "max_calls_total":       null
+  },
+  "issued_via": "federation|onboarding|manual|relay"
+}
+```
+
+`sub` MAY be `"*"` for a bearer-style token (anyone with the token may use it). Used sparingly — only for federation proxies where the actual subject is unknown at issuance time.
+
+### 3.3 Signature
+
+`Ed25519(base64url(header) + "." + base64url(payload))`. Final form:
+
+```
+hntoken://v1/<base64url(header)>.<base64url(payload)>.<base64url(signature)>
+```
+
+Total length: ~600–800 bytes typical.
+
+---
+
+## 4. Public API
+
+### 4.1 `hearthnet/identity/tokens.py`
+
+```python
+# hearthnet/identity/tokens.py
+from dataclasses import dataclass
+
+@dataclass(frozen=True)
+class TokenScope:
+    capabilities:           list[str]               # e.g. ["rag.query@1.0"]
+    params_constraints:     dict[str, list[str]]   # e.g. {"corpus": ["..."]}
+    rate_limit_per_minute:  int
+    max_calls_total:        int | None
+
+@dataclass(frozen=True)
+class CapabilityToken:
+    """The fully decoded token, ready for verification."""
+    issuer:       str
+    subject:      str                                # "*" for bearer
+    audience:     str | None
+    issued_at:    int                                 # unix seconds
+    expires_at:   int
+    not_before:   int
+    jti:          str                                 # ULID
+    scope:        TokenScope
+    issued_via:   str                                 # "federation"|"onboarding"|...
+    signature:    bytes                               # raw 64 bytes
+
+    @property
+    def is_bearer(self) -> bool: ...
+
+    def is_active(self, now: int | None = None) -> bool: ...
+
+    def covers(self, capability_name: str, version: tuple[int, int],
+               params: dict | None = None) -> bool:
+        """True iff scope includes the capability and (if params_constraints set) every requested param value is in the allow-list."""
+
+def issue_token(
+    issuer_kp: KeyPair,
+    subject: str,
+    scope: TokenScope,
+    *,
+    ttl_seconds: int = TOKEN_DEFAULT_TTL_SECONDS,
+    audience: str | None = None,
+    issued_via: str = "manual",
+    not_before_offset: int = 0,
+) -> tuple[CapabilityToken, str]:
+    """Build, sign, encode. Returns (token, encoded_str)."""
+
+def encode_token(tok: CapabilityToken, header_signature: bytes) -> str:
+    """Render to 'hntoken://v1/...'."""
+
+def decode_token(text: str) -> CapabilityToken:
+    """Parse + structural validation only. Does NOT verify the signature.
+       Raises TokenError on malformed input."""
+
+def verify_token(
+    tok: CapabilityToken,
+    *,
+    expected_audience: str | None = None,
+    revocation_cache: 'RevocationCache | None' = None,
+    now: int | None = None,
+    community_manifest: CommunityManifest,
+) -> None:
+    """Verify signature against issuer's pubkey, expiry, nbf, audience,
+       revocation, and that the issuer is currently a community member
+       (not revoked at the issuer's community level).
+       Raises TokenError with specific code."""
+
+class RevocationCache:
+    """In-memory + persisted (SQLite) cache of revoked JTIs.
+       Authoritative source is the event log."""
+
+    def __init__(self, db_path: Path):
+        ...
+
+    def add(self, jti: str, revoked_at: int) -> None: ...
+    def is_revoked(self, jti: str) -> bool: ...
+    def hydrate_from_log(self, event_log: EventLog) -> int:
+        """Read all auth.token.revoked events; bring cache up to date.
+        Returns rows added."""
+
+class TokenError(Exception):
+    """code in {
+       'token_invalid','token_expired','token_not_yet_valid',
+       'token_signature_bad','token_audience_mismatch',
+       'token_revoked','token_scope_insufficient',
+       'token_issuer_revoked','token_malformed'}"""
+    code: str
+```
+
+### 4.2 `hearthnet/services/auth/service.py`
+
+```python
+# hearthnet/services/auth/service.py
+class AuthService:
+    """Registers auth.token.issue / revoke / introspect capabilities."""
+
+    name    = "auth"
+    version = "1.0"
+
+    def __init__(
+        self,
+        author_kp: KeyPair,
+        event_log: EventLog,
+        community_manifest_provider: Callable[[], CommunityManifest],
+        revocation_cache: RevocationCache,
+    ):
+        ...
+
+    def capabilities(self) -> list[tuple[CapabilityDescriptor, Callable, ParamsPredicate]]:
+        """Registers: auth.token.issue@1.0, auth.token.revoke@1.0, auth.token.introspect@1.0."""
+
+    async def start(self) -> None:
+        """Hydrate the revocation cache from event log."""
+
+    async def stop(self) -> None: ...
+    def health(self) -> dict: ...
+
+    # --- handlers ---
+
+    async def handle_issue(self, req: RouteRequest) -> dict:
+        """CAP2 §4.5. Build a CapabilityToken, sign with author_kp, emit auth.token.issued event."""
+
+    async def handle_revoke(self, req: RouteRequest) -> dict:
+        """CAP2 §4.6. Verify caller is issuer (or 'trusted'). Append auth.token.revoked event."""
+
+    async def handle_introspect(self, req: RouteRequest) -> dict:
+        """CAP2 §4.7. Self-only. Returns active status and scope."""
+```
+
+---
+
+## 5. Behaviour
+
+### 5.1 Token-bearer call lifecycle
+
+```
+caller hits any capability endpoint with:
+  X-HearthNet-Token: hntoken://v1/...
+  (and optionally X-HearthNet-Signature)
+  ↓
+X01 transport extracts and decodes
+  ↓
+verify_token(...) — signature, expiry, audience, revocation
+  ↓
+on success:
+  caller_effective_identity = token.subject (or token.issuer if subject == "*")
+  scope_check (does token cover this capability?)
+  ↓
+bus.handle_call() with the effective caller
+  ↓
+record token usage in metrics: hearthnet_token_calls_total{issuer, scope_match}
+```
+
+### 5.2 Co-existence with per-request signing
+
+A request MAY carry both `X-HearthNet-Signature` and `X-HearthNet-Token`:
+
+- Signature: proves *who* is making this exact call right now
+- Token: proves they're *allowed* to (via delegation)
+
+The token's `sub` MUST equal the signature's `From` NodeID, unless `sub == "*"`. Mismatch → `invalid_signature`.
+
+This combination is the normal mode for federation: a federated peer's anchor signs with their key (signature) AND carries a token issued by their community's anchor delegating "rag.query is OK".
+
+### 5.3 Issuance authority
+
+A node may issue a token iff:
+
+- The capabilities in scope are ones the issuer's community offers (or grants via federation)
+- TTL ≤ `policy.capability_token_ttl_seconds` (community-wide policy bound)
+- The issuer is a `member` (level ≥ member) of the community
+
+The handler enforces these before signing.
+
+### 5.4 Revocation
+
+A token is revoked by appending `auth.token.revoked` to the event log:
+
+- Issuer may revoke their own tokens
+- A `trusted` member may revoke any token (operator override)
+- The community root can revoke any token
+
+Once the revoke event is in the log, all gossip-receiving nodes update their `RevocationCache`. Until that propagates, a revoked token may still be honoured briefly — design accepts up to 60 seconds of lag.
+
+### 5.5 Bearer tokens (`sub == "*"`)
+
+Used sparingly:
+
+- Federation proxy tokens: peer community gets one bearer token to make federated calls; rotation every 24h
+- Mobile push tokens (M22): one bearer token tied to a `PushDeviceID`, longer TTL
+
+Bearer tokens trade convenience for less revocability granularity. The `jti` is still unique so a specific bearer can be killed.
+
+### 5.6 Replay protection
+
+Tokens are not single-use. Replay is mitigated by:
+- Short TTL (default 1h)
+- Audience binding (`aud` field): server rejects if `aud` ≠ ours
+- Rate-limit budget (`scope.rate_limit_per_minute`)
+- Revocation if abuse detected
+
+For one-shot tokens (e.g. password-reset-style flows), set `max_calls_total: 1` and the server tracks usage via a per-jti counter.
+
+### 5.7 Token-on-token (delegation chains)
+
+Phase 2: **forbidden**. A token holder cannot issue new tokens. This avoids a delegation tree we cannot audit.
+
+Phase 3 may add bounded delegation with a `delegates: int` counter.
+
+---
+
+## 6. Storage
+
+### 6.1 Revocation cache table
+
+```sql
+CREATE TABLE IF NOT EXISTS token_revocations (
+  jti          TEXT PRIMARY KEY,
+  revoked_at   INTEGER NOT NULL,
+  reason       TEXT,
+  via_event_id TEXT
+);
+CREATE INDEX IF NOT EXISTS idx_revocations_time ON token_revocations(revoked_at);
+```
+
+### 6.2 Rate-limit counters
+
+Per-(jti, minute) sliding window in memory. Persisted only when capacity-exceeded events fire (for audit).
+
+---
+
+## 7. Errors
+
+`TokenError` → wire mapping:
+
+| TokenError code | Wire code | HTTP |
+|-----------------|-----------|------|
+| `token_malformed` | `bad_request` | 400 |
+| `token_invalid` | `token_invalid` | 401 |
+| `token_signature_bad` | `token_invalid` | 401 |
+| `token_expired` | `token_expired` | 410 |
+| `token_not_yet_valid` | `token_expired` | 410 |
+| `token_audience_mismatch` | `unauthorized` | 401 |
+| `token_revoked` | `token_revoked` | 401 |
+| `token_scope_insufficient` | `token_scope_insufficient` | 403 |
+| `token_issuer_revoked` | `revoked` | 403 |
+
+---
+
+## 8. Configuration
+
+From [X04](../../cross-cutting/X04-config.md) (extension):
+
+```python
+config.auth.enabled                = True
+config.auth.token_default_ttl_seconds = TOKEN_DEFAULT_TTL_SECONDS
+config.auth.token_max_ttl_seconds  = TOKEN_MAX_TTL_SECONDS
+config.auth.allow_bearer_tokens    = True
+config.auth.federated_only_bearer  = True  # bearer tokens only issued for federation context
+```
+
+---
+
+## 9. Tests
+
+### Unit
+- `test_token_encode_decode_roundtrip`
+- `test_token_under_800_bytes`
+- `test_token_signature_verified`
+- `test_token_expired_rejected`
+- `test_token_audience_mismatch_rejected`
+- `test_token_scope_covers_exact_match`
+- `test_token_scope_params_constraint_filtered`
+- `test_revocation_event_updates_cache`
+- `test_bearer_token_with_star_subject`
+
+### Integration
+- `test_federated_call_with_token_succeeds`
+- `test_revoked_token_rejected_within_60_seconds`
+- `test_rate_limit_per_token_enforced`
+- `test_mobile_client_token_authenticates`
+
+---
+
+## 10. Cross-references
+
+| What | Where |
+|------|-------|
+| Token wire format | [CAP2 §6.2](../CAPABILITY_CONTRACT_v2.md) |
+| Token-bearer requests | [CAP2 §5.2](../CAPABILITY_CONTRACT_v2.md) |
+| `auth.token.*` capabilities | [CAP2 §4.5–4.7](../CAPABILITY_CONTRACT_v2.md) |
+| Used by federation | [M14 §5](M14-federation.md) |
+| Used by relay tier | [M15 §4](M15-relay-tier.md) |
+| Used by mobile client | [M22 §4](M22-mobile-native.md) |
+| Phase 1 identity primitives | [M01](../../modules/M01-identity.md) |
+
+---
+
+## 11. Open questions
+
+1. **Audience as community vs node** — Phase 2 uses community as audience. Should single-node audience be supported (one-call-to-one-node tokens)? Probably yes; adds `aud_kind: "community"|"node"`. Defer.
+2. **JWE for confidential scope** — current scope is in cleartext. Some scope values are sensitive (corpus names). Wrap payload in JWE? Defer; out of scope MVP for tokens.
+3. **Hardware-bound tokens** — Phase 3 idea: token bound to a TPM-attested device.
+4. **Token-on-token (delegation)** — explicitly Phase 3.

docs/p2_p3/M17-ocr.md

@@ -0,0 +1,305 @@
+# M17 — OCR Service
+
+**Spec version:** v1.0 (Phase 2)
+**Depends on:** M03 (bus), M07 (blobs, for reading image/PDF inputs and storing extracted text), M11 (embedding, when integrating with M05 RAG), X04 (config), X03 (observability)
+**Depended on by:** M05 RAG (ingest of scanned PDFs), M20 vision (img.describe can fall back to OCR for text-heavy images)
+
+---
+
+## 1. Responsibility
+
+Provide `ocr.image@1.0` and `ocr.pdf@1.0`. Wrap several OCR backends so the bus can route between them by document type and language. Specifically engineered to handle:
+
+- Modern German printed text (Tesseract)
+- Handwriting (TrOCR / Microsoft Florence-OCR)
+- Historical scripts — Sütterlin, Kurrent, Latin, Arabic, Cyrillic (Christof's multilingual harness)
+- Mixed-language documents (auto-detection)
+
+---
+
+## 2. File layout
+
+```
+hearthnet/services/ocr/
+├── __init__.py
+├── service.py            # OcrService
+└── backends/
+    ├── __init__.py
+    ├── base.py           # OcrBackend Protocol
+    ├── tesseract.py      # Tesseract via pytesseract
+    ├── trocr.py          # Microsoft TrOCR via transformers
+    ├── multilingual.py   # Christof's self-improving harness (CHURRO, olmOCR-2)
+    └── florence_ocr.py   # Florence-2 OCR mode (overlap with M20)
+```
+
+---
+
+## 3. Public API
+
+### 3.1 `backends/base.py`
+
+```python
+# hearthnet/services/ocr/backends/base.py
+from dataclasses import dataclass
+
+@dataclass(frozen=True)
+class OcrBlock:
+    text:        str
+    bbox:        tuple[int, int, int, int]   # (x, y, w, h) in pixel coords
+    confidence:  float                       # 0..1
+    language:    str | None
+
+@dataclass(frozen=True)
+class OcrPageResult:
+    page:             int                    # 1-indexed
+    text:             str                    # concatenated, reading order
+    blocks:           list[OcrBlock]
+    languages:        list[str]              # detected, ordered by prevalence
+    confidence_mean:  float
+    ms:               int
+
+class OcrBackend(Protocol):
+    name:               str       # "tesseract" | "trocr" | "multilingual" | "florence_ocr"
+    languages_supported: list[str] # ISO 639-2 codes: "deu","eng","lat","ara","rus", ...
+    supports_handwriting: bool
+    max_image_pixels:   int
+
+    async def warm(self) -> None: ...
+    async def close(self) -> None: ...
+
+    async def ocr_image(
+        self,
+        image_bytes: bytes,
+        *,
+        languages: list[str] | None,        # None → auto-detect
+        preprocess: dict | None = None,     # {deskew, denoise, dpi}
+    ) -> OcrPageResult: ...
+
+    async def ocr_pdf_page(
+        self,
+        pdf_bytes: bytes,
+        *,
+        page: int,
+        languages: list[str] | None,
+        preprocess: dict | None = None,
+    ) -> OcrPageResult: ...
+
+    def health(self) -> dict: ...
+```
+
+### 3.2 Concrete backends
+
+| File | Class | Notes |
+|------|-------|-------|
+| `backends/tesseract.py` | `TesseractBackend(min_confidence: float = 0.5)` | Languages: any installed traineddata. Subprocess via pytesseract. |
+| `backends/trocr.py` | `TrocrBackend(model: str = "microsoft/trocr-large-handwritten", device: str = "auto")` | Handwriting; CUDA preferred. |
+| `backends/multilingual.py` | `MultilingualHarnessBackend(model: str = "self-improving-ocr-v1", device: str = "auto", harness_dir: Path)` | Christof's harness (CHURRO, olmOCR-2, retrieval-augmented correction, Kurrent/Sütterlin/Latin/Arabic/Cyrillic). Configured via `harness_dir`. |
+| `backends/florence_ocr.py` | `FlorenceOcrBackend(model: str = "microsoft/Florence-2-large")` | Reuses M20 vision backend in OCR mode. |
+
+`MultilingualHarnessBackend` is the headline integration for Christof's existing work. It exposes the same `OcrBackend` interface and lets the harness's internal page-level VLMs do the heavy lifting.
+
+### 3.3 `service.py`
+
+```python
+# hearthnet/services/ocr/service.py
+class OcrService:
+    name    = "ocr"
+    version = "1.0"
+
+    def __init__(self, config: OcrConfig, blob_store: BlobStore, event_log: EventLog):
+        self._backends: dict[str, OcrBackend] = self._build_backends(config)
+
+    def capabilities(self) -> list[tuple[CapabilityDescriptor, Callable, ParamsPredicate]]:
+        """One ocr.image entry per backend; one ocr.pdf entry per backend.
+        params include backend name and supported languages."""
+
+    async def start(self) -> None: ...
+    async def stop(self) -> None: ...
+    def health(self) -> dict: ...
+
+    # --- handlers ---
+
+    async def handle_image(self, req: RouteRequest) -> dict:
+        """CAP2 §4.8.
+        1. Resolve image_cid via blob_store
+        2. Pick backend from params.backend
+        3. Run; build response"""
+
+    async def handle_pdf(self, req: RouteRequest) -> AsyncIterator[dict]:
+        """CAP2 §4.9.
+        1. Resolve doc_cid
+        2. For each page in page_range:
+           emit 'progress' frame
+           emit 'page' frame
+        3. If store_text:true, write concatenated text as new blob; emit done with text_cid"""
+```
+
+### 3.4 `params_compatible` predicate
+
+```python
+def params_compatible(offered: dict, requested: dict) -> bool:
+    # backend must match if specified
+    if "backend" in requested and requested["backend"] != offered.get("backend"):
+        return False
+    # all requested languages must be supported by this backend
+    requested_langs = set(requested.get("languages", []))
+    offered_langs   = set(offered.get("languages_supported", []))
+    return requested_langs.issubset(offered_langs)
+```
+
+---
+
+## 4. Behaviour
+
+### 4.1 Auto-language detection
+
+If `languages` is omitted or set to `["auto"]`:
+
+1. Sample 3 random pages
+2. Run lightweight script detection (Tesseract `osd`)
+3. Choose top 2 scripts
+4. Re-run with that language set
+
+Backends that don't support `osd` fall back to a fixed default (configured per backend).
+
+### 4.2 Preprocessing pipeline
+
+`preprocess` dict supports:
+- `deskew: bool` — straighten image
+- `denoise: bool` — bilateral filter
+- `binarize: bool` — Otsu threshold
+- `dpi: int` — target resolution; upscale if lower
+- `contrast_normalise: bool`
+
+Default: `{"deskew": true, "denoise": false}`. Heavy preprocessing slows ingest meaningfully; only enable per document.
+
+### 4.3 Quality estimation
+
+Each page result reports `confidence_mean`. Below 0.6, the service emits a `low_quality` warning frame and recommends:
+- Trying a different backend (e.g. switch from Tesseract to multilingual harness for historic text)
+- Raising DPI
+- Re-scanning
+
+### 4.4 Integration with RAG
+
+[M05 §10 open question 4](../../modules/M05-rag.md) is now answered:
+
+```
+RagService.handle_ingest receives a scanned PDF (mime_type=image/scanned-pdf or detected)
+  → bus.call("ocr.pdf", (1,0), {input:{doc_cid:..., store_text:true}})
+  → receive text_cid
+  → ingest the text_cid blob (which is now extracted plaintext) as normal
+  → emit rag.document.ingested event (with metadata noting ocr_backend used)
+```
+
+The OCR text is stored as a separate blob, content-addressed. Re-ingestion is idempotent.
+
+### 4.5 Page-range and parallelism
+
+`page_range: [1, 50]` lets callers process partial documents. Pages are OCR'd serially within one call. For very large PDFs, callers should split into ranges and call concurrently — the bus enforces per-capability concurrency.
+
+`OCR_MAX_PAGES_PER_REQUEST = 50` is the hard ceiling per call.
+
+### 4.6 PDF text-layer detection
+
+Before OCR'ing, the service checks if the PDF has an extractable text layer (via `pypdf`). If yes and confidence is decent (heuristic), it returns the text-layer content directly — much cheaper than OCR. Caller can force OCR with `force_ocr: true`.
+
+### 4.7 Christof's multilingual harness integration
+
+The `MultilingualHarnessBackend` wraps Christof's existing self-improving OCR pipeline:
+
+- Internal models: CHURRO (page-level VLM), olmOCR-2 (page-level VLM)
+- Retrieval-augmented correction over a script-specific corpus
+- Kurrent + Sütterlin support for German historical documents
+- Latin / Arabic / Cyrillic script recognition
+
+Configuration:
+
+```python
+config.ocr.multilingual_harness_dir = Path("/srv/ocr-harness")
+config.ocr.multilingual_max_pages_concurrent = 2
+```
+
+The harness is GPU-intensive. On CPU-only nodes, it deregisters itself at startup.
+
+---
+
+## 5. Storage and lifecycle
+
+- Input image/PDF: fetched from blob store via CID
+- Output text: optionally stored as a new blob (`store_text: true`)
+- Side-effect: `ocr.document.indexed` event in the community log (carries text_cid for downstream replication)
+
+OCR backends do NOT cache results inside themselves. Reuse comes from caching at the RAG/blob layer (same `doc_cid` → already-extracted-text blob).
+
+---
+
+## 6. Errors
+
+| Condition | Wire code |
+|-----------|-----------|
+| Unknown backend | `not_found` |
+| Languages not supported by any backend | `bad_request` |
+| Image too large (> max_image_pixels) | `bad_request` |
+| Page-range exceeds document | `bad_request` |
+| > OCR_MAX_PAGES_PER_REQUEST | `bad_request` |
+| Backend crash | `internal_error` |
+| GPU OOM on multilingual | `capacity_exceeded` (with retry_after) |
+
+---
+
+## 7. Configuration
+
+```python
+config.ocr.enabled                 = True
+config.ocr.backends = [
+    OcrBackendConfig(name="tesseract", languages=["deu","eng","fra","lat"]),
+    OcrBackendConfig(name="trocr", model="microsoft/trocr-large-handwritten"),
+    OcrBackendConfig(name="multilingual", harness_dir=Path("/srv/ocr-harness")),
+]
+config.ocr.default_dpi             = OCR_DEFAULT_DPI    # 300
+config.ocr.max_pages_per_request   = OCR_MAX_PAGES_PER_REQUEST
+config.ocr.text_layer_first        = True
+```
+
+Constants: `OCR_DEFAULT_DPI`, `OCR_MAX_PAGES_PER_REQUEST`.
+
+---
+
+## 8. Tests
+
+### Unit
+- `test_descriptor_schema_validates_meta_schema`
+- `test_params_compatible_language_subset`
+- `test_text_layer_short_circuits_when_present`
+- `test_force_ocr_bypasses_text_layer`
+- `test_low_quality_emits_warning_frame`
+
+### Integration
+- `test_tesseract_german_print` (with a known sample)
+- `test_trocr_handwriting_sample`
+- `test_multilingual_kurrent_sample` (if harness installed)
+- `test_rag_ingest_scanned_pdf_end_to_end`
+- `test_ocr_pdf_progress_frames`
+
+---
+
+## 9. Cross-references
+
+| What | Where |
+|------|-------|
+| `ocr.*` wire | [CAP2 §4.8–4.9](../CAPABILITY_CONTRACT_v2.md) |
+| Blob store dependency | [M07 §3](../../modules/M07-file-blobs.md) |
+| RAG integration | [M05 §10 q4](../../modules/M05-rag.md) — now resolved |
+| Vision overlap (Florence-2 OCR mode) | [M20 §4.3](M20-vision.md) |
+| `ocr.document.indexed` event | [CAP2 §7.1](../CAPABILITY_CONTRACT_v2.md) |
+| Christof's harness | external project; this module is the integration surface |
+
+---
+
+## 10. Open questions
+
+1. **Multilingual harness auto-update.** The harness self-improves; should the model versions be event-logged so we can replay deterministically? Yes — record the harness version hash in each `ocr.document.indexed` event.
+2. **Manuscript-quality preprocessing.** Some historic documents need bespoke preprocessing (e.g. ink-bleed removal). Phase 2.5 might add a `preprocess_profile` enum.
+3. **Reading order from layout.** Currently we trust the backend's reading order. For multi-column documents, an explicit layout model (LayoutLMv3) might help. Phase 3.
+4. **Streaming OCR for very large images.** Currently atomic. Could tile and stream. Defer.

docs/p2_p3/M18-translation.md

@@ -0,0 +1,225 @@
+# M18 — Translation Service
+
+**Spec version:** v1.0 (Phase 2)
+**Depends on:** M03 (bus), X04 (config), X03 (observability), `transformers`, `torch`
+**Depended on by:** UI marketplace + chat (one-click translate), M19 STT (with `translate_to_en=true`)
+
+---
+
+## 1. Responsibility
+
+Provide `trans.text@1.0`. Translate between languages, with strong emphasis on:
+- German ↔ English (default)
+- German ↔ Plattdeutsch (Niederrhein-specific, Christof's domain)
+- Major European languages
+- Optionally Arabic, Turkish, Russian, Ukrainian — useful in refugee-context emergencies
+
+---
+
+## 2. File layout
+
+```
+hearthnet/services/translation/
+├── __init__.py
+├── service.py
+└── backends/
+    ├── __init__.py
+    ├── base.py
+    ├── nllb.py             # facebook/nllb-200-distilled-600M
+    └── plattdeutsch.py     # specialised fine-tune, optional
+```
+
+---
+
+## 3. Public API
+
+### 3.1 `backends/base.py`
+
+```python
+@dataclass(frozen=True)
+class TranslationResult:
+    text:       str
+    from_lang:  str            # ISO 639-1
+    to_lang:    str
+    confidence: float           # 0..1 if backend supports; else 1.0 placeholder
+    ms:         int
+
+class TranslationBackend(Protocol):
+    name:               str
+    languages_pairs:    list[tuple[str, str]]   # supported (from, to) pairs
+    max_chars:          int
+
+    async def warm(self) -> None: ...
+    async def close(self) -> None: ...
+
+    async def translate(
+        self,
+        text: str,
+        *,
+        from_lang: str,        # "auto" supported
+        to_lang: str,
+        domain: str | None,
+    ) -> TranslationResult: ...
+
+    def detect_language(self, text: str) -> str | None: ...
+
+    def health(self) -> dict: ...
+```
+
+### 3.2 Concrete backends
+
+```python
+class NllbBackend(TranslationBackend):
+    """facebook/nllb-200-distilled-600M (or larger variants).
+       200+ language pairs out of the box."""
+
+    def __init__(
+        self,
+        model: str = "facebook/nllb-200-distilled-600M",
+        device: str = "auto",
+        max_chars: int = TRANSLATION_MAX_CHARS,
+    ):
+        ...
+
+class PlattdeutschBackend(TranslationBackend):
+    """Optional specialised fine-tune.
+       If a Plattdeutsch fine-tune is present in models_dir, registers de↔nds pair.
+       Otherwise no-op (the backend reports zero language pairs and is filtered out)."""
+
+    def __init__(
+        self,
+        models_dir: Path,
+        device: str = "auto",
+    ):
+        ...
+```
+
+### 3.3 `service.py`
+
+```python
+class TranslationService:
+    name    = "translation"
+    version = "1.0"
+
+    def __init__(self, config: TranslationConfig):
+        self._backends: list[TranslationBackend] = self._build_backends(config)
+
+    def capabilities(self) -> list[tuple[CapabilityDescriptor, Callable, ParamsPredicate]]:
+        """One trans.text entry per backend. params declare languages_pairs."""
+
+    async def start(self) -> None: ...
+    async def stop(self) -> None: ...
+    def health(self) -> dict: ...
+
+    async def handle_translate(self, req: RouteRequest) -> dict:
+        """CAP2 §4.10."""
+```
+
+### 3.4 `params_compatible` predicate
+
+```python
+def params_compatible(offered: dict, requested: dict) -> bool:
+    if "backend" in requested and requested["backend"] != offered.get("backend"):
+        return False
+    pair = (requested.get("from"), requested.get("to"))
+    if pair[0] == "auto":
+        # auto-detect; backend must support at least one source → target pair
+        return any(t == pair[1] for (_, t) in offered.get("languages_pairs", []))
+    return pair in offered.get("languages_pairs", [])
+```
+
+---
+
+## 4. Behaviour
+
+### 4.1 Auto-detection
+
+`from: "auto"`:
+1. Call `detect_language(text)` (NLLB has internal language detection)
+2. Substitute detected lang
+3. Translate
+
+### 4.2 Domain hints
+
+`domain: "everyday" | "medical" | "legal" | "emergency"` is advisory. NLLB ignores it; specialised fine-tunes may use it.
+
+### 4.3 Niederrhein focus
+
+`PlattdeutschBackend` is Christof's local interest. When installed:
+
+- Registers pairs `("de", "nds")` and `("nds", "de")`
+- Optionally `("en", "nds")` if fine-tune extends
+- Used by the marketplace UI's "auf Platt" button in [M08 settings](../../modules/M08-ui.md) ext
+
+### 4.4 Length limits
+
+- Single request: ≤ `TRANSLATION_MAX_CHARS` (4000)
+- For longer texts, callers chunk by paragraph and recombine
+
+### 4.5 Batching
+
+Internal: requests within 100 ms batched up to 8 strings per forward pass. Improves GPU utilisation. Demultiplexed back. Transparent to callers.
+
+### 4.6 Caching
+
+In-memory LRU cache `(text_hash, from, to) → result`, max 10k entries. Big wins for marketplace UI which re-translates same posts on every refresh.
+
+---
+
+## 5. Errors
+
+| Condition | Wire code |
+|-----------|-----------|
+| Pair not supported by any backend | `not_found` |
+| Text too long | `bad_request` |
+| Detection failed | `bad_request` |
+| Backend OOM | `capacity_exceeded` |
+
+---
+
+## 6. Configuration
+
+```python
+config.translation.enabled       = True
+config.translation.backends = [
+    TranslationBackendConfig(name="nllb", model="facebook/nllb-200-distilled-600M", device="auto"),
+    TranslationBackendConfig(name="plattdeutsch", models_dir=Path("~/.hearthnet/models/plattdeutsch")),
+]
+```
+
+Constants: `TRANSLATION_MAX_CHARS`.
+
+---
+
+## 7. Tests
+
+### Unit
+- `test_descriptor_schema_validates`
+- `test_params_compatible_pair_must_match`
+- `test_auto_detect_substitutes_source_lang`
+- `test_text_too_long_rejected`
+- `test_cache_hit_returns_immediately`
+
+### Integration
+- `test_german_to_english_quality` (BLEU above floor)
+- `test_plattdeutsch_pair_registered_when_finetune_present`
+- `test_marketplace_one_click_translate_end_to_end`
+
+---
+
+## 8. Cross-references
+
+| What | Where |
+|------|-------|
+| `trans.text@1.0` wire | [CAP2 §4.10](../CAPABILITY_CONTRACT_v2.md) |
+| STT translate-to-EN feature | [M19 §4.3](M19-stt-tts.md) |
+| Marketplace one-click | [M08 ext](../../modules/M08-ui.md) |
+| Niederrhein context | Christof's domain |
+
+---
+
+## 9. Open questions
+
+1. **Fine-tune in-the-loop.** A community could fine-tune the Plattdeutsch model on its own corpus over time. Reserved.
+2. **Document-level translation.** Currently per-string. Document-coherence translation (better than chunked) is Phase 3.
+3. **Glossary support.** Domain glossaries (technical terms, names) preserved across translation. Phase 2.5.

docs/p2_p3/M19-stt-tts.md

@@ -0,0 +1,331 @@
+# M19 — Speech I/O (STT + TTS)
+
+**Spec version:** v1.0 (Phase 2)
+**Depends on:** M03 (bus), M07 (blobs, for audio I/O), X04 (config), X03 (observability), `openai-whisper`, `TTS` (Coqui XTTS-v2), `edge-tts` libs
+**Depended on by:** M08 UI (voice query button), M22 mobile (voice notes), M18 (STT can chain into translation)
+
+---
+
+## 1. Responsibility
+
+Two capabilities:
+
+- `stt.transcribe@1.0` — audio → text, with optional translate-to-English
+- `tts.synthesize@1.0` — text → audio
+
+Two services in the same module because they share the speech domain and often pair (voice query → STT → LLM → TTS).
+
+---
+
+## 2. File layout
+
+```
+hearthnet/services/speech/
+├── __init__.py
+├── stt_service.py
+├── tts_service.py
+└── backends/
+    ├── __init__.py
+    ├── base.py            # SttBackend, TtsBackend protocols
+    ├── whisper.py         # OpenAI Whisper local
+    ├── whisper_remote.py  # HF inference API alternative
+    ├── xtts.py            # Coqui XTTS-v2 (cloned voices)
+    └── edge_tts.py        # Microsoft Edge-TTS (Christof has existing pipeline)
+```
+
+---
+
+## 3. STT — public API
+
+### 3.1 `backends/base.py` (STT)
+
+```python
+@dataclass(frozen=True)
+class SttSegment:
+    start_seconds:  float
+    end_seconds:    float
+    text:           str
+    language:       str
+    speaker:        str | None     # only if diarization enabled
+    confidence:     float | None
+
+@dataclass(frozen=True)
+class SttResult:
+    segments:        list[SttSegment]
+    language:        str
+    duration_seconds: float
+    ms:              int
+
+class SttBackend(Protocol):
+    name:        str
+    models:      list[str]            # "tiny" | "base" | "small" | "medium" | "large-v3"
+    languages_supported: list[str]    # ISO 639-1
+    supports_diarization: bool
+
+    async def warm(self, model: str) -> None: ...
+    async def close(self) -> None: ...
+
+    async def transcribe(
+        self,
+        audio_bytes: bytes,
+        *,
+        model: str,
+        language: str | None,         # "auto" handled by caller
+        diarize: bool,
+        translate_to_en: bool,
+    ) -> AsyncIterator[SttSegment]:
+        """Yields segments as they are produced. Backend may produce in big chunks
+        or near-realtime depending on model + hardware."""
+
+    def health(self) -> dict: ...
+```
+
+### 3.2 `stt_service.py`
+
+```python
+class SttService:
+    name    = "stt"
+    version = "1.0"
+
+    def __init__(self, config: SpeechConfig, blob_store: BlobStore):
+        ...
+
+    def capabilities(self) -> list[tuple[CapabilityDescriptor, Callable, ParamsPredicate]]:
+        """One stt.transcribe per (backend, model) combo."""
+
+    async def start(self) -> None: ...
+    async def stop(self) -> None: ...
+    def health(self) -> dict: ...
+
+    async def handle_transcribe(self, req: RouteRequest) -> AsyncIterator[dict]:
+        """CAP2 §4.11.
+        1. Fetch audio blob by CID
+        2. Verify duration ≤ STT_MAX_AUDIO_SECONDS
+        3. Stream segments
+        4. Emit done with total stats"""
+```
+
+### 3.3 Concrete STT backends
+
+```python
+class WhisperBackend(SttBackend):
+    """Local Whisper via openai-whisper or faster-whisper."""
+
+    def __init__(self, models_dir: Path, default_model: str = "large-v3", device: str = "auto"):
+        ...
+
+class WhisperRemoteBackend(SttBackend):
+    """HF Inference API. requires_internet=True. Used as fallback when local Whisper not available."""
+
+    def __init__(self, model: str = "openai/whisper-large-v3", token_env: str = "HF_TOKEN"):
+        ...
+```
+
+---
+
+## 4. TTS — public API
+
+### 4.1 `backends/base.py` (TTS)
+
+```python
+@dataclass(frozen=True)
+class TtsResult:
+    audio_format:    str         # "ogg_vorbis" | "mp3" | "wav"
+    sample_rate:     int         # Hz
+    duration_seconds: float
+    total_bytes:     int
+    ms:              int
+
+class TtsBackend(Protocol):
+    name:        str
+    voices:      list[str]
+    languages_supported: list[str]
+    formats_supported: list[str]
+    cloned_voices_supported: bool
+
+    async def warm(self, voice: str) -> None: ...
+    async def close(self) -> None: ...
+
+    async def synthesize(
+        self,
+        text: str,
+        *,
+        voice: str,
+        language: str,
+        speed: float,                # 0.5..2.0; 1.0 default
+        output_format: str,          # "ogg_vorbis"|"mp3"|"wav"
+        chunk_size_bytes: int = 16384,
+    ) -> AsyncIterator[bytes]:
+        """Yields raw audio chunks."""
+
+    def health(self) -> dict: ...
+```
+
+### 4.2 `tts_service.py`
+
+```python
+class TtsService:
+    name    = "tts"
+    version = "1.0"
+
+    def __init__(self, config: SpeechConfig):
+        ...
+
+    def capabilities(self) -> list[tuple[CapabilityDescriptor, Callable, ParamsPredicate]]:
+        """One tts.synthesize per (backend, voice) pair (or backend-only if many voices)."""
+
+    async def start(self) -> None: ...
+    async def stop(self) -> None: ...
+    def health(self) -> dict: ...
+
+    async def handle_synthesize(self, req: RouteRequest) -> AsyncIterator[dict]:
+        """CAP2 §4.12.
+        1. Validate text length ≤ TTS_MAX_TEXT_CHARS
+        2. Pick backend and voice
+        3. Stream chunks (base64 in 'chunk' frame)
+        4. Emit done with metadata"""
+```
+
+### 4.3 Concrete TTS backends
+
+```python
+class XttsBackend(TtsBackend):
+    """Coqui XTTS-v2 (Christof has the pipeline from his podcast generator).
+       Supports voice cloning via reference audio."""
+
+    def __init__(
+        self,
+        model: str = "tts_models/multilingual/multi-dataset/xtts_v2",
+        voices_dir: Path = Path("~/.hearthnet/voices"),
+        device: str = "auto",
+    ):
+        ...
+
+class EdgeTtsBackend(TtsBackend):
+    """Microsoft Edge-TTS — requires internet, many voices, very natural.
+       Used as default when xtts is too slow on a node."""
+
+    def __init__(self, default_voice: str = "de-DE-KatjaNeural"):
+        ...
+```
+
+---
+
+## 5. Behaviour
+
+### 5.1 STT streaming
+
+For long audio:
+- Local Whisper produces segments incrementally (~real time on a 4090, slower on CPU)
+- Service emits one SSE `segment` frame per finalised segment
+- Final `done` frame includes total duration and full language detection
+
+### 5.2 STT max length
+
+`STT_MAX_AUDIO_SECONDS = 300`. Longer audio: caller chunks into 5-minute segments and concatenates results. Caller's responsibility to manage cross-chunk speaker continuity.
+
+### 5.3 Voice cloning (XTTS)
+
+`XttsBackend` supports voice cloning when given a reference audio file:
+
+```python
+config.tts.cloned_voices = [
+    ClonedVoiceConfig(name="hannes_v1", reference_path=Path("~/.hearthnet/voices/hannes-3s.wav"))
+]
+```
+
+Each cloned voice is registered as a separate `voice` entry in the descriptor params. Cloning happens once at startup; serves quickly thereafter.
+
+**Privacy note:** Voice cloning is powerful and risky. Communities SHOULD policy-restrict who can register cloned voices (suggested: `trust_required="anchor"` for voice cloning). MVP allows any member; document the risk.
+
+### 5.4 Audio format negotiation
+
+- Input STT: any common format Whisper accepts (mp3, ogg, wav, m4a). Service normalises via `ffmpeg`.
+- Output TTS: `ogg_vorbis` default (smallest), `mp3` widely-compatible, `wav` lossless.
+
+### 5.5 Edge-TTS internet dependency
+
+`EdgeTtsBackend` requires internet. Deregistered automatically by [M09](../../modules/M09-emergency.md) when offline. XTTS local backend continues to work.
+
+### 5.6 STT → TTS chain (voice assistant pattern)
+
+The voice query button in M08 UI ext:
+```
+mic → audio blob via M07 → stt.transcribe → text
+text → llm.chat → response text
+response text → tts.synthesize → audio chunks → speaker
+```
+
+This is composed at the UI layer, not internally in the speech services.
+
+### 5.7 Christof's existing pipeline reuse
+
+Christof has an established XTTS-v2 + Edge-TTS podcast generator pipeline. The `XttsBackend` and `EdgeTtsBackend` are designed to be drop-ins for that pipeline, sharing the same models directory.
+
+---
+
+## 6. Errors
+
+| Condition | Wire code |
+|-----------|-----------|
+| Audio > STT_MAX_AUDIO_SECONDS | `bad_request` |
+| Text > TTS_MAX_TEXT_CHARS | `bad_request` |
+| Unknown voice | `not_found` |
+| Audio decode failed (corrupt blob) | `bad_request` |
+| Backend GPU OOM | `capacity_exceeded` |
+
+---
+
+## 7. Configuration
+
+```python
+config.speech.enabled              = True
+config.speech.stt_backends = [
+    SttBackendConfig(name="whisper", default_model="large-v3", device="auto"),
+]
+config.speech.tts_backends = [
+    TtsBackendConfig(name="xtts", voices_dir=Path("~/.hearthnet/voices")),
+    TtsBackendConfig(name="edge_tts", default_voice="de-DE-KatjaNeural"),
+]
+config.speech.cloned_voices = []   # list[ClonedVoiceConfig]
+```
+
+Constants: `STT_MAX_AUDIO_SECONDS`, `TTS_MAX_TEXT_CHARS`.
+
+---
+
+## 8. Tests
+
+### Unit
+- `test_stt_descriptor_per_model`
+- `test_tts_descriptor_per_voice`
+- `test_stt_max_duration_rejected`
+- `test_tts_max_length_rejected`
+
+### Integration
+- `test_whisper_transcribes_de_audio` (test asset)
+- `test_xtts_synthesises_then_decodes_to_correct_duration`
+- `test_voice_chain_stt_llm_tts` — end-to-end
+- `test_edge_tts_deregistered_when_offline`
+
+---
+
+## 9. Cross-references
+
+| What | Where |
+|------|-------|
+| `stt.transcribe@1.0` wire | [CAP2 §4.11](../CAPABILITY_CONTRACT_v2.md) |
+| `tts.synthesize@1.0` wire | [CAP2 §4.12](../CAPABILITY_CONTRACT_v2.md) |
+| Voice query UI | M08 ext |
+| Mobile voice notes | [M22 §4](M22-mobile-native.md) |
+| Translation chain | [M18](M18-translation.md) |
+| Emergency dereg for internet-bound backends | [M09 §5.2](../../modules/M09-emergency.md) |
+
+---
+
+## 10. Open questions
+
+1. **Streaming STT (mic input → live caption)** — Phase 2.5. Requires WebSocket and a different backend init pattern.
+2. **Real-time TTS (sub-100ms first audio)** — XTTS is 500ms+; piper-tts is fast but limited voices. Phase 3.
+3. **Speaker enrollment** — explicit "this is who I am" speech sample so diarization can label by name. Phase 2.5.
+4. **Audio at-rest privacy** — should voice notes be E2E? [M23](M23-e2e-encryption.md) supports it; default ON for chat attachments.

docs/p2_p3/M20-vision.md

@@ -0,0 +1,369 @@
+# M20 — Vision Services
+
+**Spec version:** v1.0 (Phase 2)
+**Depends on:** M03 (bus), M07 (blobs), M04 (LLM, extended), X04 (config), X03 (observability)
+**Depended on by:** M08 UI (image describe in ask tab; generate in tools), M21 tools (vision used by tool-augmented LLM), M17 OCR (Florence-2 OCR mode shared)
+
+---
+
+## 1. Responsibility
+
+Two capability families:
+
+- `img.describe@1.0` — given an image CID, produce a caption, tags, object list, or OCR
+- `img.generate@1.0` — given a prompt, generate an image
+
+Plus: extend the LLM `llm.chat@2.0` request schema with **multimodal content** (text + image_cid in messages). The multimodal path goes through M04 backends that declare `modalities: ["text", "vision"]`. M20 is responsible for providing the vision backends those LLMs depend on.
+
+Christof's existing pipelines: Florence-2 (describe), FLUX.1-dev with LoRAs (generate), MiniCPM-V (multimodal). All wired in.
+
+---
+
+## 2. File layout
+
+```
+hearthnet/services/image/
+├── __init__.py
+├── describe_service.py
+├── generate_service.py
+└── backends/
+    ├── __init__.py
+    ├── base.py             # ImageDescribeBackend, ImageGenerateBackend
+    ├── florence2.py        # Microsoft Florence-2 large
+    ├── minicpm_v.py        # OpenBMB MiniCPM-V (also usable for chat-with-vision)
+    ├── flux.py             # black-forest-labs FLUX.1-dev with LoRA support
+    └── stable_diffusion.py # Optional SD-XL fallback
+```
+
+---
+
+## 3. Public API — describe
+
+### 3.1 `backends/base.py`
+
+```python
+@dataclass(frozen=True)
+class ImageDescription:
+    caption:    str
+    detailed_caption: str | None
+    tags:       list[str]
+    objects:    list[dict]                  # [{label, bbox, confidence}]
+    ocr_text:   str | None
+    language:   str
+    ms:         int
+
+class ImageDescribeBackend(Protocol):
+    name:               str
+    tasks_supported:    list[str]           # subset of {"caption","detailed_caption","ocr","objects","tags"}
+    languages:          list[str]
+    max_pixels:         int
+
+    async def warm(self) -> None: ...
+    async def close(self) -> None: ...
+
+    async def describe(
+        self,
+        image_bytes: bytes,
+        *,
+        task: str,
+        language: str = "en",
+    ) -> ImageDescription: ...
+
+    def health(self) -> dict: ...
+```
+
+### 3.2 `describe_service.py`
+
+```python
+class ImageDescribeService:
+    name    = "image.describe"
+    version = "1.0"
+
+    def __init__(self, config: VisionConfig, blob_store: BlobStore):
+        ...
+
+    def capabilities(self) -> list[tuple[CapabilityDescriptor, Callable, ParamsPredicate]]:
+        """One img.describe per backend. Params include backend name and tasks_supported."""
+
+    async def start(self) -> None: ...
+    async def stop(self) -> None: ...
+    def health(self) -> dict: ...
+
+    async def handle_describe(self, req: RouteRequest) -> dict:
+        """CAP2 §4.13."""
+```
+
+### 3.3 Concrete describe backends
+
+```python
+class Florence2Backend(ImageDescribeBackend):
+    def __init__(self, model: str = "microsoft/Florence-2-large", device: str = "auto"):
+        ...
+
+class MinicpmVBackend(ImageDescribeBackend):
+    """Used both standalone (img.describe) and as an LLM vision backend (M04 extension)."""
+    def __init__(self, model: str = "openbmb/MiniCPM-V-2_6", device: str = "auto"):
+        ...
+```
+
+---
+
+## 4. Public API — generate
+
+### 4.1 `backends/base.py`
+
+```python
+@dataclass(frozen=True)
+class GenerationResult:
+    image_bytes:    bytes
+    width:          int
+    height:         int
+    format:         str             # "png" | "webp" | "jpg"
+    seed:           int
+    ms:             int
+
+class ImageGenerateBackend(Protocol):
+    name:               str
+    models:             list[str]
+    loras_available:    list[str]
+    max_resolution:     tuple[int, int]
+    min_resolution:     tuple[int, int]
+    supports_negative_prompt: bool
+
+    async def warm(self, model: str) -> None: ...
+    async def close(self) -> None: ...
+
+    async def generate(
+        self,
+        prompt: str,
+        *,
+        model: str,
+        lora: str | None,
+        negative_prompt: str | None,
+        width: int,
+        height: int,
+        steps: int,
+        seed: int | None,
+        progress_cb: Callable[[int, int], None] | None = None,
+    ) -> GenerationResult: ...
+
+    def health(self) -> dict: ...
+```
+
+### 4.2 `generate_service.py`
+
+```python
+class ImageGenerateService:
+    name    = "image.generate"
+    version = "1.0"
+
+    def __init__(self, config: VisionConfig, blob_store: BlobStore):
+        ...
+
+    def capabilities(self) -> list[tuple[CapabilityDescriptor, Callable, ParamsPredicate]]:
+        """One img.generate per (backend, model) combo. params declare loras_available."""
+
+    async def start(self) -> None: ...
+    async def stop(self) -> None: ...
+    def health(self) -> dict: ...
+
+    async def handle_generate(self, req: RouteRequest) -> AsyncIterator[dict]:
+        """CAP2 §4.14.
+        1. Generate (streaming progress frames)
+        2. Store resulting image as blob
+        3. Emit done with image_cid"""
+```
+
+### 4.3 Concrete generate backends
+
+```python
+class FluxBackend(ImageGenerateBackend):
+    """FLUX.1-dev with LoRA support; Christof's existing pipeline."""
+    def __init__(
+        self,
+        model: str = "black-forest-labs/FLUX.1-dev",
+        device: str = "auto",
+        loras_dir: Path = Path("~/.hearthnet/loras"),
+    ):
+        ...
+
+class StableDiffusionBackend(ImageGenerateBackend):
+    """SD-XL fallback for nodes with smaller GPUs."""
+    def __init__(self, model: str = "stabilityai/stable-diffusion-xl-base-1.0", device: str = "auto"):
+        ...
+```
+
+---
+
+## 5. Multimodal LLM extension (M04 hook)
+
+### 5.1 Message content array
+
+In `llm.chat@2.0` (CAP2 §4.23), each `messages[].content` may be a list:
+
+```json
+[
+  {"type": "text",  "text": "Was ist auf diesem Bild?"},
+  {"type": "image", "image_cid": "blake3:..."}
+]
+```
+
+Backends that declare `modalities: ["text", "vision"]` in their descriptor must handle the array form. Backends that don't either:
+- Are skipped by the router (params_compatible returns False when message contains image and `modalities ⊉ {"vision"}`)
+- Or fall back: extract text content only, ignore images (worse UX; not recommended)
+
+### 5.2 Vision-capable backends in M04
+
+These M04 backends gain a `modalities: ["text","vision"]` declaration in Phase 2:
+
+| Backend | Vision support |
+|---------|----------------|
+| `MinicpmVBackend` (M04 entry — same model as M20's describe) | Yes; native multimodal |
+| `AnthropicApiBackend` | Yes; Claude vision via API |
+| `OpenAiApiBackend` | Yes; GPT-4V |
+| `Llava` (new, optional) | Yes; LLaVA via llama.cpp |
+| `LlamaCppBackend` | Yes if model is multimodal (LLaVA-format) |
+| `OllamaBackend` | Yes for vision models |
+| Others | No |
+
+The `M04.LlmService._build_backends` constructs these with their vision flag.
+
+### 5.3 Image preprocessing
+
+For LLM context, images are:
+- Loaded from blob store via CID
+- Resized to backend's preferred resolution (e.g. 1024×1024 for MiniCPM-V)
+- Encoded base64 or sent as bytes per backend's protocol
+
+This is opaque to the caller — the multimodal `messages` array is the contract.
+
+---
+
+## 6. Behaviour
+
+### 6.1 Image describe lifecycle
+
+```
+caller → bus.call("img.describe", (1,0), {input:{image_cid:..., task:"detailed_caption"}})
+  → ImageDescribeService.handle_describe
+       → blob_store.read_blob_bytes(image_cid)
+       → backend.describe(bytes, task=...)
+       → return ImageDescription serialised
+```
+
+### 6.2 Image generate lifecycle
+
+```
+caller → bus.stream("img.generate", (1,0), {input:{prompt:"...", steps:20}})
+  → ImageGenerateService.handle_generate
+       → backend.generate(...) with progress_cb
+       → for each step: emit 'progress' frame
+       → on completion: blob_store.write_blob(image)
+       → emit 'done' frame with image_cid
+```
+
+### 6.3 Safety filters
+
+- `img.generate` prompts pass through a configurable safety filter list (regex blocklist + optional LLM-based classifier)
+- Generation of identifiable persons is blocked by default (configurable: `config.vision.allow_identifiable_persons`)
+- NSFW filter on output (Stable Diffusion has built-in; FLUX needs separate model)
+- Failed safety → `bad_request` with `reason: "safety_filter"`
+
+### 6.4 LoRA management
+
+`FluxBackend.loras_available` lists LoRAs found in `loras_dir`. Caller can request a specific LoRA in `params.lora`. Loading a LoRA takes a few seconds on first use; cached thereafter.
+
+Christof's existing LoRAs (local-style, sketches, etc.) drop into the `loras_dir`. The backend auto-discovers them.
+
+### 6.5 GPU pressure
+
+Vision models are heavy. Recommended:
+
+- One Florence-2 instance per node (always-loaded)
+- FLUX/SD only loaded on-demand (warm on first request, kept hot for 5 minutes)
+- `max_concurrent = 1` for FLUX; `2` for describe backends
+
+These limits are declared in the capability descriptor so the bus throttles correctly.
+
+### 6.6 Multimodal LLM call routing
+
+When a user sends a multimodal message:
+
+```
+UI → bus.stream("llm.chat", (2,0), {input:{messages:[{role:"user", content:[{type:"text",text:"..."},{type:"image",image_cid:"..."}]}]}})
+  → Router.route filters candidates to those with modalities ⊇ {"vision"}
+  → picks best (e.g. MinicpmV local, fall back to Anthropic API)
+  → backend handles base64 / image-token-injection internally
+```
+
+If no vision-capable backend is online, the call returns `not_found` with a helpful `alt_capabilities` hint pointing to describe-then-text-only fallback (UI can offer this).
+
+---
+
+## 7. Errors
+
+| Condition | Wire code |
+|-----------|-----------|
+| Unknown task | `bad_request` |
+| Image too large | `bad_request` |
+| Prompt safety violation | `bad_request` (reason=safety_filter) |
+| LoRA not found | `not_found` |
+| GPU OOM | `capacity_exceeded` |
+| Backend missing for requested task | `not_implemented` |
+
+---
+
+## 8. Configuration
+
+```python
+config.vision.enabled                 = True
+config.vision.describe_backends = [
+    DescribeBackendConfig(name="florence2", model="microsoft/Florence-2-large", device="auto"),
+    DescribeBackendConfig(name="minicpm_v", model="openbmb/MiniCPM-V-2_6", device="auto"),
+]
+config.vision.generate_backends = [
+    GenerateBackendConfig(name="flux", model="black-forest-labs/FLUX.1-dev",
+                          loras_dir=Path("~/.hearthnet/loras"), device="auto"),
+]
+config.vision.allow_identifiable_persons = False
+config.vision.safety_blocklist_file       = None   # optional regex file
+```
+
+---
+
+## 9. Tests
+
+### Unit
+- `test_describe_descriptor_per_backend`
+- `test_safety_filter_blocks_known_pattern`
+- `test_lora_discovery`
+- `test_oom_returns_capacity_exceeded`
+
+### Integration
+- `test_florence2_caption_sample` (test image)
+- `test_flux_generate_with_lora_progress_frames`
+- `test_multimodal_llm_routes_to_vision_backend`
+- `test_describe_then_text_fallback_when_no_vision_llm`
+
+---
+
+## 10. Cross-references
+
+| What | Where |
+|------|-------|
+| `img.*` wire | [CAP2 §4.13–4.14](../CAPABILITY_CONTRACT_v2.md) |
+| Multimodal `llm.chat@2.0` | [CAP2 §4.23](../CAPABILITY_CONTRACT_v2.md) |
+| LLM service extension | M04 (extended in Phase 2 — see [00-OVERVIEW §1](../00-OVERVIEW.md)) |
+| OCR overlap | [M17 §3.2 florence_ocr](M17-ocr.md) |
+| Christof's pipelines | external, this is the integration |
+
+---
+
+## 11. Open questions
+
+1. **Video** — Phase 3 considers `video.describe` and `video.generate` (LTX-Video). Not in Phase 2.
+2. **Image editing (inpainting)** — Phase 2.5: `img.edit@1.0` capability. Reserved.
+3. **Control nets (depth, edge, pose)** — Phase 2.5.
+4. **3D generation** — Phase 3 with TripoSR or similar.
+5. **Safety filter quality** — regex blocklist is weak. An LLM-as-judge classifier is better but adds latency. Configurable; default off.
+6. **LoRA stacking** — caller specifies multiple `loras: ["...","..."]`. Implementable but adds attack surface (prompt-LoRA combos). Defer.

docs/p2_p3/M21-tool-calls.md

@@ -0,0 +1,317 @@
+# M21 — Tool Calls (LLM Tool-Use)
+
+**Spec version:** v1.0 (Phase 2)
+**Depends on:** M03 (bus), M04 (LLM, extended), X06 (WebSocket, for in-stream tool loops), X04 (config), X03 (observability)
+**Depended on by:** M08 UI (ask tab gains tool-augmented mode), M22 mobile (same), any future agent applications
+
+---
+
+## 1. Responsibility
+
+Let the LLM call other HearthNet capabilities mid-generation. Specifically:
+
+- Declare `tools` in the `llm.chat@2.0` request
+- Receive `tool_call_delta` and `tool_call` stream frames from the LLM
+- Execute each tool call against the bus
+- Feed results back into the LLM via `tool_result` (WebSocket) or via a follow-up `llm.chat` call (SSE)
+- Provide `llm.tools.call@1.0` as a convenience that wraps the bus dispatch
+
+This module is **a protocol + helper**, not a service in the usual sense. The actual LLM work lives in M04; M21 documents how the tool flow is structured and provides utilities both sides need.
+
+---
+
+## 2. File layout
+
+```
+hearthnet/services/llm/
+└── tools.py             # ToolDefinition, ToolCall, ToolResult, ToolExecutor
+
+hearthnet/services/auth/    (already in M16)
+# the auth service also registers llm.tools.call@1.0 as a wrapper capability
+```
+
+`tools.py` is small (~250 LOC). It lives inside the LLM service package because tool usage is intrinsic to chat completion.
+
+---
+
+## 3. Public API
+
+### 3.1 Data types
+
+```python
+# hearthnet/services/llm/tools.py
+from dataclasses import dataclass
+from typing import AsyncIterator, Callable
+
+@dataclass(frozen=True)
+class ToolDefinition:
+    """A tool the caller offers to the LLM.
+       Translated by the LLM backend into its native tool format."""
+    name:               str              # short identifier visible to the LLM
+    description:        str              # human-readable, drives LLM selection
+    parameters_schema:  dict             # JSON Schema for arguments
+    bound_capability:   str | None       # if set, ToolExecutor dispatches via bus
+    bound_version:      tuple[int, int] | None
+    side_effects:       bool             # "is this a write?" — affects retry semantics
+
+@dataclass(frozen=True)
+class ToolCall:
+    """A request from the LLM to execute a tool."""
+    id:           str                    # opaque, generated by the LLM
+    name:         str
+    arguments:    dict                   # validated against parameters_schema
+
+@dataclass(frozen=True)
+class ToolResult:
+    """The result of executing a ToolCall, fed back to the LLM."""
+    tool_call_id: str
+    name:         str
+    content:      str | dict             # serialisable; if dict, becomes JSON
+    is_error:     bool
+```
+
+### 3.2 `ToolExecutor`
+
+```python
+class ToolExecutor:
+    """Wraps the orchestration loop: forward LLM tool calls to the bus,
+       collect results, re-inject into the LLM."""
+
+    def __init__(
+        self,
+        bus: CapabilityBus,
+        tools: list[ToolDefinition],
+        *,
+        max_iterations: int = 6,
+        per_tool_timeout_seconds: int = 30,
+    ):
+        ...
+
+    @property
+    def native_definitions(self) -> list[dict]:
+        """Returns tools in the request schema's format (CAP2 §4.23 input.tools)."""
+
+    async def dispatch(self, call: ToolCall) -> ToolResult:
+        """Validate call.arguments against the tool's parameters_schema.
+        If bound_capability: bus.call(bound_capability, bound_version, {input: call.arguments}).
+        Returns ToolResult. Catches and surfaces errors as is_error=True."""
+
+    async def run_chat_with_tools(
+        self,
+        chat_request_body: dict,
+        *,
+        stream_to: Callable[[dict], Awaitable[None]] | None = None,
+    ) -> dict:
+        """Orchestrator helper. Loops:
+        1. call bus.stream("llm.chat", (2,0), body)
+        2. accumulate text + tool_call frames
+        3. on tool_call_complete: dispatch, append tool_result message
+        4. re-call llm.chat with extended messages
+        5. stop when no more tool calls OR max_iterations reached
+        Returns the final assistant message."""
+```
+
+### 3.3 Wire-level frames (recap of CAP2 §4.23 and §5.1 with the tool flow)
+
+LLM emits:
+
+```
+event: token
+data: {"text":"I'll search "}
+
+event: tool_call_delta
+data: {"id":"tc_1","name":"rag.query","arguments_delta":"{\"query\":\""}
+
+event: tool_call_delta
+data: {"id":"tc_1","arguments_delta":"Regenwasser\""}
+
+event: tool_call
+data: {"id":"tc_1","name":"rag.query","arguments":{"query":"Regenwasser","corpus":"niederrhein-emergency"}}
+```
+
+Caller dispatches and replies (over WebSocket OR by re-calling `llm.chat` with the tool result added to messages):
+
+WebSocket:
+```
+client → {"type":"tool_result","tool_call_id":"tc_1","body":{"chunks":[...]}}
+```
+
+SSE fallback:
+```
+caller re-calls llm.chat with messages = original_messages + [
+    {"role":"assistant","content":"...","tool_calls":[{"id":"tc_1","name":"rag.query","arguments":{...}}]},
+    {"role":"tool","tool_call_id":"tc_1","content":"<JSON of tool result>"}
+]
+```
+
+Both paths converge: LLM continues and eventually emits `done`.
+
+---
+
+## 4. Behaviour
+
+### 4.1 Tool selection heuristics
+
+The LLM picks tools based on:
+- Tool descriptions (descriptive English/German helps)
+- `tool_choice` parameter:
+  - `"auto"` (default): LLM decides
+  - `"none"`: forbid tool use even if tools are declared
+  - `"required"`: must call at least one tool
+  - `{"name":"rag.query"}`: must call specifically this tool
+
+Backends translate these to their native API.
+
+### 4.2 Built-in tools
+
+When `ToolExecutor` is instantiated by the UI, it can auto-include a set of standard tools bound to common bus capabilities:
+
+| Tool name | Bound to | Use case |
+|-----------|----------|----------|
+| `search_corpus` | `rag.query@1.0` | Search a corpus |
+| `list_corpora` | `rag.list_corpora@1.0` | What's available |
+| `translate` | `trans.text@1.0` | Translate snippets |
+| `find_neighbour` | (custom — list peers in current community) | "Wer ist da?" |
+| `list_marketplace` | `market.list@1.0` | Active posts |
+| `describe_image` | `img.describe@1.0` | Inspect uploaded images |
+| `transcribe_audio` | `stt.transcribe@1.0` | Voice-input chained |
+
+These are *suggested defaults*. Real applications pick what fits.
+
+### 4.3 Validation
+
+`ToolExecutor.dispatch` validates `call.arguments` against `parameters_schema` before calling the bus. Invalid args → `ToolResult(is_error=True, content="invalid_arguments: ...")`. The LLM sees the error and typically self-corrects.
+
+### 4.4 Iteration limits
+
+`max_iterations` (default 6) prevents runaway tool loops. After the limit, `ToolExecutor.run_chat_with_tools` injects a final `tool` message saying "iteration limit reached; finalise your answer" and forces `tool_choice="none"` on the next call.
+
+### 4.5 Side-effect tools
+
+Tools where `side_effects: True` (like `market.post`, `chat.send`) require explicit confirmation. By default, `ToolExecutor` raises `ToolError("requires_confirmation")` on side-effect calls, expecting the orchestrator (UI) to present a confirmation dialog.
+
+UI flow:
+```
+LLM emits tool_call to market.post
+ToolExecutor sees side_effects=True
+emits a 'confirmation_required' frame upstream
+UI shows "Allow LLM to post this?"
+user clicks yes → orchestrator calls ToolExecutor.dispatch_confirmed(call)
+```
+
+### 4.6 Parallel tool calls
+
+LLMs (Claude, GPT-4) can emit multiple `tool_call` frames in one turn. `ToolExecutor` dispatches them in parallel (bounded by `max_concurrent=4`). Results are submitted together in the next LLM turn.
+
+### 4.7 Tool call composition (tools that call tools)
+
+A `bound_capability` may itself be `llm.tools.call@1.0`. This allows defining higher-level tools as compositions of bus capabilities + LLM reasoning. Recursion limit = `max_iterations`.
+
+### 4.8 Trust and tokens
+
+Tool dispatch goes through the bus and inherits the caller's trust level. The LLM cannot escalate by emitting a tool call — the tool inherits the caller's permissions. For cross-community tool calls, the caller must hold an appropriate token (M16).
+
+### 4.9 LLM backend translation
+
+Backends translate `ToolDefinition` to their native protocol:
+
+| Backend | Native format |
+|---------|--------------|
+| `AnthropicApiBackend` | Anthropic Messages tools |
+| `OpenAiApiBackend` | OpenAI function calling |
+| `OllamaBackend` (some models) | Ollama tool calls |
+| `LlamaCppBackend` (with grammar) | JSON-Schema grammar constraint |
+| `MinicpmVBackend` | MiniCPM tool format |
+| `NemotronBackend` | OpenAI-compatible |
+| `OpenBmbBackend` | OpenAI-compatible |
+| Others | Tools ignored; backend emits a notice on `tool_choice="required"` |
+
+This translation lives inside each backend's `chat()` method.
+
+---
+
+## 5. `llm.tools.call@1.0` capability
+
+Convenience wrapper. Used when a caller wants to invoke a bus capability as if it were a tool result, without going through the LLM:
+
+Already specified in [CAP2 §4.24](../CAPABILITY_CONTRACT_v2.md).
+
+The handler lives in `M04.LlmService.handle_tools_call`:
+
+```python
+async def handle_tools_call(self, req: RouteRequest) -> dict:
+    """1. Validate target_body against target_capability's request schema (via bus.schema)
+       2. bus.call(target_capability, target_version, target_body)
+       3. Return result"""
+```
+
+Mostly used by orchestrators that want a single audit-trail capability for "tool execution".
+
+---
+
+## 6. Configuration
+
+```python
+config.llm.tools_enabled            = True
+config.llm.tools_max_iterations     = 6
+config.llm.tools_per_tool_timeout_seconds = 30
+config.llm.tools_max_parallel       = 4
+config.llm.tools_default_set        = ["search_corpus","list_corpora","translate","list_marketplace"]
+config.llm.tools_require_confirmation_for_side_effects = True
+```
+
+---
+
+## 7. Errors
+
+| Condition | Wire code |
+|-----------|-----------|
+| Tool not in declared set | `bad_request` |
+| Tool arguments fail schema | `bad_request` |
+| Tool execution timed out | `timeout` |
+| Tool returned `internal_error` | propagated as `internal_error` |
+| Iteration limit reached | (graceful — final answer forced) |
+| Caller's token doesn't cover bound capability | `token_scope_insufficient` |
+
+---
+
+## 8. Tests
+
+### Unit
+- `test_tool_definition_to_native_format` — per backend
+- `test_dispatch_validates_arguments`
+- `test_side_effect_tool_requires_confirmation`
+- `test_iteration_limit_forces_finalisation`
+- `test_parallel_tool_calls_collected`
+
+### Integration
+- `test_search_corpus_tool_used_for_grounded_answer` — LLM is asked a question, calls rag.query, answers
+- `test_translate_chain` — user types in DE, LLM uses trans.text tool internally
+- `test_market_post_requires_confirmation`
+- `test_recursive_tool_call_limited_by_max_iterations`
+
+### Manual
+- Confirm Anthropic Claude, OpenAI GPT-4, Ollama Mistral, MiniCPM-V all produce well-formed tool_call frames on the same test prompt.
+
+---
+
+## 9. Cross-references
+
+| What | Where |
+|------|-------|
+| `llm.chat@2.0` tools field | [CAP2 §4.23](../CAPABILITY_CONTRACT_v2.md) |
+| Tool-call stream frames | [CAP2 §5.1, X06 §6.6](../cross-cutting/X06-websocket.md) |
+| `llm.tools.call@1.0` | [CAP2 §4.24](../CAPABILITY_CONTRACT_v2.md) |
+| M04 backend extensions | M04 (extended in Phase 2) |
+| Token scope for cross-community tool dispatch | [M16 §5.2](M16-tokens.md) |
+| Confirmation UI hook | M08 ext |
+
+---
+
+## 10. Open questions
+
+1. **Tool result streaming.** Currently a tool result is atomic. For long-running tool calls (e.g. `img.generate`), the LLM has to wait. Phase 2.5 may stream tool progress back.
+2. **Tool memoisation.** Repeated `search_corpus(q)` in one chat could be cached. Defer.
+3. **Tool authority lineage.** When a tool is called by an LLM running on Node A on behalf of User on Node B, which token does the tool inherit? Currently the user's. This may be insufficient for federation. Phase 2.5.
+4. **Tool calls that issue tokens.** Could a tool be "issue me a token for capability X"? Probably yes; specify carefully to avoid privilege escalation. Defer.
+5. **Tool selection telemetry.** Which tools does the LLM actually pick? Useful for tuning descriptions. Log to trace ring buffer; surface in observability dashboard.

docs/p2_p3/M22-mobile-native.md

@@ -0,0 +1,325 @@
+# M22 — Mobile Native Client
+
+**Spec version:** v1.0 (Phase 2)
+**Depends on:** M01 (identity), M15 (relay tier for push and NAT traversal), M16 (tokens for app auth), M23 (E2E for chat), X06 (WebSocket for live updates), the entire Phase 1 bus protocol as a wire client
+**Depended on by:** end users on iOS/Android — first non-web HearthNet surface
+
+---
+
+## 1. Responsibility
+
+Native mobile application that:
+
+- Onboards into a community (scan invite QR or paste invite blob)
+- Stores keys in the device's secure enclave (iOS Keychain, Android Keystore)
+- Receives push notifications via the relay tier (M15)
+- Calls the community's anchor(s) via the bus protocol over HTTP/SSE or WebSocket
+- Provides UI for chat (1:1 + group), marketplace, ask (LLM), community feed
+- Operates fully when the user's anchor is reachable; degrades gracefully when not
+
+This module specifies **the contract between the mobile client and the rest of HearthNet**. The actual Flutter codebase lives in a separate repo (`/mobile-native`) and is not Python.
+
+---
+
+## 2. File layout
+
+```
+mobile-native/                          # separate Flutter project
+├── pubspec.yaml
+├── README.md
+├── lib/
+│   ├── main.dart
+│   ├── onboarding/                     # invite scan, key gen
+│   ├── identity/                       # key storage, signing
+│   │   ├── secure_storage.dart
+│   │   └── signing.dart
+│   ├── bus/                            # protocol client
+│   │   ├── http_client.dart
+│   │   ├── ws_client.dart
+│   │   └── sse_client.dart
+│   ├── crypto/                         # E2E using cryptography_flutter / libsodium bindings
+│   │   ├── x25519.dart
+│   │   ├── ratchet.dart
+│   │   └── envelope.dart
+│   ├── push/                           # APNs / FCM hookup
+│   │   └── subscriber.dart
+│   ├── ui/                             # screens
+│   │   ├── chat.dart
+│   │   ├── marketplace.dart
+│   │   ├── ask.dart
+│   │   └── community.dart
+│   └── settings/
+├── ios/
+├── android/
+└── tests/
+
+hearthnet/mobile/                        # Python-side helper (in the main package)
+├── __init__.py
+├── invite.py                            # mobile-targeted invite QR generation
+└── push_authority.py                    # bus-side service for mobile push token registry
+```
+
+The Python `hearthnet/mobile/` package contains the anchor-side helpers used by the existing community. The Flutter code is its own world; this spec governs the wire contract it must implement.
+
+---
+
+## 3. Onboarding flow
+
+```
+User installs HearthNet app
+  ↓
+App: "Scan invite QR or paste invite link"
+  ↓
+On scan: parse hnvite:// blob (Phase 1 §13)
+  ↓
+App generates Ed25519 keypair via libsodium binding
+  Persists private key in iOS Keychain (kSecAttrAccessibleAfterFirstUnlock)
+  or Android Keystore (KeyProperties.AUTH_REQUIRED if biometric set)
+  ↓
+App calls bus.call("onboard.complete", input={
+    "invite_token": "...",
+    "node_id_full": "<our new ed25519 pub>",
+    "device_class": "mobile",
+    "display_name": "<user-entered>",
+    "platform": "ios|android",
+})
+  ↓
+Anchor processes invite (M13), emits node.joined event
+  ↓
+App fetches community manifest (signed); pins it
+  ↓
+App registers for push:
+  - obtain APNs/FCM device token from OS
+  - bus.call("relay.push.register", input={device_token, platform})
+  ↓
+App publishes E2E prekey bundle (e2e.prekeys.published event)
+  ↓
+Ready to use
+```
+
+---
+
+## 4. Bus protocol on mobile
+
+### 4.1 Same wire as desktop
+
+The mobile client speaks the same HTTP/SSE/WebSocket protocol as Phase 1 anchors. It is a **client** that calls into anchors/services in its community; it does not host capabilities itself (mostly — see §4.4).
+
+### 4.2 Endpoint selection
+
+The mobile client maintains an ordered list of endpoints:
+
+1. **Cached anchor endpoints** from last successful manifest fetch
+2. **Local network discovery** (mDNS via [Bonjour iOS / NSD Android]) — only when on Wi-Fi
+3. **Relay tier** (M15) — for NAT traversal when on cellular
+4. **DHT lookup** (X05) — last resort
+
+Per call, the client tries endpoints in order; first success wins. Persistent failures bubble up as `relay_unreachable` or `network_unreachable`.
+
+### 4.3 Reconnection
+
+WebSocket connections are persistent for tool-call loops and live pubsub. On disconnect, the client reconnects with exponential backoff (1s, 2s, 4s, ..., capped at 60s). Reconnect re-subscribes to all active topics.
+
+### 4.4 Mobile-as-callable
+
+In Phase 2, the mobile client does NOT register capabilities back into the community. It's purely a caller. Phase 3 may allow simple things (`market.list` from local cache while offline).
+
+### 4.5 Token-bearer mode
+
+For background-fetch (when the app is suspended), the OS may run a brief task. Background tasks use a **bearer token** from M16 (issued at onboarding, refreshed on each app foreground). The token has scope:
+
+```json
+{
+  "capabilities": ["chat.fetch","marketplace.list"],
+  "rate_limit_per_minute": 30
+}
+```
+
+This avoids needing the user's biometric to unlock the private key for background polling.
+
+---
+
+## 5. Push notifications
+
+### 5.1 What triggers a push
+
+When the following events occur in the community, anchors send a push to relevant subscribed mobile devices:
+
+- `chat.message.sent` where recipient is the mobile user
+- `chat.thread.message.sent` where mobile user is a thread member
+- `marketplace.post.created` where post matches user's subscribed categories
+- `community.alert` (broadcast emergency alert)
+- `node.joined` (subscribed users only)
+
+### 5.2 Push payload shape
+
+Per [M15 §5.4](M15-relay-tier.md), the payload is minimal:
+
+```json
+{
+  "event_type": "chat.message.sent",
+  "sender_short": "7H4G-...",
+  "preview": "Hallo Jana, ich bring..."     // optional cleartext preview if not E2E
+}
+```
+
+For E2E messages, the preview is absent and the app must fetch + decrypt on open.
+
+### 5.3 iOS vs Android specifics
+
+**iOS:** APNs payload with `aps.alert.title` and `aps.alert.body`. Background mode enabled to fetch on receive (`content-available: 1`).
+
+**Android:** FCM `data` message (not `notification` — we control display). Handled by the app's `FirebaseMessagingService`.
+
+### 5.4 Quiet hours
+
+User-configurable. Push silenced 22:00–07:00 by default; emergency alerts override.
+
+### 5.5 Mute and per-thread settings
+
+Per-thread mute, per-category marketplace silence. Stored in `mobile.preferences` event (self-only, encrypted-at-rest on device).
+
+---
+
+## 6. Secure key storage
+
+### 6.1 iOS — Keychain Services
+
+```dart
+// lib/identity/secure_storage.dart (sketch)
+const _accessibility = 'kSecAttrAccessibleAfterFirstUnlock';
+
+Future<void> storePrivateKey(String label, Uint8List bytes) async {
+  await KeychainAccess.setData(
+    label: label,
+    data: bytes,
+    accessibility: _accessibility,
+    accessControl: AccessControl.userPresence,  // biometric on key use
+  );
+}
+```
+
+### 6.2 Android — Keystore
+
+Hardware-backed if available (StrongBox on modern Pixels), else TEE.
+
+```dart
+final cipher = await CryptographyFlutter.aesGcm(
+  keyId: 'hearthnet_identity_v1',
+  requireAuth: AuthMethod.biometric,
+);
+```
+
+### 6.3 Backup
+
+Private keys are NEVER backed up via iCloud / Google Backup. App-level backup uses an encrypted export blob (user-chosen passphrase) that the user is expected to save out-of-band (e.g. password manager, written down).
+
+`config.mobile.cloud_backup_allowed = false` enforced.
+
+### 6.4 Lost device
+
+If the user loses the device, they:
+
+1. Wait out the device's session — eventually messages stop delivering
+2. From another device, call `node.revoke` on this NodeID (anchor co-signs)
+3. The revoked NodeID is then blacklisted; the lost phone, even if recovered, can't authenticate
+
+This is identical to the Phase 1 node revocation flow.
+
+---
+
+## 7. UI surface (mobile)
+
+Mirrors the web UI ([M08](../../modules/M08-ui.md)) but native:
+
+| Tab | Content |
+|-----|---------|
+| **Chat** | 1:1 conversations + group threads. Live updates via WS. Voice notes optional (record → STT → send transcript + audio attachment). |
+| **Market** | List + post; image attachments via camera/gallery. |
+| **Ask** | LLM chat. Tool-augmented mode available. Voice input button. |
+| **Community** | Member list, recent events, federation peers. |
+| **Settings** | Push prefs, language, backup, advanced. |
+
+All UI is plain Material/Cupertino — no exotic frameworks. Matches Christof's preference for boring, durable tech.
+
+---
+
+## 8. Configuration
+
+Mobile-side (lives in app):
+
+```dart
+const config = {
+  'community_id': '<from invite>',
+  'anchor_endpoints': [/* from manifest */],
+  'relay_url': 'https://relay.hearthnet.de',
+  'push_enabled': true,
+  'quiet_hours': {'start': '22:00', 'end': '07:00'},
+  'background_fetch_minutes': 15,
+};
+```
+
+Anchor-side (Python, for the push_authority service):
+
+```python
+config.mobile.push_enabled              = True
+config.mobile.push_categories_marketplace = ["essentials","emergency"]
+config.mobile.push_quiet_hours_default  = ("22:00","07:00")
+```
+
+---
+
+## 9. Errors
+
+| Condition | UI presentation |
+|-----------|-----------------|
+| No network | Offline banner; queued sends |
+| Anchor unreachable | "Your community anchor is offline. Retrying..." |
+| Relay unreachable | Falls back to direct; warns if all fail |
+| Token expired | Silent refresh; only surface if refresh fails |
+| Push delivery failed | No UI; logged for diagnostics |
+| Manifest signature mismatch | Hard block; re-onboard required |
+
+---
+
+## 10. Tests
+
+### Flutter side
+- Widget tests for each tab
+- Integration test for onboarding flow with a mock anchor
+- E2E test using a real anchor in CI (Linux runner running hearthnet)
+
+### Anchor-side Python
+- `test_push_subscription_recorded`
+- `test_push_dispatch_on_chat_message_sent`
+- `test_quiet_hours_silences_non_emergency`
+- `test_revocation_revokes_mobile_session`
+
+### Manual
+- iOS + Android smoke tests on physical devices
+- Background-fetch verified across 30-minute suspensions
+
+---
+
+## 11. Cross-references
+
+| What | Where |
+|------|-------|
+| Bus protocol | [Phase 1 CAP §5](../../CAPABILITY_CONTRACT.md) |
+| Push relay tier | [M15 §5.4](M15-relay-tier.md) |
+| Token-bearer auth | [M16 §5.5](M16-tokens.md) |
+| E2E chat | [M23](M23-e2e-encryption.md) |
+| WebSocket | [X06](../cross-cutting/X06-websocket.md) |
+| Invite blobs | [Phase 1 CAP §13](../../CAPABILITY_CONTRACT.md) |
+| Web UI (mirror) | [M08](../../modules/M08-ui.md) |
+
+---
+
+## 12. Open questions
+
+1. **Flutter vs React Native vs native (Swift + Kotlin).** Choosing Flutter for shared codebase and Christof's stated preference for boring/durable stacks. Reconsider if Flutter's keychain support is shaky.
+2. **End-to-end encryption library in Flutter.** Need a libsodium binding that matches the Python side bit-exactly. `flutter_sodium` is well-maintained; verify on both platforms.
+3. **Background fetch reliability.** iOS throttles aggressively. We accept "best effort"; push is the real delivery mechanism.
+4. **Offline mode depth.** Mobile-only LLM (small Phi-3 / Gemma 2B) is Phase 3.
+5. **Web push for PWA.** Could the same flow target a PWA (no native app)? Yes, with FCM web push; documented but not built in Phase 2.
+6. **Family-share licence.** Christof might want to ship the iOS app to family members under his account; App Store policy permits this within Family Sharing.

docs/p2_p3/M23-e2e-encryption.md

@@ -0,0 +1,474 @@
+# M23 — End-to-End Encryption
+
+**Spec version:** v1.0 (Phase 2)
+**Depends on:** M01 (identity, key derivation), M07 (blobs, for prekey bundles), X02 (events, for prekey/session events), X04 (config), `pynacl`
+**Depended on by:** M10 chat (extended), M25 group chat, optionally M07 file encryption
+
+---
+
+## 1. Responsibility
+
+Provide end-to-end encryption between community members for:
+
+- **1:1 chat** (via M10 extension): every chat message encrypted with a per-sender ratchet
+- **Group chat** (M25): per-thread sender keys
+- **File envelopes** (M07 extension): chunks optionally wrapped in a per-recipient envelope
+
+The cryptographic design borrows from Signal but stays simpler:
+
+- **X3DH** for initial key agreement (one identity key + one signed prekey + one one-time prekey)
+- **Double Ratchet** for per-session forward-secrecy
+- **Sender keys** (Signal-style) for group threads
+- **Per-blob envelope** for file encryption
+
+This module owns the crypto primitives and session state. M10 and M25 own the message protocol that calls into M23.
+
+---
+
+## 2. File layout
+
+```
+hearthnet/crypto/
+├── __init__.py
+├── kem.py             # X25519 key agreement (X3DH)
+├── ratchet.py         # Double Ratchet, per-session
+├── sender_keys.py     # Group sender keys (M25 helper)
+├── envelope.py        # File envelope encryption (chunks)
+└── prekeys.py         # Prekey bundle storage and publication
+```
+
+The `hearthnet/crypto/` directory is a NEW top-level package in Phase 2.
+
+---
+
+## 3. Public API
+
+### 3.1 `kem.py` — X3DH
+
+```python
+# hearthnet/crypto/kem.py
+from dataclasses import dataclass
+
+@dataclass(frozen=True)
+class X25519KeyPair:
+    private: bytes        # 32 bytes
+    public:  bytes        # 32 bytes "x25519:<base64>"
+
+def x25519_generate() -> X25519KeyPair: ...
+def x25519_dh(our_priv: bytes, their_pub: bytes) -> bytes:
+    """Computes the shared secret. 32 bytes."""
+
+@dataclass(frozen=True)
+class PrekeyBundle:
+    """What a recipient publishes so senders can establish a session without them being online."""
+    node_id_full:        str
+    identity_pub:        bytes               # long-lived; derived from Ed25519 identity via SignedConversion
+    signed_prekey_pub:   bytes
+    signed_prekey_sig:   bytes               # Ed25519 signature over signed_prekey_pub by identity Ed25519 key
+    one_time_prekeys_pub: list[bytes]        # depleted on use
+    published_at:        int                 # unix seconds
+
+def derive_identity_x25519_from_ed25519(ed_kp: KeyPair) -> X25519KeyPair:
+    """Use the standard nacl conversion. Single x25519 identity key per device."""
+
+def build_prekey_bundle(
+    ed_kp: KeyPair,
+    *,
+    num_one_time: int = E2E_PREKEY_BUNDLE_SIZE,
+) -> tuple[PrekeyBundle, X25519KeyPair, list[X25519KeyPair]]:
+    """Returns (bundle, signed_prekey_full, one_time_prekeys_full).
+       Caller persists the private halves; publishes only the bundle."""
+
+def x3dh_initiator(
+    our_identity_x: X25519KeyPair,
+    our_ephemeral: X25519KeyPair,
+    their_bundle: PrekeyBundle,
+) -> tuple[bytes, dict]:
+    """Returns (shared_secret, session_init_message).
+       session_init_message includes our identity_pub + ephemeral_pub + used_otp_index."""
+
+def x3dh_responder(
+    our_identity_x: X25519KeyPair,
+    our_signed_prekey: X25519KeyPair,
+    our_one_time_prekey: X25519KeyPair | None,
+    their_identity_pub: bytes,
+    their_ephemeral_pub: bytes,
+) -> bytes:
+    """Returns shared_secret."""
+```
+
+### 3.2 `ratchet.py` — Double Ratchet
+
+```python
+# hearthnet/crypto/ratchet.py
+@dataclass
+class RatchetState:
+    """One session, one direction. There are two per session (send + receive)."""
+    root_key:      bytes
+    chain_key:     bytes
+    counter:       int
+    epoch:         int
+    skipped_messages: dict[tuple[int, int], bytes]  # (epoch, counter) → message_key
+    dh_keypair:    X25519KeyPair | None
+    remote_dh_pub: bytes | None
+
+@dataclass
+class RatchetSession:
+    """A bidirectional encrypted session between two NodeIDs."""
+    peer_node_id_full: str
+    send_state:    RatchetState
+    recv_state:    RatchetState
+
+    def is_established(self) -> bool: ...
+
+def init_session_initiator(shared_secret: bytes, peer_dh_pub: bytes) -> RatchetSession: ...
+def init_session_responder(shared_secret: bytes, our_dh_kp: X25519KeyPair) -> RatchetSession: ...
+
+@dataclass(frozen=True)
+class RatchetMessageHeader:
+    dh_pub:     bytes       # current sender's DH pub
+    epoch:      int
+    counter:    int
+
+def encrypt_message(
+    session: RatchetSession,
+    plaintext: bytes,
+    *,
+    aad: bytes = b"",
+) -> tuple[RatchetMessageHeader, bytes]:
+    """Returns (header, ciphertext). Mutates session.send_state."""
+
+def decrypt_message(
+    session: RatchetSession,
+    header: RatchetMessageHeader,
+    ciphertext: bytes,
+    *,
+    aad: bytes = b"",
+) -> bytes:
+    """Verifies + decrypts. Mutates session.recv_state.
+       Tolerates up to E2E_RATCHET_MAX_OUT_OF_ORDER out-of-order messages via skipped_messages."""
+
+class RatchetError(Exception):
+    """code in {
+       'session_not_established','decrypt_failed','out_of_order_too_far',
+       'message_too_old','aad_mismatch'}"""
+    code: str
+
+# Persistence: sessions are serialised via dataclasses_json into a SQLite table.
+```
+
+### 3.3 `sender_keys.py` — Group ratchet
+
+```python
+# hearthnet/crypto/sender_keys.py
+@dataclass
+class SenderKeyState:
+    """Per (group, sender) sender-key chain. Each sender broadcasts a chain key
+       to all group members at thread create / join."""
+    thread_id:       str
+    sender_node_id:  str
+    chain_key:       bytes
+    counter:         int
+    signature_keypair: tuple[bytes, bytes] | None    # ed25519, for message signing
+
+@dataclass
+class GroupSession:
+    """One per thread; holds all members' SenderKeyStates."""
+    thread_id:    str
+    sender_keys:  dict[str, SenderKeyState]    # sender_node_id → state
+
+def init_sender_key(thread_id: str, sender_node_id: str) -> SenderKeyState: ...
+def encrypt_for_group(session: GroupSession, sender_node_id: str, plaintext: bytes) -> tuple[dict, bytes]: ...
+def decrypt_for_group(session: GroupSession, header: dict, ciphertext: bytes) -> bytes: ...
+
+def serialise_sender_key_distribution(state: SenderKeyState) -> bytes:
+    """Serialise a sender key for sending to other group members.
+       MUST be sent inside a pairwise Double Ratchet session, not in cleartext."""
+
+def consume_sender_key_distribution(bytes_blob: bytes, session: GroupSession) -> None: ...
+```
+
+### 3.4 `envelope.py` — File envelope
+
+```python
+# hearthnet/crypto/envelope.py
+@dataclass(frozen=True)
+class FileEnvelopeHeader:
+    recipient_node_ids: list[str]
+    wrapped_keys:       dict[str, bytes]      # node_id_short → wrapped symmetric key
+    nonce:              bytes                  # 12 bytes
+    chunk_size_bytes:   int
+
+def encrypt_blob_for(
+    recipients: list[str],                    # NodeIDs full
+    plaintext: bytes,
+    sender_kp: KeyPair,
+    sessions_provider: Callable[[str], RatchetSession | None],
+) -> tuple[FileEnvelopeHeader, bytes]:
+    """1. Generate random 32-byte symmetric key
+    2. For each recipient: encrypt key with their ratchet session (one-shot use of next message key)
+    3. ChaCha20-Poly1305-encrypt plaintext with the symmetric key + nonce
+    4. Return (header, ciphertext)"""
+
+def decrypt_blob_for_self(
+    header: FileEnvelopeHeader,
+    ciphertext: bytes,
+    our_node_id_full: str,
+    sessions_provider: Callable[[str], RatchetSession | None],
+) -> bytes:
+    """1. Find our wrapped key in header.wrapped_keys
+    2. Decrypt symmetric key via sender's ratchet session
+    3. ChaCha20-Poly1305-decrypt"""
+```
+
+### 3.5 `prekeys.py` — Publication and consumption
+
+```python
+# hearthnet/crypto/prekeys.py
+class PrekeyStore:
+    """Persists this node's private prekey material and the bundles we've consumed for others."""
+
+    def __init__(self, db_path: Path):
+        ...
+
+    def publish_self(
+        self,
+        ed_kp: KeyPair,
+        event_log: EventLog,
+        *,
+        num_one_time: int = E2E_PREKEY_BUNDLE_SIZE,
+    ) -> None:
+        """1. Build PrekeyBundle (kem.build_prekey_bundle)
+        2. Persist private halves locally
+        3. Emit e2e.prekeys.published event with public halves"""
+
+    def get_peer_bundle(self, peer_node_id_full: str) -> PrekeyBundle | None:
+        """Look in local cache; if absent, fetch from peer via bus.
+        Returns one bundle including one consumable one-time prekey if available."""
+
+    def consume_one_time_prekey(self, our_otp_index: int) -> X25519KeyPair | None:
+        """Server-side: when someone uses one of our one-time prekeys, return + remove it."""
+
+    def refill_one_time_prekeys_if_low(self, ed_kp: KeyPair, event_log: EventLog) -> int:
+        """If fewer than E2E_PREKEY_BUNDLE_SIZE / 4 remain, publish a new bundle.
+        Returns count added."""
+```
+
+---
+
+## 4. Behaviour
+
+### 4.1 Session establishment lifecycle (1:1)
+
+```
+Alice wants to send Bob an encrypted message; no session exists.
+  ↓
+PrekeyStore.get_peer_bundle("ed25519:bob") → fetch from Bob's most recent e2e.prekeys.published event
+  ↓
+x3dh_initiator(alice_identity_x, alice_ephemeral, bob_bundle) → (shared_secret, init_msg)
+  ↓
+init_session_initiator(shared_secret, bob_signed_prekey_pub) → RatchetSession
+  ↓
+encrypt_message(session, plaintext) → (header, ciphertext)
+  ↓
+Alice sends: chat.message.sent event with data.body = {
+   "e2e": true,
+   "header": { x3dh_init: init_msg, ratchet_header: header },
+   "ciphertext": "<base64>"
+}
+  ↓
+Bob receives event.
+   x3dh_responder(...) → shared_secret
+   init_session_responder(...) → RatchetSession
+   decrypt_message(...) → plaintext
+   Emit e2e.session.established event so Alice can clean up retries
+  ↓
+Subsequent messages: just header + ciphertext (no x3dh_init).
+```
+
+### 4.2 Group session establishment
+
+```
+Thread creator emits chat.thread.created with members and an ed25519:thread_signing_root.
+  ↓
+Each member generates a SenderKeyState for themselves in this thread.
+  ↓
+Each member, in a pairwise loop, sends their sender key distribution to each other member
+   inside their 1:1 ratchet sessions (so non-thread-members never see it).
+  ↓
+Once everyone has everyone's sender keys, encrypt/decrypt happens with sender keys
+   (chain ratchet only; no DH ratchet on the group session itself).
+```
+
+When a member is added later, the inviter must re-distribute all existing senders' current chain states to the new member (rewinds to the message they should start being able to read — usually the current state, not history).
+
+When a member is removed, existing sender keys are still known to them. **All members must rotate their sender keys** to achieve forward secrecy after removal. UI prompts this.
+
+### 4.3 Out-of-order messages
+
+Up to `E2E_RATCHET_MAX_OUT_OF_ORDER` (32) skipped message keys are cached per session. Beyond that, `out_of_order_too_far` is raised; the message is dropped and the sender notified (out-of-band) that they should rekey.
+
+### 4.4 Rekeying
+
+After `E2E_RATCHET_REKEY_AFTER_MESSAGES` messages on the same DH ratchet, the next message includes a new DH ephemeral. Standard Double Ratchet behaviour. Transparent to users.
+
+### 4.5 Session loss recovery
+
+If a node's session state is lost (disk corruption, fresh install with same keys), the peer doesn't know — messages will fail to decrypt. Recovery flow:
+
+1. Decrypting node returns `e2e_decrypt_failed` via pubsub
+2. Sending node sees this and re-initiates X3DH
+3. New session replaces old; resends recent messages
+
+UI shows "session was reset" so users know context might have been lost.
+
+### 4.6 Identity X25519 derivation
+
+We derive a per-device X25519 identity key from the Ed25519 identity key, using libsodium's `crypto_sign_ed25519_pk_to_curve25519`. This way:
+
+- Only one identity key to maintain
+- Anyone with the public Ed25519 (in the community manifest) can derive the X25519 pub
+- Signed prekey signatures use the Ed25519 key (already established as device identity)
+
+### 4.7 Prekey publication
+
+Each node publishes a fresh `e2e.prekeys.published` event on startup if their last one is > 24h old. The event contains:
+
+- `identity_pubkey` (X25519 form)
+- `signed_prekey` (with Ed25519 signature)
+- `one_time_prekeys[]` (up to `E2E_PREKEY_BUNDLE_SIZE` = 20)
+
+Consumers find a peer's bundle by reading their latest `e2e.prekeys.published` event from the log.
+
+### 4.8 What is NOT E2E
+
+Even with M23 active:
+- Event envelope (sender, recipient, lamport, event_type, wall_clock) is cleartext within the community
+- Signatures over events remain valid for community-level audit
+- Message *metadata* leaks to community members (who talked to whom and when), just not content
+
+This is intentional: communities are trust roots; complete anonymity within a community is not a goal.
+
+### 4.9 File envelope
+
+For file blobs, `encrypt_blob_for(recipients, plaintext, ...)` produces a single ciphertext, with a small per-recipient header. Senders pick recipients explicitly (e.g. group thread members for an attachment). Bystanders cannot decrypt even if they fetch the blob via M07 `file.read`.
+
+The blob's CID is the hash of the **ciphertext**, so the same plaintext sent to different recipient sets has different CIDs. Costs more storage; needed for security.
+
+---
+
+## 5. Persistence
+
+### 5.1 Sessions table
+
+```sql
+CREATE TABLE ratchet_sessions (
+  peer_node_id_full TEXT PRIMARY KEY,
+  session_blob      BLOB NOT NULL,        -- serialised RatchetSession
+  established_at    INTEGER NOT NULL,
+  last_used         INTEGER NOT NULL
+);
+```
+
+### 5.2 Group sessions table
+
+```sql
+CREATE TABLE group_sessions (
+  thread_id        TEXT PRIMARY KEY,
+  session_blob     BLOB NOT NULL,
+  updated_at       INTEGER NOT NULL
+);
+```
+
+### 5.3 Prekey private halves
+
+```sql
+CREATE TABLE prekey_private (
+  kind TEXT NOT NULL,                     -- 'identity'|'signed_prekey'|'one_time'
+  index_or_id TEXT NOT NULL,              -- '0' for identity; 'spk_v1' for signed; otp index for OTPs
+  private_key BLOB NOT NULL,
+  consumed_at INTEGER,                    -- only set for one-time, when used
+  PRIMARY KEY (kind, index_or_id)
+);
+```
+
+Files locked at 0600. Backed up nightly via `hearthnet export` (encrypted with user passphrase).
+
+---
+
+## 6. Errors
+
+`RatchetError` codes (M23-internal):
+- `session_not_established`
+- `decrypt_failed`
+- `out_of_order_too_far`
+- `message_too_old`
+- `aad_mismatch`
+
+Wire mapping per [CAP2 §9](../CAPABILITY_CONTRACT_v2.md):
+- `e2e_session_missing` ← `session_not_established`
+- `e2e_decrypt_failed` ← `decrypt_failed`, `aad_mismatch`
+- `ratchet_out_of_order` ← `out_of_order_too_far`
+
+---
+
+## 7. Configuration
+
+```python
+config.e2e.enabled                       = True
+config.e2e.chat_default_enabled          = True       # new 1:1 chats default to E2E
+config.e2e.group_default_enabled         = True
+config.e2e.file_default_enabled          = False      # opt-in per blob
+config.e2e.prekey_refill_count           = E2E_PREKEY_BUNDLE_SIZE
+config.e2e.rekey_after_messages          = E2E_RATCHET_REKEY_AFTER_MESSAGES
+config.e2e.max_out_of_order              = E2E_RATCHET_MAX_OUT_OF_ORDER
+```
+
+---
+
+## 8. Tests
+
+### Unit
+- `test_x25519_dh_symmetric`
+- `test_x3dh_initiator_responder_agree`
+- `test_ratchet_encrypt_decrypt_roundtrip`
+- `test_ratchet_out_of_order_within_window`
+- `test_ratchet_out_of_order_too_far_rejected`
+- `test_rekey_after_n_messages`
+- `test_group_sender_key_distribution_pairwise_only`
+- `test_blob_envelope_recipient_only_can_decrypt`
+
+### Integration
+- `test_two_node_first_message_x3dh_session_persists`
+- `test_session_recovery_after_disk_wipe`
+- `test_group_add_member_can_decrypt_subsequent`
+- `test_group_remove_member_cannot_decrypt_after_rotation`
+- `test_file_envelope_2_recipients`
+
+### Adversarial
+- `test_replay_old_ratchet_message_rejected`
+- `test_modified_ciphertext_decrypt_fails`
+- `test_one_time_prekey_consumed_once`
+
+---
+
+## 9. Cross-references
+
+| What | Where |
+|------|-------|
+| `e2e.*` events | [CAP2 §7](../CAPABILITY_CONTRACT_v2.md) |
+| Encrypted chat body envelope | [CAP2 §1.1, §7.2 chat.message.sent](../CAPABILITY_CONTRACT_v2.md) |
+| Chat service hook | [M10 ext](../../modules/M10-chat.md) — Phase 2 extension |
+| Group chat | [M25](M25-group-chat.md) |
+| File envelope use | [M07 ext](../../modules/M07-file-blobs.md) |
+| Identity key conversion | [M01](../../modules/M01-identity.md) |
+
+---
+
+## 10. Open questions
+
+1. **Post-quantum readiness.** X25519 + ChaCha20-Poly1305 is not PQ-safe. Hybrid (X25519 + ML-KEM-768) is Phase 3.
+2. **Verification of session identity.** Signal does safety numbers; HearthNet can do the same. UI ergonomics deferred.
+3. **Multi-device per identity.** If a user has anchor + mobile + laptop, do they share keys or have separate ones? Currently separate (each device is a separate NodeID; group threads include all of them). Could unify with a "linked devices" Phase 3 feature.
+4. **Forward secrecy on group membership change.** Current spec asks members to rotate sender keys on removal. UX of forcing this needs design.
+5. **Cryptographic auditing.** This module should be reviewed by a real cryptographer before going to civil-defence pilots. Listed in `THREAT_MODEL_v2.md`.

docs/p2_p3/M24-rerank.md

@@ -0,0 +1,227 @@
+# M24 — Reranking Service
+
+**Spec version:** v2.0
+**Depends on:** [M03 Capability Bus](../../modules/M03-capability-bus.md), [M01 Identity](../../modules/M01-identity.md), [X03 Observability](../../cross-cutting/X03-observability.md)
+**Depended on by:** [M05 RAG](../../modules/M05-rag.md) (extension), [M06 Marketplace](../../modules/M06-marketplace.md) (extension)
+
+---
+
+## 1. Responsibility
+
+Re-score a candidate list of documents against a query using a cross-encoder, producing a higher-precision ordering than dense retrieval alone can deliver.
+
+The capability is intentionally narrow: take query + N short docs, return ranked list. The service does **not** retrieve documents, does **not** fetch from blobs, does **not** know about corpora. Callers (typically `rag.query` and `market.search`) do retrieval first, then ask the reranker to refine the top 100.
+
+This is the smallest service in Phase 2 — one model, one method, no streaming — and the most underrated. Adding it to the RAG pipeline lifts answer quality more than any other Phase 2 module.
+
+---
+
+## 2. File layout
+
+```
+hearthnet/services/rerank/
+├── __init__.py
+├── service.py            # RerankService — capability registration
+├── selection.py          # Picks a backend; loads on demand
+└── backends/
+    ├── base.py           # RerankBackend ABC
+    ├── bge_reranker.py   # BGE-reranker-v2-m3 (default)
+    └── cross_encoder.py  # Sentence-transformers fallback
+```
+
+---
+
+## 3. Public API
+
+### 3.1 `RerankBackend` (ABC)
+
+```python
+class RerankBackend(Protocol):
+    name: str            # e.g. "BAAI/bge-reranker-v2-m3"
+    max_doc_chars: int   # truncate longer docs
+
+    async def score(self, query: str, documents: list[str]) -> list[float]:
+        """Return one score per document, same length and order."""
+        ...
+
+    async def health(self) -> dict[str, Any]:
+        """Return {"ok": bool, "loaded": bool, "model_id": ..., ...}."""
+        ...
+```
+
+### 3.2 `BgeRerankerBackend`
+
+```python
+class BgeRerankerBackend:
+    def __init__(self, model_id: str = "BAAI/bge-reranker-v2-m3", device: str = "auto", max_batch: int = 32):
+        ...
+
+    async def score(self, query: str, documents: list[str]) -> list[float]:
+        # Tokenise (query, doc) pairs in batches of max_batch
+        # Forward pass; pooled logit becomes the score
+        # Higher = more relevant
+        ...
+```
+
+### 3.3 `RerankService`
+
+```python
+class RerankService:
+    """Bus-facing facade.  Picks backend, enforces RERANK_MAX_DOCS, emits metrics."""
+
+    def __init__(self, bus: CapabilityBus, settings: RerankSettings, observability: Observability):
+        ...
+
+    async def start(self) -> None:
+        # Register `rerank.text@1.0` on the bus
+        ...
+
+    async def rerank_text(self, body: RerankRequest) -> RerankResponse:
+        # 1. Validate len(documents) <= RERANK_MAX_DOCS (else bad_request)
+        # 2. Pick backend per `body.params.model` or default
+        # 3. Truncate docs to backend.max_doc_chars
+        # 4. Call backend.score
+        # 5. Sort descending, take top_k (or all)
+        # 6. Emit `rerank.latency_ms` metric
+        ...
+```
+
+### 3.4 Request / response dataclasses
+
+```python
+@dataclass
+class RerankDoc:
+    id: str
+    text: str
+
+@dataclass
+class RerankRequest:
+    query:      str
+    documents:  list[RerankDoc]
+    top_k:      int = 10
+    params:     dict[str, Any] = field(default_factory=dict)   # {"model": "..."}
+
+@dataclass
+class RerankedDoc:
+    id:    str
+    score: float
+
+@dataclass
+class RerankResponse:
+    ranked: list[RerankedDoc]
+    meta:   dict[str, Any]
+```
+
+---
+
+## 4. Behaviour
+
+### 4.1 Backend selection
+
+`params.model` is matched against installed backends (key = HuggingFace model id). Default is `BAAI/bge-reranker-v2-m3` because it handles ≥100 languages including German and Latin (relevant for the OCR'd historical doc corpus).
+
+If `params.model` is supplied but unknown → return `bad_request` with the list of installed backends.
+
+### 4.2 Cold start
+
+Backend is loaded lazily on first call. First call latency budget: ≤ 60s on the RTX 5090 (model ~2 GB on disk). Subsequent calls: ≤ 200ms for 50 docs at ~512 chars each.
+
+The service publishes `model_loaded` and `model_loading` health states; `rerank.text` calls during loading wait up to `RERANK_LOAD_TIMEOUT_SECONDS` (default 60) then return `unavailable`.
+
+### 4.3 Score semantics
+
+Scores are **raw logits**, not normalised probabilities. They are comparable within a single call but not across calls or backends. Callers MUST NOT compare a 0.91 score from BGE to a 0.91 from cross-encoder/ms-marco — different scales.
+
+### 4.4 Truncation
+
+Documents longer than `backend.max_doc_chars` (default 2048) are truncated. The service logs `rerank.docs_truncated` counter. Truncation is from the right; callers who care about specific spans should pre-summarise or chunk before passing in.
+
+### 4.5 No streaming
+
+`rerank.text@1.0` is non-streaming. Even at 100 docs the latency is well under 1s on GPU. If a Phase-3 use case demands streaming (e.g. 1000-doc reranks for academic search), introduce `rerank.text@2.0` with `progress` frames; do not retrofit v1.
+
+### 4.6 Integration with RAG (M05 extension)
+
+`rag.query` in Phase 2 grows an internal pipeline:
+
+```
+1. Hybrid retrieval (dense + BM25) → top 100 candidates
+2. Optional call to rerank.text@1.0 → top 10
+3. Pass top 10 to llm.chat as context
+```
+
+The hop to `rerank.text` is done via the bus, not via direct import. This keeps the policy ("which model?", "is reranking available?") in the service and out of the RAG core.
+
+If `rerank.text@1.0` is unavailable in the local mesh, RAG falls back to dense scores alone and logs `rag.rerank_skipped` counter (not an error).
+
+### 4.7 Integration with Marketplace (M06 extension)
+
+`market.search` follows the same pattern when the query is natural-language. For tag-based queries it skips reranking.
+
+---
+
+## 5. Errors
+
+| Code | Cause |
+|------|-------|
+| `bad_request` | `len(documents) > RERANK_MAX_DOCS`, empty query, malformed payload |
+| `unavailable` | Backend loading or hardware unavailable |
+| `model_not_found` | Requested `params.model` is not installed |
+
+`unavailable` is retryable; the other two are not.
+
+---
+
+## 6. Configuration
+
+```toml
+[services.rerank]
+enabled                  = true
+default_model            = "BAAI/bge-reranker-v2-m3"
+device                   = "auto"             # "auto" | "cuda" | "cpu"
+max_batch                = 32
+max_doc_chars            = 2048
+load_timeout_seconds     = 60
+trust_required           = "member"
+```
+
+Behind a feature flag: when `enabled=false`, the capability simply does not register and RAG falls back to dense-only.
+
+---
+
+## 7. Tests
+
+### 7.1 Unit
+- Sorting: scores `[0.1, 0.9, 0.5]` produce ranked order `[1, 2, 0]`
+- Truncation: 4000-char doc gets truncated to 2048 before scoring
+- `top_k` honoured; returns at most `top_k` results
+- Bad request when `documents=[]` or `len > RERANK_MAX_DOCS`
+
+### 7.2 Integration
+- End-to-end: `rag.query` with reranking vs without, on the niederrhein-emergency corpus, asserts at least one expected document moves into top 3 with rerank that wasn't there without
+- Cross-language: German query, mixed German/English candidates, BGE reranker should put the German candidate first when relevance is equal
+
+### 7.3 Performance
+- 100 docs @ 1024 chars: p50 ≤ 300ms on RTX 5090; p95 ≤ 600ms
+- CPU fallback (no GPU): p50 ≤ 4s for 50 docs (acceptable; degraded)
+
+### 7.4 Failure-mode
+- Backend crash mid-call: caller receives `unavailable`; service self-heals on next call
+- Concurrent calls: 20 parallel reranks should not deadlock; backend serialises behind a single semaphore
+
+---
+
+## 8. Cross-references
+
+- Capability spec: [CAPABILITY_CONTRACT_v2 §4.15](../CAPABILITY_CONTRACT_v2.md#415-reranktext10)
+- Used by: M05 RAG extension, M06 Marketplace extension
+- Observability: emits `rerank.calls_total`, `rerank.latency_ms`, `rerank.docs_truncated`, `rerank.errors_total{code}`
+
+---
+
+## 9. Open questions
+
+1. **Reciprocal rank fusion** with dense scores as the alternative when rerank is unavailable — worth implementing in M05 as the fallback path?
+2. **ColBERT-style late interaction** — heavier model, higher quality. Worth a second backend, or wait for Phase 3 to evaluate?
+3. **Reranker for code/diff content** — different model family (e.g. `BAAI/bge-code-reranker`). Should `params.model` selection be auto-inferred from query/doc content?
+4. **Caching** — query+doc-pair hash → score, evict LRU. Worth it for repeated queries in chat-driven RAG sessions, or premature optimisation?

docs/p2_p3/M25-group-chat.md

@@ -0,0 +1,293 @@
+# M25 — Group Chat
+
+**Spec version:** v2.0
+**Depends on:** [M10 Chat 1:1](../../modules/M10-chat.md), [M23 E2E Encryption](M23-e2e-encryption.md), [M16 Capability Tokens](M16-tokens.md), [M03 Capability Bus](../../modules/M03-capability-bus.md), [X02 Event Log](../../cross-cutting/X02-events.md)
+**Depended on by:** UI (web + M22 mobile), [M14 Federation](M14-federation.md) (cross-community threads)
+
+---
+
+## 1. Responsibility
+
+Multi-party threaded conversations with the same guarantees as 1:1 chat: end-to-end encryption (optional but default on), event-log-anchored history, no central server required, members can come and go.
+
+A thread is a long-lived object identified by a ULID. It has an authoritative member list maintained in the event log, an encryption "group session" (M23 sender keys), and message history. Threads do not currently support reactions, replies, threading-within-thread, or rich content — those are explicit non-goals for Phase 2 and may arrive in Phase 3 once usage informs design.
+
+Group threads are the substrate the **Nachbarschaftshilfe** use case wants: a Sankt-Martins-Comité planning thread, a "Wer hat Werkzeug?" workshop thread, a household coordination thread between Christof, Jana, and grandparents.
+
+---
+
+## 2. File layout
+
+```
+hearthnet/services/chat/
+├── thread_service.py    # ThreadService — capability registration & dispatch
+├── thread_views.py      # Materialised views: thread list, member list, history
+├── thread_store.py      # Read-only projections; not the source of truth
+├── group_session.py     # Wraps M23 sender keys for a thread
+└── moderation.py        # Phase-2: remove-member, archive — minimal
+```
+
+---
+
+## 3. Public API
+
+### 3.1 Dataclasses
+
+```python
+@dataclass(frozen=True)
+class Thread:
+    thread_id:        ThreadID
+    name:             str
+    created_at:       datetime
+    created_by:       NodeID
+    members:          frozenset[NodeID]
+    e2e_enabled:      bool
+    ratchet_root:     str | None      # x25519 pubkey of group session root, None if cleartext
+    archived:         bool
+
+@dataclass(frozen=True)
+class ThreadMessage:
+    event_id:    EventID
+    thread_id:   ThreadID
+    client_id:   ClientID
+    sender:      NodeID
+    sent_at:     datetime
+    body:        str | None           # cleartext if e2e_enabled=False
+    encrypted:   EncryptedPayload | None
+    attachments: list[Attachment]
+    delivered_to: frozenset[NodeID]   # tracked via chat.thread.message.delivered events
+```
+
+### 3.2 `ThreadService`
+
+```python
+class ThreadService:
+    """Capability handlers for chat.thread.*"""
+
+    def __init__(
+        self,
+        bus:                CapabilityBus,
+        event_log:          EventLog,
+        identity:           Identity,
+        encryption:         EncryptionService,   # M23
+        view_store:         ThreadViewStore,
+        observability:      Observability,
+    ): ...
+
+    async def start(self) -> None:
+        # Registers: chat.thread.create, .send, .history, .leave, .add_member, .archive
+        ...
+
+    # --- handlers (selected) ---
+    async def create(self, body: CreateThreadBody) -> CreateThreadResult: ...
+    async def send(self, body: SendThreadBody)   -> SendThreadResult: ...
+    async def history(self, body: HistoryBody)   -> HistoryResult: ...
+    async def leave(self, body: LeaveBody)       -> LeaveResult: ...
+    async def add_member(self, body: AddMemberBody) -> AddMemberResult: ...
+    async def archive(self, body: ArchiveBody)   -> ArchiveResult: ...
+```
+
+### 3.3 `ThreadViewStore`
+
+```python
+class ThreadViewStore:
+    """Read model.  Backed by SQLite; rebuilt from the event log on cold start."""
+
+    def list_for_member(self, node_id: NodeID) -> list[Thread]: ...
+    def get_thread(self, thread_id: ThreadID) -> Thread | None: ...
+    def get_messages(self, thread_id: ThreadID, since_lamport: int = 0, limit: int = 200) -> list[ThreadMessage]: ...
+    def members_of(self, thread_id: ThreadID) -> frozenset[NodeID]: ...
+
+    # Internal — subscribed to the event log:
+    async def apply(self, event: Event) -> None: ...
+```
+
+### 3.4 `GroupSession`
+
+Thin wrapper around M23 sender keys; one per thread.
+
+```python
+class GroupSession:
+    def __init__(self, thread_id: ThreadID, ratchet: SenderKeyRatchet): ...
+
+    def encrypt(self, plaintext: bytes) -> EncryptedPayload: ...
+    def decrypt(self, sender: NodeID, payload: EncryptedPayload) -> bytes: ...
+    def rekey(self) -> None: ...
+    def add_member(self, new_member: NodeID, their_identity_pubkey: bytes) -> None: ...
+    def remove_member(self, leaving_member: NodeID) -> None: ...
+```
+
+---
+
+## 4. Behaviour
+
+### 4.1 Thread creation
+
+`chat.thread.create@1.0` flow:
+
+1. Caller emits `chat.thread.created` event into the event log with:
+   - `thread_id` (newly minted ULID)
+   - initial member list
+   - `e2e_enabled` flag
+   - if e2e_enabled: a freshly generated `ratchet_root_pubkey` and a per-member encrypted **sender key** payload (see [M23 §6.3](M23-e2e-encryption.md))
+2. Each member's node sees the event arrive, decrypts the sender key payload addressed to itself, and constructs the GroupSession.
+3. The view store materialises a new Thread row.
+
+If any member is offline at creation, they will receive the event when they next sync. Their GroupSession constructs lazily on first decrypt.
+
+### 4.2 Sending
+
+`chat.thread.send@1.0` flow:
+
+1. Verify caller is in `Thread.members`.
+2. If `e2e_enabled`, encrypt body with the GroupSession's current sender key. The ciphertext is opaque to the event log — even other community members who are not in the thread cannot read it.
+3. Emit `chat.thread.message.sent` event.
+4. The event reaches all members (regular event-log propagation, no thread-specific transport).
+5. Each member's GroupSession decrypts; the message appears in their UI.
+
+### 4.3 Membership changes
+
+#### Adding a member
+
+1. Any existing member can issue `chat.thread.add_member` (Phase 2; later phases may add policies like "only admin can add").
+2. The caller's GroupSession is **rekeyed**: a new sender key is generated, encrypted under each existing member's pubkey and the new member's pubkey, and emitted in the `chat.thread.member.added` event.
+3. The new member cannot read **prior** messages — they joined at the new epoch. (This is by design and standard for sender-key group encryption: forward-secrecy is preserved.) Old messages remain encrypted with the old sender key, which the new member never sees.
+
+#### Removing a member
+
+`chat.thread.remove_member` (or self-leave via `chat.thread.leave`):
+
+1. Emit `chat.thread.member.removed`.
+2. The remaining members rekey the GroupSession (similar to add but excluding the removed member). New messages are not readable by the removed member.
+3. The removed member's UI marks the thread as "you left" and stops decrypting incoming messages. Their event log still contains old messages they can still read; they just can't read new ones.
+
+### 4.4 History
+
+`chat.thread.history@1.0`:
+
+- **Self-only** capability (you can only ask for history of threads you're a member of).
+- Returns from local view store. No cross-node query needed — every member already has the events.
+- Pagination by `since_lamport` + `limit`. Messages return in **logical (Lamport) order**, not wall-clock order, to match what other members will see.
+
+### 4.5 Read-receipts / delivery tracking
+
+Each member's node emits `chat.thread.message.delivered` (lightweight, no payload beyond `event_id` reference) when they materialise a message. UI shows "delivered to 4/5" by counting these events. Optional — `policy.chat.delivery_receipts_enabled` (default true) controls whether they're emitted.
+
+### 4.6 Archiving
+
+`chat.thread.archived` is a soft state. Archived threads are hidden from the default thread list, no longer rekey on membership change, and no longer accept sends. Members can still read history. An archived thread can be unarchived by any member.
+
+There is no "delete thread". Events are immutable. A thread that is archived and whose messages are all expired (via X02 retention policies) becomes effectively gone.
+
+### 4.7 Attachments
+
+`attachments` carry `cid` (blob CID) and `name`. The blob itself is uploaded via `file.put` separately. Members of the thread are by definition authorised to fetch the blob — the bus enforces this via a capability token issued automatically when sending an attachment in an E2E thread:
+
+```
+On send-with-attachment:
+  1. Service issues a short-lived (24h) token via M16 with:
+       scope.capabilities = ["file.fetch@1.0"]
+       scope.params_constraints.cid = [attachment.cid]
+       audience = thread.members  (excluding self)
+  2. Token is included in the encrypted message body.
+  3. Recipients use the token when fetching the blob from whichever node holds it.
+```
+
+This avoids the "file is restricted but everyone in the thread should access it" coordination problem.
+
+### 4.8 Federation of threads
+
+A thread MAY include members from federated communities. Mechanics:
+
+- The thread's `community_id` (the one in event headers) is the *creator's* community.
+- Members from federated communities subscribe to the thread's events via the standard federation event-bridge (see [M14 §6](M14-federation.md)).
+- Federated members are full participants — they can send, leave, be removed — provided the federation manifest grants `chat.thread.send@1.0`.
+- The view store on a federated member's node carries the foreign-community thread alongside their local-community threads, distinguished by `Thread.community_id` field for UI purposes.
+
+If federation is revoked, foreign members are silently removed from the thread on the next rekey.
+
+### 4.9 Throughput and limits
+
+- `THREAD_MAX_MEMBERS = 200` (Phase-2 conservative; larger groups should be a different module).
+- `THREAD_MAX_MESSAGE_BYTES = 64 * 1024` for the cleartext body.
+- `THREAD_RATE_LIMIT_PER_SENDER_PER_MINUTE = 60` (anti-spam, enforced by ThreadService).
+- Beyond these → `bad_request` or `too_many_requests`.
+
+---
+
+## 5. Errors
+
+| Code | Cause |
+|------|-------|
+| `bad_request` | Empty member list, malformed body, member list contains caller twice |
+| `unauthorized` | Caller not a member of the thread (for send/history/leave/add) |
+| `not_found` | `thread_id` unknown |
+| `e2e_session_missing` | Caller has no GroupSession yet (sender keys not received) |
+| `e2e_decrypt_failed` | Local key state corrupt; UI should prompt for a manual rekey |
+| `too_many_requests` | Rate limit exceeded |
+| `policy_violation` | E.g. trying to add member outside of federation scope |
+
+---
+
+## 6. Configuration
+
+```toml
+[services.chat.thread]
+enabled                          = true
+max_members                      = 200
+max_message_bytes                = 65536
+rate_limit_per_sender_per_minute = 60
+delivery_receipts_enabled        = true
+allow_federated_members          = true
+
+[services.chat.thread.archival]
+auto_archive_after_days_idle     = 0   # 0 = never auto-archive
+```
+
+---
+
+## 7. Tests
+
+### 7.1 Unit
+- Create thread with 3 members; verify GroupSession is constructable by each member from the `chat.thread.created` payload
+- Send + decrypt round-trip
+- Add member; old messages remain undecryptable for them, new ones work
+- Remove member; their session can't decrypt new messages
+- Self-leave; cleanup is graceful (no orphan state)
+- History pagination: 1000 messages, fetch 200 + 200 + 200... covers all
+
+### 7.2 Integration
+- Three nodes on one LAN form a thread; messages propagate via gossip
+- Same with one member partitioned; their replay on reconnect works
+- E2E on/off threads coexist; switching one to the other is not supported (must create a new thread)
+- Federation: a federated peer's node receives the `chat.thread.created` event via the bridge and constructs a working GroupSession
+
+### 7.3 Adversarial
+- A non-member tries to call `chat.thread.send` → `unauthorized`
+- A non-member subscribes to `chat.thread.message.<id>` pubsub: receives encrypted blobs they can't decrypt (no information leak beyond traffic patterns and member list)
+- Replay: replaying an old `chat.thread.message.sent` event by IP-level adversary is rejected by per-message nonce in the E2E header
+- Rekey storm: 100 sequential add/remove operations finish within 30s on the dev rig; no deadlock
+
+### 7.4 Performance
+- 50-member thread, 1 msg/s: p95 deliver-to-decrypt latency < 500ms on LAN
+- History fetch of 10,000 messages: < 2s on SSD
+
+---
+
+## 8. Cross-references
+
+- Capability spec: [CAPABILITY_CONTRACT_v2 §4.16–4.19](../CAPABILITY_CONTRACT_v2.md)
+- Encryption primitives: [M23 §6 sender keys](M23-e2e-encryption.md)
+- Event types: [CAPABILITY_CONTRACT_v2 §7.1](../CAPABILITY_CONTRACT_v2.md#71-new-event-types)
+- Federation: [M14 §6](M14-federation.md)
+
+---
+
+## 9. Open questions
+
+1. **Reactions / replies / rich content** — explicitly out of Phase 2. Worth a survey of community use before designing. (Likely Phase 3 add-on, gated on "are people actually asking for it?")
+2. **Per-thread retention policy** — currently inherits the community-wide retention. Different threads might want different policies (planning thread = 30 days, household chat = forever).
+3. **Read-only threads** (announcements) — pseudo-thread where only one member can send. Worth a flag or worth a dedicated capability?
+4. **Thread search** — could plug into `rag.*`. Indexing of decrypted message text would be opt-in per thread; raises privacy concerns.
+5. **Cross-thread mentions / linking** — e.g. "see thread X for context". Probably as a UI affordance (markdown link), not a protocol feature.
+6. **Disappearing messages** — Signal-style auto-expiry per-thread. Useful for sensitive coordination; adds complexity. Phase 3 candidate.

docs/p2_p3/M26-distributed-inference.md

@@ -0,0 +1,325 @@
+# M26 — Distributed Inference
+
+**Spec version:** v3.0 — *experimental*
+**Depends on:** [X08 Tensor Transport](../cross-cutting/X08-tensor-transport.md), [X06 WebSocket](../../phase-2/cross-cutting/X06-websocket.md), [M04 LLM](../../modules/M04-llm.md), [M16 Tokens](../../phase-2/modules/M16-tokens.md), [X03 Observability](../../cross-cutting/X03-observability.md)
+**Depended on by:** Optional `experimental.distributed_llm.chat` backend in M04
+
+---
+
+## 1. Responsibility
+
+Run a single LLM forward pass across multiple machines on the same LAN (or, in extremely careful setups, a single federation). Take a 7B model that doesn't fit on any one anchor's GPU and split it: anchor A holds layers 0–7, anchor B holds 8–15, anchor C holds 16–23, etc. A request orchestrator chains them and streams tokens back to the user.
+
+This is a **research module**. It exists for two reasons:
+
+1. **Resilience.** When a community's biggest GPU breaks, the next-biggest fleet of GPUs can still serve mid-sized models cooperatively.
+2. **Reach.** A community of three households, each with consumer hardware, can collectively run a model none of them could run alone.
+
+It is explicitly **not** for serving production user-facing LLM traffic at scale. The latency is worse than local inference (typically 2–4× per token), the orchestration is fragile (one shard offline = retry the whole pipeline), and the GPU memory savings come at significant complexity cost. Communities should default to local inference; this module exists for the cases where local isn't enough.
+
+---
+
+## 2. Non-goals (loud and clear)
+
+- **Large models.** 70B-class models are out of scope. The math says you'd need ten 24 GB GPUs to host one, which is the wrong problem for a neighbourhood mesh to solve.
+- **Cross-WAN sharding.** Inference across the public internet is uneconomical (latency, bandwidth). Limit to same LAN or same-VPN federation.
+- **Heterogeneous shards across model versions.** All shards in a pipeline must serve the **exact same model and weights checksum**. No partial-model recovery.
+- **Replacing local inference.** When `policy.research.enable = false`, this module is inert.
+
+---
+
+## 3. File layout
+
+```
+hearthnet/distributed_inference/
+├── __init__.py
+├── shard.py              # Shard, ShardDescriptor, ShardServer
+├── pipeline.py           # Pipeline, PipelineOrchestrator
+├── routing.py            # Picks a set of shards that cover [0..N] layers
+├── health.py             # Heartbeats, failover detection
+└── backends/
+    ├── base.py
+    ├── petals_like.py    # uses bigscience/petals client/server primitives
+    └── small_model_layered.py  # custom impl for small models (≤ 3B)
+```
+
+---
+
+## 4. Public API
+
+### 4.1 Dataclasses
+
+```python
+@dataclass(frozen=True)
+class ShardDescriptor:
+    shard_id:        ShardID         # "<model_id>:<lo>-<hi>"
+    model_id:        str             # HF model id
+    weights_sha256:  str             # full model weights hash; shards must match
+    layer_range:     tuple[int,int]  # inclusive
+    vram_required_mb: int
+    max_concurrent_streams: int
+    host:            NodeID
+    endpoint:        Endpoint        # ws://...
+    advertised_at:   datetime
+
+@dataclass
+class Pipeline:
+    pipeline_id:     str
+    model_id:        str
+    weights_sha256:  str
+    total_layers:    int
+    ordered_shards:  list[ShardDescriptor]
+    established_at:  datetime
+
+@dataclass
+class ShardHealth:
+    shard_id:        ShardID
+    online:          bool
+    last_seen:       datetime
+    p95_latency_ms:  float
+    queue_depth:     int
+```
+
+### 4.2 `ShardServer`
+
+```python
+class ShardServer:
+    """Hosts one contiguous shard.  Loaded on demand; lazy-evictable under memory pressure."""
+
+    def __init__(self, descriptor: ShardDescriptor, model_loader: ModelLoader, settings: ShardSettings): ...
+
+    async def start(self) -> None:
+        # Load weights for the layer range; register `experimental.distributed_llm.shard.serve` on the bus
+        ...
+
+    async def forward(self, activations_in: TensorChunkStream) -> TensorChunkStream:
+        """The hot path.  Receives activations, runs layers, emits activations."""
+        ...
+
+    async def health(self) -> ShardHealth: ...
+
+    async def evict(self) -> None:
+        """Free VRAM; triggered by host memory manager."""
+        ...
+```
+
+### 4.3 `PipelineOrchestrator`
+
+```python
+class PipelineOrchestrator:
+    """
+    Chooses shards to cover the model's layers, opens streams to each, and
+    pumps activations through them in order.  Handles failover.
+    """
+
+    def __init__(
+        self,
+        bus:           CapabilityBus,
+        router:        ShardRouter,
+        health:        ShardHealthTracker,
+        observability: Observability,
+    ): ...
+
+    async def chat(
+        self,
+        request:  LlmChatRequest,
+        params:   DistributedChatParams,
+    ) -> AsyncIterator[StreamFrame]:
+        # 1. Resolve a Pipeline covering all layers of the target model
+        # 2. Open WS streams to each shard via X08 tensor transport
+        # 3. For each token step:
+        #    embedding → shard 0 → shard 1 → ... → shard N → token sample
+        # 4. Yield `token_delta` frames; emit `shard_status` and `shard_failover` diagnostics
+        # 5. On any shard failure, attempt re-routing once; if that fails and
+        #    `params.fallback_to_local`, fall back to local inference and emit a
+        #    `pipeline_aborted` frame
+        ...
+```
+
+### 4.4 `ShardRouter`
+
+```python
+class ShardRouter:
+    """
+    Given a model_id and an `experimental.shard.advertised` event stream,
+    pick a covering set of shards minimising:
+      - total network hops
+      - max per-shard queue depth
+      - chance of overlap with the caller's own GPU (avoid self-as-shard)
+    """
+
+    def __init__(self, store: ShardStore, settings: RoutingSettings): ...
+
+    async def pick(self, model_id: str, weights_sha256: str) -> Pipeline: ...
+    async def repick(self, pipeline: Pipeline, exclude: set[ShardID]) -> Pipeline: ...
+```
+
+---
+
+## 5. Behaviour
+
+### 5.1 Shard advertisement and discovery
+
+A node hosting a shard emits `experimental.shard.advertised` events into the community event log. The event carries `ShardDescriptor` fields plus a timestamp. Advertisements expire after `DISTRIBUTED_SHARD_HEALTH_TIMEOUT_S * 4` (default 120s); shard hosts must re-advertise via heartbeat.
+
+When a node opts out (`policy.research.enable=false`), it does not emit advertisements. Existing advertisements expire normally.
+
+The shard store is a local read model built from these events, indexed by `(model_id, weights_sha256, layer_range)`.
+
+### 5.2 Pipeline construction
+
+`ShardRouter.pick`:
+
+1. Filter advertisements to those matching `model_id` and `weights_sha256`.
+2. Greedy cover: starting from layer 0, pick the shard with the lowest queue depth that includes the next uncovered layer; advance the cursor; repeat. Returns failure if any layer is uncoverable.
+3. Prefer shards on the same LAN if possible (LAN advertisements have a lower "hop weight" metric attached by Discovery).
+4. Avoid sharding to **self** as the first shard — embedding + sampling should stay on the orchestrator.
+
+Constructed pipelines are not persisted; they're per-call.
+
+### 5.3 Forward pass
+
+Per token:
+
+```
+[orchestrator] embedding → [shard 0] layers 0..7 → [shard 1] layers 8..15 → ... → [orchestrator] sample
+```
+
+Activations flow as **fp16 tensors** by default (configurable to fp32 for debugging). Each hop is a WebSocket binary frame stream (see [X08](../cross-cutting/X08-tensor-transport.md)). The orchestrator interleaves token-N and token-N+1: as soon as shard 0 finishes token N, the orchestrator pushes token N+1's embedding into shard 0 while shard 1 is still processing N. This pipeline parallelism approaches the latency of the longest-latency shard at steady state.
+
+### 5.4 Failure handling
+
+If a shard's stream errors or stalls past `DISTRIBUTED_SHARD_HEALTH_TIMEOUT_S`:
+
+1. The orchestrator emits a `shard_status` frame with `status:"degraded"`.
+2. Calls `router.repick(pipeline, exclude={failed_shard_id})`.
+3. If repick succeeds, opens a fresh stream to the replacement and emits `shard_failover` frame. **In-flight tokens are restarted** (no mid-token recovery).
+4. If repick fails and `params.fallback_to_local`, the orchestrator silently restarts the call as a local-only `llm.chat@2.0` against any local model that matches.
+5. Else: emit `pipeline_aborted` frame and return `shard_unavailable`.
+
+`DISTRIBUTED_FALLBACK_TO_LOCAL_AFTER_FAILURES` (default 2): if failover happens that many times in one call, give up and fall back to local.
+
+### 5.5 Streaming and backpressure
+
+Tensor-chunk streams use a window of `TENSOR_FLOW_CONTROL_WINDOW` chunks (default 16). Each chunk is at most `TENSOR_CHUNK_BYTES` (1 MB). If the downstream shard's send queue fills, the orchestrator pauses upstream until ACKs drain. See [X08 §4](../cross-cutting/X08-tensor-transport.md).
+
+### 5.6 Concurrency
+
+A shard's `max_concurrent_streams` is honoured strictly. If the orchestrator's call would exceed it, the orchestrator picks a different shard (via `router.repick`) rather than queuing.
+
+A shard's GPU memory budget is enforced by the shard host's own resource manager; a shard exceeding its budget gets evicted and re-advertises with `vram_required_mb` updated next time it loads.
+
+### 5.7 Models supported
+
+Phase 3 launches with two backend choices:
+
+| Backend | Models | Notes |
+|---------|--------|-------|
+| `small_model_layered` | Qwen2.5-{1.5B,3B,7B}, Llama-3.2-{1B,3B}, MiniCPM-3 | Custom HearthNet impl; PyTorch model surgery to expose per-layer forward |
+| `petals_like` | (vendored from BigScience Petals) | Optional; only if user installs `hearthnet[petals]` extra |
+
+The `small_model_layered` backend handles models up to roughly 7B parameters cleanly; beyond that the activation transport becomes the bottleneck.
+
+### 5.8 Security boundary
+
+A shard host receives activation tensors which **leak training data residue**. Treat activations as sensitive: do not log them, do not persist, do not retain past forward pass. Per-call signed authentication; the caller's identity is recorded in metrics but not in logs of tensor contents.
+
+A malicious shard could degrade outputs subtly. Detection is hard in general; the orchestrator does **basic sanity checks** (norm bounds, NaN/Inf detection) but cannot detect adversarial corruption. Communities should only enable distributed inference among members they trust as much as they trust the LLM service operator.
+
+### 5.9 Privacy threat surface
+
+A shard sees the activations of every request routed through it. With effort, a shard host can reconstruct approximate input text (especially the prompt) from activations of intermediate layers. This is **a real concern, not a theoretical one**.
+
+Mitigations (none perfect):
+- Restrict participation to members at trust level `trusted` or higher.
+- Mix activations with a small amount of noise at the orchestrator (research; not yet implemented).
+- Use this module only for queries the requester would already trust the community with.
+
+### 5.10 Observability
+
+Per call, emit:
+- `distributed_inference.pipeline_construct_ms`
+- `distributed_inference.first_token_ms`
+- `distributed_inference.tokens_per_second`
+- `distributed_inference.shard_latency_ms{shard_id}` histograms
+- `distributed_inference.failovers_total`
+- `distributed_inference.fallback_to_local_total`
+
+---
+
+## 6. Errors
+
+| Code | Cause |
+|------|-------|
+| `experimental_disabled` | `policy.research.enable=false` |
+| `shard_unavailable` | No shard covers a required layer range, or all candidates are at max concurrency |
+| `pipeline_stalled` | No progress within timeout |
+| `weights_mismatch` | A shard's advertised `weights_sha256` differs from requested |
+| `bad_request` | Unknown model, malformed pipeline params |
+
+---
+
+## 7. Configuration
+
+```toml
+[research.distributed_inference]
+enabled                       = false
+backend                       = "small_model_layered"
+max_shards_per_request        = 16
+shard_health_timeout_seconds  = 30
+fallback_to_local             = true
+activation_dtype              = "fp16"   # "fp16" | "fp32"
+allow_self_as_shard           = false
+max_concurrent_pipelines      = 4
+
+[research.distributed_inference.host]
+serve_shards                  = false
+shard_eviction_idle_seconds   = 600
+shard_max_vram_mb             = 20000
+```
+
+---
+
+## 8. Tests
+
+### 8.1 Unit
+- ShardRouter cover algorithm: 16-layer model + 3 advertised shards (0-7, 4-11, 8-15) → picks {0-7, 8-15}, ignores overlap shard
+- Sanity bounds on activations: NaN injection triggers `pipeline_stalled` (via failed health check on subsequent chunk)
+- Pipeline construction with weights mismatch → `weights_mismatch`
+
+### 8.2 Integration (LAN)
+- Two-node setup, 1.5B model split as 0-7 / 8-15; happy-path tokens/sec measured; baseline single-machine inference also measured; ratio reported (expect 0.4–0.6× local)
+- Shard host kill mid-stream; failover to a third node; total call still succeeds; latency penalty bounded
+- Concurrent two-pipeline test on three nodes; no deadlock; per-call latency degrades < 2×
+
+### 8.3 Adversarial
+- Malicious shard returns garbage activations: orchestrator's NaN/Inf detector catches the call; metric `distributed_inference.shard_corruption_detected_total` increments; pipeline aborts
+- Slowloris shard (returns one chunk per second): `pipeline_stalled` after timeout; failover succeeds
+
+### 8.4 Performance budget
+- 3B model, 2-shard pipeline, RTX 5090 + RTX 4090: ≥ 8 tokens/sec sustained
+- First-token latency ≤ 800ms
+- Construction-to-first-byte ≤ 500ms
+- Tensor-chunk overhead per hop ≤ 25ms p95
+
+---
+
+## 9. Cross-references
+
+- Capability spec: [CAPABILITY_CONTRACT_v3 §4.1–4.3](../CAPABILITY_CONTRACT_v3.md)
+- Tensor transport: [X08](../cross-cutting/X08-tensor-transport.md)
+- Base LLM service: [M04](../../modules/M04-llm.md)
+- Trust levels: [M01](../../modules/M01-identity.md)
+
+---
+
+## 10. Open research questions
+
+1. **Activation privacy.** Can we add fast-to-compute noise that preserves inference accuracy but defeats activation-inversion attacks? Cite the Geiping et al. inversion paper as the threat baseline.
+2. **Mid-token recovery.** Currently a shard failure restarts the in-flight token. Could we use micro-checkpointing (every K tokens) to recover without a restart? Latency cost?
+3. **Heterogeneous shards.** Could a 4090 host the early layers (heavier compute per layer) and a 3060 the later ones, while remaining balanced? Probably yes — automated load assignment is the research question.
+4. **Async pipeline.** Currently the orchestrator interleaves at the token level. Could it interleave at the layer level (one shard processes token N+2 while another processes N+1) for higher throughput? In theory yes; coordination protocol unclear.
+5. **Mixed local + distributed.** When the orchestrator could host some layers itself (it has a GPU), should it? When? Currently `allow_self_as_shard=false`. A heuristic that considers compute headroom would be richer.
+6. **Adversarial detection.** Beyond NaN/Inf, can we cheaply detect activation tampering by comparing to a small "shadow inference" on a tiny model? Cost vs. benefit unclear.
+7. **Pricing / incentive.** A shard host pays in GPU time. A community-internal token-economy is explicitly out of scope (00-OVERVIEW §8). But a *reputational* signal — "this anchor served 4000 shard-tokens this week" — could be helpful. Should it be a metric?
+8. **Backend strategy.** `petals_like` vs `small_model_layered`: which delivers better quality / latency / robustness for our target models? An honest A/B is the answer.

docs/p2_p3/M27-moe-routing.md

@@ -0,0 +1,378 @@
+# M27 — MoE Expert Routing
+
+**Spec version:** v3.0 — *experimental*
+**Depends on:** [M03 Capability Bus](../../modules/M03-capability-bus.md), [M04 LLM](../../modules/M04-llm.md), [M10 Chat](../../modules/M10-chat.md), [M25 Group Chat](../../phase-2/modules/M25-group-chat.md), [X02 Event Log](../../cross-cutting/X02-events.md), [M16 Tokens](../../phase-2/modules/M16-tokens.md)
+**Depended on by:** Optional routing layer in M03 Bus dispatcher
+
+---
+
+## 1. Responsibility
+
+"MoE" here means **Mixture of Experts** in a generalised sense: given a question, route to the best expert. The experts can be:
+
+- a **model** running locally on some node ("Llama-3.2-3B is good at code"),
+- a **service** capability ("the niederrhein-emergency RAG corpus knows this"),
+- a **human** with declared expertise ("Maria in Issum has organised Sankt Martins for 20 years"),
+- an **external** API or another community via federation.
+
+The router takes a query summary + tags, asks "which expert would do this best", returns top-K candidates with scores. The caller chooses one (or the system chooses automatically with a confidence threshold) and the chosen expert serves the request.
+
+This module is a research bet that **knowing-who-to-ask is more valuable than scaling-the-model**. A 3B model that knows when to defer to a neighbour with first-hand knowledge will outperform a 70B model that has to confabulate. Whether that bet pays off is exactly what the research is for.
+
+---
+
+## 2. Non-goals
+
+- **Auction-style routing.** Experts do not bid; there is no money flowing. Routing is by score, not price.
+- **Mandatory routing.** The router is a *recommender*. The caller can always choose to run a query against a default LLM. MoE routing is opt-in per capability or per call.
+- **Replacing RAG.** RAG still does retrieval inside a corpus. The router decides *which corpus* (or which non-RAG expert) — different layer.
+- **Routing without consent.** A human expert never gets pinged unless they have explicitly registered availability for the topic.
+
+---
+
+## 3. File layout
+
+```
+hearthnet/moe/
+├── __init__.py
+├── router.py             # MoeRouter — the capability handler
+├── scorer.py             # Learned scoring model; rule-based fallback
+├── expert_registry.py    # Tracks registered experts and their declared topics
+├── human_in_the_loop.py  # Coordinates handoff to a human; manages timeouts
+└── feedback.py           # Records routing outcomes to train the scorer
+```
+
+---
+
+## 4. Public API
+
+### 4.1 Dataclasses
+
+```python
+@dataclass(frozen=True)
+class ExpertDescriptor:
+    expert_id:      ExpertID            # "human:<NodeID>" | "model:<id>" | "service:<cap_name>" | "external:<url>"
+    kind:           ExpertKind
+    topics:         frozenset[str]
+    capabilities:   frozenset[str]      # for kind=service: which bus capability to invoke
+    availability:   AvailabilityWindow
+    consent_to_route: bool
+    score_bias:     float = 0.0         # operator nudge; positive favours, negative dispels
+    registered_at:  datetime
+    expires_at:     datetime | None
+
+@dataclass
+class RouteCandidate:
+    expert_id:      ExpertID
+    kind:           ExpertKind
+    score:          float
+    expected_latency_minutes: int       # for humans, hours; for models, seconds
+    rationale:      str
+    name:           str | None          # for display
+
+@dataclass
+class RouteResult:
+    candidates:     list[RouteCandidate]
+    rationale:      str
+    routed_at:      datetime
+
+@dataclass
+class Handoff:
+    handoff_id:     str
+    expert_id:      ExpertID
+    context_summary: str
+    initiated_at:   datetime
+    deadline_at:    datetime
+    state:          Literal["pending","accepted","declined","completed","timed_out"]
+    thread_id:      ThreadID | None
+```
+
+### 4.2 `MoeRouter`
+
+```python
+class MoeRouter:
+    """Capability handler for experimental.moe.route@1.0"""
+
+    def __init__(
+        self,
+        bus:        CapabilityBus,
+        registry:   ExpertRegistry,
+        scorer:     Scorer,
+        feedback:   FeedbackStore,
+        settings:   MoeSettings,
+        observability: Observability,
+    ): ...
+
+    async def start(self) -> None: ...
+
+    async def route(self, body: RouteBody) -> RouteResult: ...
+    async def handoff(self, body: HandoffBody) -> Handoff: ...
+    async def feedback_outcome(self, handoff_id: str, outcome: Outcome) -> None: ...
+```
+
+### 4.3 `Scorer`
+
+```python
+class Scorer(Protocol):
+    name: str
+
+    def score(
+        self,
+        request_summary: str,
+        tags:            list[str],
+        candidates:      list[ExpertDescriptor],
+        context:         ScoringContext,
+    ) -> list[float]:
+        """Return one score per candidate, same order."""
+        ...
+
+class RuleBasedScorer:
+    """The default scorer.  Pure rules: tag overlap, recency of expert activity, availability, bias.  No ML."""
+    ...
+
+class LearnedScorer:
+    """A small classifier trained on feedback outcomes.  Off by default; activated once
+       MOE_ROUTER_TRAIN_MIN_EXAMPLES historical handoffs with outcomes are recorded."""
+    ...
+```
+
+### 4.4 `ExpertRegistry`
+
+```python
+class ExpertRegistry:
+    """
+    Materialised view of `experimental.moe.expert.registered` events.
+    Indexed by topic for fast routing.
+    """
+
+    def __init__(self, event_log: EventLog): ...
+
+    def by_topic(self, tag: str) -> list[ExpertDescriptor]: ...
+    def by_id(self, expert_id: ExpertID) -> ExpertDescriptor | None: ...
+    def available_now(self) -> list[ExpertDescriptor]: ...
+```
+
+### 4.5 `HumanInTheLoopCoordinator`
+
+```python
+class HumanInTheLoopCoordinator:
+    """
+    Manages handoff-to-human flows.
+    - Sends a chat invitation (M25 thread) to the chosen human.
+    - Awaits acceptance within HANDOFF_RESPONSE_DEADLINE_MINUTES.
+    - Falls back to next-best candidate if declined or timed out.
+    - Stores audit trail (who routed where, why, outcome).
+    """
+
+    def __init__(self, bus: CapabilityBus, thread_service: ThreadService, registry: ExpertRegistry, settings: HitlSettings): ...
+
+    async def initiate(self, handoff: Handoff) -> None: ...
+    async def on_response(self, handoff_id: str, accepted: bool) -> None: ...
+    async def on_completion(self, handoff_id: str, outcome: Outcome) -> None: ...
+```
+
+---
+
+## 5. Behaviour
+
+### 5.1 Expert registration
+
+A node calls `experimental.moe.expert.register@1.0` to register an expert. For `kind="human"`:
+
+- The caller must be the human in question (self-registration) OR be an anchor registering on behalf of a member with explicit consent token.
+- `consent_to_route=true` is mandatory; humans without it are silently excluded from routing.
+- Topics are free-form strings, lowercased, kebab-case. The registry will compute embeddings of topics so that `"sankt_martins"` and `"sankt martins"` match.
+
+For `kind="model"` or `kind="service"`:
+
+- The node hosting the model/service self-registers.
+- Topics describe what the model is good at, e.g. `["code","python"]` or `["niederrhein-history","local-genealogy"]`.
+
+For `kind="external"`:
+
+- An anchor registers a third-party endpoint (HF Inference, OpenAI, Anthropic, another HearthNet community via federation).
+- External experts are not consulted unless `policy.research.moe_allow_external=true`.
+
+### 5.2 Routing flow
+
+`experimental.moe.route@1.0`:
+
+1. Caller submits `{request_summary, tags, top_k}`.
+2. The registry returns all currently-available experts whose topics overlap `tags` or whose semantic similarity to `request_summary` is above `MOE_TOPIC_SIMILARITY_THRESHOLD` (default 0.55).
+3. Scorer scores each candidate.
+4. Apply score biases (per-expert operator nudges, per-community policy).
+5. Sort descending; return top-K with rationales.
+6. The caller decides whether to:
+   - Route automatically (if top score ≥ `MOE_AUTO_ROUTE_THRESHOLD`, default 0.85),
+   - Present the user with the candidate list to choose,
+   - Or fall back to default LLM.
+
+### 5.3 Handoff to a human expert
+
+`experimental.moe.expert.handoff@1.0`:
+
+1. The coordinator creates a new E2E group thread (M25) with the requester + chosen human.
+2. The requester's question (or its summary) is posted to the thread.
+3. The chosen human receives a notification.
+4. If the human accepts within `HANDOFF_RESPONSE_DEADLINE_MINUTES` (default 60, configurable), the thread proceeds normally.
+5. If declined or timed out: the coordinator silently picks the next candidate (or falls back to a model). The requester is informed once total wait exceeds `HANDOFF_WAIT_BUDGET_MINUTES` (default 30).
+
+The human's UI shows handoffs as a low-priority "questions for you" inbox; they are not interruptive notifications by default. Policy can flip this for community types where instant response matters (e.g. civdef pilot, M31).
+
+### 5.4 Routing model + RAG hybrid
+
+A common pattern: a chat session begins, the user asks a question. The orchestrator:
+
+1. Sends `request_summary` (the question + recent thread context) to the router.
+2. If the top expert is a `service` expert pointing at `rag.query@1.0` against a specific corpus → run RAG against that corpus.
+3. If the top is a `model` expert → run `llm.chat` against that model.
+4. If the top is a `human` expert and the user explicitly opted in to human handoff → initiate handoff.
+5. Else default model.
+
+This is opt-in; the chat UI surfaces a "ask a neighbour?" affordance when human handoff is a credible option.
+
+### 5.5 Feedback loop
+
+After every routed call, the caller (or a UI signal: "did this answer your question?") records an `Outcome`:
+
+```python
+@dataclass
+class Outcome:
+    handoff_id:     str
+    helpful:        bool | None            # None if no signal
+    user_rating:    int | None             # 1–5
+    completion_time_seconds: float | None
+    handed_off_again: bool                 # the user asked the question elsewhere
+```
+
+Feedback is stored in `FeedbackStore` (SQLite). Once `MOE_ROUTER_TRAIN_MIN_EXAMPLES` (default 200) outcomes exist, the `LearnedScorer` becomes available and is retrained every `MOE_ROUTER_RETRAIN_EVERY_HOURS` (default 24). Communities can flip back to `RuleBasedScorer` at any time.
+
+### 5.6 Privacy in routing
+
+Routing is **observable** by definition — the request summary is sent to the router, which inspects it to pick an expert. Implications:
+
+- The router lives on the **caller's own node**; the request summary is not transmitted off-node for routing.
+- When the chosen expert is on a different node, the request body is sent over the bus as usual (signed, optionally E2E if it's a chat thread).
+- The router does not log full request summaries. It logs `tags`, the candidate list, and the chosen expert. The summary is held in memory for the duration of the call.
+- For handoff to humans, the human sees the actual question — they need to. The handoff event in the audit trail records "request handed off to X", not the question's content.
+
+### 5.7 Cross-community routing (federation)
+
+A federation manifest can include the scope `moe.route@1.0`. In that case the router can include experts from federated communities in its candidate list. Cross-community handoff:
+
+- Initiates a federated thread (M25 + M14): the thread's events are bridged to the federated community.
+- The expert's identity (e.g. "Lukas from Geldern") is visible to the requester.
+- Federation scope must include `chat.thread.send@1.0` and `chat.thread.history@1.0` for the thread to function.
+
+### 5.8 Operational policy
+
+Communities can configure:
+
+```yaml
+moe:
+  enabled: true
+  auto_route_threshold: 0.85
+  topic_similarity_threshold: 0.55
+  human_handoff:
+    enabled: true
+    response_deadline_minutes: 60
+    wait_budget_minutes: 30
+    allowed_during_quiet_hours: false   # no human pings 22:00–06:00 local
+  external_experts: false
+  cross_community: false
+```
+
+### 5.9 Anti-abuse
+
+- **Rate limit per requester:** `MOE_REQUESTS_PER_USER_PER_HOUR` (default 60). Prevents one user from spamming the human-expert pool.
+- **Per-expert cooldown:** an expert is not offered the same user's request within `MOE_PER_EXPERT_COOLDOWN_MINUTES` (default 30).
+- **Decline penalty:** an expert who declines 3 handoffs in a row gets temporarily marked `availability=false` until they update their registration.
+
+---
+
+## 6. Errors
+
+| Code | Cause |
+|------|-------|
+| `experimental_disabled` | Research not enabled |
+| `bad_request` | Empty `request_summary`, malformed tags |
+| `not_found` | `expert_id` does not exist (in `handoff`) |
+| `handoff_declined` | Chosen expert declined and no fallback was permitted |
+| `handoff_timed_out` | No response within deadline |
+| `policy_violation` | Cross-community handoff but federation does not allow |
+
+---
+
+## 7. Configuration
+
+```toml
+[research.moe]
+enabled                       = false
+scorer                        = "rule_based"   # "rule_based" | "learned"
+auto_route_threshold          = 0.85
+topic_similarity_threshold    = 0.55
+top_k_default                 = 3
+requests_per_user_per_hour    = 60
+per_expert_cooldown_minutes   = 30
+allow_external                = false
+allow_cross_community         = false
+
+[research.moe.human_handoff]
+enabled                       = true
+response_deadline_minutes     = 60
+wait_budget_minutes           = 30
+allowed_during_quiet_hours    = false
+quiet_hours_start             = "22:00"
+quiet_hours_end               = "06:00"
+
+[research.moe.learned_scorer]
+train_min_examples            = 200
+retrain_every_hours           = 24
+model_kind                    = "logistic_regression"   # small, interpretable
+```
+
+---
+
+## 8. Tests
+
+### 8.1 Unit
+- RuleBasedScorer: tag-overlap dominance test (4 candidates, exact tag match scores highest)
+- Availability filter: expert with `availability` window not covering "now" is excluded
+- Cooldown: same user calls twice within `per_expert_cooldown_minutes` → second call excludes that expert
+
+### 8.2 Integration
+- Three-node community: two humans + one model registered as experts for `{cooking, niederrhein-history}`. Query about Sankt-Martins-Lieder → human expert chosen; handoff flow completes.
+- Handoff decline: chosen expert declines, fallback picks next candidate; user sees a single thread experience without knowing about the decline.
+- Cross-community: federation manifest grants `moe.route`; query routed to an expert in the federated community; thread bridged correctly.
+
+### 8.3 Adversarial
+- Spam: one user submits 100 routes in 10 minutes → rate-limit blocks at #60, returns `too_many_requests`.
+- Decline-storm: an expert declines 10 in a row → after the third, that expert is auto-unavailable; not offered as candidate until they re-register.
+- Score injection: a community member tries to set `score_bias=999` on their own expert record → registration rejects (caller must be anchor for `score_bias` outside `[-1, 1]`).
+
+### 8.4 UX
+- Top-K presentation in chat UI: candidates show as a 3-button affordance under the user's question; user picks one; thread morphs accordingly.
+- Outcome capture: thumbs-up/down on the answer records an `Outcome`; visible in router metrics dashboard.
+
+---
+
+## 9. Cross-references
+
+- Capability spec: [CAPABILITY_CONTRACT_v3 §4.4–4.6](../CAPABILITY_CONTRACT_v3.md)
+- Group chat (handoff substrate): [M25](../../phase-2/modules/M25-group-chat.md)
+- Federation: [M14](../../phase-2/modules/M14-federation.md)
+
+---
+
+## 10. Open research questions
+
+1. **What signal predicts a good route?** Tag overlap is shallow. Embeddings of past handoffs vs current request might do better. The `LearnedScorer` is the placeholder; the actual feature engineering is unsolved.
+2. **Calibration.** Is a score of 0.85 actually 85% likely to be a good route? Reliability diagrams from feedback data needed.
+3. **Negative experts.** Should the router learn that "Llama-3.2 is *bad* at Niederrhein-Plattdeutsch" and avoid it? Currently only positive scores.
+4. **Cold-start.** A new community has no feedback data and no `LearnedScorer`. Bootstrapping by federation (borrowing experts from a more-mature peer)?
+5. **Human consent UX.** What is the right number of handoffs per week before it becomes a burden? Per-person ceiling, per-community ceiling, dynamic?
+6. **Privacy of the rationale.** Should the rationale ("Maria worked on Sankt Martins for 20 years") be visible to the requester? It can reveal information about Maria. Default: rationale is shown to requester only when the expert opts in to that.
+7. **Refusal protocol.** When a model expert "refuses" (e.g. "I cannot answer this"), should the router re-route, or trust the refusal? Mistaken refusals are a known LLM failure mode.
+8. **Expert overlap.** Two experts both register for `{sankt_martins}`. Both equally good. What's the tiebreaker that doesn't always favour the same person? Round-robin? Random? Both score 0.85 — caller chooses?
+9. **Network effects.** As more people register as experts, does the score signal get diluted? Empirical question.
+10. **Audit and review.** A community might want a quarterly "who was routed to most, on what topics" view — for fairness, for spotting overworked experts. UX for surfacing this respectfully.

docs/p2_p3/M28-fedlearn.md

@@ -0,0 +1,348 @@
+# M28 — Federated Learning (LoRA Aggregation)
+
+**Spec version:** v3.0 — *experimental*
+**Depends on:** [M03 Capability Bus](../../modules/M03-capability-bus.md), [M04 LLM](../../modules/M04-llm.md), [M14 Federation](../../phase-2/modules/M14-federation.md), [X02 Event Log](../../cross-cutting/X02-events.md), [M16 Tokens](../../phase-2/modules/M16-tokens.md), [X06 WebSocket](../../phase-2/cross-cutting/X06-websocket.md)
+**Depended on by:** nothing in MVP — opt-in research feature
+
+---
+
+## 1. Responsibility
+
+Federated learning of small **LoRA adapters** on top of a shared base model. Each node trains locally on its own data, sends only the **adapter weight deltas** (not raw data, not full weights) to an aggregator, and receives back an averaged adapter that subsequent nodes can use or further refine.
+
+The bet: a 3B-parameter base model with a *community-tuned* LoRA adapter ("how people in our village actually phrase things, what jargon our Feuerwehr uses, what the local agricultural calendar looks like") is more useful for the community than a generic 3B model, and we can do this without any node ever shipping its private data off-box.
+
+This module deliberately stays at LoRA scope only. Full fine-tunes, distillation, and continual pre-training are explicitly out — both because they are bandwidth-hostile and because the privacy story for full-weight federation is significantly harder.
+
+---
+
+## 2. Non-goals
+
+- **Federating raw data.** Never. Training data stays on the node that owns it.
+- **Full fine-tunes.** LoRA only. If a use case truly needs more, that's a different research project.
+- **Cross-base-model aggregation.** All participants in a round must run the same base model at the same quantisation. Heterogeneous aggregation is open research.
+- **Mandatory participation.** Every node decides per-round whether to join. There is no "you must contribute back" rule.
+- **Aggregator centralisation.** Any node can host an aggregator. There is no privileged aggregator role.
+- **Hiding participation.** Whether you joined a round is visible to other participants in that round; only your data and your gradients are private.
+
+---
+
+## 3. File layout
+
+```
+hearthnet/fedlearn/
+├── __init__.py
+├── coordinator.py        # Orchestrates a round: announce, gather, aggregate, distribute
+├── participant.py        # Local-side: respond to round announcements, train, submit
+├── trainer.py            # Wraps M04 LLM in a LoRA training loop (peft + bitsandbytes)
+├── aggregator.py         # FedAvg with optional secure aggregation
+├── delta.py              # Serialise/deserialise LoRA deltas (state-dict subset)
+├── privacy.py            # Optional DP-noise injection and gradient clipping
+└── manifest.py           # Round manifest: base model id, hyperparams, signature
+```
+
+---
+
+## 4. Public API
+
+### 4.1 Dataclasses
+
+```python
+RoundID = NewType("RoundID", str)        # ULID
+
+@dataclass(frozen=True)
+class RoundManifest:
+    round_id:        RoundID
+    coordinator:     NodeID
+    base_model_id:   str                  # exact model id from M04 ("qwen2.5:3b-instruct-q4_K_M")
+    base_model_sha:  str                  # SHA-256 of base weights; mismatch = exclusion
+    lora_target_modules: tuple[str, ...]  # which linear layers carry LoRA (e.g. "q_proj","v_proj")
+    lora_rank:       int                  # 4 ≤ r ≤ FEDLEARN_MAX_LORA_RANK
+    lora_alpha:      int
+    lora_dropout:    float
+    train_steps:     int                  # max local SGD steps per participant
+    learning_rate:   float
+    batch_size:      int
+    seed:            int                  # for deterministic init of LoRA matrices
+    dp_noise_scale:  float                # 0.0 = off
+    clip_norm:       float                # gradient clip; must be > 0 if DP on
+    min_participants: int                 # round aborts if fewer participants submit
+    max_participants: int
+    deadline:        datetime             # UTC; submissions after this dropped
+    topic:           str                  # free-form: "niederrhein-emergency", "village-chat"
+    consent_text:    str                  # human-readable; participant must accept
+    coordinator_sig: bytes                # detached Ed25519 over the manifest
+
+@dataclass
+class ParticipantSubmission:
+    round_id:        RoundID
+    participant:     NodeID
+    delta_bytes:     bytes                # serialised LoRA state-dict
+    delta_sha:       str
+    num_samples:     int                  # for weighted FedAvg
+    train_loss:      float                # for telemetry only
+    submitted_at:    datetime
+    signature:       bytes                # Ed25519 over (round_id, participant, delta_sha, num_samples)
+
+@dataclass
+class RoundResult:
+    round_id:        RoundID
+    aggregated_delta_sha: str
+    n_participants:  int
+    total_samples:   int
+    aggregator:      NodeID
+    completed_at:    datetime
+    manifest_sha:    str
+    download_url:    str                  # capability bus uri for fetching the aggregated delta
+```
+
+### 4.2 Capabilities
+
+```python
+async def fedlearn_round_announce(manifest: RoundManifest) -> RoundID
+async def fedlearn_round_list(topic: str | None = None) -> list[RoundManifest]
+async def fedlearn_round_join(round_id: RoundID, consent: bool) -> JoinReceipt
+async def fedlearn_round_submit(submission: ParticipantSubmission) -> SubmitReceipt
+async def fedlearn_round_status(round_id: RoundID) -> RoundStatus
+async def fedlearn_round_finalize(round_id: RoundID) -> RoundResult     # coordinator-only
+async def fedlearn_adapter_fetch(sha: str) -> bytes
+async def fedlearn_adapter_apply(sha: str, scope: Literal["session","node"]) -> ApplyReceipt
+```
+
+All capabilities are in the `experimental.fedlearn.*` namespace and only registered on the bus when `experimental.fedlearn = true` in the node config.
+
+### 4.3 Coordinator class
+
+```python
+class RoundCoordinator:
+    def __init__(self,
+                 bus: CapabilityBus,
+                 event_log: EventLog,
+                 llm: LLMService,
+                 fedlearn_config: FedLearnConfig): ...
+
+    async def announce_round(self, draft: RoundManifestDraft) -> RoundID: ...
+    async def collect_submissions(self, round_id: RoundID) -> list[ParticipantSubmission]: ...
+    async def aggregate(self, round_id: RoundID) -> bytes: ...
+    async def finalize_and_publish(self, round_id: RoundID) -> RoundResult: ...
+
+    # internal
+    async def _validate_submission(self, sub: ParticipantSubmission, manifest: RoundManifest) -> None: ...
+    async def _emit(self, evt: Event) -> None: ...
+```
+
+### 4.4 Participant class
+
+```python
+class RoundParticipant:
+    def __init__(self,
+                 bus: CapabilityBus,
+                 event_log: EventLog,
+                 llm: LLMService,
+                 data_provider: TrainingDataProvider,
+                 fedlearn_config: FedLearnConfig): ...
+
+    async def consider_round(self, manifest: RoundManifest) -> Decision: ...
+    async def train(self, manifest: RoundManifest) -> ParticipantSubmission: ...
+    async def submit(self, submission: ParticipantSubmission) -> SubmitReceipt: ...
+    async def apply_aggregated(self, result: RoundResult, scope: Literal["session","node"]) -> ApplyReceipt: ...
+```
+
+### 4.5 Aggregator
+
+```python
+class FedAvgAggregator:
+    def __init__(self, manifest: RoundManifest): ...
+
+    def add(self, submission: ParticipantSubmission, delta: dict[str, Tensor]) -> None: ...
+    def aggregate(self) -> dict[str, Tensor]: ...      # weighted by num_samples
+
+class SecureFedAvgAggregator(FedAvgAggregator):
+    """Optional: pairwise masking so the aggregator sees only the sum, never individual deltas."""
+    def __init__(self, manifest: RoundManifest, mask_scheme: Literal["additive_pairwise"] = "additive_pairwise"): ...
+```
+
+### 4.6 Privacy helpers
+
+```python
+def clip_gradient(state_dict: dict[str, Tensor], max_norm: float) -> dict[str, Tensor]
+def add_dp_noise(state_dict: dict[str, Tensor], scale: float, rng: Generator) -> dict[str, Tensor]
+def epsilon_estimate(scale: float, clip: float, n_steps: int, batch: int, dataset_size: int) -> float
+```
+
+---
+
+## 5. Behaviour
+
+### 5.1 Round lifecycle
+
+```
+ANNOUNCED ──join──▶ JOINED ──train──▶ TRAINED ──submit──▶ SUBMITTED ──┐
+   │                                                                  │
+   │                              ┌──────────── aggregate ◀───────────┘
+   │                              ▼
+   └────deadline reached────▶ AGGREGATING ──finalize──▶ COMPLETED
+                                                  │
+                                                  └──min_participants not met──▶ ABORTED
+```
+
+State transitions are recorded as events (`fedlearn.round.*`) on the coordinator's event log. Participants see their own state mirrored via subscription.
+
+### 5.2 Manifest signing
+
+Manifest is canonicalised (JCS, like federation manifests in M14 §5.2), then signed Ed25519 by the coordinator's node key. Participants must verify the signature before training. A manifest with an invalid signature is dropped silently and logged as a security event (`security.signature.invalid`).
+
+### 5.3 Consent flow
+
+When `fedlearn.round.join` is called, the participant module must:
+
+1. Check `experimental.fedlearn` is enabled in node config. If not → `experimental_disabled`.
+2. Display `manifest.consent_text` to the operator via the M11 Notifications path. The operator must explicitly accept. The acceptance is stored as a signed `fedlearn.consent.granted` event.
+3. Verify coordinator signature. If invalid → `signature_invalid` (we deliberately don't say *whose* signature; bystanders learn nothing useful).
+4. Check `base_model_sha` against the locally-installed base model. If mismatch → `base_model_mismatch`. Do not download a different base on demand; this is a hard error.
+5. Check resource budget: estimate VRAM and disk for the training run from `lora_rank * len(target_modules) * hidden_size`. If insufficient → `insufficient_resources`.
+6. If all checks pass → emit `fedlearn.round.joined`, return `JoinReceipt`.
+
+### 5.4 Local training
+
+The trainer wraps M04's LLM handle in a HuggingFace `peft.LoraConfig` and uses `bitsandbytes` 4-bit base + fp16 LoRA matrices. Training data is provided by an injected `TrainingDataProvider` — the module never reaches into other modules' storage. Typical providers:
+
+- `ChatHistoryProvider` (asks M10 for redacted, consented chat turns),
+- `KBProvider` (asks M07 for documents tagged for training),
+- `CustomFileProvider` (operator-curated training set).
+
+After `train_steps` steps or convergence (loss plateau over a window), the trainer extracts the LoRA state-dict, applies optional gradient clipping and DP noise (if `manifest.dp_noise_scale > 0`), serialises, signs, and returns a `ParticipantSubmission`.
+
+### 5.5 Aggregation
+
+The default aggregator is weighted **FedAvg**: each adapter weight is weighted by `num_samples` and averaged across submissions. After aggregation, the coordinator emits `fedlearn.round.aggregated` and stores the aggregated delta via the capability bus (using the same content-addressed file path that M06 Files uses).
+
+If the round was declared with `secure=true` in the draft, `SecureFedAvgAggregator` is used: each participant pair establishes an additive mask, masks cancel in the sum, and the aggregator never sees individual deltas. This costs an extra round-trip between participants before submission (the *mask exchange phase*) and requires `min_participants ≥ 3`.
+
+### 5.6 Distribution
+
+The aggregated adapter is published as a content-addressed file. Participants who joined the round get a `fedlearn.round.completed` event with the SHA. They can choose to:
+
+- **Session apply** — load into a single LLM session via M04 (`llm.session.apply_adapter`),
+- **Node apply** — install as the default adapter for the node (requires explicit operator action),
+- **Discard** — do nothing.
+
+Non-participants can also fetch and apply adapters they trust. There is no DRM and no whitelist: the aggregated delta is just a file with a SHA.
+
+### 5.7 Failure modes
+
+- **Coordinator vanishes mid-round:** participants wait until `deadline`, then any participant can call `fedlearn.round.finalize_takeover(round_id)` which constructs the aggregated delta from received submissions and re-publishes. The takeover is signed by the takeover-node and is visible as such.
+- **A participant submits garbage:** validation in `_validate_submission` checks tensor shapes, dtypes, finite-ness (no NaN/Inf), and that the delta is structurally a valid LoRA state-dict for the manifest's `lora_target_modules`. Garbage submissions are dropped and logged.
+- **Sybil flooding:** all participants must be authenticated with M01 identity and the manifest can require a minimum reputation/trust score (this is open research — for v3.0 the field exists in the manifest but is not yet enforced).
+- **Adversarial gradient (poisoning):** out of scope for v3.0; documented in Open Research Questions §10.
+
+---
+
+## 6. Errors
+
+| Code                            | When                                                                |
+|---------------------------------|---------------------------------------------------------------------|
+| `experimental_disabled`         | Caller invokes a fedlearn capability with the flag off              |
+| `signature_invalid`             | Manifest or submission signature does not verify                    |
+| `base_model_mismatch`           | Local base model SHA differs from manifest                          |
+| `insufficient_resources`        | Estimated VRAM/disk exceeds budget                                  |
+| `consent_required`              | join() called without an explicit consent record                    |
+| `round_full`                    | `max_participants` reached                                          |
+| `round_closed`                  | Submission after deadline                                           |
+| `delta_invalid`                 | Submitted state-dict fails structural validation                    |
+| `fedlearn_aggregation_failed`   | Aggregation produced NaN/Inf or insufficient submissions            |
+| `fedlearn_min_participants_unmet` | Round closes with fewer than `min_participants` valid submissions |
+| `fedlearn_aggregator_unreachable` | finalize() called while coordinator is offline and takeover not triggered |
+| `adapter_not_found`             | `fedlearn.adapter.fetch` for an unknown SHA                         |
+
+---
+
+## 7. Configuration
+
+```python
+@dataclass(frozen=True)
+class FedLearnConfig:
+    enabled:                   bool = False               # master switch; default off
+    max_lora_rank:             int  = FEDLEARN_MAX_LORA_RANK              # 64
+    max_lora_target_modules:   int  = FEDLEARN_MAX_LORA_TARGET_MODULES    # 8
+    max_train_steps:           int  = FEDLEARN_MAX_TRAIN_STEPS            # 1000
+    max_round_participants:    int  = FEDLEARN_MAX_PARTICIPANTS           # 32
+    min_round_participants:    int  = FEDLEARN_MIN_PARTICIPANTS           # 3
+    dp_noise_scale_default:    float = FEDLEARN_DP_NOISE_SCALE_DEFAULT    # 0.0 (off)
+    clip_norm_default:         float = FEDLEARN_CLIP_NORM_DEFAULT         # 1.0
+    submission_max_bytes:      int  = FEDLEARN_SUBMISSION_MAX_BYTES       # 64 MiB
+    require_secure_aggregation: bool = False
+    auto_apply_aggregated:     bool = False               # never auto-apply by default
+    training_vram_budget_mb:   int  = 8192
+    training_disk_budget_mb:   int  = 4096
+```
+
+All `FEDLEARN_*` constants live in `hearthnet/constants.py` so a single source of truth governs both validation and documentation generation.
+
+---
+
+## 8. Tests
+
+### 8.1 Unit
+
+- `test_manifest_canonicalisation_stable` — re-encoding does not change SHA.
+- `test_manifest_signature_roundtrip`.
+- `test_delta_serialisation_roundtrip` — tensors preserve dtype and shape.
+- `test_fedavg_weighted_arithmetic` — manually averaged deltas match aggregator output to within fp16 noise.
+- `test_dp_noise_zero_is_identity` — `add_dp_noise(d, scale=0.0)` is a no-op.
+- `test_clip_gradient_norm` — post-clip norm ≤ `max_norm`.
+- `test_secure_aggregation_masks_cancel` — sum of masks across all pairs is zero.
+
+### 8.2 Property
+
+- Across random shapes, `fedavg([d, d, d]) == d`.
+- Across random submissions, `fedavg(submissions)` is finite when all inputs are finite.
+
+### 8.3 Integration
+
+- Two-node loopback round on a 0.5B base model: announce → join → train (synthetic data, 10 steps) → submit → aggregate → apply. Aggregated adapter must be loadable and must not blow up perplexity by more than 2x on a held-out set (sanity, not quality).
+- Coordinator-failure round: simulate coordinator going offline after submissions received; takeover by another participant produces an aggregated delta with the same SHA.
+- Sybil-defence stub: round with `min_participants=3` and only 2 valid submissions aborts with `fedlearn_min_participants_unmet`.
+
+### 8.4 Negative
+
+- Wrong base SHA → `base_model_mismatch`.
+- Submission with NaN in one tensor → `delta_invalid`.
+- Submission missing one of the target modules → `delta_invalid`.
+- Manifest signed by an untrusted identity → `signature_invalid`.
+- Disabled flag → `experimental_disabled` even for read-only queries.
+
+---
+
+## 9. Cross-references
+
+- **Phase 1 M04 LLM** — provides the local model handle, exposes `llm.session.apply_adapter` and `llm.adapter.list`.
+- **Phase 1 M07 Knowledge Base** — `KBProvider` reads tagged documents for training.
+- **Phase 2 M14 Federation** — federated rounds across communities use the federation transport for manifest distribution and submission. Cross-community rounds require both communities' DPOs to sign the round consent.
+- **Phase 2 M16 Tokens** — round participation tokens (`fedlearn-participant` scope) are issued by the coordinator and bound to a single round.
+- **Phase 2 M25 Group Chat** — `village-chat` rounds typically draw training data from group chat history (consented turns only).
+- **Phase 3 M30 Evidence/EBKH** — aggregated adapters can be tracked as claims in the evidence graph; "adapter X improved perplexity on held-out set Y" is a `claim.assert`.
+
+---
+
+## 10. Open research questions
+
+1. **Gradient poisoning defence.** Coordinated malicious participants can submit deltas that, when aggregated, degrade or backdoor the adapter. Median-based aggregation (Krum, trimmed mean) is a partial defence; an authenticated-data attestation (per-submission proof that gradients were computed on real, non-cherry-picked data) is the harder question. v3.0 ships FedAvg only; v3.1 may add Krum behind a flag.
+
+2. **Heterogeneous base models.** Today, every participant in a round must run the same base model at the same quantisation. Cross-base aggregation (e.g., projecting LoRA from Qwen-3B-Q4 to Qwen-3B-Q5 or even Qwen-3B → Qwen-7B) is open. The naive approach (re-projecting via a translation matrix learnt from a calibration set) loses accuracy quickly.
+
+3. **Adaptive DP-noise.** Fixed `dp_noise_scale` is crude. Per-round noise calibration as a function of `min_participants` and `lora_rank` would tighten the privacy/utility tradeoff. Out of scope for v3.0.
+
+4. **Reputation-weighted FedAvg.** Weighting submissions by `num_samples * trust_score` instead of `num_samples` alone. Requires a credible trust signal, which the broader HearthNet design has not yet committed to.
+
+5. **Continual rounds.** Today each round produces a stand-alone adapter. Stacking rounds (round N tunes on top of round N-1's aggregate) raises questions about drift, fairness, and rollback. Probably belongs in a future M28b.
+
+6. **Cross-task adapters.** A `niederrhein-emergency` adapter and a `village-chat` adapter are trained separately. Whether they can be cleanly combined at inference time (LoRA composition) is a known-hard problem and explicitly not promised here.
+
+7. **Hardware-class fairness.** A round held by a participant with an RTX 5090 might exclude phone-class participants by setting `train_steps` too high. A "ranked tier" with separate aggregations per tier is one possibility. Currently the manifest is a single-tier flat artefact.
+
+8. **Audit of training data.** Even though raw data never leaves the node, the *fact that training happened on consented data* is currently un-auditable from the outside. A future zero-knowledge attestation of "this delta was computed on N samples each tagged training=true" would be useful. Out of scope.
+
+---
+
+*Last updated: spec v3.0.*

docs/p2_p3/M29-lora-beacons.md

@@ -0,0 +1,322 @@
+# M29 — LoRa Hardware Beacons
+
+**Spec version:** v3.0 — *experimental*
+**Depends on:** [M03 Capability Bus](../../modules/M03-capability-bus.md), [M02 Transport](../../modules/M02-transport.md), [X02 Event Log](../../cross-cutting/X02-events.md), [M11 Notifications](../../modules/M11-notifications.md), [M01 Identity](../../modules/M01-identity.md)
+**Depended on by:** Civil Defense (M31) optionally consumes beacon-presence signals
+
+---
+
+## 1. Responsibility
+
+Optional **out-of-band presence and panic-button channel** over 868 MHz LoRa hardware. When the internet is out, when the cellular network is down, when the power grid is wobbly — the LoRa stack still carries a 32-byte "I exist" ping or a tiny panic message between neighbours up to a few kilometres apart.
+
+This module is explicitly **not a data channel**. No AI traffic, no chat content, no file transfer. The bandwidth is laughably small (sub-100 bytes per minute per node in a normal duty-cycle regime), the latency is awful, and the airwave is shared. What LoRa is good at is "I'm still here, and the gateway in the next village is still reachable."
+
+The module exposes a small set of capabilities for sending and receiving beacons, mapping LoRa device IDs to HearthNet identities, and surfacing the resulting connectivity graph to the rest of the stack as a fallback signal. Hardware is a USB-attached LoRa stick (RFM95W, sx1276, sx1262 chipsets) bridged via serial.
+
+---
+
+## 2. Non-goals
+
+- **General-purpose meshing of HearthNet over LoRa.** Bandwidth and duty cycle make this impossible at any useful scale.
+- **Encryption of beacon contents.** Beacons carry identity hash + sequence + minimal flags only. Anything sensitive belongs on a different channel.
+- **Replacing TETRA/BOS.** Emergency services have their own radio. This is a *neighbour-to-neighbour* fallback, not a replacement for professional emergency comms.
+- **Hardware abstraction layer for every LoRa chipset.** v3.0 supports a small whitelist (RFM95W, sx1276, sx1262 via Meshtastic-firmware sticks). Others are open contributions.
+- **Long-distance routing.** No multi-hop store-and-forward in v3.0. A beacon goes one hop or it doesn't go.
+- **Legal interpretation of national radio regulations.** Each operator is responsible for complying with their local rules (BNetzA in Germany, ETSI in EU, FCC in US). The module enforces *configured* duty-cycle limits but cannot enforce the law on the operator.
+
+---
+
+## 3. File layout
+
+```
+hearthnet/lora/
+├── __init__.py
+├── service.py           # LoraBeaconService — the capability handler
+├── serial_bridge.py     # USB-serial framing to the LoRa stick
+├── frame.py             # Encode/decode the 32-byte beacon frame
+├── duty_cycle.py        # Track airtime, enforce duty-cycle limits
+├── peer_map.py          # LoRa device ID ↔ NodeID mapping (with TOFU verification)
+└── adapters/
+    ├── __init__.py
+    ├── meshtastic.py    # Meshtastic firmware stick
+    ├── rfm95w.py        # Bare RFM95W via serial-port gateway firmware
+    └── sx126x.py        # sx1262 module
+```
+
+---
+
+## 4. Public API
+
+### 4.1 Dataclasses
+
+```python
+LoraBeaconID = NewType("LoraBeaconID", str)         # device-local sequence + prefix
+LoraDeviceID = NewType("LoraDeviceID", str)         # hardware ID from the stick
+
+@dataclass(frozen=True)
+class LoraBeacon:
+    beacon_id:       LoraBeaconID
+    sender_hash:     bytes                # 4-byte truncated SHA-256 of sender NodeID
+    sequence:        int                  # u16, wraps
+    flags:           int                  # u8 bit-field; see §5.2
+    rssi:            int | None           # dBm, on receive only
+    snr:             float | None         # dB, on receive only
+    timestamp:       datetime             # local clock at decode
+
+@dataclass(frozen=True)
+class LoraPeer:
+    node_id:         NodeID
+    device_id:       LoraDeviceID
+    sender_hash:     bytes
+    first_seen:      datetime
+    last_seen:       datetime
+    rssi_recent:     int | None
+    verified_tofu:   bool                 # True after operator confirmation
+
+@dataclass(frozen=True)
+class DutyCycleStatus:
+    region:          Literal["EU868","US915","AS923"]
+    window_seconds:  int
+    airtime_used_ms: int
+    airtime_budget_ms: int                # e.g. 36000 ms in EU868 1% window
+    next_tx_allowed_at: datetime
+```
+
+### 4.2 Capabilities
+
+All under `experimental.lora.*`:
+
+```python
+async def lora_status() -> LoraStatus
+async def lora_beacon_send(flags: int = 0) -> LoraBeaconID
+async def lora_panic_send() -> LoraBeaconID                 # sets FLAG_PANIC; bypasses normal pacing
+async def lora_peer_list() -> list[LoraPeer]
+async def lora_peer_verify(device_id: LoraDeviceID, node_id: NodeID) -> VerifyReceipt
+async def lora_recent_beacons(since: datetime | None = None) -> list[LoraBeacon]
+async def lora_duty_cycle() -> DutyCycleStatus
+async def lora_subscribe_beacons() -> AsyncIterator[LoraBeacon]
+```
+
+### 4.3 Service class
+
+```python
+class LoraBeaconService:
+    def __init__(self,
+                 bus: CapabilityBus,
+                 event_log: EventLog,
+                 notifications: NotificationService,
+                 identity: IdentityService,
+                 config: LoraConfig): ...
+
+    async def start(self) -> None: ...                 # opens serial, begins RX loop
+    async def stop(self) -> None: ...
+    async def send_beacon(self, flags: int = 0) -> LoraBeaconID: ...
+    async def send_panic(self) -> LoraBeaconID: ...
+    async def on_frame_received(self, raw: bytes, rssi: int, snr: float) -> None: ...
+    async def _drain_rx(self) -> None: ...
+    def duty_cycle_status(self) -> DutyCycleStatus: ...
+```
+
+### 4.4 Serial bridge
+
+```python
+class SerialBridge:
+    def __init__(self, port: str, baud: int = 115200, adapter: LoraAdapter = ...): ...
+    async def open(self) -> None: ...
+    async def close(self) -> None: ...
+    async def write(self, frame: bytes) -> None: ...
+    async def read(self) -> AsyncIterator[bytes]: ...
+
+class LoraAdapter(Protocol):
+    """Per-chipset/firmware framing rules."""
+    name: str
+    def encode_tx(self, payload: bytes) -> bytes: ...
+    def decode_rx(self, raw: bytes) -> tuple[bytes, int, float]: ...  # payload, rssi, snr
+    def at_init_commands(self) -> list[bytes]: ...
+```
+
+---
+
+## 5. Behaviour
+
+### 5.1 Beacon frame
+
+Strictly 32 bytes, big-endian:
+
+```
+offset  size  field
+ 0      1     version (currently 0x01)
+ 1      4     sender_hash (SHA-256(NodeID)[:4])
+ 5      2     sequence (u16, wraps)
+ 7      1     flags
+ 8      1     reserved (0x00)
+ 9      4     unix_seconds (sender's clock, u32; informational only)
+13      19    payload (currently zero-padded; reserved for future use)
+```
+
+No payload content is carried beyond identity-hash + flags + clock. The flags field carries:
+
+```python
+FLAG_PANIC          = 0x01   # urgent attention requested
+FLAG_OK             = 0x02   # explicit "I'm fine" (operator pressed an OK button)
+FLAG_GATEWAY        = 0x04   # this node has an alternate transport currently up
+FLAG_LOW_BATTERY    = 0x08   # device-level low-battery indicator
+FLAG_RESERVED_*     = 0x10..0x80
+```
+
+Frames are not encrypted. Frames *are* not anonymous either — the sender hash is small enough to collide (4 bytes), but stable enough that a passive observer can correlate beacons from the same sender over time. This is documented and acceptable for the threat model: LoRa airwaves are observable by construction.
+
+### 5.2 RX path
+
+1. `SerialBridge` yields raw frames as they arrive.
+2. `LoraAdapter.decode_rx` peels off the chipset framing and returns the 32-byte payload + RSSI + SNR.
+3. `service.on_frame_received` validates: length == 32, version == 0x01, sender_hash plausibly maps to a known or unknown peer.
+4. If `sender_hash` matches a verified peer in `peer_map`, the beacon is recorded against that peer.
+5. If `sender_hash` is unknown, a `lora.peer.unknown` event is emitted with a TOFU verification prompt for the operator.
+6. If `FLAG_PANIC` is set, a high-priority notification is raised via M11 regardless of peer-verification status.
+7. The beacon is published on the bus subscription `experimental.lora.beacon.received`.
+
+### 5.3 TX path and duty cycle
+
+Beaconing follows a fixed cadence `LORA_BEACON_PERIOD_SECONDS` (default 600 = 10 minutes). Each transmission's airtime is computed from spreading factor and bandwidth (typical: SF9, BW125 → ~165 ms per 32-byte frame) and added to the duty-cycle window.
+
+The duty-cycle window enforces the region's regulation:
+
+| Region | Window | Budget |
+|--------|--------|--------|
+| EU868 | 3600 s | 36 s (1%) |
+| US915 | 3600 s | unlimited (FHSS) but config still applies |
+| AS923 | 3600 s | 36 s (1%) |
+
+If a normal `send_beacon` call would exceed the budget, it is **deferred** until the budget allows. `send_panic` ignores the duty-cycle limit (regulations universally permit emergency transmissions). The operator is told via notification that the duty-cycle override was used and the event log records `lora.duty_cycle.overridden`.
+
+### 5.4 Peer mapping (TOFU)
+
+The first time a `sender_hash` is received, the module emits a notification: *"A new LoRa peer with hash 0xABCD1234 was heard. Do you recognise this device?"* The operator can:
+
+- **Verify by NodeID** — provide a HearthNet NodeID; the module checks that `SHA-256(NodeID)[:4] == sender_hash` and stores the verified mapping.
+- **Mark as unknown** — store the hash with no NodeID; future beacons from this hash will still be tracked but flagged unknown.
+- **Block** — drop all beacons from this hash; never prompt again.
+
+Hash collisions (two different NodeIDs producing the same 4-byte hash) are possible but unlikely. When two operators independently verify the same hash to different NodeIDs, the conflict is surfaced as a `lora.peer.conflict` event for manual resolution.
+
+### 5.5 Beacon-presence signal
+
+Other modules can subscribe to `experimental.lora.beacon.received` to incorporate "this peer is alive on LoRa even though the internet says they're offline" into their own logic. M31 Civil Defense in particular uses this to corroborate that a target node is alive during an outage incident.
+
+The presence signal is *advisory*: a node that beacons on LoRa is alive in the radio sense, but that says nothing about whether the operator is responsive or whether higher-layer services are available there.
+
+### 5.6 Failure modes
+
+- **No stick attached or USB error:** `lora.status()` reports `unavailable`. The module starts in a disabled state; no errors are raised on startup, only logged.
+- **Stick attached but firmware mismatch:** `at_init_commands` fail; the adapter raises `lora_hardware_unsupported` and the service stays disabled.
+- **Receive flood:** the RX queue is bounded (`LORA_RX_QUEUE_MAX` default 256). Overflow drops oldest entries and emits a `lora.rx.dropped` event.
+- **Clock skew:** beacons carry the sender's clock, but the receiver never trusts it for ordering — local arrival timestamp is authoritative.
+- **Adversarial flooding:** an attacker on 868 MHz can spam frames; the duty-cycle limits *us* but not *them*. The service rate-limits beacons per `sender_hash` at the RX side (`LORA_PEER_RX_MAX_PER_MINUTE`, default 20) to avoid filling notifications. Excess beacons from one hash are dropped silently after the rate limit; this is a known DoS vector and documented in §10.
+
+---
+
+## 6. Errors
+
+| Code                          | When                                                             |
+|-------------------------------|------------------------------------------------------------------|
+| `experimental_disabled`       | Capability called with the flag off                              |
+| `lora_hardware_unavailable`   | No stick present or serial port not opened                       |
+| `lora_hardware_unsupported`   | Adapter init failed; firmware not whitelisted                    |
+| `lora_duty_cycle_exhausted`   | Non-panic send requested with budget at zero and override off    |
+| `lora_peer_unknown`           | `lora.peer.verify` for a sender_hash we've never seen            |
+| `lora_peer_conflict`          | verify() would create a (hash → two distinct NodeIDs) mapping    |
+| `lora_frame_malformed`        | RX frame fails structural validation                             |
+
+---
+
+## 7. Configuration
+
+```python
+@dataclass(frozen=True)
+class LoraConfig:
+    enabled:                bool = False
+    serial_port:            str = "/dev/ttyUSB0"     # also Windows COM4, etc.
+    serial_baud:            int = 115200
+    adapter:                Literal["meshtastic","rfm95w","sx126x"] = "meshtastic"
+    region:                 Literal["EU868","US915","AS923"] = "EU868"
+    spreading_factor:       int = 9                  # 7..12; higher = more range, less rate
+    bandwidth_khz:          int = 125
+    coding_rate_denom:      int = 5                  # 4/5
+    tx_power_dbm:           int = 14                 # legal max for EU868
+    beacon_period_seconds:  int = LORA_BEACON_PERIOD_SECONDS_DEFAULT       # 600
+    panic_burst_count:      int = 3                  # PANIC sends this many frames rapid-fire
+    panic_burst_gap_ms:     int = 800
+    rx_queue_max:           int = LORA_RX_QUEUE_MAX                        # 256
+    peer_rx_max_per_minute: int = LORA_PEER_RX_MAX_PER_MINUTE              # 20
+    tofu_auto_accept:       bool = False             # never auto-trust new hashes by default
+    duty_cycle_override_for_panic: bool = True
+```
+
+Constants live in `hearthnet/constants.py`.
+
+---
+
+## 8. Tests
+
+### 8.1 Unit
+
+- `test_frame_encode_decode_roundtrip` — random payloads encode to exactly 32 bytes and round-trip.
+- `test_sender_hash_matches_nodeid` — `SHA-256(NodeID)[:4]` matches the field in the encoded frame.
+- `test_duty_cycle_tracks_airtime` — synthetic transmissions accumulate; budget drains; recovers over time.
+- `test_panic_overrides_duty_cycle` — `send_panic` succeeds at zero budget when override is enabled.
+- `test_panic_blocked_when_override_disabled` — `send_panic` returns `lora_duty_cycle_exhausted` when override is off.
+- `test_peer_rx_rate_limit` — 30 frames from one hash within a minute → only 20 surface.
+
+### 8.2 Integration (loopback)
+
+- Mock `SerialBridge` echoes TX as RX after a configurable delay. Verify a sent beacon shows up in `recent_beacons` and on the subscription.
+- Two simulated nodes (separate SerialBridges connected via an in-memory channel) — A sends, B receives, B's peer_map contains A after TOFU verification, RSSI/SNR are populated.
+
+### 8.3 Hardware-in-the-loop (optional)
+
+- With a real LoRa stick, send N beacons and verify duty-cycle accounting matches what the firmware reports.
+- Range test: two sticks at increasing distance; record packet-loss vs distance.
+
+### 8.4 Negative
+
+- Disabled flag → all capabilities return `experimental_disabled`.
+- No serial port → `lora_hardware_unavailable` on status.
+- Truncated frame → `lora_frame_malformed`, dropped.
+- Conflicting verify → `lora_peer_conflict`.
+
+---
+
+## 9. Cross-references
+
+- **Phase 1 M01 Identity** — `SHA-256(NodeID)[:4]` is the sender hash; verification uses M01's NodeID type.
+- **Phase 1 M02 Transport** — LoRa is *not* a Transport in the M02 sense. It does not carry capability-bus traffic; it lives parallel to M02 as an alternative signalling channel. The two share no code.
+- **Phase 1 M11 Notifications** — high-priority panic-beacon notifications and TOFU prompts route through M11.
+- **Phase 1 X02 Event Log** — `lora.*` events.
+- **Phase 3 M31 Civil Defense** — beacon-presence is one corroborating signal for "is the target node alive" during an incident.
+- **Phase 3 X09 Conformance Suite** — LoRa is an optional capability; conformance tests use a mock serial bridge.
+
+---
+
+## 10. Open research questions
+
+1. **Mesh routing.** Multi-hop store-and-forward over LoRa is well-explored in the Meshtastic project. Whether HearthNet should adopt it (and inherit the bandwidth tradeoffs) or keep one-hop simplicity is unsettled. Probably belongs in M29b.
+
+2. **Authenticated beacons.** Adding even a 4-byte MAC would let receivers reject forged sender-hashes. This costs payload space we don't have today. A 64-byte frame variant (`version 0x02`) with HMAC-truncated-to-8-bytes is the obvious extension.
+
+3. **DoS robustness.** Per-hash rate limiting is naive; an attacker just rotates hashes. The defence on 868 MHz is mostly the regulatory duty-cycle and physical proximity, neither of which we control in software. Documented as a known limitation.
+
+4. **Sleep-and-wake duty cycles.** Battery-powered nodes (a panic button by the bedside) want to sleep most of the time and wake on demand. Class-A/B/C LoRaWAN-style scheduling is the standard answer. Out of scope for v3.0.
+
+5. **Chipset coverage.** v3.0 supports a small whitelist. Each new chipset is an adapter shaped exactly like the existing ones; contributors are encouraged.
+
+6. **GPS integration.** Many LoRa sticks ship with a GPS module. We deliberately did not surface location data in v3.0 — location is privacy-sensitive and the use case is unclear. A future `FLAG_HAS_GPS` + paired side-channel might make sense for civil-defence scenarios.
+
+7. **Integration with civil-defence radio.** TETRA-BOS and BOS-Digitalfunk are professional networks we have no business interoperating with. But a *unidirectional* "did the BOS station broadcast a known alert" listener might be useful. Legally complex.
+
+8. **Network coding.** When multiple nearby nodes beacon, the airwave fills. Cooperative beacon scheduling (so neighbours don't transmit on top of each other) is a fun problem. Currently each node beacons independently and collisions are accepted.
+
+---
+
+*Last updated: spec v3.0.*

docs/p2_p3/M30-evidence-ebkh.md

@@ -0,0 +1,384 @@
+# M30 — Evidence Graph & EBKH Integration
+
+**Spec version:** v3.0 — *experimental*
+**Depends on:** [M03 Capability Bus](../../modules/M03-capability-bus.md), [X02 Event Log](../../cross-cutting/X02-events.md), [M01 Identity](../../modules/M01-identity.md), [M06 Files](../../modules/M06-files.md), [M07 Knowledge Base](../../modules/M07-knowledge-base.md), [M16 Tokens](../../phase-2/modules/M16-tokens.md), [M21 Tool Calls](../../phase-2/modules/M21-tool-calls.md)
+**Depended on by:** M31 Civil Defense (alerts may carry an evidence chain); RAG quality reports
+
+---
+
+## 1. Responsibility
+
+A **content-addressed claim graph** layered alongside the append-only event log, plus an integration adapter for Christof's existing **EBKH v3+** (event-sourced knowledge hub with OSINT capabilities, PostGIS, and the class-reference graph already described in his upstream work).
+
+The event log answers "what happened, in what order". The evidence graph answers a different question: "what is asserted, by whom, on what basis, and what counterclaims exist?" This is necessary because in a community AI mesh, *information has provenance* — a claim about when the Sankt Martins parade starts, where the local emergency assembly point is, or what the Volksbank's IBAN is — must be traceable to a source, must be cross-checkable, and must support dispute. The event log doesn't do that; it just records that someone said something.
+
+The module is small in line count but conceptually load-bearing: it defines what counts as evidence, what a claim is, how claims compose, and how the rest of the stack queries provenance. The EBKH adapter wires this to Christof's already-built PostGIS + OSINT infrastructure so the two systems converge rather than duplicate.
+
+---
+
+## 2. Non-goals
+
+- **Replacing the event log.** The event log is still the source of truth for *what happened*. The evidence graph is a *derived* view over claims extracted from events plus claims asserted directly.
+- **Adjudicating truth.** The module records claims, disputes, and attestations. It does not decide who is right.
+- **General-purpose knowledge graph.** This is not a Wikidata clone. The schema is deliberately narrow: claim, source, attestation, dispute, derivation.
+- **Replacing RAG.** RAG retrieves passages. The evidence graph annotates those passages with provenance and lets a downstream summariser say "according to source X (last verified Y)". Different layer.
+- **OSINT collection.** EBKH already collects from external sources; this module *integrates* with that collector, it does not duplicate it.
+- **Mandatory adoption.** RAG and chat can still operate without consulting the evidence graph. The graph is queried when callers explicitly want provenance.
+
+---
+
+## 3. File layout
+
+```
+hearthnet/evidence/
+├── __init__.py
+├── service.py            # EvidenceService — capability handler
+├── claim.py              # Claim, ClaimSource, Attestation, Dispute, Derivation dataclasses
+├── store.py              # ClaimStore — append-only with content-addressed ClaimIDs
+├── query.py              # Provenance traversal: trace(), neighbours(), conflicts()
+├── extractor.py          # Pulls claims out of events (chat, KB ingest, federation)
+├── ebkh_adapter.py       # Bridge to Christof's EBKH v3+ via JSON-RPC
+└── trust.py              # Per-source trust scoring (advisory, no enforcement)
+```
+
+---
+
+## 4. Public API
+
+### 4.1 Dataclasses
+
+```python
+ClaimID = NewType("ClaimID", str)             # SHA-256 over canonical claim record
+SourceID = NewType("SourceID", str)           # "url:https://...", "node:<NodeID>", "doc:<FileID>", "ebkh:<entity_uri>"
+
+EvidenceLevel = Literal[
+    "unverified",       # claim made, no corroboration
+    "cited",            # claim has at least one source
+    "cross_referenced", # ≥2 independent sources agree
+    "attested",         # an identified party with skin in the game has signed
+    "disputed",         # at least one counterclaim with sources
+]
+
+@dataclass(frozen=True)
+class ClaimSource:
+    source_id:    SourceID
+    accessed_at:  datetime
+    content_sha:  str | None              # SHA of the cited content if available
+    excerpt:      str | None              # short verbatim excerpt; max 280 chars
+
+@dataclass(frozen=True)
+class Claim:
+    claim_id:        ClaimID
+    subject:         str                   # canonical subject string
+    predicate:       str                   # e.g. "starts_at", "located_at", "operated_by"
+    object:          str                   # canonical object string
+    asserted_by:     NodeID
+    asserted_at:     datetime
+    sources:         tuple[ClaimSource, ...]
+    confidence:      float                 # asserter's self-rated confidence [0,1]
+    derived_from:    tuple[ClaimID, ...]   # parent claims if this is a derivation
+    signature:       bytes                 # asserter's Ed25519 signature
+
+@dataclass(frozen=True)
+class Attestation:
+    claim_id:     ClaimID
+    attester:     NodeID
+    attested_at:  datetime
+    rationale:    str                      # human-readable "I know this because..."
+    role:         str                      # "first-hand witness", "official record holder", "expert"
+    signature:    bytes
+
+@dataclass(frozen=True)
+class Dispute:
+    claim_id:        ClaimID               # the claim being disputed
+    counterclaim_id: ClaimID               # the claim made in response
+    disputer:        NodeID
+    disputed_at:     datetime
+    rationale:       str
+    signature:       bytes
+
+@dataclass(frozen=True)
+class ProvenanceTrace:
+    claim:           Claim
+    sources:         tuple[ClaimSource, ...]
+    attestations:    tuple[Attestation, ...]
+    disputes:        tuple[Dispute, ...]
+    derivation_tree: tuple[Claim, ...]     # walked depth-first, deduplicated
+    evidence_level:  EvidenceLevel
+    trust_score:     float                 # advisory; from trust.py
+```
+
+### 4.2 Capabilities
+
+All under `experimental.evidence.*`:
+
+```python
+async def evidence_claim_assert(draft: ClaimDraft) -> ClaimID
+async def evidence_claim_dispute(claim_id: ClaimID, counterclaim: ClaimDraft, rationale: str) -> ClaimID
+async def evidence_claim_attest(claim_id: ClaimID, role: str, rationale: str) -> AttestationReceipt
+async def evidence_claim_get(claim_id: ClaimID) -> Claim
+async def evidence_claim_query(subject: str | None = None,
+                                predicate: str | None = None,
+                                object: str | None = None,
+                                min_evidence: EvidenceLevel = "unverified") -> list[Claim]
+async def evidence_provenance_trace(claim_id: ClaimID, max_depth: int = 5) -> ProvenanceTrace
+async def evidence_subject_summary(subject: str) -> SubjectSummary
+async def evidence_ebkh_sync(direction: Literal["pull","push","bidi"] = "bidi") -> SyncReport
+```
+
+### 4.3 Service class
+
+```python
+class EvidenceService:
+    def __init__(self,
+                 bus: CapabilityBus,
+                 event_log: EventLog,
+                 identity: IdentityService,
+                 store: ClaimStore,
+                 extractor: ClaimExtractor,
+                 ebkh: EbkhAdapter | None,
+                 trust: TrustScorer,
+                 config: EvidenceConfig): ...
+
+    async def assert_claim(self, draft: ClaimDraft) -> ClaimID: ...
+    async def dispute(self, claim_id: ClaimID, counterclaim: ClaimDraft, rationale: str) -> ClaimID: ...
+    async def attest(self, claim_id: ClaimID, role: str, rationale: str) -> AttestationReceipt: ...
+    async def trace(self, claim_id: ClaimID, max_depth: int) -> ProvenanceTrace: ...
+    async def summarise_subject(self, subject: str) -> SubjectSummary: ...
+    async def evidence_level(self, claim_id: ClaimID) -> EvidenceLevel: ...
+```
+
+### 4.4 Claim store
+
+```python
+class ClaimStore:
+    async def put(self, claim: Claim) -> ClaimID: ...        # idempotent on ClaimID
+    async def get(self, claim_id: ClaimID) -> Claim | None: ...
+    async def by_subject(self, subject: str) -> list[Claim]: ...
+    async def by_triple(self, subject: str, predicate: str, object: str | None) -> list[Claim]: ...
+    async def disputes_of(self, claim_id: ClaimID) -> list[Dispute]: ...
+    async def attestations_of(self, claim_id: ClaimID) -> list[Attestation]: ...
+    async def derivatives_of(self, claim_id: ClaimID) -> list[Claim]: ...
+```
+
+The store is append-only. A "retraction" is itself a claim (`predicate="retracted"`, `object="<claim_id>"`) and is treated as a special kind of dispute by the trace algorithm.
+
+### 4.5 Extractor
+
+```python
+class ClaimExtractor:
+    """Watches the event log and proposes claims from candidate events.
+
+    Proposals are *suggestions*, not auto-asserted. A claim only enters
+    the store when an identified asserter signs it.
+    """
+    async def consume(self, evt: Event) -> list[ClaimDraft]: ...
+    def register_pattern(self, predicate: str, matcher: Callable[[Event], ClaimDraft | None]) -> None: ...
+```
+
+Patterns shipped in v3.0:
+
+- KB ingest event → `claim(doc_sha, "contains_text", text_hash)` with the doc as source.
+- Tool-call event (M21) with HTTP fetch → `claim(url, "served", content_sha)` with the URL as source.
+- Federation manifest event → `claim(remote_node, "advertises_capability", cap_name)` with the manifest as source.
+- LoRa beacon (M29) reception → *not* auto-extracted; presence is logged but not claimed.
+
+### 4.6 EBKH adapter
+
+```python
+class EbkhAdapter:
+    def __init__(self, endpoint: str, token: str, postgis_dsn: str | None = None): ...
+
+    async def push_claim(self, claim: Claim) -> EbkhRef: ...
+    async def pull_entity(self, entity_uri: str) -> list[Claim]: ...
+    async def query_spatial(self, bbox: Bbox, predicate: str | None = None) -> list[Claim]: ...
+    async def sync(self, direction: Literal["pull","push","bidi"]) -> SyncReport: ...
+```
+
+The adapter speaks EBKH's JSON-RPC over HTTPS with an Ed25519-bound bearer token (issued via M16). Spatial queries piggy-back on EBKH's PostGIS layer — useful for civil-defence claims like "Sammelplatz is at geom(...)". For nodes without EBKH installed, the adapter is `None` and capabilities still function on the local claim store only.
+
+### 4.7 Trust scoring
+
+`TrustScorer` produces an advisory `[0,1]` score for a source. The function is intentionally simple and visible:
+
+```python
+class TrustScorer:
+    def score_source(self, source: ClaimSource, context: TrustContext) -> float: ...
+    def score_asserter(self, node_id: NodeID, context: TrustContext) -> float: ...
+```
+
+Inputs include: how long the source has been known, how many of its prior claims were not disputed, whether it's signed by a verified identity, whether it's in an operator-curated allowlist or blocklist. The score is **always shown alongside the claim, never hidden**, and never causes a claim to be omitted from query results — only re-ranked. Operators can override individual scores.
+
+---
+
+## 5. Behaviour
+
+### 5.1 Canonicalisation and ClaimID
+
+A claim's identity is its `ClaimID`, defined as:
+
+```
+ClaimID = base32-no-pad( SHA-256( JCS({
+    subject, predicate, object,
+    asserted_by, asserted_at_iso8601,
+    sources: [{source_id, accessed_at_iso, content_sha} ...],
+    confidence_5dp,
+    derived_from: [...sorted ClaimIDs...],
+}) ) )
+```
+
+The signature is *not* part of the ClaimID — a different asserter making the identical claim would produce a different signature but the same record. To distinguish, we use `Claim.asserted_by` and the signature ensures non-repudiation. A claim asserted twice by the same node at the same instant with the same sources is genuinely the same claim and the store deduplicates.
+
+### 5.2 Evidence level computation
+
+```
+unverified       = no sources
+cited            = ≥1 source
+cross_referenced = ≥2 sources, distinct source_ids, ≥1 not from asserter's own node
+attested         = ≥1 attestation with role in {"first-hand witness","official record holder"}
+disputed         = ≥1 unretracted dispute by a node with trust_score ≥ EVIDENCE_DISPUTE_MIN_TRUST
+```
+
+Levels are not mutually exclusive in nature, but the API returns the *strongest applicable level* with `disputed` taking precedence over everything else if present. This way callers default to "show that there's a dispute" rather than burying it under a stronger-sounding label.
+
+### 5.3 Provenance trace algorithm
+
+`trace(claim_id, max_depth)` does a depth-first walk over `derived_from` edges, deduplicates by ClaimID, and collects every source, attestation, and dispute encountered. The walk stops at `max_depth` (default 5) or at a cycle (cycles shouldn't exist by construction, but we guard anyway).
+
+The result is a flat tuple in topological order from root claim outward. UI is expected to render this as a tree or a list, with disputes inlined wherever they occur.
+
+### 5.4 Subject summary
+
+`summarise_subject(subject)` is the workhorse for the rest of the stack. It returns:
+
+- All claims with this subject, grouped by predicate.
+- For each predicate, the strongest claim by evidence level and trust score.
+- All disputes affecting this subject.
+- A flat list of distinct sources contributing.
+
+This is what a RAG pipeline calls to add provenance to its retrieved passages, and what civil-defence (M31) calls to verify a target before publishing an alert.
+
+### 5.5 EBKH sync
+
+`evidence.ebkh.sync` runs in three modes:
+
+- **pull** — fetch claims for subjects in our local store from EBKH, add as new claims (asserted by the EBKH node identity).
+- **push** — send our locally-asserted claims to EBKH; EBKH stores them tagged with our node ID.
+- **bidi** — both, in that order.
+
+Sync is idempotent. Each side stores the other-side's claim records; nothing is overwritten. Conflicts (same triple, different sources) become co-existing claims, and disputes can be raised normally.
+
+EBKH's existing PostGIS schema is reused for spatial predicates. The adapter does *not* try to model the full EBKH schema in our claim graph; it surfaces what is asked for and lets EBKH remain the authoritative store for OSINT-collected material.
+
+### 5.6 Failure modes
+
+- **Claim signature invalid** on receipt from another node → reject; emit `security.signature.invalid`.
+- **Dispute on a non-existent claim** → `claim_not_found`.
+- **Cyclic derivation** → reject the new claim; `evidence_cycle_detected`. (This can only happen via malicious crafting; honest derivation cannot cycle.)
+- **EBKH unreachable** during sync → return a `SyncReport` with `partial=true` and the unreachable error; do not fail the calling operation.
+
+---
+
+## 6. Errors
+
+| Code                              | When                                                              |
+|-----------------------------------|-------------------------------------------------------------------|
+| `experimental_disabled`           | Capability called with the flag off                               |
+| `claim_not_found`                 | Operation references a ClaimID we don't have                      |
+| `claim_signature_invalid`         | Signature doesn't verify against asserter's identity              |
+| `evidence_cycle_detected`         | Proposed claim's derivation chain forms a cycle                   |
+| `evidence_contradiction`          | (advisory) two claims with the same triple but opposite objects   |
+| `ebkh_unavailable`                | EBKH endpoint not configured or unreachable                       |
+| `trust_below_threshold`           | (advisory) attached to results; not an error condition by itself  |
+
+`evidence_contradiction` is *advisory* — returned in query results as a flag, not raised as an exception. The system never silently picks a winner.
+
+---
+
+## 7. Configuration
+
+```python
+@dataclass(frozen=True)
+class EvidenceConfig:
+    enabled:                  bool = False
+    auto_extract:             bool = True              # let extractor propose drafts
+    extract_patterns:         tuple[str, ...] = ("kb_ingest","tool_fetch","federation_manifest")
+    claim_ttl_days:           int  = EVIDENCE_CLAIM_TTL_DAYS_DEFAULT          # 365
+    trust_default:            float = 0.5
+    dispute_min_trust:        float = EVIDENCE_DISPUTE_MIN_TRUST               # 0.3
+    ebkh_endpoint:            str | None = None
+    ebkh_token_scope:         str = "evidence-sync"
+    ebkh_sync_interval_minutes: int = 60
+    max_provenance_depth:     int = EVIDENCE_MAX_PROVENANCE_DEPTH              # 8
+    summary_max_predicates:   int = 32
+```
+
+Constants live in `hearthnet/constants.py`. `claim_ttl_days` does not delete claims — it marks them as stale for query purposes; the actual record is permanent.
+
+---
+
+## 8. Tests
+
+### 8.1 Unit
+
+- `test_claim_id_canonicalisation` — re-ordering source list or whitespace changes do not affect ClaimID.
+- `test_claim_signature_roundtrip`.
+- `test_evidence_level_disputed_wins` — a claim with two sources *and* a dispute returns `disputed`.
+- `test_provenance_trace_dedup` — diamond derivation graph yields each ancestor once.
+- `test_extractor_kb_ingest_pattern` — KB ingest event produces a draft with the right predicate.
+- `test_retraction_is_dispute` — a retraction shows up in `disputes_of`.
+
+### 8.2 Property
+
+- For random claims, `evidence_level(c)` is monotonic when adding sources/attestations and falls to `disputed` on adding a dispute.
+- For random derivation DAGs, `trace(root)` yields exactly the reachable set.
+
+### 8.3 Integration
+
+- KB ingest → claim drafted → operator asserts → query returns it.
+- Dispute lifecycle: assert claim, attest it, dispute it, see `evidence_level=disputed`, retract the dispute (as a dispute-of-the-dispute), verify level returns to `attested`.
+- EBKH adapter against a mock JSON-RPC endpoint: round-trip a spatial claim, verify the bbox query returns it.
+- Federated extraction: a federation manifest event from M14 produces a claim about advertised capabilities, which is then visible to subject-query `summarise_subject(<remote_node_id>)`.
+
+### 8.4 Negative
+
+- Cyclic derived_from input → `evidence_cycle_detected`.
+- Claim signed by an unknown identity → `claim_signature_invalid`.
+- EBKH endpoint configured but unreachable → `sync` returns partial; capability does not raise.
+
+---
+
+## 9. Cross-references
+
+- **Phase 1 M01 Identity** — every claim is signed; signatures verified against M01.
+- **Phase 1 M06 Files** — `doc:<FileID>` sources resolve to content-addressed files.
+- **Phase 1 M07 Knowledge Base** — KB ingest events feed the extractor.
+- **Phase 1 X02 Event Log** — `evidence.claim.*`, `evidence.dispute.*`, `evidence.attestation.*` events.
+- **Phase 2 M21 Tool Calls** — fetched URLs are extracted as claims for downstream provenance.
+- **Phase 3 M31 Civil Defense** — alerts carry a top-level claim ID, allowing recipients to trace why an alert was issued.
+- **Phase 3 X09 Conformance Suite** — provenance-trace correctness is part of the experimental suite.
+- **External: EBKH v3+** — Christof's existing event-sourced knowledge hub; PostGIS-backed; this module is the integration point.
+
+---
+
+## 10. Open research questions
+
+1. **Claim semantics.** "subject/predicate/object" is intentionally loose. Whether to adopt RDF, JSON-LD, or a custom ontology with a controlled vocabulary is unsettled. v3.0 accepts free strings and ships a small recommended vocabulary in the docs.
+
+2. **Trust composition.** When a derived claim depends on three parent claims of varying trust, what's the derived trust? Min, product, weighted-average? Currently the trust scorer ignores derivation. A future version may compose explicitly.
+
+3. **Dispute escalation.** Today a dispute is a single counter-claim. In practice, communities will want threaded discussion attached to disputes. Whether this belongs here or in M10/M25 chat is a design call.
+
+4. **Time-bound claims.** "The Volksbank opens at 9:00" is true on weekdays but not Sundays. The schema has no first-class temporal modality. A pragmatic workaround is encoding temporal qualifiers into the predicate ("opens_at_weekday"), but a proper temporal logic layer would be cleaner. Out of scope.
+
+5. **Confidentiality.** Some claims are sensitive (a neighbour's medical condition, a Feuerwehr member's home address). The current model has no claim-level access control. The capability-bus tokens (M16) can scope access to the evidence service entirely, but not at a per-claim granularity. Open.
+
+6. **OSINT integration boundaries.** EBKH ingests from external feeds. When does an external feed become a sufficiently authoritative source to upgrade evidence level? The pragmatic stance in v3.0 is "operator decides via the trust allowlist". A future version may automate this.
+
+7. **Visualisation.** Provenance trees get big fast. A graph visualisation widget (probably d3 in plain HTML) would help operators. Specced but unbuilt.
+
+8. **Federated claim propagation.** Two communities federate, and one asserts a claim relevant to the other. Should it auto-mirror? Today, no — claims propagate only when explicitly queried via `ebkh_sync` or fetched on demand. A push model would be possible but worsens consent.
+
+---
+
+*Last updated: spec v3.0.*

docs/p2_p3/M31-civil-defense.md

@@ -0,0 +1,410 @@
+# M31 — Civil Defense (NRW Bevölkerungsschutz Pilot)
+
+**Spec version:** v3.0 — *experimental*
+**Depends on:** [M03 Capability Bus](../../modules/M03-capability-bus.md), [X02 Event Log](../../cross-cutting/X02-events.md), [M01 Identity](../../modules/M01-identity.md), [M11 Notifications](../../modules/M11-notifications.md), [M14 Federation](../../phase-2/modules/M14-federation.md), [M16 Tokens](../../phase-2/modules/M16-tokens.md), [M30 Evidence](./M30-evidence-ebkh.md), [M29 LoRa Beacons](./M29-lora-beacons.md), [M22 Mobile Native](../../phase-2/modules/M22-mobile-native.md)
+**Depended on by:** nothing — terminal module; civil defense is a downstream consumer of everything else
+
+---
+
+## 1. Responsibility
+
+A scoped pilot for **NRW Bevölkerungsschutz**: integrate HearthNet with the role structures that Germany's civil-defence ecosystem actually uses (THW, DRK, Feuerwehr, Katastrophenschutz) so that during an incident, role-certified members can publish authenticated alerts, coordinate locally, and produce a tamper-evident audit trail that survives legal review.
+
+This module is deliberately **regional and regulated**. It does *not* try to be a global civil-defence platform. It encodes the role taxonomy, certificate semantics, and audit-retention rules that apply in Nordrhein-Westfalen, with hooks for other German Länder and EU regions to plug in later. The pilot lives in Issum and the Niederrhein because that's where Christof can actually walk into a Feuerwehrhaus and get this tested with humans who will use it under stress.
+
+Where the rest of HearthNet aims for "soft consensus across neighbours", this module aims for "hard provenance, signed by an authority, retained per legal mandate". Different ergonomics. Different threat model.
+
+---
+
+## 2. Non-goals
+
+- **Replacing official alert systems.** NINA, KATWARN, Cell-Broadcast, and BOS radio remain the authoritative channels. M31 is *complementary* — it works when official channels are degraded, congested, or geographically miss the affected area, and it carries the *local context* that mass-broadcast systems can't.
+- **Issuing legally binding evacuation orders.** Those come from the Krisenstab and are out of any AI-mediated system's authority.
+- **Modelling every German Land.** v3.0 targets NRW; M31 has a region adapter so others can be added, but the module ships with NRW only.
+- **Replacing TETRA-BOS.** Professional emergency-services radio is its own thing. We coexist; we don't interop.
+- **Automatic identity verification of certificate holders.** A role certificate carries who issued it and who it was issued to. *Verifying* that a holder is who they claim is the issuer's responsibility, not ours. We check the signature chain; we don't re-do the background check.
+- **Persistent geolocation of helpers.** We record where alerts target and where reported incidents are. We do not continuously track helpers' phones.
+
+---
+
+## 3. File layout
+
+```
+hearthnet/civdef/
+├── __init__.py
+├── service.py             # CivilDefenseService — capability handler
+├── alert.py               # Alert, AlertEnvelope, AlertSeverity dataclasses
+├── role.py                # RoleCertificate, role schemas per region
+├── audit.py               # Tamper-evident audit chain + export
+├── regions/
+│   ├── __init__.py
+│   ├── nrw.py             # NRW role taxonomy & issuer trust roots
+│   └── _stubs.py          # other Länder placeholders
+├── target.py              # Geographic / role / channel targeting
+└── ack.py                 # Acknowledgement collection
+```
+
+---
+
+## 4. Public API
+
+### 4.1 Dataclasses
+
+```python
+AlertID = NewType("AlertID", str)            # ULID
+AlertSeverity = Literal["info","advisory","warning","emergency","extreme"]
+
+@dataclass(frozen=True)
+class RoleCertificate:
+    cert_id:         str
+    holder:          NodeID
+    role:            str                       # canonical role, e.g. "DE.NRW.THW.OV.Leiter"
+    region:          str                       # "DE.NRW.KreisKleve"
+    issuer:          NodeID                    # issuing authority's HearthNet identity
+    issuer_chain:    tuple[NodeID, ...]        # chain back to a trust root
+    issued_at:       datetime
+    expires_at:      datetime
+    scopes:          frozenset[str]            # what this cert is allowed to do
+    signature:       bytes
+    revocation_url:  str | None
+
+@dataclass(frozen=True)
+class AlertTarget:
+    region:          str                       # "DE.NRW.KreisKleve.Issum"
+    bbox:            Bbox | None               # optional precise geo target
+    roles:           tuple[str, ...]           # which roles should see this; empty = public
+    channels:        tuple[Literal["push","lora","federation","local"], ...]
+
+@dataclass(frozen=True)
+class Alert:
+    alert_id:        AlertID
+    severity:        AlertSeverity
+    title:           str                       # ≤ 80 chars
+    body:            str                       # ≤ 1000 chars
+    target:          AlertTarget
+    instructions:    tuple[str, ...]           # short imperative lines
+    published_at:    datetime
+    expires_at:      datetime
+    publisher:       NodeID
+    publisher_role:  str
+    publisher_cert:  str                       # cert_id
+    evidence_claim:  ClaimID | None            # link to M30 claim chain if relevant
+    correlation_id:  str | None                # links to NINA/KATWARN ID if mirrored
+    signature:       bytes                     # publisher signs the alert
+    issuer_attestation: bytes | None           # optional co-sign by a higher-tier issuer
+
+@dataclass(frozen=True)
+class AlertEnvelope:
+    alert:           Alert
+    federation_hops: tuple[NodeID, ...]        # forward path for audit
+    received_at:     datetime
+    received_via:    Literal["bus","federation","lora_signal","manual"]
+
+@dataclass(frozen=True)
+class Ack:
+    alert_id:        AlertID
+    acker:           NodeID
+    acked_at:        datetime
+    status:          Literal["received","acting","need_help","standing_down","mistaken"]
+    note:            str                       # ≤ 280 chars
+    signature:       bytes
+
+@dataclass(frozen=True)
+class AuditEntry:
+    seq:             int                       # monotonic per audit chain
+    alert_id:        AlertID
+    event:           str                       # "published","forwarded","acked","mirrored","cancelled"
+    actor:           NodeID
+    at:              datetime
+    payload_sha:     str
+    prev_sha:        str                       # chain-link to previous audit entry
+    signature:       bytes
+```
+
+### 4.2 Capabilities
+
+All under `experimental.civdef.*`:
+
+```python
+async def civdef_alert_publish(draft: AlertDraft) -> AlertID
+async def civdef_alert_cancel(alert_id: AlertID, reason: str) -> CancelReceipt
+async def civdef_alert_list(active_only: bool = True,
+                            severity_min: AlertSeverity = "info") -> list[Alert]
+async def civdef_alert_get(alert_id: AlertID) -> AlertEnvelope
+async def civdef_alert_subscribe(target_filter: AlertTarget | None = None) -> AsyncIterator[AlertEnvelope]
+async def civdef_alert_ack(alert_id: AlertID, status: AckStatus, note: str = "") -> AckReceipt
+async def civdef_alert_acks(alert_id: AlertID) -> list[Ack]
+async def civdef_role_register(cert: RoleCertificate) -> RegisterReceipt
+async def civdef_role_list() -> list[RoleCertificate]
+async def civdef_role_revoke(cert_id: str, reason: str) -> RevokeReceipt    # issuer-only
+async def civdef_audit_export(alert_id: AlertID | None = None,
+                              since: datetime | None = None,
+                              format: Literal["jsonl","pdf"] = "jsonl") -> bytes
+```
+
+### 4.3 Service class
+
+```python
+class CivilDefenseService:
+    def __init__(self,
+                 bus: CapabilityBus,
+                 event_log: EventLog,
+                 identity: IdentityService,
+                 notifications: NotificationService,
+                 federation: FederationService,
+                 evidence: EvidenceService | None,
+                 region: RegionAdapter,
+                 audit_store: AuditChainStore,
+                 config: CivDefConfig): ...
+
+    async def publish_alert(self, draft: AlertDraft, publisher_cert: RoleCertificate) -> AlertID: ...
+    async def cancel_alert(self, alert_id: AlertID, reason: str, by_cert: RoleCertificate) -> None: ...
+    async def receive_alert(self, envelope: AlertEnvelope) -> None: ...
+    async def register_role(self, cert: RoleCertificate) -> None: ...
+    async def revoke_role(self, cert_id: str, by_cert: RoleCertificate, reason: str) -> None: ...
+    async def ack(self, alert_id: AlertID, status: AckStatus, note: str) -> AckReceipt: ...
+    async def export_audit(self, ...) -> bytes: ...
+```
+
+### 4.4 Region adapter
+
+```python
+class RegionAdapter(Protocol):
+    region_code: str
+    trust_roots: tuple[NodeID, ...]            # public keys of recognised issuers
+    role_schema: dict[str, RoleSpec]           # role name → spec
+    audit_retention_years: int
+    mandatory_severity_minimums: dict[str, AlertSeverity]  # role → max severity it can publish
+
+    def validate_role(self, cert: RoleCertificate) -> None: ...
+    def validate_alert(self, draft: AlertDraft, publisher_cert: RoleCertificate) -> None: ...
+```
+
+`regions/nrw.py` ships the NRW taxonomy with roles drawn from real-world structure: `DE.NRW.<Kreis>.<Gemeinde>.<Org>.<Role>`, e.g. `DE.NRW.Kleve.Issum.Feuerwehr.Wehrleiter`, `DE.NRW.Kleve.THW.OV.Leiter`, `DE.NRW.Kleve.DRK.Ortsverein.Bereitschaftsleiter`, `DE.NRW.Kleve.KatS.Stabsleiter`. Each role declares maximum severity it may publish, geographic scope it may target, and whether it may co-sign cross-org alerts.
+
+### 4.5 Audit chain store
+
+```python
+class AuditChainStore:
+    """Append-only, signed, hash-chained audit log.
+
+    Retention is governed by config.audit_retention_years; default is 10 (NRW pragmatic baseline,
+    operator must confirm against current Landesarchivgesetz at deployment time).
+    """
+    async def append(self, entry: AuditEntry) -> None: ...
+    async def latest(self) -> AuditEntry | None: ...
+    async def get_range(self, start_seq: int, end_seq: int) -> list[AuditEntry]: ...
+    async def verify_chain(self, start: int = 0, end: int | None = None) -> VerifyReport: ...
+    async def export(self, ...) -> bytes: ...
+```
+
+---
+
+## 5. Behaviour
+
+### 5.1 Role certification
+
+Role certificates form a chain to a regional trust root. NRW's trust roots are configured at deployment time and should match published issuer keys (Innenministerium NRW, the Kreis Kleve administration, etc. — note that as of v3.0 these *do not* publish HearthNet-compatible keys; the pilot uses a substitute issuance ceremony where the local Wehrleiter signs certificates after manual identity verification, and a clear migration path to real institutional keys is documented).
+
+A certificate may be:
+
+- **Issued** — signed by an authority that itself chains to a trust root.
+- **Active** — within validity window and not revoked.
+- **Revoked** — explicitly revoked by issuer; revocation is itself signed and appended to the audit chain.
+- **Expired** — past `expires_at`.
+
+Service operations that require a role check the certificate at every invocation. Revocations propagate via federation; a node receiving a revocation must, on next receipt of an alert signed by the revoked cert, refuse delivery and emit `civdef.alert.dropped.revoked`.
+
+### 5.2 Alert publication
+
+```
+publish_alert(draft, cert):
+  1. cert.holder must equal self.identity                         → else civdef_cert_not_owned
+  2. cert active, not revoked, not expired                        → else civdef_cert_invalid
+  3. region.validate_role(cert)                                   → else civdef_cert_unrecognised
+  4. region.validate_alert(draft, cert) (severity / scope match)  → else civdef_cert_out_of_scope
+  5. Construct Alert with publisher_role from cert.role
+  6. Sign Alert with self.identity
+  7. (optional) collect issuer_attestation if config requires co-sign
+  8. Append to audit chain: event="published"
+  9. Emit civdef.alert.published event
+ 10. Distribute:
+     - "local"      → notifications via M11 to local subscribers
+     - "push"       → mobile-native delivery via M22
+     - "federation" → M14 forwarding to federated nodes matching target.region
+     - "lora"       → if M29 enabled, set FLAG_PANIC on the next beacon as a presence-of-alert signal
+ 11. Optionally mirror to evidence graph (M30) as a claim record
+ 12. Return AlertID
+```
+
+If the publisher loses connectivity mid-publish, the audit-chain `published` entry has already been appended locally, so the alert is recoverable on reconnect and re-distributes from there. Idempotent on AlertID.
+
+### 5.3 Targeting
+
+`AlertTarget` is a set of orthogonal filters:
+
+- **region** — hierarchical region code; matches by prefix (`DE.NRW.Kleve` matches `DE.NRW.Kleve.Issum`).
+- **bbox** — optional geographic bounding box (overrides region for the precise area).
+- **roles** — empty means public; non-empty restricts visibility to certificate holders of those roles.
+- **channels** — which delivery mechanisms to use.
+
+A receiving node filters on its own identity's location, registered roles, and active subscriptions. The filter is enforced **client-side at delivery** as well as **publisher-side at distribution**, so a node that mis-claims a role doesn't expose role-only content (the federation forwarder uses publisher-side filtering when forwarding `roles`-restricted alerts).
+
+### 5.4 Acknowledgements
+
+When a role-targeted alert arrives, the recipient may ack with a status:
+
+- `received` — read confirmation.
+- `acting` — operationally taking action (e.g., Feuerwehr en route).
+- `need_help` — recipient cannot act; help requested.
+- `standing_down` — alert handled, recipient disengages.
+- `mistaken` — the recipient believes this alert is in error; an attached `note` should explain.
+
+Acks are signed, appended to the audit chain, and visible to the publisher via `civdef.alert.acks(alert_id)`. Public alerts (no `roles` filter) suppress acks unless `config.allow_public_ack=true` — to prevent ack floods on widely-distributed alerts.
+
+### 5.5 Cancellation
+
+Cancellation requires a certificate with cancel scope (typically the original publisher or a same-or-higher role in the same region). A cancellation:
+
+1. Records the cancellation in the audit chain.
+2. Emits `civdef.alert.cancelled` to all original delivery channels.
+3. Marks the alert inactive in `civdef_alert_list` queries (`active_only=true`).
+
+The original alert is not deleted. Audit retention applies to the cancellation as well.
+
+### 5.6 Audit chain
+
+The audit chain is an append-only, hash-chained, signed log specific to this module. Each entry's `prev_sha` is the SHA-256 of the previous entry's canonicalised body, creating a tamper-evident chain. `verify_chain` walks from genesis (or a checkpoint) verifying signatures and hashes; failure raises `civdef_audit_chain_broken` and is surfaced as a high-priority operator notification.
+
+Audit entries cover: alert published, alert forwarded (with federation hop), alert acked, alert cancelled, role certificate registered, role certificate revoked, audit chain checkpointed. Export produces `jsonl` (machine-readable, default) or `pdf` (operator-readable for legal review, generated via the public `pdf` skill).
+
+Retention is governed by `CIVDEF_AUDIT_RETENTION_YEARS` (default 10 — operator must validate against current NRW Landesarchivgesetz at deployment; the constant is the recommendation, not the law).
+
+### 5.7 Federation interaction
+
+Alerts cross federation boundaries via M14. The federation manifest must declare `civdef` as an advertised capability; otherwise the alert is not forwarded into the neighbouring community. Forwarding nodes append themselves to `AlertEnvelope.federation_hops` for audit, but do not re-sign the alert (the publisher's signature is the source of truth). The receiving community independently audits the alert against its own role schemas; if the publisher's role is not recognised, the alert is delivered with a `civdef.alert.foreign_role` flag and is *not* surfaced as a high-severity push.
+
+### 5.8 LoRa interaction
+
+LoRa beacons (M29) carry no alert content; they carry only presence. When the local node receives a `severity ∈ {emergency, extreme}` alert and LoRa is enabled, the node sets `FLAG_PANIC` on its next beacon and increases beacon cadence to the panic-burst configured in M29. This is a *signal* that something is happening, not a *content* channel. Receivers must consult bus or notifications for the actual alert content.
+
+### 5.9 Failure modes
+
+- **Publisher's cert revoked after publish, before propagation completes**: federation forwarders that have received the revocation drop the in-flight alert; nodes that have not yet seen the revocation propagate normally. Eventually consistent; documented limitation.
+- **Audit chain corruption** (disk failure, manual tampering): `verify_chain` detects; the module enters degraded mode where new publishes are blocked until an operator acknowledges and re-checkpoints. Reads continue.
+- **Trust root key compromise**: out of scope for v3.0 to *recover* automatically; documented incident response: revoke all certs chaining to the compromised root, rotate root, reissue.
+- **Mass-ack flood**: `allow_public_ack=false` default; per-alert ack rate-limit `CIVDEF_ACK_MAX_PER_MINUTE_PER_NODE`.
+
+---
+
+## 6. Errors
+
+| Code                              | When                                                              |
+|-----------------------------------|-------------------------------------------------------------------|
+| `experimental_disabled`           | Capability called with the flag off                               |
+| `civdef_cert_not_owned`           | Publish/ack with a cert whose holder ≠ caller's identity          |
+| `civdef_cert_invalid`             | Certificate expired, revoked, or signature broken                 |
+| `civdef_cert_unrecognised`        | Issuer chain doesn't terminate at a configured trust root         |
+| `civdef_cert_out_of_scope`        | Cert's role/region doesn't authorise the requested action         |
+| `civdef_alert_not_found`          | Operation references an unknown AlertID                           |
+| `civdef_alert_target_invalid`     | Target region/bbox malformed or outside the issuer's scope        |
+| `civdef_audit_chain_broken`       | Hash or signature mismatch in the audit chain                     |
+| `civdef_role_revoked`             | Operation attempted with a revoked certificate                    |
+| `civdef_region_unsupported`       | No region adapter loaded for the requested region                 |
+| `civdef_ack_rate_limited`         | Ack rate exceeded for this alert from this node                   |
+
+---
+
+## 7. Configuration
+
+```python
+@dataclass(frozen=True)
+class CivDefConfig:
+    enabled:                     bool = False
+    region:                      str = "DE.NRW"
+    audit_retention_years:       int = CIVDEF_AUDIT_RETENTION_YEARS                # 10
+    require_issuer_cosign:       dict[AlertSeverity, bool] = field(default_factory=lambda: {
+        "info": False, "advisory": False, "warning": False,
+        "emergency": True, "extreme": True,
+    })
+    allow_public_ack:            bool = False
+    ack_max_per_minute_per_node: int = CIVDEF_ACK_MAX_PER_MINUTE_PER_NODE           # 5
+    federation_forward:          bool = True
+    lora_panic_signal:           bool = True
+    severity_push_threshold:     AlertSeverity = "warning"   # below this, no mobile push
+    trust_roots_extra:           tuple[NodeID, ...] = ()      # operator-added roots
+    region_adapter_overrides:    dict[str, str] = field(default_factory=dict)
+```
+
+Constants centralised in `hearthnet/constants.py`.
+
+---
+
+## 8. Tests
+
+### 8.1 Unit
+
+- `test_role_cert_chain_to_root` — cert with valid chain → accepted; broken chain → rejected.
+- `test_role_cert_expired` — past `expires_at` → `civdef_cert_invalid`.
+- `test_alert_signature_roundtrip`.
+- `test_target_region_prefix_match` — `DE.NRW.Kleve` matches `DE.NRW.Kleve.Issum`, not `DE.NRW.Wesel`.
+- `test_audit_chain_link` — appending entries chains correctly; `verify_chain` returns ok.
+- `test_audit_chain_tamper_detected` — flip a byte in the middle; `verify_chain` reports the break.
+- `test_severity_cap_per_role` — Wehrleiter publishing `extreme` → `civdef_cert_out_of_scope` if schema caps at `emergency`.
+- `test_revocation_propagates` — revoke cert; subsequent alerts from that cert dropped.
+
+### 8.2 Integration
+
+- Two-node alert flow: node A (Wehrleiter cert) publishes `warning` alert targeting `DE.NRW.Kleve.Issum`; node B (resident in Issum, no cert) receives via M11 push.
+- Role-targeted alert: A publishes alert with `roles=("DE.NRW.Kleve.THW.OV.Leiter",)`; B (without cert) does not receive; C (with cert) does.
+- Federation: A publishes in community X; X federates to Y; Y's resident D receives with `federation_hops=[X]`.
+- Cancellation: A cancels; B's alert list moves it to inactive.
+- Audit export: publish, ack, cancel; export `jsonl`; round-trip parses and `verify_chain` passes.
+
+### 8.3 Negative / adversarial
+
+- Forged cert chain (random issuer key) → `civdef_cert_unrecognised`.
+- Targeting `DE.BY` (outside NRW) from an NRW-only cert → `civdef_alert_target_invalid`.
+- Ack flood beyond rate limit → `civdef_ack_rate_limited`.
+- Tampered audit chain → publish blocked until operator re-checkpoint.
+
+### 8.4 Tabletop
+
+- Manual scenarios with Issum Feuerwehr volunteers: simulated Hochwasser event, simulated grid outage, simulated industrial incident on the A57. Goals: latency from alert publication to first ack, false-positive ack rate, operator-perceived clarity of UI under stress.
+
+---
+
+## 9. Cross-references
+
+- **Phase 1 M01 Identity** — every cert, alert, ack, and audit entry is signed against M01 identities.
+- **Phase 1 M11 Notifications** — alerts surface via notifications with priority mapped from `severity`.
+- **Phase 2 M14 Federation** — alerts cross community boundaries via federation.
+- **Phase 2 M16 Tokens** — cert validation reuses M16's signature primitives; alert distribution endpoints require `civdef-receive` scoped tokens.
+- **Phase 2 M22 Mobile Native** — mobile push for `severity ≥ severity_push_threshold`.
+- **Phase 3 M29 LoRa Beacons** — `FLAG_PANIC` corroboration during emergencies.
+- **Phase 3 M30 Evidence** — alerts may carry an `evidence_claim` ClaimID; recipients can `evidence.provenance.trace` to see the reasoning chain.
+- **Phase 3 X09 Conformance Suite** — civdef has a dedicated conformance section because of audit-chain integrity requirements.
+
+---
+
+## 10. Open research questions
+
+1. **Real institutional keys.** v3.0 uses substitute issuance because NRW authorities do not (yet) publish HearthNet-compatible keys. The migration path — getting the Innenministerium or Kreis Kleve to publish keys and sign initial role certs — is a political process, not a technical one. Documented; out of code scope.
+
+2. **NINA / KATWARN bridge.** A read-only mirror that pulls public NINA alerts and republishes them locally with a `correlation_id` is plausible and would be valuable. Whether it's M31's job or a separate bridge module is undecided.
+
+3. **Multi-Land schema.** The NRW role taxonomy is concrete; Bayern, Niedersachsen, Hessen each have variations (especially around KatS structures). A community-contributed `regions/` directory is the plan; v3.0 ships only NRW.
+
+4. **Co-signing UX.** When `require_issuer_cosign=true` for emergencies, the publisher must obtain a co-signature from a higher-tier issuer. Latency-sensitive. A pre-delegated "emergency co-sign authority" mechanism (similar to OCSP-stapling for certs) is the obvious extension. Not in v3.0.
+
+5. **Public-ack ergonomics.** Public alerts with `allow_public_ack=true` would let citizens self-report ("I am safe", "I need help"), but the failure modes (ack flood, false reports) are severe enough that v3.0 defaults this off. A future tier with rate limits and ack-content moderation is plausible.
+
+6. **Legal retention.** `CIVDEF_AUDIT_RETENTION_YEARS=10` is the operator-friendly default. Actual legal retention varies (NRW Landesarchivgesetz, federal data retention rules for civil-defence records, GDPR exceptions for vital interests). The deployment guide must explicitly walk operators through this; we cannot guess from code.
+
+7. **Cross-border alerts.** Issum borders the Netherlands. An alert about a Dutch industrial incident might originate from a Dutch system. Cross-border interop is interesting and outside v3.0 scope. The `region` adapter pattern doesn't preclude it.
+
+8. **Drills and false-alarm semantics.** A drill should look real enough to be useful and clearly different enough to not panic non-participants. A `drill=true` flag on Alert is the obvious addition; v3.0 omits it pending feedback from real drill rehearsals.
+
+---
+
+*Last updated: spec v3.0.*

docs/p2_p3/M32-protocol-standard.md

@@ -0,0 +1,323 @@
+# M32 — Protocol Standardisation & Conformance
+
+**Spec version:** v3.0 — *experimental*
+**Depends on:** [M03 Capability Bus](../../modules/M03-capability-bus.md), [X02 Event Log](../../cross-cutting/X02-events.md), [M14 Federation](../../phase-2/modules/M14-federation.md), [X09 Conformance Suite](../cross-cutting/X09-conformance-suite.md)
+**Depended on by:** anyone building an alternate implementation of HearthNet
+
+---
+
+## 1. Responsibility
+
+Turn the HearthNet specs from "Christof's working code with documentation" into something a **second team** could implement compatibly. This module is the bookkeeping for that: a versioned protocol document set, a conformance reporting capability, governance for how the spec changes, and a registry of known implementations and their conformance levels.
+
+The premise is that long-term, a single implementation is fragile. If HearthNet only works as long as one person maintains it, it doesn't survive. Standardising the protocol (the wire formats, the capability contracts, the federation semantics) so other implementations can interoperate is the path to durability — and it sets up the social and legal scaffolding (versioning policy, change process, conformance claims) that other projects will need to take HearthNet seriously.
+
+This is not "rewrite HearthNet as an RFC". It's "package what already exists in a form that can be cited, versioned, and conformance-tested". The reference implementation is HearthNet itself; the protocol document is the contract; the conformance suite (X09) is the proof.
+
+---
+
+## 2. Non-goals
+
+- **Submitting to IETF / W3C in v3.0.** That's a multi-year governance process and is out of scope. We make ourselves *ready* for it by structuring the documents and adopting RFC-style versioning conventions, but we don't file anything.
+- **Patent licensing.** There are no patented techniques in HearthNet's core (capability bus, event log, federation, etc. — all build on well-known primitives). We document this assumption but do not run a patent review.
+- **Trademark of "HearthNet".** Out of scope. If the protocol survives, someone (probably ki-fusion-labs.de) eventually claims the name; v3.0 doesn't deal with that.
+- **Conformance certification with a fee structure.** Conformance reports are self-published and free. No "HearthNet Inside™ certification programme".
+- **Backward compatibility forever.** Protocol versions can have breaking changes between major versions, with documented migration. Within a minor version, compatibility is required.
+
+---
+
+## 3. File layout
+
+The module is unusual in that most of its content lives *outside* the `hearthnet/` Python package — in a top-level `protocol/` directory at the repository root.
+
+```
+protocol/                                # repo-root sibling of hearthnet/
+├── README.md
+├── VERSION                              # current protocol version (e.g. 3.0.0)
+├── CHANGELOG.md
+├── governance.md                        # change process, decision rights
+├── versioning.md                        # semver-with-twists rules
+├── reference-implementations.md         # registry
+├── core/
+│   ├── 01-identity-and-addressing.md
+│   ├── 02-transport.md
+│   ├── 03-capability-bus.md
+│   ├── 04-event-log.md
+│   ├── 05-tokens.md
+│   ├── 06-federation.md
+│   └── ...
+└── experimental/
+    ├── 30-evidence.md
+    └── 31-civil-defense.md
+
+hearthnet/protocol/
+├── __init__.py
+├── service.py                           # ProtocolService — registry and report capability
+├── registry.py                          # In-memory + persisted registry of known impls
+└── report.py                            # Conformance report dataclass + serialiser
+```
+
+The `protocol/` directory is the **specification artefact** — versioned alongside code but conceptually independent. The Python `hearthnet/protocol/` module is the thin runtime surface that lets a HearthNet node expose its conformance information on the bus.
+
+---
+
+## 4. Public API
+
+### 4.1 Dataclasses
+
+```python
+@dataclass(frozen=True)
+class ProtocolVersion:
+    major:  int
+    minor:  int
+    patch:  int
+    suffix: str = ""        # "", "rc1", "experimental"
+
+    def __str__(self) -> str: ...        # "3.0.0" or "3.0.0-rc1"
+    def is_compatible_with(self, other: ProtocolVersion) -> bool: ...   # same major
+
+@dataclass(frozen=True)
+class ImplementationDescriptor:
+    name:           str                  # e.g. "hearthnet-reference"
+    vendor:         str                  # e.g. "ki-fusion-labs.de"
+    version:        str                  # implementation version, e.g. "0.4.2"
+    protocol_versions: tuple[ProtocolVersion, ...]   # which protocol versions supported
+    homepage_url:   str | None
+    contact:        str | None
+
+@dataclass(frozen=True)
+class ConformanceReport:
+    implementation:  ImplementationDescriptor
+    protocol_version: ProtocolVersion
+    suite_version:   str                   # X09 conformance suite version
+    ran_at:          datetime
+    sections: dict[str, SectionResult]     # section name → result
+    overall: Literal["pass","fail","partial","skipped"]
+    signature: bytes                       # implementation signs its own report
+
+@dataclass(frozen=True)
+class SectionResult:
+    name:          str
+    total:         int
+    passed:        int
+    failed:        int
+    skipped:       int
+    failures:      tuple[FailureDetail, ...]
+```
+
+### 4.2 Capabilities
+
+```python
+async def protocol_version_list() -> list[ProtocolVersion]
+async def protocol_self_describe() -> ImplementationDescriptor
+async def protocol_conformance_report(suite_version: str | None = None) -> ConformanceReport
+async def protocol_registry_list() -> list[ImplementationDescriptor]
+async def protocol_registry_announce(descriptor: ImplementationDescriptor) -> AnnounceReceipt
+```
+
+These are stable (non-experimental) capabilities — the protocol must include its own self-description and conformance-reporting capability, otherwise interop is impossible.
+
+### 4.3 Service class
+
+```python
+class ProtocolService:
+    def __init__(self,
+                 bus: CapabilityBus,
+                 event_log: EventLog,
+                 federation: FederationService,
+                 registry: ImplementationRegistry,
+                 conformance_runner: ConformanceRunner | None,
+                 config: ProtocolConfig): ...
+
+    def supported_versions(self) -> list[ProtocolVersion]: ...
+    def self_descriptor(self) -> ImplementationDescriptor: ...
+    async def run_conformance(self, suite_version: str | None = None) -> ConformanceReport: ...
+    async def announce(self, descriptor: ImplementationDescriptor) -> None: ...   # to local registry + federation
+    async def registry_list(self) -> list[ImplementationDescriptor]: ...
+```
+
+### 4.4 Implementation registry
+
+```python
+class ImplementationRegistry:
+    """Local registry of known implementations.
+
+    Populated by:
+      - self (this node, on startup)
+      - federation peers' announcements
+      - operator-curated additions
+    """
+    async def upsert(self, descriptor: ImplementationDescriptor, source: NodeID) -> None: ...
+    async def list(self) -> list[ImplementationDescriptor]: ...
+    async def known_by_name(self, name: str) -> list[ImplementationDescriptor]: ...
+```
+
+---
+
+## 5. Behaviour
+
+### 5.1 Versioning policy
+
+Protocol versions follow **semver with explicit stability tiers**:
+
+- **Major** (`X.0.0`): breaking changes to wire formats or capability contracts. New major version requires explicit migration documentation. Old majors remain readable for migration purposes for at least 2 years.
+- **Minor** (`X.Y.0`): additive — new capabilities, new event types, new optional fields. Backward compatibility within the major is required: a `3.0` impl talking to a `3.2` impl must work, with the `3.0` side ignoring new fields/events.
+- **Patch** (`X.Y.Z`): clarification, typo fixes, no functional change.
+- **Suffix** `-experimental`: capabilities in the `experimental.*` namespace; can change without bumping major.
+
+Each protocol document carries a frontmatter `protocol-version: 3.X.Y` that names the smallest version that contains it. The `protocol/VERSION` file is the current latest. The `CHANGELOG.md` lists every diff between versions.
+
+### 5.2 Stability tiers
+
+Capabilities are tagged with one of:
+
+- `stable` — frozen at the major version; any change is a breaking change.
+- `provisional` — expected to become stable; minor-version breaking changes allowed with deprecation period.
+- `experimental` — `experimental.*` namespace; may change or vanish.
+
+The `protocol_self_describe` capability reports which capabilities the implementation supports and at which tier. A capability marked `stable` in the protocol but implemented at `experimental` tier in the node is a configuration error and is logged at startup.
+
+### 5.3 Conformance reporting
+
+A `ConformanceReport` is the artefact produced by running the X09 conformance suite against a running node. The report is:
+
+- **Self-signed** by the implementation — there is no central authority that "certifies" reports.
+- **Reproducible** — the suite version is in the report; running the same suite against the same impl should produce equivalent results modulo timestamps.
+- **Public** — implementations are encouraged to publish their reports openly (in their repo, on their website, federated via the registry).
+- **Honest about partial conformance** — `partial` is a valid outcome and is more useful than a misleading `pass`.
+
+Reports do not expire, but a report from suite version `1.0.0` is not equivalent to a report from `2.0.0`. The X09 suite versions independently of the protocol.
+
+### 5.4 Implementation registry & federation
+
+Each HearthNet node announces its `ImplementationDescriptor` to its federation peers on connect. Peers add it to their local registry. Operators can query their local registry for "who else is out there" via `protocol_registry_list`.
+
+The registry is *advisory*. There is no trust beyond "this peer claimed this descriptor at this time". A peer claiming to be an implementation it isn't is a security incident, but the registry doesn't authenticate vendor names — only that the node signed its descriptor.
+
+### 5.5 Governance (documented, not enforced)
+
+The `protocol/governance.md` document describes how protocol changes happen:
+
+1. **Proposal**: any contributor writes a "change note" as a PR to `protocol/`. Includes motivation, exact spec diff, migration story, and conformance impact.
+2. **Discussion**: open period (default 4 weeks) for review.
+3. **Decision**: maintainers (initially just Christof, ideally expanding to a small group) accept, reject, or request revision. Rejections are logged with rationale.
+4. **Merge**: accepted change merges to `protocol/main` with version bump per §5.1 rules.
+5. **Release**: tagged release of the protocol document set independent of code releases.
+
+This is a process document; the module does not *enforce* governance technically. Enforcement is social.
+
+### 5.6 Reference implementations registry
+
+`protocol/reference-implementations.md` is a living document listing known implementations. v3.0 entries:
+
+- **hearthnet-reference** (Python, this codebase). Status: complete, all stable + provisional + experimental capabilities.
+- (placeholder for a second impl when one exists).
+
+Adding an implementation to this document requires a PR demonstrating at minimum a passing conformance report for `core` sections of the X09 suite at the current major version.
+
+### 5.7 Migration documentation
+
+Each major version transition ships a `protocol/migration/X-to-Y.md` document. v3.0 includes:
+
+- `protocol/migration/2-to-3.md` — placeholder for now; v3.0 introduces only experimental capabilities, so a strict v2→v3 migration is effectively a no-op for stable code paths.
+
+### 5.8 Failure modes
+
+- **Mismatched protocol versions in federation**: handled by M14 federation manifest version negotiation. M32 itself doesn't intervene at runtime; it just reports.
+- **Conformance suite not present**: `protocol.conformance.report` returns `skipped` overall with reason `suite_not_installed`.
+- **Conflicting registry entries** (same `name` claimed by two distinct vendors): both stored; the registry list returns both; operators decide.
+
+---
+
+## 6. Errors
+
+| Code                          | When                                                              |
+|-------------------------------|-------------------------------------------------------------------|
+| `protocol_version_unknown`    | Operation references a protocol version not in our table         |
+| `protocol_suite_not_installed`| Conformance report requested but X09 not available               |
+| `protocol_descriptor_invalid` | Announcement with malformed descriptor                            |
+| `protocol_unsupported_capability` | Federation negotiation finds no compatible major version       |
+
+---
+
+## 7. Configuration
+
+```python
+@dataclass(frozen=True)
+class ProtocolConfig:
+    enabled:                  bool = True       # this one is enabled by default
+    supported_versions:       tuple[str, ...] = ("3.0.0",)
+    default_announce_version: str = "3.0.0"
+    descriptor:               ImplementationDescriptor = field(default_factory=lambda: ImplementationDescriptor(
+        name="hearthnet-reference",
+        vendor="ki-fusion-labs.de",
+        version="0.4.2",
+        protocol_versions=(ProtocolVersion(3, 0, 0),),
+        homepage_url="https://ki-fusion-labs.de/hearthnet",
+        contact=None,
+    ))
+    announce_to_federation:   bool = True
+    conformance_auto_run_on_startup: bool = False
+    registry_max_entries:     int = 4096
+```
+
+---
+
+## 8. Tests
+
+### 8.1 Unit
+
+- `test_protocol_version_compat` — same major compatible; different major not.
+- `test_descriptor_signature_roundtrip`.
+- `test_registry_upsert_idempotent`.
+- `test_conformance_report_signed` — self-signed report's signature verifies against the implementation's identity.
+- `test_protocol_version_parse` — `"3.0.0"`, `"3.0.0-rc1"`, `"3.0.0-experimental"` parse correctly.
+
+### 8.2 Integration
+
+- Two nodes (same impl, same version): announce → registry shows both.
+- Two nodes (same impl, different versions): registry shows both; federation negotiates highest compatible.
+- Run conformance suite against the reference impl: must pass `core/*` sections by definition (the suite is built to match the spec).
+
+### 8.3 Spec-document tests
+
+- `test_protocol_documents_present` — every protocol document referenced in `protocol/README.md` exists.
+- `test_protocol_version_consistent` — `protocol/VERSION` matches the `default_announce_version` in code.
+- `test_changelog_format` — `protocol/CHANGELOG.md` parses as a sequence of versioned entries with semver-valid ordering.
+
+### 8.4 Negative
+
+- Malformed descriptor → `protocol_descriptor_invalid`.
+- Federation peer announces protocol `99.0.0` → registry stores it, federation negotiation declines.
+
+---
+
+## 9. Cross-references
+
+- **All Phase 1, 2, 3 modules** — they collectively *are* the protocol. M32's job is the meta-layer.
+- **Phase 2 M14 Federation** — federation manifest carries protocol version; negotiation uses M32's compat rules.
+- **Phase 3 X09 Conformance Suite** — produces the data that M32's report capability surfaces.
+
+---
+
+## 10. Open research questions
+
+1. **Independent implementation.** The protocol is real only when a second implementation exists. v3.0 ships only the reference. A small "minimal HearthNet" written in Go or Rust as a contrast implementation would prove the spec is implementable from the documents alone. Concrete next step, but not in v3.0.
+
+2. **Formal verification of wire formats.** TLS-style formal proofs of the federation handshake and capability-bus dispatch would be valuable. Out of v3.0 scope; documented as a research direction.
+
+3. **Governance bootstrapping.** "Christof decides" is fine for now and honest about the project's state. Transitioning to a multi-maintainer model needs a path — a Tech Steering Committee, a foundation, or simply a documented succession plan. Currently undefined.
+
+4. **Standards-body engagement.** If the protocol matures, IETF (for federation/transport) and W3C (for capability-bus semantics if they look RPC-like) are plausible homes. v3.0 deliberately avoids premature standards engagement; the bar is "second implementation exists and is interoperable".
+
+5. **Legal entity.** ki-fusion-labs.de is currently the operating entity. Whether a separate legal entity (e.V., foundation) is needed for a multi-vendor protocol is a real question. Out of code scope.
+
+6. **Trademark and naming.** "HearthNet" as a trademark is undefined; the protocol could be renamed to something more obviously generic at standardisation time. The reference impl can keep the name.
+
+7. **Optionality flags vs separate profiles.** A node might support `core` only, or `core + federation`, or everything. Whether to model this as per-capability optionality (current approach) or named profiles (`HearthNet-Core`, `HearthNet-Federated`, `HearthNet-Civdef`) is a design question that needs feedback from a second impl team.
+
+8. **Conformance suite drift.** The X09 suite is the source of truth for "what does conformance mean"; the protocol documents describe the *intent*. When the two disagree, currently the suite wins (because it's executable). This is pragmatic but not principled. A future version may flip this and use the suite to *test* the documents, with the documents as primary.
+
+---
+
+*Last updated: spec v3.0.*

docs/p2_p3/X05-dht.md

@@ -0,0 +1,374 @@
+# X05 — Distributed Hash Table (DHT)
+
+**Spec version:** v1.0 (Phase 2)
+**Depends on:** M01 (identity), X01 (transport), X04 (config), X03 (observability)
+**Depended on by:** M14 (federation discovery), M07 ext (background blob replication via content routing), M02 ext (cross-LAN peer discovery), M15 (relay bootstrap)
+
+---
+
+## 1. Responsibility
+
+Provide a Kademlia-style DHT over the internet that lets:
+
+- A node find peers of its own community across LANs (cross-LAN extension of M02)
+- A node find sources of a specific CID across communities (extension of M07's local source index)
+- A node bootstrap into a federation (find an anchor of community X without knowing its IP)
+
+Out of scope:
+- Permanent storage in the DHT — DHT entries are TTL'd advertisements only
+- Anonymity (no onion routing — there's no anonymity goal)
+- Sybil resistance — communities are the trust roots; DHT is an unreliable hint layer
+
+---
+
+## 2. File layout
+
+```
+hearthnet/dht/
+├── __init__.py
+├── kademlia.py        # KademliaNode, routing table
+├── routing.py         # FindNode, FindValue, Store, Ping RPCs
+├── storage.py         # local DHT k/v store with TTL
+└── bootstrap.py       # bootstrap peer list, NAT-aware reachability
+```
+
+---
+
+## 3. Concepts
+
+### 3.1 Key space
+
+XOR-distance over a 256-bit key space. Keys are derived as:
+
+- For peers: `key = blake3(node_id_full)[:32]`
+- For CIDs: `key = blake3(cid_string)[:32]`
+- For communities: `key = blake3(community_id_full)[:32]`
+
+### 3.2 Bucket structure
+
+Standard Kademlia: 256 buckets of size `DHT_REPLICATION_K = 8` (from Phase 2 constants). Concurrent lookups: `DHT_ALPHA = 3`.
+
+### 3.3 Values stored
+
+The DHT does **not** store community state. It stores small, signed advertisements:
+
+| Value type | Key | Value (signed) | TTL |
+|------------|-----|----------------|-----|
+| Peer presence | `blake3(node_id)` | `{endpoints, community_id, expires_at}` | matches manifest TTL (30s) |
+| CID source | `blake3(cid)` | `{node_id, last_seen}` | 1 hour |
+| Community bootstrap | `blake3(community_id)` | `{anchor_node_ids, endpoints, manifest_url}` | 24 hours |
+
+The DHT is a **hint cache**. Authoritative state lives in community event logs.
+
+---
+
+## 4. Public API
+
+### 4.1 `kademlia.py`
+
+```python
+# hearthnet/dht/kademlia.py
+from dataclasses import dataclass
+
+@dataclass(frozen=True)
+class DhtContact:
+    node_key:    bytes            # 32 bytes
+    node_id_full: str
+    endpoint:    Endpoint
+    last_seen:   float
+
+@dataclass(frozen=True)
+class DhtValue:
+    """A stored advertisement. The payload is a signed dict."""
+    key:        bytes
+    payload:    dict              # has 'signature' field
+    expires_at: int               # unix seconds
+
+class KademliaNode:
+    """One node's view of the DHT.
+       Provides high-level find_node / find_value / store APIs."""
+
+    def __init__(
+        self,
+        kp: KeyPair,
+        endpoint: Endpoint,
+        transport_client: HttpClient,
+        bootstrap_endpoints: list[Endpoint],
+    ):
+        ...
+
+    async def start(self) -> None:
+        """Bootstrap: ping bootstrap_endpoints, populate routing table."""
+
+    async def stop(self) -> None: ...
+
+    # --- public lookups ---
+
+    async def find_node(self, target_key: bytes) -> list[DhtContact]:
+        """Return the k closest contacts to target_key."""
+
+    async def find_value(self, key: bytes) -> list[DhtValue]:
+        """Return values stored at this key (or empty)."""
+
+    async def store(self, value: DhtValue) -> int:
+        """Replicate to k closest nodes. Returns count of successful stores."""
+
+    # --- maintenance ---
+
+    async def refresh_buckets(self) -> None:
+        """Per DHT_REFRESH_SECONDS: ping a random key in each bucket to liveness-check it."""
+
+    async def republish_values(self) -> None:
+        """Per DHT_REPUBLISH_SECONDS: re-store our own advertisements so TTL doesn't expire them."""
+
+    # --- introspection ---
+
+    def routing_table_size(self) -> int: ...
+    def stored_values(self) -> int: ...
+```
+
+### 4.2 `routing.py`
+
+Wire RPCs exposed by the bus transport (X01) as additional endpoints under `/dht/v1/`:
+
+| Endpoint | Method | Purpose |
+|----------|--------|---------|
+| `/dht/v1/ping` | POST | Liveness, exchange contact info |
+| `/dht/v1/find_node` | POST | Return k closest contacts to a key |
+| `/dht/v1/find_value` | POST | Return values at a key OR closest contacts if absent |
+| `/dht/v1/store` | POST | Accept a value into local storage (if we're among the k closest) |
+
+```python
+# hearthnet/dht/routing.py
+async def serve_ping(req: dict) -> dict: ...
+async def serve_find_node(req: dict, kademlia: KademliaNode) -> dict: ...
+async def serve_find_value(req: dict, kademlia: KademliaNode) -> dict: ...
+async def serve_store(req: dict, kademlia: KademliaNode) -> dict: ...
+
+# Request / response shapes documented inline in each function.
+```
+
+### 4.3 `storage.py`
+
+```python
+# hearthnet/dht/storage.py
+class DhtStore:
+    """Local key-value store with TTL eviction.
+       Backing: SQLite in <DATA>/dht/store.sqlite."""
+
+    def __init__(self, db_path: Path):
+        ...
+
+    def put(self, value: DhtValue) -> bool:
+        """Idempotent. Returns True if stored (we're in k closest), False if rejected."""
+
+    def get(self, key: bytes) -> list[DhtValue]:
+        """Return non-expired values for this key."""
+
+    def evict_expired(self) -> int: ...
+
+    def size(self) -> int: ...
+```
+
+### 4.4 `bootstrap.py`
+
+```python
+# hearthnet/dht/bootstrap.py
+
+DEFAULT_BOOTSTRAP_NODES: list[Endpoint] = [
+    # Filled at packaging time with community-run bootstrap endpoints.
+    # Christof's relay.hearthnet.de will be a default.
+]
+
+async def is_reachable(endpoint: Endpoint, timeout_seconds: float = 5) -> bool:
+    """Send a ping; return True if responded."""
+
+async def discover_external_ip() -> str | None:
+    """Use STUN against a public STUN server to learn our external IP.
+       Used by relay-assisted bootstrap to advertise reachable endpoints."""
+```
+
+---
+
+## 5. Behaviour
+
+### 5.1 Advertisement lifecycle
+
+```
+Node starts → KademliaNode.start()
+  → ping bootstrap_endpoints; build initial routing table
+  → store our peer presence: store(DhtValue(blake3(node_id), {...}, ttl=30s))
+  → store community bootstrap: store(DhtValue(blake3(community_id), {anchors, ...}, ttl=24h))
+  → for each pinned CID: store(DhtValue(blake3(cid), {node_id, ...}, ttl=1h))
+  ↓
+Every MANIFEST_REPUBLISH_INTERVAL_SECONDS: re-store peer presence
+Every DHT_REPUBLISH_SECONDS: re-store all our advertisements
+Every DHT_REFRESH_SECONDS: refresh routing table buckets
+```
+
+### 5.2 Lookup integration with M02
+
+When [M02 PeerRegistry](../../modules/M02-discovery.md) doesn't find a peer for a known community on the LAN:
+
+```python
+# M02 extension (Phase 2)
+async def find_remote_peers(community_id: str) -> list[PeerRecord]:
+    if dht is None:
+        return []
+    contacts = await dht.find_value(blake3(community_id))
+    candidates = [parse_community_bootstrap(v.payload) for v in contacts]
+    return await fetch_manifests_and_filter(candidates, community_id)
+```
+
+### 5.3 Lookup integration with M07
+
+When [M07 TransferManager](../../modules/M07-file-blobs.md) needs sources for a CID and the local `file.cid.advertised` index is empty:
+
+```python
+# M07 extension (Phase 2)
+async def find_remote_sources(cid: str) -> list[str]:  # NodeIDs
+    contacts = await dht.find_value(blake3(cid))
+    return [parse_source_advert(v.payload).node_id for v in contacts]
+```
+
+### 5.4 Signature requirement on stored values
+
+Every DHT value's `payload` must contain a `signature` field signed by the advertiser. Receivers reject values whose signature does not validate against the advertiser's claimed NodeID. Cost is small; protection is essential — without it, anyone can poison the DHT.
+
+### 5.5 NAT traversal hooks
+
+The DHT itself does not do hole-punching. It cooperates with [M15 Relay Tier](../M15-relay-tier.md):
+
+- If our advertised endpoint is unreachable (NAT'd), we additionally advertise `via_relay: "<relay_url>"` in the value payload
+- Peers wanting to reach us see the relay hint and route through it
+- Direct peer-to-peer over NAT (STUN/TURN) is Phase 3
+
+### 5.6 Privacy of the DHT
+
+The DHT is a public-internet-facing component (by definition). It leaks:
+- Which NodeIDs exist
+- Which communities exist
+- Which CIDs are popular
+
+It does **not** leak:
+- The contents of any blob
+- The contents of community event logs
+- Who's actually a member of a community (membership is in the signed manifest, fetched out of band)
+
+This is acceptable for a system whose goal is community resilience, not anonymity.
+
+### 5.7 Anti-spam
+
+- Per-source rate limit on `store` calls: max 100 per minute per node
+- Stored value size cap: 4 KB
+- Per-bucket eviction prefers values with higher signature reputation (Phase 3)
+
+### 5.8 Bootstrap reachability
+
+`bootstrap_endpoints` (from config) are tried in order. If all fail, the node logs a warning and continues with mDNS+UDP only. The DHT is best-effort.
+
+---
+
+## 6. Wire format (request/response examples)
+
+### 6.1 `POST /dht/v1/find_value`
+
+Request:
+```json
+{
+  "key":      "blake3:<hex of 32 bytes>",
+  "from":     "ed25519:<our NodeID>",
+  "trace_id": "01HXR...",
+  "signature": "ed25519:<over the above three fields canonicalised>"
+}
+```
+
+Response (value found):
+```json
+{
+  "values": [
+    {
+      "key":     "blake3:...",
+      "payload": {"node_id":"...","endpoints":[...],"signature":"ed25519:..."},
+      "expires_at": 1717942800
+    }
+  ]
+}
+```
+
+Response (not found, get closer contacts):
+```json
+{
+  "values":  [],
+  "closer":  [{"node_id_full":"ed25519:...","endpoint":{"host":"...","port":7080}}, "..."]
+}
+```
+
+---
+
+## 7. Errors
+
+`DhtError`:
+
+- `bootstrap_failed` — no bootstrap endpoint reachable
+- `lookup_timeout` — couldn't find value or contacts within DHT_LOOKUP_TIMEOUT
+- `store_unauthorized` — payload signature invalid
+- `value_too_large` — > 4 KB
+- `rate_limited` — per-source store rate exceeded
+
+These don't always map to wire codes — most DHT activity is internal to the node. When they bubble up to a caller, `dht_lookup_failed` is the wire code.
+
+---
+
+## 8. Configuration
+
+```python
+config.dht.enabled                  = False  # opt-in; phase 1 default off
+config.dht.bootstrap_endpoints      = [...]
+config.dht.public_endpoint_override = None   # for nodes behind NAT, manual override
+config.dht.advertise_cids           = True   # also advertise pinned CIDs
+config.dht.advertise_community      = True
+```
+
+Constants used: `DHT_REPLICATION_K=8`, `DHT_ALPHA=3`, `DHT_REFRESH_SECONDS=3600`, `DHT_REPUBLISH_SECONDS=86400`.
+
+---
+
+## 9. Tests
+
+### Unit
+- `test_xor_distance_metric`
+- `test_routing_table_insert_eviction`
+- `test_signed_value_verification`
+- `test_unsigned_value_rejected`
+- `test_ttl_eviction`
+
+### Integration
+- `test_three_node_dht_find_value` — three KademliaNodes in process, store, find
+- `test_bootstrap_picks_up_existing_dht`
+- `test_partition_then_reconnect_converges`
+- `test_value_republish_keeps_alive`
+
+### Property-based
+- `test_kademlia_eventual_consistency_under_churn` (Hypothesis-driven)
+
+---
+
+## 10. Cross-references
+
+| What | Where |
+|------|-------|
+| Used by federation bootstrap | [M14 §4.3](../modules/M14-federation.md) |
+| Used by background blob replication | M07 ext (see [00-OVERVIEW §1](../00-OVERVIEW.md)) |
+| Wire error code | `dht_lookup_failed` in [CAP2 §9](../CAPABILITY_CONTRACT_v2.md) |
+| Phase 1 alternative (mDNS/UDP) | [M02](../../modules/M02-discovery.md) |
+| Phase 3 sybil resistance | TBD |
+
+---
+
+## 11. Open questions
+
+1. **libp2p reuse vs custom Python.** libp2p has a Python port but it's heavyweight. A focused 1000-LOC Kademlia matches our needs and stays auditable. Decision: custom for now; can swap.
+2. **NAT hole punching.** Currently relay-only. STUN/TURN integration is Phase 3.
+3. **Public DHT vs federated DHTs.** Should the DHT itself be federated (per-community DHT joined via cross-sig)? Maybe. Defer.
+4. **Onion routing.** Out of scope. HearthNet has no anonymity goal.

docs/p2_p3/X06-websocket.md

@@ -0,0 +1,316 @@
+# X06 — WebSocket Upgrade
+
+**Spec version:** v1.0 (Phase 2)
+**Depends on:** X01 (transport), X03 (observability), `websockets` Python library
+**Depended on by:** X01 transport server (in-place extension), M21 (tool-call loops), M25 (group chat live), M22 (mobile push delivery)
+
+---
+
+## 1. Responsibility
+
+Add bidirectional WebSocket transport alongside the existing HTTP/1.1 + SSE in [X01](../../cross-cutting/X01-transport.md). Use cases:
+
+- Tool-call loops in `llm.chat` where the server needs to ask the client to execute a tool mid-stream
+- Live pubsub topics that fan out many messages per second (group chat, federation heartbeats)
+- Mobile clients on flaky cellular where reconnect is expensive
+
+WebSockets do **not** replace the request/response model. They are an *upgrade* available on specific endpoints when both ends support v2 contract.
+
+---
+
+## 2. File layout
+
+```
+hearthnet/transport/
+└── websocket.py        # WebSocket server-side handler + client-side wrapper
+```
+
+Single file; the protocol is small.
+
+---
+
+## 3. Endpoints supporting upgrade
+
+| Endpoint | Behaviour |
+|----------|-----------|
+| `/bus/v1/call` | When `Upgrade: websocket` present and capability descriptor supports streaming, upgrade and use frame protocol on the WS instead of SSE |
+| `/pubsub/v1/subscribe` | When upgraded, server pushes messages on topic without long-polling |
+| `/sync/v1/events` | NOT upgraded — sync is bursty and short-lived; HTTP fits |
+
+---
+
+## 4. WebSocket frame protocol
+
+WebSocket frames carry the **same JSON event/data envelope** as SSE. This is deliberate — handlers can be written once and dispatched to either transport.
+
+### 4.1 Outbound (server → client)
+
+Each WebSocket message is one JSON object:
+
+```json
+{"event": "token", "data": {"text": "Hallo "}, "seq": 12}
+```
+
+`seq` is monotonic per-stream from the server. Used for backpressure ACKs.
+
+### 4.2 Inbound (client → server)
+
+Two kinds of messages:
+
+#### Backpressure ACK
+
+```json
+{"type":"ack","upto":8}
+```
+
+#### Tool result (mid-stream)
+
+```json
+{"type":"tool_result","tool_call_id":"tc_01HXR...","body":{...}}
+```
+
+Used in tool-call loops (see [M21](../modules/M21-tool-calls.md)).
+
+#### Cancel
+
+```json
+{"type":"cancel"}
+```
+
+Cleanly stops the current operation. Server must abort within 200 ms and emit a final `error` or `done` frame.
+
+### 4.3 Control frames
+
+Standard WebSocket pings/pongs. `WEBSOCKET_PING_SECONDS = 30` between pings.
+
+---
+
+## 5. Public API
+
+### 5.1 Server side
+
+```python
+# hearthnet/transport/websocket.py
+class WebSocketSession:
+    """Wraps a WebSocket connection from the server's perspective."""
+
+    def __init__(self, ws: WebSocket, kp: KeyPair):
+        ...
+
+    @property
+    def closed(self) -> bool: ...
+    @property
+    def remote_node_id(self) -> str: ...
+
+    async def emit(self, event: str, data: dict) -> None:
+        """Send a frame; respect flow control."""
+
+    async def emit_token(self, token: dict) -> None: ...
+    async def emit_progress(self, current: int, total: int, stage: str) -> None: ...
+    async def emit_error(self, code: ErrorCode, **kwargs) -> None: ...
+    async def emit_done(self, **meta) -> None: ...
+
+    async def receive(self) -> WsClientFrame | None:
+        """Block until a client frame arrives, or None on close."""
+
+    async def close(self, code: int = 1000) -> None: ...
+
+@dataclass(frozen=True)
+class WsClientFrame:
+    type:         str          # "ack" | "tool_result" | "cancel"
+    data:         dict
+```
+
+### 5.2 Client side
+
+```python
+class WebSocketClient:
+    """Used by HttpClient (X01) when stream() is called with `prefer_ws=True`."""
+
+    def __init__(
+        self,
+        url: str,
+        kp: KeyPair,
+        community_id: str,
+        pinned_certs: PinnedCerts,
+    ):
+        ...
+
+    async def open(self) -> None: ...
+    async def close(self) -> None: ...
+
+    async def send_call(
+        self,
+        capability: str,
+        version: str,
+        body: dict,
+        *,
+        trace_id: str,
+    ) -> None:
+        """Initial call frame. Authentication via X-HearthNet-* headers
+        and a signed call-envelope sent as the first WS message."""
+
+    async def __aiter__(self) -> AsyncIterator[Frame]:
+        """Yields Frame objects (same shape as SSE Frame)."""
+
+    async def send_tool_result(self, tool_call_id: str, body: dict) -> None: ...
+    async def send_ack(self, upto: int) -> None: ...
+    async def cancel(self) -> None: ...
+```
+
+### 5.3 Upgrade negotiation on the server
+
+X01's [HttpServer](../../cross-cutting/X01-transport.md) gets a small dispatch shim:
+
+```python
+# in hearthnet/transport/server.py (Phase 2 extension)
+async def dispatch_call(request: Request):
+    if request.headers.get("upgrade") == "websocket" and capability_supports_stream(...):
+        return await dispatch_via_websocket(request)
+    else:
+        return await dispatch_via_sse_or_json(request)
+```
+
+`capability_supports_stream` checks the descriptor's `stream_schema` is not None.
+
+---
+
+## 6. Behaviour
+
+### 6.1 Handshake
+
+```
+client → GET /bus/v1/call
+         Connection: Upgrade
+         Upgrade: websocket
+         Sec-WebSocket-Protocol: hearthnet-bus.v2
+         (other X-HearthNet-* headers)
+  ↓
+server: validates capability + initial signature
+        responds 101 Switching Protocols if v2 capable
+        responds 426 Upgrade Required (with downgrade hint) if not v2
+  ↓
+client sends first message: signed call envelope
+        {"type":"call","envelope":{...},"signature":"ed25519:..."}
+  ↓
+server: validates signature, dispatches to bus
+  ↓
+server streams response frames; client streams ACKs / tool_results / cancels
+```
+
+### 6.2 Flow control
+
+Same window-based FC as SSE (`STREAM_WINDOW_FRAMES = 16`, ACK every 8). Server checks `flow_control.send()` before each emit; client sends `ack` messages every 8 received frames.
+
+### 6.3 Idle handling
+
+If no message in either direction for `WEBSOCKET_IDLE_CLOSE_SECONDS` (120s), server closes with code 1000. Client may reopen.
+
+### 6.4 Failure modes
+
+| Symptom | Behaviour |
+|---------|-----------|
+| Client disconnect mid-stream | Server's task receives `CancelledError`, aborts the underlying capability within 200ms |
+| Network drop | Either side's WS library raises; current stream is `error`-terminated locally |
+| Server overload | Server may decline upgrade with 503 + retry hint; client falls back to SSE |
+| Protocol version mismatch | Server replies 426 with `Sec-WebSocket-Protocol` listing supported versions |
+
+### 6.5 Pubsub via WS
+
+Subscribing to a topic via WS:
+
+```
+client GET /pubsub/v1/subscribe?topic=marketplace.post.created
+       Upgrade: websocket
+  ↓
+server upgrades; sends backlog (if `since_seq` provided) then live messages
+  ↓
+each message: {"event":"published","data":{...},"seq":N}
+  ↓
+client sends ACKs to allow server to advance flow control
+```
+
+This replaces the long-polling pattern from Phase 1 §8 for clients that hold the connection. The long-poll endpoint remains for non-WS clients.
+
+### 6.6 Tool-call loop (used by [M21](../modules/M21-tool-calls.md))
+
+```
+server emits:
+  {"event":"token","data":{"text":"..."}}
+  {"event":"tool_call_delta","data":{"id":"tc_1","name":"rag.query","arguments_delta":"..."}}
+  ...
+  {"event":"tool_call","data":{"id":"tc_1","arguments":{"query":"...","corpus":"..."}}}
+client must respond:
+  {"type":"tool_result","tool_call_id":"tc_1","body":{...result of bus.call("rag.query",...)...}}
+server continues:
+  {"event":"token","data":{"text":"Based on the documents..."}}
+  ...
+  {"event":"done","data":{...}}
+```
+
+Without WebSocket, the SSE-only fallback is for the *caller* (UI) to execute the tool and re-call `llm.chat` with the tool result added to messages. Both paths work; WS is more efficient.
+
+---
+
+## 7. Errors
+
+`WebSocketError` codes (local domain):
+
+- `upgrade_refused` — server returned 426 or 503
+- `version_unsupported` — protocol mismatch
+- `idle_timeout`
+- `bad_frame` — malformed JSON or invalid `type`
+
+On the wire, errors carried inside the WS as `event: error` frames map to the standard wire codes in [CAP §9](../../CAPABILITY_CONTRACT.md).
+
+---
+
+## 8. Configuration
+
+```python
+config.transport.websocket_enabled       = True
+config.transport.websocket_idle_close_seconds = WEBSOCKET_IDLE_CLOSE_SECONDS
+config.transport.websocket_ping_seconds  = WEBSOCKET_PING_SECONDS
+```
+
+---
+
+## 9. Tests
+
+### Unit
+- `test_ws_frame_shape_matches_sse`
+- `test_signed_call_envelope_first_message`
+- `test_invalid_signature_closes_connection`
+- `test_idle_close_after_timeout`
+
+### Integration
+- `test_two_node_ws_call_round_trip`
+- `test_ws_stream_tokens_then_done`
+- `test_ws_tool_result_inline`
+- `test_ws_cancel_within_200ms`
+- `test_ws_fallback_to_sse_when_426`
+- `test_pubsub_via_ws_backlog_plus_live`
+
+### Chaos
+- `test_ws_dropped_packet_recovery` (using `tc`)
+
+---
+
+## 10. Cross-references
+
+| What | Where |
+|------|-------|
+| Endpoint upgrade | [CAP2 §5.1](../CAPABILITY_CONTRACT_v2.md) |
+| Frame protocol shared with SSE | [X01 §6](../../cross-cutting/X01-transport.md), [CAP §5.3](../../CAPABILITY_CONTRACT.md) |
+| Tool-call loop | [M21](../modules/M21-tool-calls.md) |
+| Mobile client benefits | [M22 §5](../modules/M22-mobile-native.md) |
+| Phase 3 considerations (WebTransport / QUIC) | TBD |
+
+---
+
+## 11. Open questions
+
+1. **HTTP/3 / WebTransport** — Phase 3 candidate; better on mobile, doesn't need TCP setup time on reconnect.
+2. **Binary frames** — JSON works; binary CBOR could save bytes. Defer until profiling shows it matters.
+3. **Multiplexing many capability calls on one WS** — currently one WS per call. Multiplex possible but adds complexity. Defer.
+4. **WSS certificate handling** — same TLS pinning as HTTPS; works because WS goes over the same TLS connection.

docs/p2_p3/X07-federated-metrics.md

@@ -0,0 +1,330 @@
+# X07 — Federated Metrics
+
+**Spec version:** v2.0
+**Depends on:** [X03 Observability](../../cross-cutting/X03-observability.md), [M14 Federation](../modules/M14-federation.md), [M16 Tokens](../modules/M16-tokens.md), [X04 Config](../../cross-cutting/X04-config.md)
+**Depended on by:** Operator dashboards (out of band), federation health UI
+
+---
+
+## 1. Responsibility
+
+Take the per-node Prometheus metrics produced by [X03](../../cross-cutting/X03-observability.md) and aggregate them, with consent, into a community-level view — and, where federation grants it, into a federation-level view.
+
+X03 gives each node a private view of itself. X07 gives:
+
+- **The community founder** a dashboard like "how healthy is the mesh today, where are the hot spots, what's the GPU saturation across all anchors?"
+- **A federated peer** a much narrower view — opt-in, aggregated, no per-node identifiers — like "Geldern reports 18 active members and 4.2k events/day".
+
+The design rule is: **less information at greater distance**. Per-node detail stays on the node. The community sees aggregates. Federated peers see anonymised aggregates. There is no global "every node and what it does" surface.
+
+---
+
+## 2. File layout
+
+```
+hearthnet/observability/
+├── federated.py              # FederatedMetricsExporter & Aggregator
+├── otlp_export.py            # Optional OpenTelemetry OTLP push
+├── aggregation_views.py      # SQL-like views over time-series
+└── consent.py                # Per-metric publish consent
+```
+
+---
+
+## 3. Public API
+
+### 3.1 `FederatedMetricsExporter`
+
+```python
+class FederatedMetricsExporter:
+    """
+    Pulls metrics from the local Prometheus registry, applies consent rules,
+    and publishes aggregated subsets either:
+      - to the community's aggregator anchor (mesh-internal)
+      - to an external OTLP collector (optional, off by default)
+      - to federated peers via the bus
+    """
+
+    def __init__(
+        self,
+        observability:   Observability,
+        consent:         ConsentPolicy,
+        bus:             CapabilityBus,
+        settings:        FederatedMetricsSettings,
+    ): ...
+
+    async def start(self) -> None: ...
+    async def stop(self) -> None: ...
+
+    # Triggered by tick; publishes to internal bus topic
+    async def publish_community(self) -> None: ...
+
+    # Triggered when federated peer requests
+    async def publish_federated(self, peer_community_id: NodeID) -> AggregatedSnapshot: ...
+
+    # OTLP push, off by default
+    async def push_otlp(self, endpoint: str) -> None: ...
+```
+
+### 3.2 `MetricsAggregator`
+
+Runs on the **aggregator anchor** (any anchor designated by community policy; default is the founder's node):
+
+```python
+class MetricsAggregator:
+    """
+    Subscribes to `observability.metrics.tick.*` events from all members,
+    keeps a 7-day rolling window, exposes:
+      - GET /metrics/community   (Prometheus format, community-wide)
+      - capability `observability.community_snapshot@1.0`
+    """
+
+    def __init__(self, bus: CapabilityBus, event_log: EventLog, store: TimeSeriesStore): ...
+
+    async def start(self) -> None: ...
+
+    async def community_snapshot(self) -> CommunityMetrics: ...
+    async def federated_snapshot(self, peer_id: NodeID) -> AggregatedSnapshot: ...
+```
+
+### 3.3 Snapshot dataclasses
+
+```python
+@dataclass
+class NodeMetricsTick:
+    """What each node publishes every METRICS_TICK_SECONDS (default 60)."""
+    node_id:            NodeID
+    timestamp:          datetime
+    cpu_pct:            float
+    mem_used_mb:        int
+    mem_total_mb:       int
+    gpu_pct:            float | None
+    gpu_mem_used_mb:    int | None
+    disk_used_gb:       float
+    disk_total_gb:      float
+    capability_calls_per_min: dict[str, int]  # by capability
+    error_rate_per_min:       dict[str, float]
+    p95_latency_ms_by_cap:    dict[str, float]
+    online_seconds:     int                    # since last restart
+
+@dataclass
+class CommunityMetrics:
+    """Aggregated over the community.  Has per-node detail (members see members)."""
+    timestamp:           datetime
+    nodes_total:         int
+    nodes_online:        int
+    nodes:               list[NodeMetricsTick]
+    capability_calls_per_min_total: dict[str, int]
+    events_per_min:      int
+    storage_used_gb:     float
+    federation_links:    int
+
+@dataclass
+class AggregatedSnapshot:
+    """For federated peers.  No per-node detail, no identifiers, banded values."""
+    timestamp:                    datetime
+    community_id:                 NodeID
+    nodes_online_band:            str             # "10-20", "20-50", etc.
+    daily_active_members_band:    str
+    capability_calls_per_day_top: list[tuple[str, str]]   # [(cap, band)]
+    error_rate_band:              str
+    federation_links_count:       int
+```
+
+### 3.4 `ConsentPolicy`
+
+```python
+@dataclass
+class ConsentPolicy:
+    """
+    Loaded from policy.yaml.  Controls what leaves the node.
+    """
+    publish_to_community:  set[str]   # metric names included in NodeMetricsTick
+    publish_to_federated:  set[str]   # subset, applied to AggregatedSnapshot
+    publish_to_external:   bool       # OTLP push on/off
+    aggregation_min_nodes: int        # don't expose a metric unless ≥ N nodes contribute
+    banding:               dict[str, list[int]]  # metric → bucket edges
+```
+
+---
+
+## 4. Behaviour
+
+### 4.1 Tick lifecycle
+
+Every `METRICS_TICK_SECONDS` (default 60s) each node:
+
+1. Snapshots its local Prometheus registry.
+2. Filters per `ConsentPolicy.publish_to_community`.
+3. Constructs a `NodeMetricsTick`.
+4. Publishes to bus topic `observability.metrics.tick.<community_id>` over WebSocket pubsub (efficient: many small messages, low latency).
+5. Also writes a local rolling-window copy for debug.
+
+The aggregator anchor subscribes to the topic, ingests into its time-series store, and computes `CommunityMetrics` on demand.
+
+### 4.2 Aggregator selection
+
+The community policy contains:
+
+```yaml
+observability:
+  aggregator_anchor: ed25519:<NodeID>   # optional; if absent, any anchor self-elects
+  aggregator_failover_seconds: 600
+```
+
+If the configured aggregator is offline for `aggregator_failover_seconds`, another anchor self-elects (lowest NodeID hash wins). A live community-wide view tolerates the aggregator going offline; nodes keep publishing ticks and a new aggregator picks up where the old one left off (with a brief gap).
+
+### 4.3 What gets exposed to whom
+
+| Metric category | Self | Other members | Aggregator anchor | Federated peers | External OTLP |
+|------------------|------|----------------|-------------------|------------------|---------------|
+| CPU / mem / GPU per-node | ✅ | per policy | ✅ | ❌ | ❌ |
+| Per-capability call counts | ✅ | ✅ | ✅ | banded only | optional |
+| Per-capability latencies | ✅ | aggregated | ✅ | ❌ | ❌ |
+| Error rates | ✅ | aggregated | ✅ | banded only | optional |
+| Federation link count | ✅ | ✅ | ✅ | exact count | ❌ |
+| File counts / sizes | ✅ | ❌ | aggregated | banded | ❌ |
+| Identity of which node did what | ✅ | per policy | ❌ (anonymised on ingest) | ❌ | ❌ |
+
+The aggregator does **not** store per-node identity in its long-term time series. It computes per-node views on the fly for the founder UI but persists only anonymised aggregates after `MEMBER_DETAIL_RETENTION_HOURS` (default 24).
+
+### 4.4 Banding
+
+Federated snapshots use bands rather than exact numbers to prevent triangulation across multiple federations:
+
+```yaml
+banding:
+  nodes_online: [0, 5, 10, 20, 50, 100, 500]
+  daily_active_members: [0, 3, 10, 30, 100]
+  capability_calls_per_day: [0, 100, 1000, 10000, 100000]
+  error_rate: [0, 0.01, 0.05, 0.10]
+```
+
+Result: `"nodes_online_band": "10-20"` instead of `19`.
+
+### 4.5 OTLP push (external)
+
+Off by default. When `publish_to_external=true`:
+
+- Pushes to a configured OTLP endpoint (could be Grafana Cloud, self-hosted Tempo/Mimir, or your own collector).
+- Only metrics in `publish_to_external` set leave the node.
+- The receiver gets aggregated, banded data — same restrictions as a federated peer.
+- TLS required; OTLP headers carry an API token (set via env var, not in policy file).
+
+This is the path for an operator (Christof) who wants a single Grafana dashboard across all his bofrost-managed communities — but the protections still apply: external collector cannot reconstruct who did what.
+
+### 4.6 Trackio integration
+
+Phase 1 already supports per-node Trackio logging. X07 adds: **the aggregator** can push a community-level summary to a Trackio space, useful for hackathon demos and HF leaderboard-style displays.
+
+`policy.observability.trackio_community_space` (URL) is configurable. The aggregator anchor pushes `CommunityMetrics` rows hourly. Per-node detail is excluded from this path; only aggregates go.
+
+### 4.7 Federated peer queries
+
+A federated peer asks for our snapshot via:
+
+```
+POST /bus/v1/call
+X-HearthNet-Community: <their community>
+Capability: observability.federated_snapshot@1.0
+Body: {"input": {"window_hours": 24}}
+```
+
+Bus checks federation scope, calls the aggregator, returns `AggregatedSnapshot`. The peer's UI may display "Geldern: 10-20 nodes online, light activity today" alongside their own community.
+
+### 4.8 Cost & sizing
+
+A `NodeMetricsTick` is roughly 500 bytes JSON. At 1 tick / 60s per node, a 50-node community publishes 50 × 500B / 60s ≈ 420 B/s on the metrics topic. Negligible.
+
+The aggregator's time-series store is **DuckDB** (Phase 2 choice; SQLite would also work). Retention: 7 days at full per-node resolution, then daily roll-ups for 90 days, then weekly forever.
+
+---
+
+## 5. Errors
+
+| Code | Cause |
+|------|-------|
+| `unavailable` | Aggregator anchor offline |
+| `aggregation_too_few_nodes` | < `aggregation_min_nodes` nodes contributed; refusing to disclose |
+| `federation_forbidden` | Peer requested a metric category not in federation scope |
+| `consent_denied` | Local policy forbids this metric from leaving the node |
+
+---
+
+## 6. Configuration
+
+```toml
+[observability.federated]
+enabled                        = true
+metrics_tick_seconds           = 60
+aggregator_failover_seconds    = 600
+member_detail_retention_hours  = 24
+aggregation_min_nodes          = 3
+publish_to_external            = false
+otlp_endpoint                  = ""
+otlp_token_env                 = "OTLP_TOKEN"
+trackio_community_space        = ""
+
+[observability.federated.consent.publish_to_community]
+metrics = [
+  "node.cpu_pct", "node.mem_pct", "node.gpu_pct",
+  "node.online_seconds", "node.capability_calls_per_min",
+  "node.p95_latency_by_capability",
+]
+
+[observability.federated.consent.publish_to_federated]
+metrics = [
+  "community.nodes_online", "community.daily_active_members",
+  "community.capability_calls_top", "community.federation_links",
+]
+
+[observability.federated.consent.banding]
+"community.nodes_online"          = [0, 5, 10, 20, 50, 100]
+"community.daily_active_members"  = [0, 3, 10, 30, 100]
+"community.capability_calls_top"  = [0, 100, 1000, 10000]
+"community.error_rate"            = [0, 0.01, 0.05, 0.10]
+```
+
+---
+
+## 7. Tests
+
+### 7.1 Unit
+- Banding: value 17 with bands `[0,5,10,20,50]` returns `"10-20"`
+- Aggregation refuses when contributors < min: `aggregation_too_few_nodes`
+- Consent: a metric not in `publish_to_community` set is excluded from tick
+- AggregatedSnapshot construction strips all NodeID fields
+
+### 7.2 Integration
+- 5 nodes publish ticks for 5 minutes; aggregator's snapshot reflects 5 contributors with correct totals
+- Aggregator kill / failover: a second anchor takes over within 10 minutes, snapshot resumes
+- Federated peer requests snapshot; receives banded version; cannot infer specific node counts
+
+### 7.3 Adversarial
+- Malicious node publishes inflated counters → outlier detection drops obvious outliers (>3σ) from the aggregate
+- Federated peer requests snapshot for window the aggregator hasn't filled → `aggregation_too_few_nodes`
+- OTLP endpoint compromised: leaked data contains only banded aggregates; per-node attribution impossible
+
+### 7.4 Privacy
+- Asserts: no NodeID, IP, or device-identifying string is present in `AggregatedSnapshot`
+- Asserts: after `MEMBER_DETAIL_RETENTION_HOURS` the aggregator's persisted store contains no per-node rows
+
+---
+
+## 8. Cross-references
+
+- Capability: `observability.community_snapshot@1.0`, `observability.federated_snapshot@1.0` (introduced here, listed in [CAPABILITY_CONTRACT_v2 §3](../CAPABILITY_CONTRACT_v2.md#3-complete-new-capabilities-list))
+- Bus topic: `observability.metrics.tick.<community_id>`
+- Underlying primitives: [X03](../../cross-cutting/X03-observability.md)
+- Federation scope: [M14 §5](../modules/M14-federation.md)
+- Policy schema: [X04](../../cross-cutting/X04-config.md)
+
+---
+
+## 9. Open questions
+
+1. **Differential privacy** — adding Laplacian noise to federated snapshots. Worth it for stronger guarantees, or does banding already suffice given small N?
+2. **Federation gossip of snapshots** — should snapshots propagate transitively (A→B→C sees A's banded numbers), or strictly point-to-point? Phase-2 default: point-to-point.
+3. **Per-capability cost accounting** — exposing GPU-seconds per capability call would help operators reason about cost / who's consuming what. Reveals usage patterns; needs consent design.
+4. **Histogram vs banded scalars** — banded scalars are simple but lose distribution shape. Full Prometheus histograms with aggregated buckets might be a better federated unit. Trade-off: bytes on the wire vs richness.
+5. **Aggregator beyond a single anchor** — at large scale (100+ nodes) a single aggregator becomes a bottleneck. Sharded aggregation (per-capability-prefix?) is a Phase-3 problem.

docs/p2_p3/X08-tensor-transport.md

@@ -0,0 +1,302 @@
+# X08 — Tensor Transport
+
+**Spec version:** v3.0 — *experimental*
+**Depends on:** [X06 WebSocket](../../phase-2/cross-cutting/X06-websocket.md), [M02 Transport](../../modules/M02-transport.md), [M01 Identity](../../modules/M01-identity.md)
+**Depended on by:** [M26 Distributed Inference](../modules/M26-distributed-inference.md)
+
+---
+
+## 1. Purpose
+
+A binary, framed, flow-controlled transport for **tensor data** between HearthNet nodes — specifically the activations and gradients moved during M26 distributed inference. The text-oriented capability bus and JSON-shaped event envelopes are wrong for this traffic: tensors are large, dense, and benefit from binary representation, streaming, and explicit flow control.
+
+X08 lives parallel to the bus, not on top of it. A tensor session is *negotiated* via the bus (M26 calls `pipeline.shard.connect` which returns an X08 endpoint URL and a session token), then the actual bytes move over a dedicated WebSocket binary channel.
+
+Scope: bidirectional tensor streaming, fp16 by default, optional zstd compression above a threshold, 16-byte fixed-size headers, chunked payloads, ack-based flow control. Not a general-purpose RPC.
+
+---
+
+## 2. Non-goals
+
+- **Replacing capability-bus traffic.** Control plane stays on the bus. X08 carries data only.
+- **Persistent storage of tensors.** X08 is point-to-point, in-memory, ephemeral. Storage is the caller's job.
+- **Cross-version negotiation of the frame format.** v3.0 ships one frame format. A future version bumps the major.
+- **End-to-end encryption beyond TLS.** WebSocket runs over TLS via M02. Per-frame application-layer crypto is out of scope (the threat model doesn't require it because session establishment is authenticated, and the WSS hop is encrypted).
+- **Reliable broadcast.** Sessions are 1:1. Multi-receiver fan-out is M26's problem if it needs it.
+
+---
+
+## 3. Wire format
+
+### 3.1 Frame
+
+Every frame is a single WebSocket binary message. Frame layout (big-endian):
+
+```
+offset  size   field
+ 0      1      version (currently 0x01)
+ 1      1      frame_type
+ 2      2      reserved (must be 0x0000)
+ 4      4      session_seq      (u32, monotonic per session)
+ 8      4      payload_length   (u32, bytes of body)
+12      4      flags
+16      ...    body (payload_length bytes)
+```
+
+The header is always 16 bytes. Body is opaque to the framing layer; its interpretation depends on `frame_type`.
+
+### 3.2 Frame types
+
+```
+0x01  TENSOR_DATA          body = tensor chunk (see §3.4)
+0x02  TENSOR_END           body = empty; marks last chunk of a tensor
+0x03  ACK                  body = empty; acknowledges receipt up to session_seq
+0x04  CONTROL_NACK         body = utf-8 error reason
+0x05  CONTROL_HELLO        body = HelloMsg (json, utf-8)
+0x06  CONTROL_BYE          body = utf-8 reason, optional
+0x07  CONTROL_FLOWCTL      body = FlowCtlMsg (json, utf-8)
+0x08  CONTROL_PING         body = 8 bytes (echo nonce)
+0x09  CONTROL_PONG         body = 8 bytes (echoed nonce)
+```
+
+Frame types `0x10..0xFF` are reserved for future extensions and current implementations must close the session on unknown types.
+
+### 3.3 Flags
+
+```
+0x00000001  COMPRESSED      payload is zstd-compressed
+0x00000002  FINAL           last frame in this tensor (also implied by TENSOR_END)
+0x00000004  GRAD            payload is a gradient (informational; for telemetry)
+0x00000008  ENCRYPTED       reserved for future per-frame encryption
+0xFFFFFFF0  reserved
+```
+
+### 3.4 Tensor chunk body
+
+A `TENSOR_DATA` body is:
+
+```
+offset  size   field
+ 0      2      tensor_id        (u16, scoped to this session)
+ 2      1      dtype            (0x01=fp16, 0x02=fp32, 0x03=bf16, 0x04=int8)
+ 3      1      n_dims           (1..8)
+ 4      n_dims*4   shape         (u32 per dim, big-endian)
+ ...           data_bytes        (compressed if COMPRESSED flag set)
+```
+
+`tensor_id` lets a session carry multiple concurrent tensors (e.g., parallel pipeline stages). A given `tensor_id` may be split across multiple `TENSOR_DATA` frames and is terminated by a `TENSOR_END` with the same `tensor_id`.
+
+### 3.5 HelloMsg
+
+```json
+{
+  "session_id": "<ulid>",
+  "session_token": "<m16-token>",
+  "from": "<NodeID>",
+  "to": "<NodeID>",
+  "purpose": "pipeline.shard.forward",
+  "negotiation": {
+    "preferred_dtype": "fp16",
+    "compression": "zstd",
+    "max_chunk_bytes": 1048576,
+    "flow_window": 16
+  }
+}
+```
+
+Both parties exchange `CONTROL_HELLO` on connect; mismatched purposes or invalid tokens terminate the session with `CONTROL_BYE`.
+
+### 3.6 FlowCtlMsg
+
+```json
+{ "window": 16, "credits_added": 8 }
+```
+
+Receiver-initiated. Says "I can accept N more in-flight chunks beyond what I've already acked". See §4.3.
+
+---
+
+## 4. Behaviour
+
+### 4.1 Session lifecycle
+
+```
+CONNECT ──hello exchange──▶ READY ──tensor data──▶ STREAMING ──end/bye──▶ CLOSED
+                  │
+                  ├── auth fails ──▶ NACK ──▶ CLOSED
+                  └── timeout ──▶ CLOSED
+```
+
+A session is opened by the side that initiated the bus call (the M26 caller for forward passes; the shard server for activations sent back if reverse direction is needed). The HelloMsg `session_token` is an M16 token scoped to the bus capability that authorised this session (e.g., `pipeline-shard-forward`); the receiver validates it before accepting any `TENSOR_DATA`.
+
+### 4.2 Sequencing
+
+`session_seq` is a u32 starting at 1 and incrementing per outgoing frame from the sender. It wraps to 1 at 2^32-1 in the theoretical case but practically a single session is expected to be far below that. Wrap is supported by the protocol but is not exercised by tests.
+
+The receiver tracks the highest `session_seq` it has processed and acknowledges via `ACK` frames whose `session_seq` echoes the highest contiguous received seq.
+
+### 4.3 Flow control
+
+The receiver advertises a *credit window* in `CONTROL_FLOWCTL`. The sender may have at most `window` un-acked frames in flight. Initial window is set in `HelloMsg.negotiation.flow_window` (default `TENSOR_FLOW_CONTROL_WINDOW=16`). The receiver replenishes credits by sending `FLOWCTL` with `credits_added > 0` as it processes frames.
+
+If the sender's in-flight count reaches the window, it pauses until an `ACK` or `FLOWCTL` arrives. There is no timeout-based unblock; if the receiver disappears, the underlying WebSocket eventually closes and the session ends.
+
+### 4.4 Compression
+
+`COMPRESSED` flag is set per-frame, not per-session. The sender chooses; the receiver MUST support zstd (level 3 default). Compression is applied to the *body* (everything after the 16-byte header). The body's `payload_length` reflects the compressed size; the uncompressed shape is recovered from the tensor chunk header after decompression.
+
+Compression is enabled when the raw body exceeds `TENSOR_COMPRESSION_THRESHOLD_BYTES` (default 64 KiB). Below this, the framing overhead dominates and compression is skipped.
+
+### 4.5 Chunking
+
+A tensor larger than `TENSOR_CHUNK_BYTES` (default 1 MiB) is split into multiple `TENSOR_DATA` frames sharing the same `tensor_id`. The split is on raw-byte boundaries (after compression if compressed); the receiver concatenates raw bytes per `tensor_id` and then, on `TENSOR_END`, decompresses (if needed) and reconstructs the tensor using the shape declared in the *first* chunk for that `tensor_id`. Subsequent chunks for the same `tensor_id` repeat the dtype/shape header — the receiver MUST verify consistency or close the session with a NACK.
+
+### 4.6 Keepalive
+
+Either side may send `CONTROL_PING` at any time; the peer must respond with `CONTROL_PONG` echoing the nonce. A session with no PING/PONG and no data for `TENSOR_KEEPALIVE_SECONDS` (default 30) sends a PING; failure to respond within 2× that closes the session.
+
+### 4.7 Backpressure & cancellation
+
+A caller cancelling a pipeline operation (M26) sends `CONTROL_BYE` with a reason. The receiver may discard in-flight tensors for the cancelled session. There is no "graceful drain" — cancellation is fast and lossy.
+
+### 4.8 Failure modes
+
+- **Decompression fails**: NACK + close. The caller in M26 retries with the failover shard.
+- **Tensor shape inconsistency across chunks**: NACK + close.
+- **Auth failure on HelloMsg**: NACK + close before any data flows.
+- **Unknown frame type**: close with NACK reason `unknown_frame_type`.
+- **Sequence gap**: NACK + close. There is no out-of-order recovery; WebSocket delivers in order, so a gap means corruption.
+- **Window overrun by sender**: NACK + close — the sender violated flow control.
+
+---
+
+## 5. API
+
+X08 is a library, not a capability surface. Public Python API:
+
+```python
+class TensorSession:
+    @classmethod
+    async def connect(cls,
+                       url: str,
+                       token: AuthToken,
+                       *,
+                       purpose: str,
+                       remote: NodeID,
+                       negotiation: SessionNegotiation | None = None) -> TensorSession: ...
+    @classmethod
+    async def accept(cls,
+                      ws: WebSocket,
+                      *,
+                      expected_purpose: str,
+                      validate_token: Callable[[AuthToken], None]) -> TensorSession: ...
+
+    async def send_tensor(self, tensor_id: int, t: Tensor, *, gradient: bool = False) -> None: ...
+    async def recv_tensor(self) -> RecvTensor: ...
+    async def close(self, reason: str = "") -> None: ...
+
+    @property
+    def session_id(self) -> str: ...
+    @property
+    def stats(self) -> SessionStats: ...
+
+@dataclass(frozen=True)
+class RecvTensor:
+    tensor_id: int
+    tensor:    Tensor
+    is_grad:   bool
+
+@dataclass(frozen=True)
+class SessionStats:
+    bytes_sent:           int
+    bytes_received:       int
+    bytes_compressed_out: int
+    bytes_uncompressed_out: int
+    frames_sent:          int
+    frames_received:      int
+    rtt_estimate_ms:      float
+```
+
+Implementations: `hearthnet/transport/tensor/` houses `session.py`, `frame.py`, `flow.py`, `compress.py`.
+
+---
+
+## 6. Configuration
+
+```python
+@dataclass(frozen=True)
+class TensorTransportConfig:
+    default_dtype:                 Literal["fp16","fp32","bf16","int8"] = "fp16"
+    chunk_bytes:                   int = TENSOR_CHUNK_BYTES                 # 1048576
+    flow_control_window:           int = TENSOR_FLOW_CONTROL_WINDOW         # 16
+    compression_threshold_bytes:   int = TENSOR_COMPRESSION_THRESHOLD_BYTES # 65536
+    compression_level:             int = 3                                  # zstd
+    keepalive_seconds:             int = TENSOR_KEEPALIVE_SECONDS           # 30
+    max_session_lifetime_seconds:  int = 3600                               # hard cap
+    max_concurrent_sessions:       int = 64
+    rx_buffer_bytes_max:           int = 64 * 1024 * 1024                   # 64 MiB
+```
+
+Constants in `hearthnet/constants.py`.
+
+---
+
+## 7. Tests
+
+### 7.1 Unit
+
+- `test_frame_header_layout` — pack/unpack roundtrip for all frame types.
+- `test_tensor_chunk_body` — pack/unpack roundtrip for all dtypes and ranks.
+- `test_compression_roundtrip` — compressed body decompresses to identity.
+- `test_chunking_reassembly` — 5 MiB tensor split into 5 chunks reassembles to identical bytes.
+- `test_unknown_frame_type_closes` — receiver rejects 0xFF.
+- `test_flow_control_blocks_at_window` — sender pauses at window edge, resumes on ACK.
+- `test_seq_gap_closes` — injecting a missing seq forces NACK + close.
+
+### 7.2 Property
+
+- Random tensor shapes and dtypes: send → receive → equal modulo dtype precision.
+- Random chunk sizes that always sum to the same total: reassembly identical.
+
+### 7.3 Integration
+
+- Loopback session over an in-memory WebSocket pair: send 10 tensors of varying size, verify all received, stats consistent.
+- Two-process loopback: same as above but over a real localhost WSS.
+- Cancellation mid-stream: sender sends half a tensor, receives BYE, no further frames sent.
+- Auth failure: connect with bad token → NACK on hello.
+
+### 7.4 Negative
+
+- Send to a wrong purpose → hello mismatch → close.
+- Send oversized tensor (exceeds rx_buffer_bytes_max) → receiver NACKs with `tensor_too_large`.
+- Corrupt frame in the middle of a tensor: receiver detects via shape inconsistency or decompression failure → close.
+
+---
+
+## 8. Cross-references
+
+- **Phase 1 M02 Transport** — provides the underlying WebSocket (WSS, TLS, certificate pinning).
+- **Phase 2 X06 WebSocket** — defines the WebSocket framing and reconnection semantics that X08 layers on.
+- **Phase 2 M16 Tokens** — session tokens authorise tensor transport sessions.
+- **Phase 3 M26 Distributed Inference** — the primary consumer; defines purposes like `pipeline.shard.forward`, `pipeline.shard.backward`.
+- **Phase 3 X09 Conformance Suite** — includes optional `tensor_transport` section, only run when M26 is enabled.
+
+---
+
+## 9. Open questions
+
+1. **Per-frame encryption.** The `ENCRYPTED` flag is reserved. The use case is post-quantum hardening above TLS, or end-to-end above a federation-relay path that terminates TLS at the relay. Not in v3.0.
+
+2. **Adaptive compression.** Fixed zstd level 3 is fine for typical activations. Per-session adaptive level (lower for hot, higher for warm tensors) is plausible. Out of scope.
+
+3. **GPU-direct transport.** Activations sit in GPU memory and round-tripping through CPU memory for serialisation is wasteful. Direct GPU-to-network (NVLink/RDMA) is interesting but assumes a specific hardware topology that HearthNet doesn't have. Not in v3.0.
+
+4. **Multipath.** Sending tensor chunks over multiple parallel WebSocket sessions to bond bandwidth is appealing but complicates ordering. v3.0 sticks to one session.
+
+5. **Sequence wrap.** Practically irrelevant; correctness at wrap is asserted but not battle-tested.
+
+6. **Flow control on the wire.** Currently we layer flow control on top of WebSocket, which already has some. The duplication is intentional (we want app-level explicit windowing for backpressure into the inference scheduler) but worth revisiting.
+
+---
+
+*Last updated: spec v3.0.*

docs/p2_p3/X09-conformance-suite.md

@@ -0,0 +1,316 @@
+# X09 — Conformance Suite
+
+**Spec version:** v3.0 — *experimental*
+**Depends on:** Every other module (the suite tests them); no runtime dependency in production
+**Depended on by:** [M32 Protocol Standardisation](../modules/M32-protocol-standard.md)
+
+---
+
+## 1. Purpose
+
+A black-box, implementation-agnostic test suite that defines what "HearthNet-compliant" means in practice. The suite spins up an instance of an implementation, drives it through specified interactions, observes the wire format and the capability behaviour, and produces a `ConformanceReport` (defined in M32).
+
+Where the spec documents say "the system MUST do X", the conformance suite contains a test that observes whether the system does X. If a behaviour is described in a spec but not tested by X09, the spec wins in principle but the suite wins in practice — so we treat closing that gap as a continuous effort.
+
+The suite is designed so that an alternate implementation (a future Go or Rust HearthNet) can be tested by the same suite. This is the entire point: it makes "interoperable" a measurable property.
+
+---
+
+## 2. Non-goals
+
+- **Replacing per-module unit tests.** Each module ships its own unit and property tests as described in its spec. X09 sits one level higher and treats the implementation as a black box.
+- **Performance benchmarks.** Conformance is correctness, not speed. A future X10 may handle benchmarks.
+- **Security audits.** Out of scope. The suite includes some negative-path tests but is not a pen-test.
+- **Visual / UX testing.** The web UI is exercised only via its capability-bus and HTTP API surfaces.
+- **Locking in implementation detail.** Tests assert on observable behaviour (wire formats, capability responses, event log entries), never on internal state.
+
+---
+
+## 3. File layout
+
+The suite lives at repository root, sibling to `hearthnet/` and `protocol/`:
+
+```
+conformance/
+├── README.md
+├── VERSION                            # suite version, e.g. "1.0.0"
+├── pyproject.toml                     # standalone tool, runnable without hearthnet/
+├── runner.py                          # entry point: `python -m conformance.runner --target=...`
+├── report.py                          # builds ConformanceReport from results
+├── harness/
+│   ├── target.py                      # abstraction over a "system under test" (SUT)
+│   ├── docker_target.py               # SUT in a docker container
+│   ├── local_target.py                # SUT on the local network at a URL
+│   ├── fixtures.py                    # synthetic identities, tokens, files
+│   └── wire_capture.py                # records bus / WS traffic for diffing
+├── suites/
+│   ├── core/
+│   │   ├── identity/
+│   │   ├── transport/
+│   │   ├── bus/
+│   │   ├── events/
+│   │   ├── tokens/
+│   │   ├── files/
+│   │   ├── kb/
+│   │   └── llm/
+│   ├── services/
+│   │   ├── chat/
+│   │   ├── group_chat/
+│   │   ├── ocr/
+│   │   ├── translation/
+│   │   └── stt_tts/
+│   ├── federation/
+│   ├── experimental/
+│   │   ├── distributed_inference/
+│   │   ├── moe/
+│   │   ├── fedlearn/
+│   │   ├── evidence/
+│   │   └── civdef/
+│   └── operability/
+│       ├── shutdown_clean/
+│       ├── restart_persistence/
+│       └── observability/
+└── vectors/                           # test vectors: canonical inputs and expected outputs
+    ├── identity/
+    ├── tokens/
+    ├── federation/
+    └── tensor_transport/
+```
+
+The whole `conformance/` directory is published as part of every protocol release, with the `VERSION` file aligning with the protocol's release cadence (but versioned independently — see §4.1).
+
+---
+
+## 4. Architecture
+
+### 4.1 Suite versioning
+
+Suite version follows semver. `suite_version` in `ConformanceReport` is the suite that produced the report. A protocol version is paired with a *minimum* suite version that is sufficient to test it. Newer suite versions test more thoroughly; older suite versions may not exercise newer protocol features.
+
+### 4.2 Target abstraction
+
+```python
+class Target(Protocol):
+    """A system under test (SUT). The suite never touches the SUT's internals."""
+    base_url: str
+    admin_token: AuthToken
+
+    async def start(self) -> None: ...                 # for managed targets like docker
+    async def stop(self) -> None: ...
+    async def reset(self) -> None: ...                 # blank slate (for tests that need it)
+    async def bus_call(self, capability: str, payload: dict) -> dict: ...
+    async def event_subscribe(self, types: list[str]) -> AsyncIterator[Event]: ...
+    async def http_get(self, path: str, headers: dict | None = None) -> Response: ...
+    async def http_post(self, path: str, body: bytes, headers: dict | None = None) -> Response: ...
+    async def ws_connect(self, path: str, subprotocol: str | None = None) -> WebSocket: ...
+    async def capture_wire(self) -> WireCapture: ...   # for federation/tensor tests
+```
+
+Two concrete `Target` implementations ship:
+
+- `LocalTarget`: SUT runs as a long-lived process accessible at a known URL. Simplest; used in CI against the reference implementation.
+- `DockerTarget`: SUT runs in a docker container that the suite spawns. Useful for testing alternate implementations packaged as containers.
+
+Authors of an alternate implementation supply their own `Target` subclass if needed.
+
+### 4.3 Test format
+
+Tests are plain `pytest` cases under `suites/`. They use the target as an injected fixture:
+
+```python
+# suites/core/identity/test_node_id_format.py
+
+async def test_node_id_is_base32_no_pad(target: Target) -> None:
+    r = await target.bus_call("identity.self.describe", {})
+    node_id = r["node_id"]
+    assert re.fullmatch(r"[A-Z2-7]+", node_id), "NodeID must be base32 with no padding"
+    assert len(node_id) >= 52
+```
+
+Each test asserts at most one *spec requirement*. The test docstring names the spec section it covers; the runner uses this to produce traceability from `SectionResult.failures` back to the relevant module spec.
+
+### 4.4 Wire vectors
+
+For wire-format tests (federation manifest, tensor transport frame, token JWS envelope) the suite carries canonical byte vectors in `vectors/`. Tests assert that:
+
+- The SUT, given a known input, produces a byte-equal output (after canonicalisation where applicable).
+- The SUT, given a known byte vector, parses it without errors and produces the expected semantic content.
+
+This catches subtle interop bugs — the kind of "we both speak JSON-with-tiny-differences" issue that has historically killed federated systems.
+
+### 4.5 Report aggregation
+
+After a run, `report.py`:
+
+1. Collects per-test pass/fail/skip results.
+2. Groups by suite path → SectionResult.
+3. Computes `overall`:
+   - `pass` if all `core/*` and all `services/*` sections passed (experimental and operability may fail without affecting `pass`).
+   - `partial` if `core/*` passed but anything else failed.
+   - `fail` if any `core/*` test failed.
+   - `skipped` if no sections ran.
+4. Signs the report with the SUT's identity (the SUT signs its own report — there is no external authority).
+5. Emits `report.json` and a human-readable `report.html`.
+
+### 4.6 Reproducibility
+
+Every run produces a `run_manifest.json` containing:
+
+- Suite version, suite git commit.
+- Target type and configuration (without secrets).
+- Random seed (suite seeds all RNGs deterministically for reproducibility).
+- Test selection (which suites/tests were run vs skipped).
+- Timestamps.
+
+Replaying with the same manifest against the same SUT version must produce equivalent results modulo timestamps.
+
+---
+
+## 5. Required sections
+
+A claim of "HearthNet-compliant at protocol version 3.0.0" requires passing **every test** under:
+
+- `suites/core/identity/`
+- `suites/core/transport/`
+- `suites/core/bus/`
+- `suites/core/events/`
+- `suites/core/tokens/`
+- `suites/core/files/`
+- `suites/core/kb/` *(minimum: ingest, query)*
+- `suites/core/llm/` *(minimum: chat capability, error handling)*
+
+Plus passing the relevant *advertised-capability* sections under `suites/services/` for any service the implementation advertises. An implementation advertising `chat.thread.*` but not running `suites/services/chat/` is non-compliant by omission.
+
+Federation is required for any implementation that advertises federation; otherwise it's optional. Experimental sections are *always* optional and `partial` is a valid honest outcome.
+
+---
+
+## 6. Behaviour
+
+### 6.1 Pre-flight
+
+Before running tests, the runner:
+
+1. Confirms `target.start()` succeeded.
+2. Calls `protocol.self_describe` and `protocol.version_list` to discover what to test.
+3. Confirms `protocol_version` returned by the SUT is compatible with the suite's supported versions; if not, fails fast with `protocol_version_unsupported`.
+4. Resets the SUT (`target.reset()`).
+5. Loads vector files into memory.
+
+### 6.2 Test isolation
+
+Each test must be independent — order should not matter. Tests that need a clean slate request `target.reset()` in a fixture; tests that need shared state declare it via pytest fixtures with explicit scope.
+
+Tests use synthetic identities and tokens generated per-test, never the real operator's keys.
+
+### 6.3 Graceful skipping
+
+A test that requires a capability not advertised by the SUT is *skipped*, not failed:
+
+```python
+@requires_capability("experimental.fedlearn.round.announce")
+async def test_fedlearn_round_announce_signs_manifest(target: Target) -> None:
+    ...
+```
+
+`requires_capability` queries `protocol.self_describe`. Skipped tests appear in the report as `skipped` with a reason. They never flip `overall` to `fail`.
+
+### 6.4 Wire capture mode
+
+For federation and tensor-transport sections, the runner may attach a `WireCapture` to record the raw bytes flowing between two SUT instances (or between an SUT and the suite's own simulator). The captured frames are checked against vectors and against the schema documented in the relevant cross-cutting spec.
+
+Wire-capture mode requires the operator to have configured the SUT to log raw traffic to a known location (typically a Unix socket the suite reads). For SUTs that can't expose raw traffic, the suite falls back to behavioural assertions only and notes `partial` if wire vectors couldn't be verified.
+
+### 6.5 Operability sections
+
+`operability/` sections test resilience properties:
+
+- `shutdown_clean`: send SIGTERM (or container stop); verify no events are lost, the audit chain (M31) verifies, and the SUT restarts cleanly.
+- `restart_persistence`: data created before restart is queryable after restart.
+- `observability`: standard event types fire as expected; X03 observability conformance.
+
+### 6.6 Reporting failures
+
+Each failure records:
+
+- Spec section reference (e.g. `M14 §5.2 canonicalisation`).
+- The actual observed value or behaviour.
+- The expected value or behaviour.
+- A reproduction recipe (capability call + payload, or wire vector identifier).
+
+This is what makes a `partial` report useful: the failures are debuggable.
+
+---
+
+## 7. Configuration
+
+```python
+@dataclass(frozen=True)
+class ConformanceConfig:
+    target_kind:        Literal["local","docker","custom"] = "local"
+    target_url:         str = "http://127.0.0.1:7900"
+    target_admin_token: str | None = None         # acquired out-of-band
+    docker_image:       str | None = None
+    suite_filter:       tuple[str, ...] = ()      # glob patterns; empty = all required + advertised
+    skip_experimental:  bool = False
+    skip_operability:   bool = False
+    wire_capture:       bool = False
+    output_dir:         str = "./conformance-report"
+    parallel:           int = 1                   # 1 by default to avoid test-isolation surprises
+    seed:               int = 0xC0FFEE
+```
+
+A typical CI invocation:
+
+```
+python -m conformance.runner \
+    --target=docker \
+    --docker-image=hearthnet:latest \
+    --output-dir=./report
+```
+
+---
+
+## 8. Tests of the suite itself
+
+The suite has its own tests, kept under `conformance/tests/`:
+
+- `test_runner_smoke` — runs the suite against the reference impl, expects `overall=pass` for `core/*`.
+- `test_skip_logic` — capabilities not advertised → tests skipped, not failed.
+- `test_seed_deterministic` — given a seed, two consecutive runs produce identical reports modulo timestamps.
+- `test_report_schema` — generated `ConformanceReport` validates against the schema in `protocol/`.
+- `test_vector_integrity` — every file in `vectors/` parses with the canonical loader.
+- `test_known_partial` — the reference impl with `experimental.*` disabled produces a `partial` report (because experimental tests skip, not fail) — verify that the `overall` calculation correctly produces `pass`, since experimental skips don't flip the bit.
+
+---
+
+## 9. Cross-references
+
+- **M32 Protocol Standardisation** — consumes `ConformanceReport`; the suite is the source of "what conformance means".
+- **Every module spec** — the suite's tests reference the spec section they verify.
+- **X02 Event Log, X03 Observability** — operability tests assert on these.
+- **X06 WebSocket, X08 Tensor Transport** — wire-capture vectors live in `vectors/`.
+
+---
+
+## 10. Open questions
+
+1. **Adversarial tests.** v3.0 has minimal negative-path coverage in `core/`. A future suite version with a `security/` section that probes for known classes of mistakes (auth bypass, signature reuse, event-log forgery attempts) would be valuable. Out of scope for v3.0.
+
+2. **Conformance for partial implementations.** The current model gates `pass` on all required `core/*` passing. A future tiered model (`HearthNet-Bronze` = identity + transport + bus only; `HearthNet-Silver` adds services; `HearthNet-Gold` adds federation) is appealing for low-resource implementations. Not in v3.0.
+
+3. **Differential testing.** Once two implementations exist, running them side-by-side with the same input and asserting identical observable behaviour is the strongest interop test. The harness supports this in principle (two targets), but no tests in v3.0 actually use it because only one implementation exists.
+
+4. **Vector generation.** Today vectors are hand-curated. Tooling to *regenerate* vectors from the reference implementation and detect drift would prevent test rot. Planned, not implemented.
+
+5. **Reporting hub.** A public registry that collects published conformance reports from various implementations would help users assess interop status. Out of scope for the suite itself; M32's `protocol.registry.*` capabilities are the closest current analogue.
+
+6. **Performance regression guardrails.** Not conformance, but obviously valuable. A separate X10 (TBD) may handle this.
+
+7. **Long-haul tests.** Some bugs (memory leaks, slow drifts in audit chains) appear only after hours. The suite is built for short runs; a "soak mode" with `--duration=24h` would test these. Open.
+
+8. **Federation interop with non-HearthNet systems.** Out of scope. The suite verifies HearthNet ↔ HearthNet federation only.
+
+---
+
+*Last updated: spec v3.0.*