amoe-lora / SCHEMA.md
AbstractPhil's picture
0.2.1: band-coordinate law fixed (+invariant), dispatch-bank/cond classification fixed, schema v1.1 usage card + SCHEMA.md; code now canonical at github.com/AbstractEyes/amoe-lora
33e89d6 verified
|
Raw
History Blame Contribute Delete
6.09 kB

amoe.diffusion.anchor — the adapter file format

One file = one adapter stack for one trunk. The file carries its own documentation: a loader can tell you what the adapter is, what it was trained on, what it measurably does, how strongly to run it, and what it does not do — without any external database.

Two containers, same logical content:

container when notes
.safetensors preferred, and what ComfyUI loads meta rides in the safetensors metadata block (str→str)
.pt the campaign's original saves meta rides as a plain dict under "meta"

Tensor layout

Flat, one namespace, site index first:

blocks.{site_index}.{param_path}        # in the safetensors key space
{site_index}.{param_path}               # in memory / in the .pt payload

site_index is 0..n_sites-1 in training enumeration order.

The ordering law (read this before writing a loader)

Site order is diffusers named_modules() order, which registers down_blocksup_blocksmid_block. The mid block is LAST, not in the middle. For SD1.5 the width signature is

[320,320,640,640,1280,1280, 1280,1280,1280, 640,640,640, 320,320,320, 1280]

A denoiser's execution order is different (input → middle → output, giving …1280,1280,1280,1280… with mid at index 6). Zipping the two positionally misplaces 7 of 16 sites and still runs, producing quietly wrong images. Map by site identity, then verify the width signature — the two orders differ at indices 9 and 15, so the signature catches it. DiffusionAnchorCheckpoint.widths reads the signature off the tensors.

Metadata

Required (provenance — every file has these)

field type meaning
format str "amoe.diffusion.anchor"
version int/str 1
adapter.kind str relay · multiband3 · mono · bank · cond
substrate.family str sd15_unet · sdxl_unet · cosmos_dit
substrate.n_sites int must equal the enumerated site count at attach

Only relay and multiband3 are attachable. mono and bank are matched controls and falsified-routing evidence; cond is the Law-2 negative. Loaders may read them; attach() refuses them by design and says why.

Optional (provenance, written when known)

adapter.* spec (n_slots/K/tau/hidden for relay, rank for multiband3) · substrate.base_model_id · substrate.site_names · substrate.widths · objective.kind (eps|flow|v) · objective.shift · blob.lambda · dtype · seed · recipe.* · created · content_hash_v2 · imported_from · home_reconstructed

Optional (the usage card — schema v1.1)

What a UI renders. All optional, so raw campaign stacks stay valid.

field type meaning
display_name str human name, e.g. "SD1.5 Relay — Grounding v1 (seed 0)"
usage str what it does and when to reach for it
evidence str the measured verdict with numbers and seed status
recommended_strength float sane default for a strength slider
band_roles list[str] multiband only: role per band, LOW→HIGH
caveats list[str] what it costs, where it fails, what is unverified
license str SPDX-ish string
nc bool true = non-commercial (derived from NC weights)

DiffusionAnchorCheckpoint.card() returns exactly this block with safe defaults filled in.

Honesty rule for evidence

evidence states what was measured, at how many seeds, against what control — never a marketing claim. If a result is single-seed, it says so. If a matched control beat it on the aggregate metric, that goes in evidence or caveats, not omitted. Controls and falsified artifacts carry evidence describing what they refuted; that is their value.

Worked example

{
  "format": "amoe.diffusion.anchor",
  "version": 1,
  "display_name": "SD1.5 Multiband — Coarse-to-Fine v1 (seed 0)",
  "adapter": {"kind": "multiband3", "rank": 16},
  "substrate": {"family": "sd15_unet", "n_sites": 16,
                "base_model_id": "stable-diffusion-v1-5/stable-diffusion-v1-5"},
  "objective": {"kind": "eps"},
  "band_roles": ["fidelity/detail (LOW noise)",
                 "continuity/semantics (MID)",
                 "diversity/structure (HIGH noise)"],
  "usage": "Three sigma-band experts gated per sampling step. Lesion a band to see what it carries.",
  "evidence": "exp008, 2 seeds: band lesions surgical 3/3 — own-band damage 50-200x cross-band. A matched rank-48 monolith still edges it on aggregate eps-MSE under uniform pressure.",
  "recommended_strength": 1.0,
  "caveats": ["Trained on the stock SD1.5 eps trunk; other trunks are untested.",
              "The aggregate win belongs to the monolith control — the value here is the band structure, not the loss number."],
  "license": "MIT", "nc": false,
  "dtype": "float32", "seed": 0
}

Band gating (multiband3 only)

Band windows are cosine crossfades on s01, the normalized discrete timestep:

s01 = t / 1000            # eps trunks — t from the model's own sampling
s01 = sigma               # flow trunks — the SHIFT-warped sigma
edges = (0.35, 0.75)      # LAW constants, not tunables
xfade = 0.06

s01 is not a noise-level proxy. On the real SD1.5 schedule, 1 - alphas_cumprod[t] puts 316 of 1000 timesteps in a different band than the one the expert was trained on. In ComfyUI, get it from model_sampling.timestep(sigma) / 1000.

Loader checklist

  1. Read meta; reject if format is absent or unknown.
  2. Enumerate the host's sites; sort into training order.
  3. Assert n_sites matches, then assert the width signature matches checkpoint.widths — refuse loudly on mismatch.
  4. Cast adapters to the trunk's declared dtype (the dtype law).
  5. kind in {relay, multiband3} to attach; otherwise explain and stop.
  6. Render card() so the user sees provenance, evidence, and caveats.