Upload README.md with huggingface_hub

README.md

@@ -1,8 +1,10 @@
 ---
 license: apache-2.0
 library_name: mlx
+base_model: ByteDance-Seed/SeedVR2-7B
 tags:
   - mlx
+  - mflux
   - mlx-swift
   - super-resolution
   - image-upscaling
@@ -12,30 +14,56 @@ tags:
 pipeline_tag: image-to-image
 ---
 
-# SeedVR2-7B-sharp (MLX-Swift) — int8
+# SeedVR2-7B-sharp (MLX) — int8
 
-**int8-quantized** MLX-Swift weights for **SeedVR2-7B** (ByteDance, ICLR 2026) — one-step
-diffusion **super-resolution**. ~Half the size of fp16, near-lossless. For the
-[`seedvr2-mlx-swift`](https://github.com/xocialize/seedvr2-mlx-swift) package.
-fp16 base: [`SeedVR2-7B-mlx`](https://huggingface.co/benc0/SeedVR2-7B-sharp-mlx).
+Runtime-agnostic **int8-quantized** MLX-format weights for **SeedVR2-7B-sharp**, the **sharp** variant of ByteDance's one-step diffusion **super-resolution / restoration** model (ICLR 2026) — tuned for stronger detail sharpening, for on-device upscaling on Apple Silicon.
 
-- **Quantization:** transformer DiT Linears int8, group size 64 (the small `vid_in.proj` and the VAE are kept fp16). Declared in `config.json` (`"quantization": {"bits": 8, "group_size": 64}`); the loader applies it automatically.
-- **Files:** `transformer.safetensors` (int8, ~8.5 GB vs 16.5 GB fp16) · `vae.safetensors` (fp16) · `pos_emb.safetensors` · `config.json`.
-- **Quality:** int8 `t_out` cosine vs fp16 = **0.9997755** (the sharp fine-tune has heavier-tailed weights than the standard checkpoint's 0.9999481, costing slightly more under group quantization); end-to-end image vs the fp16 pipeline = **54.9 dB PSNR** (visually lossless; device noise alone measures ~60 dB). Reload round-trip **bit-exact (cosine 1.0)**. (int4 degrades this model family badly — use int8 on-device.)
+Not tied to any single package — these load into:
+- [`mflux`](https://github.com/filipstrand/mflux) (Python MLX, actively maintained; also the parity reference these weights were validated against),
+- [`seedvr2-mlx-swift`](https://github.com/xocialize/seedvr2-mlx-swift) (MLX-Swift; **archived/read-only since Jun 2026** but functional — MIT-licensed and forkable),
+- or any MLX code that reconstructs the same module tree (see **Format notes** below).
 
-## Usage
+fp16 base: [`SeedVR2-7B-sharp-mlx`](https://huggingface.co/benc0/SeedVR2-7B-sharp-mlx) · standard checkpoint: [`SeedVR2-7B-mlx`](https://huggingface.co/benc0/SeedVR2-7B-mlx) · 3B family: [`mlx-community/SeedVR2-3B-mlx`](https://huggingface.co/mlx-community/SeedVR2-3B-mlx)
+
+- **Files:** `transformer.safetensors` (DiT, int8, ~8.8 GB vs 16.5 GB fp16) · `vae.safetensors` (3D-causal-conv VAE, fp16) · `pos_emb.safetensors` (precomputed text embedding) · `config.json`.
+- **Architecture (vs 3B):** vid_dim 3072 (2560), 24 heads (20), 36 layers (32), all layers multimodal, plain MLP (SwiGLU), rope_dim 64.
+- **Quality:** int8 `t_out` cosine vs fp16 = **0.9997755** (the sharp fine-tune has heavier-tailed weights than the standard checkpoint's 0.9999481, costing slightly more under group quantization); **end-to-end image vs the fp16 pipeline = 54.9 dB PSNR** (visually lossless — device noise alone measures ~60 dB). Reload round-trip **bit-exact**. (int4 degrades this model family badly — use int8 on-device.)
+
+## Usage — Python (MLX / mflux)
+
+```python
+import json, mlx.core as mx, mlx.nn as nn
+from mlx.utils import tree_unflatten
+from mflux.models.seedvr2.model.seedvr2_transformer.transformer import SeedVR2Transformer
+from mflux.models.seedvr2.weights.seedvr2_weight_definition import SeedVR2WeightDefinition
+
+cfg = json.load(open("config.json"))
+tx = SeedVR2Transformer(**cfg["transformer_overrides"])
+q = cfg["quantization"]                     # {"bits": 8, "group_size": 64}
+nn.quantize(tx, group_size=q["group_size"], bits=q["bits"],
+            class_predicate=SeedVR2WeightDefinition.quantization_predicate)
+tx.update(tree_unflatten(list(mx.load("transformer.safetensors").items())))
+mx.eval(tx.parameters())
+```
+
+The full pipeline (VAE, scheduler, pre/post-processing) lives in mflux: `mflux-upscale-seedvr2 --model seedvr2-7b --image-path input.png --resolution 2x` (note: mflux's built-in downloader fetches the PyTorch source weights and converts on the fly; loading *these* pre-converted files uses the snippet above).
+
+## Usage — Swift
 
 ```swift
-import SeedVR2MLX
+import SeedVR2MLX   // github.com/xocialize/seedvr2-mlx-swift (archived/read-only, MIT — fork to maintain)
 let upscaler = try SeedVR2Upscaler(directory: weightsDir)   // detects int8 from config, applies quantize on load
-let out = upscaler.upscale(processedImage: img, seed: 42)
+let out = upscaler.upscale(processedImage: img, seed: 42)   // [-1,1], dims padded to /16
 ```
 
+## Format notes (for other MLX runtimes)
+
+- **Key naming:** mflux module hierarchy, flattened with `mlx.utils.tree_flatten` (e.g. `blocks.17.attn.proj_qkv_vid.weight`). Deterministic mapping back to ByteDance's original PyTorch names: mflux `src/mflux/models/seedvr2/weights/seedvr2_weight_mapping.py`.
+- **Layouts:** MLX conventions throughout — VAE conv weights are `(O, *K, I)`.
+- **Config:** `config.json["transformer_overrides"]` carries the 7B dims (vid_dim 3072, heads 24, num_layers 36, mm_layers 36, rope_dim 64, …) and must be passed to the transformer constructor.
+- **Conditioning:** `pos_emb.safetensors` (58×5120, fp16) is the precomputed embedding of the fixed prompt — the text encoder is eliminated from this port, so it is a mandatory `txt` input.
+- **Quantization format:** standard MLX affine group quantization (bits 8, group 64). Each quantized Linear stores packed `weight` (U32) + `scales`/`biases` (F16). Only Linears with in-dim divisible by 64 are quantized — `vid_in.proj` (in-dim 132) and the whole VAE stay fp16. Declared in `config.json` so loaders can rebuild the module structure before `update()`.
+
 ## Provenance & license
 
-**ByteDance Seed** (*SeedVR2*, ICLR 2026, [ByteDance-Seed/SeedVR](https://github.com/ByteDance-Seed/SeedVR), **Apache-2.0**)
-→ fp16 [`numz/SeedVR2_comfyUI`](https://huggingface.co/numz/SeedVR2_comfyUI) (verified bitwise against
-ByteDance's original fp32 `.pth`, 1128/1128 tensors)
-→ MLX ref [`filipstrand/mflux`](https://github.com/filipstrand/mflux) → export + int8 conversion via
-[`xocialize/seedvr2-mlx`](https://github.com/xocialize/seedvr2-mlx) tooling. Format/precision-converted
-weights (not a new model); Apache-2.0 applies.
+Chain: **ByteDance Seed** — *SeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Training* (ICLR 2026, [arXiv:2506.05301](https://arxiv.org/abs/2506.05301)), [ByteDance-Seed/SeedVR](https://github.com/ByteDance-Seed/SeedVR), **Apache-2.0** → PyTorch fp16 redistribution [`numz/SeedVR2_comfyUI`](https://huggingface.co/numz/SeedVR2_comfyUI) (`seedvr2_ema_7b_sharp_fp16.safetensors`; independently verified **bitwise** against ByteDance's original fp32 `seedvr2_ema_7b_sharp.pth` — all 1128 tensors identical after fp32→fp16 cast) → MLX reference impl [`filipstrand/mflux`](https://github.com/filipstrand/mflux) → export + int8 conversion via [`xocialize/seedvr2-mlx`](https://github.com/xocialize/seedvr2-mlx) tooling. These are format/precision-converted weight artifacts (not a new model); Apache-2.0 applies. Credit ByteDance Seed (original), cite the paper.