explicit-moe / README.md
Massyzs's picture
Model card: keep it short; setup and usage live in the GitHub README
22b63e1 verified
|
Raw History Blame Contribute Delete
5.06 kB
metadata
license: apache-2.0
base_model: linyq/kiwi-edit-5b-instruct-only-diffusers
pipeline_tag: video-to-video
tags:
  - video-editing
  - instruction-based-editing
  - mixture-of-experts
  - wan2.2
extra_gated_prompt: >-
  Access to these weights is granted manually by the authors. Please tell us
  your name, affiliation and intended use.
extra_gated_fields:
  Name: text
  Affiliation: text
  Intended use: text

explicit-moe

Code: https://github.com/massyzs/Video-Editor-MoE (scripts/moe/explicit-moe/) Optimizer state: removed (weights only).

Package contents

The weights are shipped as a tar.gz stream split into ~20 GB chunks (Hugging Face limits single files to 50 GB).

chunk size sha256
explicit-moe.tar.gz.part-00 8.81 GB a3732ceeb496e294b7ead2625ae5fa6eb12a0be24a5356d3a0988e392e8c152d

Total: 1 chunks, 8.81 GB compressed / 9.67 GB extracted.

Reassemble and verify:

huggingface-cli download Massyzs/explicit-moe --local-dir explicit-moe-package      # after your access request is approved
cd explicit-moe-package
cat explicit-moe.tar.gz.part-* | tar -xzf - -C <BASE_DIR>/ckpt/                # creates <BASE_DIR>/ckpt/explicit-moe/
cp SHA256SUMS.contents <BASE_DIR>/ckpt/ && (cd <BASE_DIR>/ckpt && sha256sum -c SHA256SUMS.contents)   # optional check

MANIFEST.json lists every payload file with its size and sha256, and every chunk with its sha256 (SHA256SUMS.parts).

Extracted files:

file size sha256
explicit-moe/fusion_moe_v2_absorb.safetensors 1.06 GB ec6ff554367b337222e7e1f9ce9ca8bf094d584958a71fdf67fd6b7b757b6394
explicit-moe/lora_experts/add/dit_query_router.safetensors 1.29 GB e89781f56ce9ae1b6378cc1a00b8ae91cf0d89c17e39e1e3fdf06c66b28fb74b
explicit-moe/lora_experts/background/dit_query_router.safetensors 1.29 GB c7ed8394bd3493168130186c21f189803131d188e968bf745ef160f3b4753fac
explicit-moe/lora_experts/remove/dit_query_router.safetensors 1.29 GB 58a8936c1f208e55266f624d97306008698eaf57de3e2ed4a73796190364a011
explicit-moe/lora_experts/replace/dit_query_router.safetensors 1.29 GB 88406dc90ce0a79b591cbcb2ce0f9762f1e97e4c5a421c1d21133bd9d88db0c1
explicit-moe/lora_experts/style/dit_query_router.safetensors 1.29 GB d5f03ea7aea848e5d108b5dc596d135400913670a0ebb4a16bba109e6c4cfde8
explicit-moe/meta.json 1038 B d407255d560faafd42249393a4f3f6dbcbd3093f0a245d6b6da978349950645f
explicit-moe/moe_v2_state.pt 2.13 GB 07929c4ed8ad28dc1560a5508ed7e6ca20d79f12e1664d860ab4cccf13fed0b0
explicit-moe/router_head_finetuned.pt 16.8 MB fc99904f4870e610c1b758a1638a0b735e095f824d9a9cc76b928bc504a3a902
explicit-moe/router_head_online.pt 16.8 MB 1695f5e0defe34a95ec9dc4bfa0bd9e41b08d602aed8bc908f66bce42f41471b

Base weights you also need

The DiT base weights are required: this package contains only the trainable parts (task experts, fusion layers, router head), which are applied on top of the frozen Kiwi-Edit 5B DiT.

All frozen base components are published, already converted to the layout this code expects, in Massyzs/kiwi-edit-5b-instruct-only-videoxfun:

huggingface-cli download Massyzs/kiwi-edit-5b-instruct-only-videoxfun --local-dir <BASE_DIR>/ckpt/kiwi-edit-instruct-only-videoxfun

See BASE_WEIGHTS.md for where each component comes from (Kiwi-Edit, Wan2.2) and its checksum.

Inference

python scripts/moe/explicit-moe/moe_v2_infer.py --base_dir <BASE_DIR> --ckpt <BASE_DIR>/ckpt/explicit-moe \
    --src_video input.mp4 --prompt "Replace the red car with a blue truck" --name edited --out_dir <OUT_DIR>
# multi-step instruction: --prompts "Remove the dog ||| Convert to oil painting style"

Single 80 GB GPU. Output: <OUT_DIR>/<name>.mp4 plus lossless PNG frames in <OUT_DIR>/frames/<name>/. moe_v2_state.pt (fp32 trainable tensors, optimizer removed) is what the loader reads; fusion_moe_v2_absorb.safetensors and router_head_finetuned.pt contain the same tensors in bf16 / as a state_dict for convenience. router_head_online.pt supplies only the router-head architecture fields.

Setup and usage (installation, where to put the downloaded weights, inputs and outputs) are described in the repository README.

Training

The training launcher is scripts/moe/explicit-moe/train_multinode.sh in the code repository: edit the configuration block at the top (BASE_DIR, data paths, node addresses) and run bash scripts/moe/explicit-moe/train_multinode.sh <node rank> on every node. scripts/moe/explicit-moe/train_smoke.sh is a single-node smoke test. The data formats are described in docs/DATA.md of the repository.

License

The released weights and code are under Apache-2.0. The frozen base components (Kiwi-Edit MLLM encoder / DiT base, Wan2.2 VAE) are not redistributed here and remain subject to their own licenses.