codemap.md

Repository Atlas: minimax-h3-repo

Project Responsibility

The denoising half of a two-Space MiniMax-H3 deployment on Hugging Face ZeroGPU: joint video + synchronized soundtrack generation, unquantized bf16. The 195.9 GiB checkpoint cannot fit one ZeroGPU Space (150 GB storage cap), so MiniMaxH3Blocks is cut at its text_encoder step. This repo holds the transformer + two autoencoders; the 62 GiB Qwen3-VL conditioner runs in the sibling multimodalart/qwen3vl-conditioner Space and is called over the gradio API on every request.

System Entry Points

• app.py — Gradio UI + request orchestration. Runs load_models() at import (77.3 GB startup load on CPU). Entrypoint per README frontmatter (app_file: app.py).
• requirements.txt — Dependency manifest. diffusers comes from a pinned WIP PR commit (refs/pull/14371/head @ 665f5782), not PyPI.
• README.md — Space config (frontmatter sdk_version must match gradio==6.20.0) + design rationale and measured performance.

File Map

File	Responsibility	Detailed Notes
app.py	Request flow + Gradio UI	generate → remote /encode → @spaces.GPU _generate → encode_video mux. Owns CANVASES (cross-Space contract), snap_frames (17n+5), placement modes (pack/lazy/offload), get_duration GPU budget estimator. Turbo LoRA: A/B factors stored on CPU (~1.2 GB), merged on-the-fly via B @ A inside @spaces.GPU.
h3_split_blocks.py	Split pipeline definition	SequentialPipelineBlocks subclasses for both halves (Conditioner/Generator) and both partitions (t2va/ref2va). Defines the wire-format outputs (prompt_embeds + text_token_tags + resolved height/width/num_frames).
h3_aoti.py	AoTI acceleration	maybe_load/patch_blocks = runtime path (strict key bf16/torch2.11/sm120/dynamic, else eager). export_block/compile_and_save/upload = build path for the separate multimodalart/minimax-h3-aoti repo (never called by the Space).
spaces_constant_binding_patch.py	AoTI constant binding	Compile-side (register_loose_tensors, write_constant_aliases) + load-side (apply_spaces_constant_binding_patch) glue preventing SIGSEGV from anonymously-lifted export constants.
requirements.txt	Pip dependencies	All pins deliberate — see notes in file.
packages.txt	Space apt packages	ffmpeg (ZeroGPU system-level, not pip).
.gitattributes	HF LFS rules	Standard Space LFS filters for model artifacts.
AGENTS.md	Agent instructions	Critical rules, dependency pins, verification steps.

Data & Control Flow

1. Browser/client calls generate(prompt, image?, last_image?, canvas, duration, steps, seed, upsample, turbo).
2. snap_frames aligns the duration to 17*n + 5 frames at 24 fps; if turbo, steps forced to 7. encode_remote calls the conditioner's /encode over gradio_client (forwarding the caller's own ZeroGPU token).
3. Conditioner returns one safetensors file carrying prompt_embeds (1, num_text_tokens, 5120) bf16 + text_token_tags int64, with resolved height/width/num_frames in metadata + the plan.
4. _generate (decorated @spaces.GPU(duration=get_duration, size=xlarge)) runs the packed-sequence denoise loop + two decoders on GPU. If turbo, A/B factors are moved to CUDA (~1.2 GB) and LoRA is merged/unmerged around the pipeline call. Returns only (video frames, audio, sampling_rate) — a @spaces.GPU return crosses a process boundary by pickling, so pipeline state stays on the card.
5. encode_video (PyAV) muxes h264 frames + stereo AAC soundtrack into an MP4 in temp storage.

Integration Points

• Cross-Space wire contract: CANVASES label strings in app.py go over the wire and must stay identical to the conditioner's table; the wire format (two tensors + metadata) must stay in lockstep. Changing either breaks the split.
• AoTI package repo: multimodalart/minimax-h3-aoti — consumed at runtime via snapshot_download when H3_AOTI=1; produced by this repo's build path (requires HF_TOKEN + same-arch GPU).
• Model repo: MiniMaxAI/MiniMax-H3 (public diffusers checkpoint, subfolders transformer/ + vae/ + audio_vae/).
• Conditioner Space: multimodalart/qwen3vl-conditioner — remote /encode endpoint, billed to the requesting user's token.
• diffusers upstream: huggingface/diffusers PR #14371 — the pinned WIP commit whose block classes h3_split_blocks.py subclasses.