Spaces:
Sleeping
Sleeping
|
Download README.md from AlayaLab/anygame-renderer: direct link, hf CLI and curl.
- Browser
- Download file 4.49 kB
-
https://huggingface.co/spaces/AlayaLab/anygame-renderer/resolve/main/README.md
- Command line
-
hf download hf://spaces/AlayaLab/anygame-renderer/README.md
-
curl -L -o README.md https://huggingface.co/spaces/AlayaLab/anygame-renderer/resolve/main/README.md
4.49 kB
| title: AnyGameRenderer | |
| emoji: 🎮 | |
| colorFrom: blue | |
| colorTo: indigo | |
| sdk: gradio | |
| sdk_version: 6.30.0 | |
| app_file: app.py | |
| pinned: false | |
| short_description: Blockout gameplay clips re-rendered as polished game video | |
| python_version: "3.12" | |
| startup_duration_timeout: 1h | |
| models: | |
| - AlayaLab/AnyGameRenderer | |
| - multimodalart/MiniMax-H3-Pruned | |
| - MiniMaxAI/MiniMax-H3 | |
| # AnyGameRenderer | |
| A demo of [`AlayaLab/AnyGameRenderer`](https://huggingface.co/AlayaLab/AnyGameRenderer), the entity-consistent | |
| generative game renderer from [*AnyGameRenderer: Entity-Consistent Generative Rendering Across Games and an | |
| Entity-Level Benchmark*](https://arxiv.org/pdf/2610.31466). Hand it the crude "programmer art" a game agent produces | |
| — a pale-gray blockout with one flat, saturated proxy color per entity class — plus the caption the paper trains on, | |
| and it returns the same gameplay re-rendered as a polished game clip: frame-aligned with the input, group counts and | |
| appearance timing intact, with a jointly generated soundtrack. One checkpoint covers shooters, kart racers, city | |
| driving, survivors-likes, turn-based battles and more. | |
| ## The request | |
| Two inputs, both required: | |
| | Slot | What it is | | |
| |---|---| | |
| | **Blockout clip** (`<Video 1>`) | an mp4 at ~24 fps, 2–15 s. 1344×768 recommended. Pale-gray geometry; one flat proxy color (red, orange, yellow, yellow-green, green, cyan, blue, purple, magenta) per character / weapon / vehicle class. No HUD, text or effects. | | |
| | **Caption** | the six-section caption the LoRA was trained on (subject_definitions, summary, retention_analysis, detailed_description, overall_soundscape, non_diegetic_music). The **caption builder** in the accordion builds it from a short spec, exactly as the paper's `make_caption.py` fills the training layout. | | |
| The three bundled examples are the paper's own — the WW1 shooter, the third-person egg heist and the city-driving | |
| clips its repo ships, with the captions built from their `spec.json` files. | |
| ## How it is deployed | |
| MiniMax-H3 is 195.9 GiB in bfloat16 and a ZeroGPU Space is evicted at 150 GB of storage, so the deployment is | |
| split the same way as its sibling `hugging-apps/minimax-h3-character-swap-lora`: this Space is the **denoising | |
| half** of `ref2va` (the `transformer_ref` partition and the two autoencoders), and the 62 GiB Qwen3-VL conditioner | |
| runs in [`multimodalart/qwen3vl-conditioner`](https://huggingface.co/spaces/multimodalart/qwen3vl-conditioner), | |
| called over the gradio API per request. `prompt_embeds` + `text_token_tags` is the whole wire format. | |
| The DiT is [`multimodalart/MiniMax-H3-Pruned`](https://huggingface.co/multimodalart/MiniMax-H3-Pruned)'s | |
| `transformer_ref` (37.5 GiB): the released partition with its AdaLN input projections folded onto their reachable | |
| rank. The AnyGameRenderer LoRA — rank 128, alpha 128, on `attn.qkv_proj`, `attn.out_proj`, `mlp.fc1`, `mlp.fc2` of | |
| every transformer block — is AdaLN-free by construction, so it applies verbatim; it is attached as a runtime PEFT | |
| adapter so the strength slider stays a per-request knob and **0.0 is the zero-shot base model**, the paper's own | |
| baseline. Loading goes through diffusers 0.41's `MiniMaxH3LoraLoaderMixin`, whose | |
| `_convert_non_diffusers_minimax_h3_lora_to_diffusers` handles this file's flattened musubi-tuner | |
| `lora_unet_blocks_…` names, folds the `.alpha` scalars in, splits the fused QKV and swaps the SwiGLU gate/value | |
| halves — the same transforms the paper's `infer.py` applies through musubi-tuner's merge. | |
| Defaults are the paper's own inference settings: 1344×768, 124 frames (the training bucket), 20 steps, seed 42, | |
| strength 1.0. The GPU reservation is a fitted cost model over the packed sequence, evaluated per request. | |
| ## Notes | |
| - Resolution is fixed to 1344×768 — what the LoRA was trained on. Length is snapped to `17n+5` frames; clips longer | |
| than 15 s are refused by the base model. | |
| - Generation is deterministic for a given seed; different seeds change appearance details, not layout. | |
| - The soundtrack is *generated*, not carried over from the blockout (which has no audio). | |
| ## Example assets and license | |
| The three example clips and their captions are from [`AlayaLab/AnyGameRenderer`](https://github.com/AlayaLab/AnyGameRenderer) | |
| (Apache-2.0), the paper's own repository. The adapter is distributed under the | |
| [MiniMax H3 Community License Agreement](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE) — read the | |
| upstream terms; nothing here extends them. | |