Upload folder using huggingface_hub
Browse files- README.md +81 -6
- app.py +71 -0
- minimax/__init__.py +3 -0
- minimax/apps/__init__.py +1 -0
- minimax/apps/gradio/__init__.py +5 -0
- minimax/apps/gradio/demos.py +76 -0
- minimax/apps/gradio/derive.py +141 -0
- minimax/apps/gradio/state.py +167 -0
- minimax/apps/gradio/ui.py +562 -0
- minimax/formatting/__init__.py +9 -0
- minimax/formatting/envelope.py +122 -0
- minimax/formatting/fields.py +46 -0
- minimax/formatting/instructions.py +126 -0
- minimax/formatting/meta_leak.py +32 -0
- minimax/formatting/postprocess.py +168 -0
- minimax/formatting/ref_repair.py +106 -0
- minimax/formatting/timestamps.py +108 -0
- minimax/modeling/__init__.py +0 -0
- minimax/modeling/generate.py +63 -0
- minimax/modeling/loader.py +80 -0
- minimax/paths.py +24 -0
- minimax/scoring/__init__.py +8 -0
- minimax/scoring/format_score.py +157 -0
- prompts/system_base.txt +27 -0
- prompts/system_base_fl2va.txt +20 -0
- prompts/system_base_i2va.txt +20 -0
- prompts/system_base_l2va.txt +20 -0
- prompts/system_base_t2va.txt +18 -0
- prompts/system_ref.txt +42 -0
- requirements.txt +10 -0
README.md
CHANGED
|
@@ -1,13 +1,88 @@
|
|
| 1 |
---
|
| 2 |
title: MiniMax H3 Prompt Enhancer
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: gradio
|
| 7 |
-
sdk_version:
|
| 8 |
-
python_version: '3.12'
|
| 9 |
app_file: app.py
|
| 10 |
pinned: false
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
title: MiniMax H3 Prompt Enhancer
|
| 3 |
+
emoji: "🎬"
|
| 4 |
+
colorFrom: purple
|
| 5 |
+
colorTo: pink
|
| 6 |
sdk: gradio
|
| 7 |
+
sdk_version: 5.49.1
|
|
|
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
+
license: other
|
| 11 |
+
short_description: Trained LLM for MiniMax H3 structured video prompts
|
| 12 |
+
tags:
|
| 13 |
+
- text-generation
|
| 14 |
+
- video
|
| 15 |
+
- prompt
|
| 16 |
+
- minimax
|
| 17 |
+
- liquid
|
| 18 |
+
- lfm2
|
| 19 |
+
- zerogpu
|
| 20 |
---
|
| 21 |
|
| 22 |
+
# MiniMax H3 Prompt Enhancer
|
| 23 |
+
|
| 24 |
+
**Not another “make my prompt longer” toy.**
|
| 25 |
+
|
| 26 |
+
This Space runs a **purpose-trained small language model** that rewrites rough ideas into **production-ready MiniMax H3 video prompts** — the same kind of structured audiovisual brief MiniMax expects: shots, camera language, diegetic sound, and score, not a bag of adjectives.
|
| 27 |
+
|
| 28 |
+
Type something half-formed like *“baker opens a shop at dawn”*. The model doesn’t slap on “cinematic, 8k, masterpiece.” It **reasons** about scene, motion, and audio, then emits the **exact field layout** used for MiniMax video generation.
|
| 29 |
+
|
| 30 |
+
---
|
| 31 |
+
|
| 32 |
+
## Why this is different
|
| 33 |
+
|
| 34 |
+
| Typical prompt “enhancers” | This one |
|
| 35 |
+
|---|---|
|
| 36 |
+
| Templates, regex, or a generic chat model | **Fine-tuned LLM** dedicated to this job |
|
| 37 |
+
| Mood words and hashtag soup | **Shot lists, camera moves, soundscape, music** |
|
| 38 |
+
| Guesswork about “what sounds good” | Aligned to **MiniMax H3 video prompt writing guidelines** |
|
| 39 |
+
| One free-form paragraph | **Structured output** MiniMax can actually consume |
|
| 40 |
+
|
| 41 |
+
Under the hood: [geocine/minimax-video-prompt-enhancer-350m](https://huggingface.co/geocine/minimax-video-prompt-enhancer-350m) — LoRA SFT on [LiquidAI/LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M), trained on teacher rewrites that follow the official MiniMax base/ref prompt schema (T2VA · I2VA · FL2VA · L2VA · full reference).
|
| 42 |
+
|
| 43 |
+
---
|
| 44 |
+
|
| 45 |
+
## What you get
|
| 46 |
+
|
| 47 |
+
Paste a rough idea (and optional reference notes). Choose duration. Hit **Generate**.
|
| 48 |
+
|
| 49 |
+
You get a full brief, for example:
|
| 50 |
+
|
| 51 |
+
- **`integrated_multimodal_description`** — style, framing, action, camera, dialogue tags
|
| 52 |
+
- **`overall_soundscape`** — ambience and physical sound
|
| 53 |
+
- **`non_diegetic_music`** — audience-only score (or N/A)
|
| 54 |
+
- For image/video refs: alignment lines, subjects, retention-style structure in **full-reference** mode
|
| 55 |
+
|
| 56 |
+
Built for people who actually ship MiniMax clips — not for prompt-cosplay.
|
| 57 |
+
|
| 58 |
+
---
|
| 59 |
+
|
| 60 |
+
## How to use
|
| 61 |
+
|
| 62 |
+
1. Write a rough user prompt (dialogue, shot notes, or a one-liner).
|
| 63 |
+
2. Optionally fill **References & frames** (first/last frame, subjects, clips, audio). The Space picks the task for you.
|
| 64 |
+
3. Set duration. Generate. Copy the structured prompt into MiniMax.
|
| 65 |
+
|
| 66 |
+
**Tip:** Short, real-user prompts work best. The model expands them into the full MiniMax format.
|
| 67 |
+
|
| 68 |
+
---
|
| 69 |
+
|
| 70 |
+
## ZeroGPU
|
| 71 |
+
|
| 72 |
+
Runs on **Hugging Face ZeroGPU** — a GPU spins up only while **Generate** runs.
|
| 73 |
+
|
| 74 |
+
- Visitors use **their** daily HF GPU quota (or PRO credits if they go over).
|
| 75 |
+
- You must be **logged in** to use ZeroGPU Spaces.
|
| 76 |
+
- No permanent GPU rental for idle time.
|
| 77 |
+
|
| 78 |
+
---
|
| 79 |
+
|
| 80 |
+
## Model & license
|
| 81 |
+
|
| 82 |
+
| | |
|
| 83 |
+
|--|--|
|
| 84 |
+
| Model | [geocine/minimax-video-prompt-enhancer-350m](https://huggingface.co/geocine/minimax-video-prompt-enhancer-350m) |
|
| 85 |
+
| Base | [LiquidAI/LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M) |
|
| 86 |
+
| License | Liquid **lfm1.0** — read the base card before commercial use |
|
| 87 |
+
|
| 88 |
+
This is a **text prompt specialist**, not a video generator. It writes the brief; MiniMax (or your pipeline) makes the frames.
|
app.py
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Hugging Face ZeroGPU Space — MiniMax Video Prompt Enhancer.
|
| 3 |
+
|
| 4 |
+
Hosts on free ZeroGPU. Visitors spend *their* daily HF GPU quota / credits
|
| 5 |
+
when they click Generate — the host is not billed for continuous GPU rental.
|
| 6 |
+
|
| 7 |
+
Model: geocine/minimax-video-prompt-enhancer-350m
|
| 8 |
+
"""
|
| 9 |
+
|
| 10 |
+
from __future__ import annotations
|
| 11 |
+
|
| 12 |
+
import os
|
| 13 |
+
import sys
|
| 14 |
+
from pathlib import Path
|
| 15 |
+
|
| 16 |
+
ROOT = Path(__file__).resolve().parent
|
| 17 |
+
if str(ROOT) not in sys.path:
|
| 18 |
+
sys.path.insert(0, str(ROOT))
|
| 19 |
+
|
| 20 |
+
import spaces
|
| 21 |
+
import torch
|
| 22 |
+
|
| 23 |
+
from minimax.apps.gradio import build_app
|
| 24 |
+
from minimax.apps.gradio.state import InferenceSession, ModelBundle
|
| 25 |
+
from minimax.modeling.loader import build_stop_ids, load_model_and_tokenizer
|
| 26 |
+
from minimax.paths import DEFAULT_MODEL
|
| 27 |
+
|
| 28 |
+
MODEL_ID = os.environ.get("MODEL_ID", DEFAULT_MODEL)
|
| 29 |
+
|
| 30 |
+
# ---------------------------------------------------------------------------
|
| 31 |
+
# ZeroGPU: place the model on `cuda` at module level (emulated until a real
|
| 32 |
+
# GPU is attached inside @spaces.GPU). Visitors pay quota on each Generate.
|
| 33 |
+
# ---------------------------------------------------------------------------
|
| 34 |
+
print(f"Loading {MODEL_ID} onto cuda (ZeroGPU)...", flush=True)
|
| 35 |
+
_model, _tokenizer = load_model_and_tokenizer(MODEL_ID)
|
| 36 |
+
_stop_ids = build_stop_ids(_tokenizer)
|
| 37 |
+
_PRELOADED = ModelBundle(
|
| 38 |
+
path=MODEL_ID, model=_model, tokenizer=_tokenizer, stop_ids=_stop_ids
|
| 39 |
+
)
|
| 40 |
+
print(f"Ready on device={next(_model.parameters()).device}", flush=True)
|
| 41 |
+
|
| 42 |
+
_orig_ensure = InferenceSession.ensure_loaded
|
| 43 |
+
_orig_generate = InferenceSession.generate
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
def _ensure_preloaded(self, model_path=None):
|
| 47 |
+
path = str(model_path or self.default_model)
|
| 48 |
+
if path in {MODEL_ID, DEFAULT_MODEL, "geocine/minimax-video-prompt-enhancer-350m"}:
|
| 49 |
+
self._bundle = _PRELOADED
|
| 50 |
+
return _PRELOADED
|
| 51 |
+
return _orig_ensure(self, model_path)
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
@spaces.GPU(duration=90)
|
| 55 |
+
def _run_generate(session, kwargs):
|
| 56 |
+
"""GPU-scoped body — real CUDA is available only here on ZeroGPU."""
|
| 57 |
+
return _orig_generate(session, **kwargs)
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
def _generate_zerogpu(self, **kwargs):
|
| 61 |
+
return _run_generate(self, kwargs)
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
InferenceSession.ensure_loaded = _ensure_preloaded # type: ignore[method-assign]
|
| 65 |
+
InferenceSession.generate = _generate_zerogpu # type: ignore[method-assign]
|
| 66 |
+
|
| 67 |
+
demo = build_app(model_path=MODEL_ID)
|
| 68 |
+
|
| 69 |
+
if __name__ == "__main__":
|
| 70 |
+
demo.queue(default_concurrency_limit=1)
|
| 71 |
+
demo.launch()
|
minimax/__init__.py
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""MiniMax video-prompt enhancer pipeline (student: LiquidAI/LFM2.5-350M)."""
|
| 2 |
+
|
| 3 |
+
__all__ = ["paths"]
|
minimax/apps/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Interactive apps (Gradio, etc.) built on the inference stack."""
|
minimax/apps/gradio/__init__.py
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Gradio UI for interactive champion testing."""
|
| 2 |
+
|
| 3 |
+
from .ui import build_app, launch
|
| 4 |
+
|
| 5 |
+
__all__ = ["build_app", "launch"]
|
minimax/apps/gradio/demos.py
ADDED
|
@@ -0,0 +1,76 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Starter examples for the Gradio app."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from minimax.formatting.envelope import BASE_TASKS, REF_TASKS
|
| 6 |
+
|
| 7 |
+
TASK_CHOICES = sorted(BASE_TASKS) + sorted(REF_TASKS)
|
| 8 |
+
|
| 9 |
+
# (task, duration, prompt, assets_text) — assets one per line
|
| 10 |
+
EXAMPLES: list[list] = [
|
| 11 |
+
[
|
| 12 |
+
"T2VA",
|
| 13 |
+
6.0,
|
| 14 |
+
(
|
| 15 |
+
"A baker opens a small street bakery at dawn, places a loaf on the counter, "
|
| 16 |
+
'and says "First batch of the morning." Cinematic, two shots.'
|
| 17 |
+
),
|
| 18 |
+
"",
|
| 19 |
+
],
|
| 20 |
+
[
|
| 21 |
+
"I2VA",
|
| 22 |
+
6.0,
|
| 23 |
+
(
|
| 24 |
+
"Continue from the first frame: she looks up from a letter at the rain-covered "
|
| 25 |
+
'train window and quietly says "I get off at the next station."'
|
| 26 |
+
),
|
| 27 |
+
"Picture 1: first frame — young woman in a dark coat by a rain train window at night, folded letter",
|
| 28 |
+
],
|
| 29 |
+
[
|
| 30 |
+
"FL2VA",
|
| 31 |
+
8.0,
|
| 32 |
+
(
|
| 33 |
+
"Shot 1: a cyclist rolls through morning fog on a riverside path. "
|
| 34 |
+
"Shot 2: she stops at a bridge and watches the mist lift. Soft documentary feel."
|
| 35 |
+
),
|
| 36 |
+
"Picture 1: first frame — cyclist entering fog\nPicture 2: last frame — bridge overlook, mist clearing",
|
| 37 |
+
],
|
| 38 |
+
[
|
| 39 |
+
"reference_generation",
|
| 40 |
+
8.0,
|
| 41 |
+
"a man in a spider-themed suit walking through fire",
|
| 42 |
+
(
|
| 43 |
+
"Picture 1: subject/scene reference — man in red-and-blue spider-themed suit\n"
|
| 44 |
+
"Picture 2: environment / secondary look reference — corridor of practical fire"
|
| 45 |
+
),
|
| 46 |
+
],
|
| 47 |
+
[
|
| 48 |
+
"keyframe_completion",
|
| 49 |
+
8.0,
|
| 50 |
+
"a fantastic steam train on a street painted in oil",
|
| 51 |
+
"Picture 1: keyframe anchor for: a fantastic steam train on a street painted in oil",
|
| 52 |
+
],
|
| 53 |
+
[
|
| 54 |
+
"video_editing",
|
| 55 |
+
8.0,
|
| 56 |
+
(
|
| 57 |
+
"Reference the coffee shop and the blonde woman; a man walks in with a Samoyed "
|
| 58 |
+
'that lunges at her cookie. She says "Hey! Watch your dog!" Sitcom, multi-shot.'
|
| 59 |
+
),
|
| 60 |
+
(
|
| 61 |
+
"Picture 1: coffee-shop interior — brick wall, orange sofa, neon sign\n"
|
| 62 |
+
"Video 1: young blonde woman in light-pink shirt\n"
|
| 63 |
+
"Video 2: young man in dark-grey hoodie\n"
|
| 64 |
+
"Picture 2: fluffy white Samoyed"
|
| 65 |
+
),
|
| 66 |
+
],
|
| 67 |
+
[
|
| 68 |
+
"video_continuation+audio_reference",
|
| 69 |
+
8.0,
|
| 70 |
+
"panoramic golden sand beach, blue tides, palms, sunset, seabirds",
|
| 71 |
+
(
|
| 72 |
+
"Video 1: source ending to continue — panoramic golden sand beach at sunset\n"
|
| 73 |
+
"Audio 1: audio continuity reference (not 1:1 copy)"
|
| 74 |
+
),
|
| 75 |
+
],
|
| 76 |
+
]
|
minimax/apps/gradio/derive.py
ADDED
|
@@ -0,0 +1,141 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Deterministic task derivation from role-tagged inputs.
|
| 2 |
+
|
| 3 |
+
The user never picks a cryptic task code (T2VA / reference_generation / …).
|
| 4 |
+
Instead they fill the *role* slots that describe what they actually have — a
|
| 5 |
+
first frame, a clip to continue, subject references, and so on — and the task
|
| 6 |
+
label falls out of which slots are populated.
|
| 7 |
+
|
| 8 |
+
Intent that isn't recoverable from the inputs (continue-vs-edit a lone clip,
|
| 9 |
+
what a single image *means*) is resolved by precedence + sensible defaults, so
|
| 10 |
+
the common cases need zero explicit choices. The derived label is surfaced back
|
| 11 |
+
to the UI so the mapping stays transparent.
|
| 12 |
+
|
| 13 |
+
Asset lines are emitted in the exact phrasing the champion trained on (verified
|
| 14 |
+
against data/train/train.jsonl), because the model conditions on both the task
|
| 15 |
+
label and the asset wording.
|
| 16 |
+
"""
|
| 17 |
+
|
| 18 |
+
from __future__ import annotations
|
| 19 |
+
|
| 20 |
+
from dataclasses import dataclass, field
|
| 21 |
+
|
| 22 |
+
# MiniMax H3 Ref2VA media caps (mirrors D:/PL/MiniMaxUI REF_LIMITS).
|
| 23 |
+
REF_LIMITS = {"image": 9, "video": 2, "audio": 2, "total": 12}
|
| 24 |
+
# Duration bounds (seconds) for the H3 checkpoint.
|
| 25 |
+
DURATION = {"min": 5, "max": 15, "default": 5, "step": 1}
|
| 26 |
+
# Max characters for the rough user prompt (matches the enhance input schema).
|
| 27 |
+
PROMPT_MAX = 7000
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
@dataclass
|
| 31 |
+
class Derivation:
|
| 32 |
+
task: str
|
| 33 |
+
assets: list[str]
|
| 34 |
+
warnings: list[str] = field(default_factory=list)
|
| 35 |
+
|
| 36 |
+
@property
|
| 37 |
+
def assets_text(self) -> str:
|
| 38 |
+
return "\n".join(self.assets)
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
def _clean(value: str | None) -> str:
|
| 42 |
+
return (value or "").strip()
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
def _lines(value: str | None) -> list[str]:
|
| 46 |
+
return [ln.strip() for ln in _clean(value).splitlines() if ln.strip()]
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
def derive(
|
| 50 |
+
*,
|
| 51 |
+
first_frame: str | None = None,
|
| 52 |
+
last_frame: str | None = None,
|
| 53 |
+
subject_refs: str | None = None,
|
| 54 |
+
keyframe: str | None = None,
|
| 55 |
+
continue_video: str | None = None,
|
| 56 |
+
edit_footage: str | None = None,
|
| 57 |
+
audio_ref: str | None = None,
|
| 58 |
+
) -> Derivation:
|
| 59 |
+
"""Map role slots to a (task, assets) pair.
|
| 60 |
+
|
| 61 |
+
Precedence (most specific first): continue → edit → subject refs → keyframe
|
| 62 |
+
→ first+last → first → last → text-only. Audio only forms a ``+audio``
|
| 63 |
+
variant for the three ref tasks that have one.
|
| 64 |
+
"""
|
| 65 |
+
first, last = _clean(first_frame), _clean(last_frame)
|
| 66 |
+
keyframe = _clean(keyframe)
|
| 67 |
+
continue_video = _clean(continue_video)
|
| 68 |
+
subjects, edits, audios = _lines(subject_refs), _lines(edit_footage), _lines(audio_ref)
|
| 69 |
+
audio_on = bool(audios)
|
| 70 |
+
|
| 71 |
+
assets: list[str] = []
|
| 72 |
+
counts = {"Picture": 0, "Video": 0, "Audio": 0}
|
| 73 |
+
|
| 74 |
+
def tag(kind: str) -> str:
|
| 75 |
+
counts[kind] += 1
|
| 76 |
+
return f"{kind} {counts[kind]}"
|
| 77 |
+
|
| 78 |
+
def add(kind: str, body: str) -> None:
|
| 79 |
+
assets.append(f"{tag(kind)}: {body}")
|
| 80 |
+
|
| 81 |
+
def audio_lines(reuse: bool) -> None:
|
| 82 |
+
for a in audios:
|
| 83 |
+
if reuse:
|
| 84 |
+
body = f"synchronized audio track of {a}"
|
| 85 |
+
else:
|
| 86 |
+
body = f"audio continuity reference (not 1:1 copy) — {a}"
|
| 87 |
+
add("Audio", body)
|
| 88 |
+
|
| 89 |
+
if continue_video:
|
| 90 |
+
add("Video", f"source ending to continue — {continue_video}")
|
| 91 |
+
if audio_on:
|
| 92 |
+
audio_lines(reuse=False)
|
| 93 |
+
task = "video_continuation+audio_reference"
|
| 94 |
+
else:
|
| 95 |
+
task = "video_continuation"
|
| 96 |
+
elif edits:
|
| 97 |
+
for clip in edits:
|
| 98 |
+
add("Video", f"source clip to edit — {clip}")
|
| 99 |
+
if audio_on:
|
| 100 |
+
audio_lines(reuse=True)
|
| 101 |
+
task = "video_editing+audio_reuse"
|
| 102 |
+
else:
|
| 103 |
+
task = "video_editing"
|
| 104 |
+
elif subjects:
|
| 105 |
+
for i, ref in enumerate(subjects):
|
| 106 |
+
role = "subject/scene reference" if i == 0 else "environment / secondary look reference"
|
| 107 |
+
add("Picture", f"{role} — {ref}")
|
| 108 |
+
if audio_on:
|
| 109 |
+
audio_lines(reuse=False)
|
| 110 |
+
task = "reference_generation+audio_reference"
|
| 111 |
+
else:
|
| 112 |
+
task = "reference_generation"
|
| 113 |
+
elif keyframe:
|
| 114 |
+
add("Picture", f"keyframe anchor for: {keyframe}")
|
| 115 |
+
task = "keyframe_completion"
|
| 116 |
+
elif first and last:
|
| 117 |
+
add("Picture", f"first frame — start state for: {first}")
|
| 118 |
+
tail = "last frame — end state after continuous motion"
|
| 119 |
+
add("Picture", f"{tail} — {last}" if last else tail)
|
| 120 |
+
task = "FL2VA"
|
| 121 |
+
elif first:
|
| 122 |
+
add("Picture", f"first frame — opening composition for: {first}")
|
| 123 |
+
task = "I2VA"
|
| 124 |
+
elif last:
|
| 125 |
+
add("Picture", f"last frame — final composition for: {last}")
|
| 126 |
+
task = "L2VA"
|
| 127 |
+
else:
|
| 128 |
+
task = "T2VA"
|
| 129 |
+
|
| 130 |
+
warnings: list[str] = []
|
| 131 |
+
img, vid, aud = counts["Picture"], counts["Video"], counts["Audio"]
|
| 132 |
+
if img > REF_LIMITS["image"]:
|
| 133 |
+
warnings.append(f"{img} images exceeds the {REF_LIMITS['image']}-image limit")
|
| 134 |
+
if vid > REF_LIMITS["video"]:
|
| 135 |
+
warnings.append(f"{vid} videos exceeds the {REF_LIMITS['video']}-video limit")
|
| 136 |
+
if aud > REF_LIMITS["audio"]:
|
| 137 |
+
warnings.append(f"{aud} audio refs exceeds the {REF_LIMITS['audio']}-audio limit")
|
| 138 |
+
if img + vid + aud > REF_LIMITS["total"]:
|
| 139 |
+
warnings.append(f"{img + vid + aud} references exceeds the {REF_LIMITS['total']} total limit")
|
| 140 |
+
|
| 141 |
+
return Derivation(task=task, assets=assets, warnings=warnings)
|
minimax/apps/gradio/state.py
ADDED
|
@@ -0,0 +1,167 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Lazy model cache for the Gradio app."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from dataclasses import dataclass
|
| 6 |
+
from pathlib import Path
|
| 7 |
+
from typing import Any
|
| 8 |
+
|
| 9 |
+
from minimax.modeling.generate import generate_one
|
| 10 |
+
from minimax.modeling.loader import build_stop_ids, load_model_and_tokenizer
|
| 11 |
+
from minimax.paths import DEFAULT_MODEL
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
def parse_assets(assets_text: str | None) -> list[str] | None:
|
| 15 |
+
if not assets_text or not str(assets_text).strip():
|
| 16 |
+
return None
|
| 17 |
+
lines = []
|
| 18 |
+
for line in str(assets_text).splitlines():
|
| 19 |
+
line = line.strip()
|
| 20 |
+
if not line or line in {"(none)", "- (none)", "- none"}:
|
| 21 |
+
continue
|
| 22 |
+
if line.startswith("- "):
|
| 23 |
+
line = line[2:].strip()
|
| 24 |
+
lines.append(line)
|
| 25 |
+
return lines or None
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
# Human-readable labels for the format-score checks, so users see what each
|
| 29 |
+
# check means (and what's missing) instead of raw internal keys.
|
| 30 |
+
_CHECK_LABELS = {
|
| 31 |
+
"nonempty": "Output is not empty",
|
| 32 |
+
"has_shot1": "Has an opening shot ([Shot 1])",
|
| 33 |
+
"within_h3_char_limit": "Within the 7000-character limit",
|
| 34 |
+
"no_instruction_leak": "Scene body is not paraphrasing the format rules",
|
| 35 |
+
"field_order": "Sections are in the required order",
|
| 36 |
+
"no_heavy_loop": "No duplicated sections",
|
| 37 |
+
"i2va_instruction": "Starts with the image-to-video alignment line",
|
| 38 |
+
"fl2va_alignment": "Starts with the first + last-frame alignment line",
|
| 39 |
+
"l2va_alignment": "Starts with the last-frame alignment line",
|
| 40 |
+
"has_shot2_optional": "Second shot present (optional)",
|
| 41 |
+
"shots_have_timestamps": "Later shots carry timestamps",
|
| 42 |
+
"shot1_no_bogus_timestamp": "Opening shot has no timestamp",
|
| 43 |
+
"summary_task_prefix": "Summary starts with a task-type tag",
|
| 44 |
+
"has_reference_label": "Cites at least one reference tag",
|
| 45 |
+
}
|
| 46 |
+
|
| 47 |
+
|
| 48 |
+
def _humanize_check(key: str) -> str:
|
| 49 |
+
if key in _CHECK_LABELS:
|
| 50 |
+
return _CHECK_LABELS[key]
|
| 51 |
+
if key.startswith("field:"):
|
| 52 |
+
name = key[len("field:") :].rstrip(":").replace("_", " ")
|
| 53 |
+
return f'Has the "{name}" section'
|
| 54 |
+
return key.replace("_", " ")
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
# Colored inline marks (Markdown can't color text; HTML spans can).
|
| 58 |
+
_OK = '<span style="color:var(--gc-accent);font-weight:700">✓</span>'
|
| 59 |
+
_MISS = '<span style="color:var(--gc-danger);font-weight:700">✗</span>'
|
| 60 |
+
|
| 61 |
+
|
| 62 |
+
def format_checks(report: dict) -> str:
|
| 63 |
+
passed = bool(report.get("pass"))
|
| 64 |
+
if passed:
|
| 65 |
+
verdict = '<span style="color:var(--gc-accent);font-weight:700">✓ Looks good</span>'
|
| 66 |
+
else:
|
| 67 |
+
verdict = '<span style="color:var(--gc-danger);font-weight:700">⚠ Needs a fix</span>'
|
| 68 |
+
chars = report.get("char_len", "?")
|
| 69 |
+
try:
|
| 70 |
+
score_pct = f"{float(report.get('score')) * 100:.0f}%"
|
| 71 |
+
except (TypeError, ValueError):
|
| 72 |
+
score_pct = str(report.get("score"))
|
| 73 |
+
lines = [f"{verdict} · Score {score_pct} · {chars} / 7000 chars"]
|
| 74 |
+
checks = report.get("checks") or {}
|
| 75 |
+
if checks:
|
| 76 |
+
# optional checks that simply weren't triggered aren't failures — don't
|
| 77 |
+
# list them under "Needs fixing"
|
| 78 |
+
missing = [k for k, v in checks.items() if not v and "optional" not in k]
|
| 79 |
+
ok = [k for k, v in checks.items() if v]
|
| 80 |
+
lines.append("")
|
| 81 |
+
if missing:
|
| 82 |
+
lines.append("**Needs fixing:**")
|
| 83 |
+
lines += [f"- {_MISS} {_humanize_check(k)}" for k in missing]
|
| 84 |
+
lines.append("")
|
| 85 |
+
lines.append("**Passed:**")
|
| 86 |
+
lines += [f"- {_OK} {_humanize_check(k)}" for k in ok]
|
| 87 |
+
else:
|
| 88 |
+
lines.append(f"**All {len(ok)} checks passed:**")
|
| 89 |
+
lines += [f"- {_OK} {_humanize_check(k)}" for k in ok]
|
| 90 |
+
notes = report.get("notes") or []
|
| 91 |
+
if notes:
|
| 92 |
+
lines.append("")
|
| 93 |
+
lines.append("**Notes:** " + "; ".join(notes))
|
| 94 |
+
return "\n".join(lines)
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
@dataclass
|
| 98 |
+
class ModelBundle:
|
| 99 |
+
path: str
|
| 100 |
+
model: Any
|
| 101 |
+
tokenizer: Any
|
| 102 |
+
stop_ids: list[int]
|
| 103 |
+
|
| 104 |
+
|
| 105 |
+
class InferenceSession:
|
| 106 |
+
"""Holds one loaded model; reloads when the path changes."""
|
| 107 |
+
|
| 108 |
+
def __init__(self, default_model: str | Path | None = None) -> None:
|
| 109 |
+
self.default_model = str(default_model or DEFAULT_MODEL)
|
| 110 |
+
self._bundle: ModelBundle | None = None
|
| 111 |
+
|
| 112 |
+
@property
|
| 113 |
+
def loaded_path(self) -> str | None:
|
| 114 |
+
return None if self._bundle is None else self._bundle.path
|
| 115 |
+
|
| 116 |
+
def ensure_loaded(self, model_path: str | None = None) -> ModelBundle:
|
| 117 |
+
# Keep Hub repo ids as plain strings (do not Path()-normalize them).
|
| 118 |
+
raw = model_path or self.default_model
|
| 119 |
+
path = str(raw)
|
| 120 |
+
if self._bundle is not None and self._bundle.path == path:
|
| 121 |
+
return self._bundle
|
| 122 |
+
model, tokenizer = load_model_and_tokenizer(path)
|
| 123 |
+
stop_ids = build_stop_ids(tokenizer)
|
| 124 |
+
self._bundle = ModelBundle(path=path, model=model, tokenizer=tokenizer, stop_ids=stop_ids)
|
| 125 |
+
return self._bundle
|
| 126 |
+
|
| 127 |
+
def generate(
|
| 128 |
+
self,
|
| 129 |
+
*,
|
| 130 |
+
prompt: str,
|
| 131 |
+
task: str,
|
| 132 |
+
duration: float,
|
| 133 |
+
assets_text: str = "",
|
| 134 |
+
model_path: str | None = None,
|
| 135 |
+
max_new_tokens: int | None = None,
|
| 136 |
+
temperature: float = 0.0,
|
| 137 |
+
top_k: int = 40,
|
| 138 |
+
repetition_penalty: float = 1.0,
|
| 139 |
+
seed: int | None = 42,
|
| 140 |
+
legacy_labels: bool = False,
|
| 141 |
+
) -> tuple[str, str]:
|
| 142 |
+
if not (prompt or "").strip():
|
| 143 |
+
return "", "Enter a user prompt first."
|
| 144 |
+
|
| 145 |
+
import torch
|
| 146 |
+
|
| 147 |
+
bundle = self.ensure_loaded(model_path)
|
| 148 |
+
if seed is not None:
|
| 149 |
+
torch.manual_seed(int(seed))
|
| 150 |
+
if torch.cuda.is_available():
|
| 151 |
+
torch.cuda.manual_seed_all(int(seed))
|
| 152 |
+
|
| 153 |
+
text, report = generate_one(
|
| 154 |
+
bundle.model,
|
| 155 |
+
bundle.tokenizer,
|
| 156 |
+
task=task,
|
| 157 |
+
duration=float(duration),
|
| 158 |
+
prompt=prompt.strip(),
|
| 159 |
+
assets=parse_assets(assets_text),
|
| 160 |
+
stop_ids=bundle.stop_ids,
|
| 161 |
+
max_new_tokens=max_new_tokens if max_new_tokens and max_new_tokens > 0 else None,
|
| 162 |
+
temperature=float(temperature),
|
| 163 |
+
top_k=int(top_k),
|
| 164 |
+
repetition_penalty=float(repetition_penalty),
|
| 165 |
+
legacy_labels=bool(legacy_labels),
|
| 166 |
+
)
|
| 167 |
+
return text, format_checks(report)
|
minimax/apps/gradio/ui.py
ADDED
|
@@ -0,0 +1,562 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Gradio Blocks UI for the MiniMax prompt enhancer."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import random
|
| 6 |
+
from pathlib import Path
|
| 7 |
+
|
| 8 |
+
from minimax.paths import CHAMPION_DIR
|
| 9 |
+
|
| 10 |
+
from .derive import DURATION, PROMPT_MAX, REF_LIMITS, derive
|
| 11 |
+
from .state import InferenceSession
|
| 12 |
+
|
| 13 |
+
# Friendly names for the deterministically derived task labels (shown in the
|
| 14 |
+
# live "Detected" badge so the role → task mapping stays transparent).
|
| 15 |
+
_TASK_HUMAN = {
|
| 16 |
+
"T2VA": "Text → video",
|
| 17 |
+
"I2VA": "First frame → video",
|
| 18 |
+
"FL2VA": "First + last frame → video",
|
| 19 |
+
"L2VA": "Last frame → video",
|
| 20 |
+
"reference_generation": "Reference-guided generation",
|
| 21 |
+
"reference_generation+audio_reference": "Reference generation + audio",
|
| 22 |
+
"keyframe_completion": "Keyframe completion",
|
| 23 |
+
"video_continuation": "Video continuation",
|
| 24 |
+
"video_continuation+audio_reference": "Video continuation + audio",
|
| 25 |
+
"video_editing": "Video editing",
|
| 26 |
+
"video_editing+audio_reuse": "Video editing + audio",
|
| 27 |
+
}
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
# GEOCINE cockpit skin — dark instrument look, Razer-green accent, sharp panels,
|
| 31 |
+
# mono uppercase labels, grid + glow backdrop. Overrides Gradio's CSS variables
|
| 32 |
+
# (verified against gradio 6.x) plus a few structural hooks.
|
| 33 |
+
_CSS = """
|
| 34 |
+
:root, .gradio-container {
|
| 35 |
+
--gc-bg:#151a16; --gc-panel:#1b211c; --gc-panel2:#212820; --gc-inset:#161c17;
|
| 36 |
+
--gc-ink:#e9efe5; --gc-muted:#97a18d; --gc-faint:#6c7666;
|
| 37 |
+
--gc-line:#2b332a; --gc-line2:#384037;
|
| 38 |
+
--gc-accent:#6bff4f; --gc-accent-bright:#44d62c; --gc-accent-ink:#06210a;
|
| 39 |
+
--gc-glow:rgba(68,214,44,.40); --gc-danger:#ef6a52;
|
| 40 |
+
--gc-mono: ui-monospace,"JetBrains Mono","Cascadia Code",Consolas,monospace;
|
| 41 |
+
|
| 42 |
+
--body-background-fill: var(--gc-bg);
|
| 43 |
+
--body-text-color: var(--gc-ink);
|
| 44 |
+
--body-text-color-subdued: var(--gc-muted);
|
| 45 |
+
--background-fill-primary: var(--gc-panel);
|
| 46 |
+
--background-fill-secondary: var(--gc-panel2);
|
| 47 |
+
--block-background-fill: var(--gc-panel);
|
| 48 |
+
--block-border-color: var(--gc-line);
|
| 49 |
+
--block-label-text-color: var(--gc-faint);
|
| 50 |
+
--block-title-text-color: var(--gc-faint);
|
| 51 |
+
--border-color-primary: var(--gc-line);
|
| 52 |
+
--border-color-accent: var(--gc-accent);
|
| 53 |
+
--input-background-fill: var(--gc-inset);
|
| 54 |
+
--input-border-color: var(--gc-line);
|
| 55 |
+
--input-border-color-focus: var(--gc-accent);
|
| 56 |
+
--color-accent: var(--gc-accent);
|
| 57 |
+
--color-accent-soft: color-mix(in srgb, var(--gc-accent-bright) 12%, transparent);
|
| 58 |
+
--button-primary-background-fill: var(--gc-accent-bright);
|
| 59 |
+
--button-primary-background-fill-hover: var(--gc-accent);
|
| 60 |
+
--button-primary-text-color: var(--gc-accent-ink);
|
| 61 |
+
--button-secondary-background-fill: var(--gc-panel2);
|
| 62 |
+
--button-secondary-background-fill-hover: var(--gc-panel);
|
| 63 |
+
--button-secondary-text-color: var(--gc-ink);
|
| 64 |
+
--button-secondary-border-color: var(--gc-line);
|
| 65 |
+
--slider-color: var(--gc-accent-bright);
|
| 66 |
+
--checkbox-background-color-selected: var(--gc-accent-bright);
|
| 67 |
+
--checkbox-border-color-selected: var(--gc-accent-bright);
|
| 68 |
+
--radius-xxl:0; --radius-xl:0; --radius-lg:0; --radius-md:0; --radius-sm:0; --radius-xs:0;
|
| 69 |
+
--block-shadow: none; --input-shadow: none;
|
| 70 |
+
--layout-gap: 10px; --block-padding: 12px 13px;
|
| 71 |
+
}
|
| 72 |
+
/* reserve the scrollbar gutter so content doesn't shift when a scrollbar
|
| 73 |
+
appears/disappears (e.g. as the output grows) */
|
| 74 |
+
html { scrollbar-gutter: stable; }
|
| 75 |
+
.gradio-container {
|
| 76 |
+
width: 100% !important; max-width: 1360px !important; margin: 0 auto !important;
|
| 77 |
+
padding: 20px clamp(16px,3vw,30px) 26px !important;
|
| 78 |
+
background:
|
| 79 |
+
radial-gradient(1200px 600px at 88% -10%, var(--gc-glow), transparent 55%),
|
| 80 |
+
linear-gradient(color-mix(in srgb, var(--gc-ink) 3%, transparent) 1px, transparent 1px),
|
| 81 |
+
linear-gradient(90deg, color-mix(in srgb, var(--gc-ink) 3%, transparent) 1px, transparent 1px),
|
| 82 |
+
var(--gc-bg) !important;
|
| 83 |
+
background-size: 100% 100%, 38px 38px, 38px 38px, 100% 100% !important;
|
| 84 |
+
}
|
| 85 |
+
/* instrument-style field labels */
|
| 86 |
+
span[data-testid="block-info"], .block-title, label > span:first-child {
|
| 87 |
+
font-family: var(--gc-mono) !important; font-size:10.5px !important;
|
| 88 |
+
letter-spacing:.16em !important; text-transform:uppercase !important;
|
| 89 |
+
color: var(--gc-faint) !important; font-weight:500 !important;
|
| 90 |
+
}
|
| 91 |
+
.info-text { color: var(--gc-muted) !important; }
|
| 92 |
+
/* control panels get the instrument border; text/markdown blocks stay bare */
|
| 93 |
+
.block { border:1px solid var(--gc-line) !important; box-shadow:none !important; }
|
| 94 |
+
.block.hide-container,
|
| 95 |
+
.block:has(> .md), .block:has(> .prose),
|
| 96 |
+
.block:has(> div > .md), .block:has(> div > .prose) {
|
| 97 |
+
border:0 !important; background:transparent !important; padding:0 !important;
|
| 98 |
+
overflow:hidden !important;
|
| 99 |
+
}
|
| 100 |
+
/* description: tighter, muted, unboxed */
|
| 101 |
+
.prose { color: var(--gc-muted) !important; font-size:13px !important; line-height:1.5 !important; }
|
| 102 |
+
.prose p { margin:3px 0 !important; }
|
| 103 |
+
.prose strong { color: var(--gc-ink) !important; }
|
| 104 |
+
/* trim default inner form gaps */
|
| 105 |
+
.form { gap:8px !important; }
|
| 106 |
+
.gap { gap:10px !important; }
|
| 107 |
+
/* columns are top-aligned: the output panel's top always lines up with the
|
| 108 |
+
left column's top, and expanding Decoding grows the left column downward
|
| 109 |
+
without shoving the right side down. Children keep their natural height
|
| 110 |
+
(no stretch) so the Generate button never balloons. */
|
| 111 |
+
.gc-main-row { align-items: flex-start !important; }
|
| 112 |
+
.gc-col-left, .gc-col-right { display:flex !important; flex-direction:column !important; }
|
| 113 |
+
/* keep panels at their natural height — the Row's equal-height stretch would
|
| 114 |
+
otherwise inflate the collapsed accordions to fill the column */
|
| 115 |
+
.gc-col-left > *, .gc-col-right > * { flex: 0 0 auto !important; }
|
| 116 |
+
/* dropdown menu: darker so it reads above the panels behind it */
|
| 117 |
+
ul.options {
|
| 118 |
+
background:#0f130e !important; border:1px solid var(--gc-line-2) !important;
|
| 119 |
+
box-shadow:0 16px 34px -14px #000 !important; border-radius:0 !important;
|
| 120 |
+
}
|
| 121 |
+
ul.options li, ul.options .item { color: var(--gc-ink) !important; }
|
| 122 |
+
ul.options li.active, ul.options li.selected,
|
| 123 |
+
ul.options .item.active, ul.options .item.selected {
|
| 124 |
+
background: color-mix(in srgb, var(--gc-accent-bright) 16%, var(--gc-inset)) !important;
|
| 125 |
+
color: var(--gc-ink) !important;
|
| 126 |
+
}
|
| 127 |
+
/* derived-task badge inside the References accordion */
|
| 128 |
+
.gc-detected, .gc-detected p {
|
| 129 |
+
font-family: var(--gc-mono) !important; font-size:11px !important;
|
| 130 |
+
letter-spacing:.04em !important; color: var(--gc-muted) !important; margin:0 !important;
|
| 131 |
+
}
|
| 132 |
+
.gc-detected code {
|
| 133 |
+
color: var(--gc-accent) !important;
|
| 134 |
+
background: color-mix(in srgb, var(--gc-accent) 12%, transparent) !important;
|
| 135 |
+
border:1px solid color-mix(in srgb, var(--gc-accent) 30%, var(--gc-line)) !important;
|
| 136 |
+
}
|
| 137 |
+
/* inside the Decoding accordion, controls stay borderless — no divider lines
|
| 138 |
+
or boxes around the sliders and number fields (the accordion keeps its own
|
| 139 |
+
outer frame from the .block rule above) */
|
| 140 |
+
.gc-decoding .block, .gc-plain .block { border:0 !important; background:transparent !important; }
|
| 141 |
+
/* accordion toggle: replace the plain ▼ glyph with a crisp accent chevron —
|
| 142 |
+
points right when collapsed, down when open */
|
| 143 |
+
.label-wrap .icon {
|
| 144 |
+
transform: none !important; font-size: 0 !important;
|
| 145 |
+
display: inline-flex !important; align-items: center; justify-content: center;
|
| 146 |
+
width: 14px !important; height: 14px !important;
|
| 147 |
+
}
|
| 148 |
+
.label-wrap .icon::after {
|
| 149 |
+
content: ''; display: block; width: 7px; height: 7px; margin-top: -2px;
|
| 150 |
+
border-right: 1.7px solid var(--gc-accent); border-bottom: 1.7px solid var(--gc-accent);
|
| 151 |
+
transform: rotate(-45deg); transition: transform .2s ease;
|
| 152 |
+
}
|
| 153 |
+
.label-wrap.open .icon::after { transform: rotate(45deg); margin-top: -4px; }
|
| 154 |
+
/* focus: a single accent-green border on every field, dark inside (no gray
|
| 155 |
+
fill), and no heavy outline/glow — consistent across text, number, dropdown
|
| 156 |
+
and slider fields */
|
| 157 |
+
:focus, :focus-visible { outline: none !important; }
|
| 158 |
+
/* text/number/dropdown fields only — NOT checkboxes, whose checked green fill
|
| 159 |
+
and checkmark would otherwise be clobbered by the dark background on focus */
|
| 160 |
+
input:not([type="checkbox"]):focus, textarea:focus, select:focus,
|
| 161 |
+
input:not([type="checkbox"]):focus-visible, textarea:focus-visible {
|
| 162 |
+
border-color: var(--gc-accent) !important;
|
| 163 |
+
background: var(--gc-inset) !important;
|
| 164 |
+
box-shadow: none !important;
|
| 165 |
+
}
|
| 166 |
+
/* checkbox: keep the accent fill on focus, no glow/ring and no blue focus
|
| 167 |
+
border (Gradio's --checkbox-border-color-focus) */
|
| 168 |
+
input[type="checkbox"]:focus, input[type="checkbox"]:focus-visible {
|
| 169 |
+
outline: none !important; box-shadow: none !important;
|
| 170 |
+
border-color: var(--gc-line) !important;
|
| 171 |
+
}
|
| 172 |
+
input[type="checkbox"]:checked,
|
| 173 |
+
input[type="checkbox"]:checked:focus,
|
| 174 |
+
input[type="checkbox"]:checked:focus-visible {
|
| 175 |
+
background-color: var(--gc-accent-bright) !important;
|
| 176 |
+
border-color: var(--gc-accent-bright) !important;
|
| 177 |
+
}
|
| 178 |
+
.wrap:focus-within, .secondary-wrap:focus-within,
|
| 179 |
+
.tab-like-container:focus-within, .label-wrap:focus-within {
|
| 180 |
+
border-color: var(--gc-accent) !important; box-shadow: none !important;
|
| 181 |
+
}
|
| 182 |
+
/* number fields (Max new tokens / Seed): border on the input, not the wrapper,
|
| 183 |
+
so they match the text fields instead of showing a hollow double box */
|
| 184 |
+
label.block.container:has(> input[type="number"]) {
|
| 185 |
+
border:0 !important; background:transparent !important; padding:0 !important;
|
| 186 |
+
display:flex !important; flex-direction:column !important; gap:6px !important;
|
| 187 |
+
}
|
| 188 |
+
input[type="number"] {
|
| 189 |
+
border:1px solid var(--gc-line) !important; background: var(--gc-inset) !important;
|
| 190 |
+
min-height:38px !important; width:100% !important;
|
| 191 |
+
font-family: var(--gc-mono) !important;
|
| 192 |
+
-moz-appearance: textfield !important; appearance: textfield !important;
|
| 193 |
+
}
|
| 194 |
+
/* disabled number field (Seed while "Randomize seed" owns it): make the greyed-out
|
| 195 |
+
state unmistakable so toggling the checkbox reads as doing something */
|
| 196 |
+
input[type="number"]:disabled {
|
| 197 |
+
opacity:0.4 !important; cursor:not-allowed !important;
|
| 198 |
+
color: var(--gc-muted, #7a7a7a) !important;
|
| 199 |
+
}
|
| 200 |
+
/* strip the native up/down spinner arrows (display:none is the reliable fix —
|
| 201 |
+
appearance:none alone still renders them in current Chrome) */
|
| 202 |
+
input[type="number"]::-webkit-inner-spin-button,
|
| 203 |
+
input[type="number"]::-webkit-outer-spin-button {
|
| 204 |
+
-webkit-appearance: none !important; appearance: none !important;
|
| 205 |
+
display: none !important; width: 0 !important; height: 0 !important; margin: 0 !important;
|
| 206 |
+
}
|
| 207 |
+
/* slider value box: one frame on the container, borderless mono input, and a
|
| 208 |
+
reset button that spans the full height with a divider (no nested double box) */
|
| 209 |
+
.tab-like-container {
|
| 210 |
+
border:1px solid var(--gc-line) !important; background: var(--gc-inset) !important;
|
| 211 |
+
overflow:hidden !important; align-items:stretch !important;
|
| 212 |
+
}
|
| 213 |
+
/* fixed input width so all three slider value boxes match (Gradio otherwise
|
| 214 |
+
auto-sizes each to its own min/max/step); 84px keeps the original width */
|
| 215 |
+
.tab-like-container input[type="number"] {
|
| 216 |
+
border:0 !important; background:transparent !important; min-height:24px !important;
|
| 217 |
+
width:84px !important; flex:0 0 84px !important; min-width:84px !important;
|
| 218 |
+
}
|
| 219 |
+
.tab-like-container button {
|
| 220 |
+
border:0 !important; border-left:1px solid var(--gc-line) !important;
|
| 221 |
+
background:transparent !important; color: var(--gc-faint) !important;
|
| 222 |
+
height:auto !important; align-self:stretch !important;
|
| 223 |
+
}
|
| 224 |
+
.tab-like-container button:hover {
|
| 225 |
+
color: var(--gc-accent) !important;
|
| 226 |
+
background: color-mix(in srgb, var(--gc-accent-bright) 10%, transparent) !important;
|
| 227 |
+
}
|
| 228 |
+
/* notched, glowing Generate button */
|
| 229 |
+
button.primary {
|
| 230 |
+
font-family: var(--gc-mono) !important; font-weight:700 !important;
|
| 231 |
+
text-transform:uppercase !important; letter-spacing:.08em !important;
|
| 232 |
+
clip-path: polygon(10px 0,100% 0,100% calc(100% - 10px),calc(100% - 10px) 100%,0 100%,0 10px) !important;
|
| 233 |
+
box-shadow: 0 6px 24px -10px var(--gc-glow) !important; border:0 !important;
|
| 234 |
+
}
|
| 235 |
+
/* hover brightens the fill; keep the dark ink text so it stays readable
|
| 236 |
+
(white-on-bright-green was invisible) */
|
| 237 |
+
button.primary:hover { color: var(--gc-accent-ink) !important; }
|
| 238 |
+
button.secondary {
|
| 239 |
+
font-family: var(--gc-mono) !important; text-transform:uppercase !important;
|
| 240 |
+
letter-spacing:.06em !important; font-size:12px !important;
|
| 241 |
+
border:1px solid var(--gc-line) !important;
|
| 242 |
+
}
|
| 243 |
+
.prose code {
|
| 244 |
+
color: var(--gc-accent) !important;
|
| 245 |
+
background: color-mix(in srgb,var(--gc-accent) 12%, transparent) !important;
|
| 246 |
+
border:1px solid color-mix(in srgb,var(--gc-accent) 30%, var(--gc-line)) !important;
|
| 247 |
+
border-radius:0 !important;
|
| 248 |
+
}
|
| 249 |
+
textarea:focus, input:not([type="checkbox"]):focus {
|
| 250 |
+
box-shadow: 0 0 0 3px var(--gc-glow) !important; border-color: var(--gc-accent) !important;
|
| 251 |
+
}
|
| 252 |
+
table thead th {
|
| 253 |
+
font-family: var(--gc-mono) !important; text-transform:uppercase !important;
|
| 254 |
+
letter-spacing:.06em !important; color: var(--gc-faint) !important;
|
| 255 |
+
}
|
| 256 |
+
/* cockpit brand bar */
|
| 257 |
+
.gc-brandbar { padding: 2px 0 6px; }
|
| 258 |
+
.gc-brand { display:flex; align-items:center; gap:12px; }
|
| 259 |
+
.gc-mark {
|
| 260 |
+
width:36px; height:36px; display:grid; place-items:center;
|
| 261 |
+
font-family: var(--gc-mono); font-weight:800; font-size:21px; line-height:1;
|
| 262 |
+
border:1px solid color-mix(in srgb, var(--gc-accent) 55%, var(--gc-line));
|
| 263 |
+
background: color-mix(in srgb, var(--gc-accent-bright) 12%, transparent);
|
| 264 |
+
color: var(--gc-accent); box-shadow: 0 0 24px -8px var(--gc-glow);
|
| 265 |
+
}
|
| 266 |
+
.gc-word {
|
| 267 |
+
font-family: var(--gc-mono); font-size:17px; font-weight:700; letter-spacing:.06em;
|
| 268 |
+
color: var(--gc-ink); display:flex; align-items:baseline; gap:10px; line-height:1;
|
| 269 |
+
}
|
| 270 |
+
.gc-word span { font-size:9px; font-weight:500; letter-spacing:.28em; color: var(--gc-accent); }
|
| 271 |
+
.gc-sub {
|
| 272 |
+
margin-top:5px; font-family: var(--gc-mono); font-size:9.5px; letter-spacing:.12em;
|
| 273 |
+
text-transform:uppercase; color: var(--gc-faint);
|
| 274 |
+
}
|
| 275 |
+
/* GEOCINE attribution appended to Gradio's footer (see _HEAD) — the text stays
|
| 276 |
+
solid green and pulses a soft glow so it reads as a live, clickable credit */
|
| 277 |
+
footer .gc-built {
|
| 278 |
+
font-family: var(--gc-mono) !important; font-size:12px; letter-spacing:.06em;
|
| 279 |
+
text-decoration:none; margin-left:8px; align-self:center; white-space:nowrap;
|
| 280 |
+
color: var(--gc-accent) !important; -webkit-text-fill-color: var(--gc-accent) !important;
|
| 281 |
+
animation: gc-shimmer 2.2s ease-in-out infinite;
|
| 282 |
+
}
|
| 283 |
+
footer .gc-built:hover { text-decoration:underline; }
|
| 284 |
+
@keyframes gc-shimmer {
|
| 285 |
+
0%, 100% { color: var(--gc-accent); -webkit-text-fill-color: var(--gc-accent); text-shadow: none; }
|
| 286 |
+
50% { color: #dcffd2; -webkit-text-fill-color: #dcffd2; text-shadow: 0 0 10px var(--gc-glow); }
|
| 287 |
+
}
|
| 288 |
+
@media (prefers-reduced-motion: reduce) { footer .gc-built { animation: none; } }
|
| 289 |
+
"""
|
| 290 |
+
|
| 291 |
+
# Injected into <head>: appends a clickable "Built by GEOCINE" link to Gradio's
|
| 292 |
+
# own footer once it mounts (a CSS ::after can't be a link).
|
| 293 |
+
_HEAD = """
|
| 294 |
+
<script>
|
| 295 |
+
(function () {
|
| 296 |
+
function addLink() {
|
| 297 |
+
var f = document.querySelector('footer');
|
| 298 |
+
if (!f) return false;
|
| 299 |
+
if (f.querySelector('.gc-built')) return true;
|
| 300 |
+
var a = document.createElement('a');
|
| 301 |
+
a.className = 'gc-built';
|
| 302 |
+
a.href = 'https://x.com/aivandroid';
|
| 303 |
+
a.target = '_blank';
|
| 304 |
+
a.rel = 'noopener noreferrer';
|
| 305 |
+
a.textContent = '\\u00B7 Built by GEOCINE';
|
| 306 |
+
f.appendChild(a);
|
| 307 |
+
return true;
|
| 308 |
+
}
|
| 309 |
+
if (!addLink()) {
|
| 310 |
+
var obs = new MutationObserver(function () { if (addLink()) obs.disconnect(); });
|
| 311 |
+
obs.observe(document.documentElement, { childList: true, subtree: true });
|
| 312 |
+
setTimeout(function () { obs.disconnect(); }, 15000);
|
| 313 |
+
}
|
| 314 |
+
})();
|
| 315 |
+
</script>
|
| 316 |
+
"""
|
| 317 |
+
|
| 318 |
+
_BRAND_HTML = """
|
| 319 |
+
<div class="gc-brandbar">
|
| 320 |
+
<div class="gc-brand">
|
| 321 |
+
<div class="gc-mark">G</div>
|
| 322 |
+
<div>
|
| 323 |
+
<div class="gc-word">MINIMAX H3 <span>PROMPT ENHANCER</span></div>
|
| 324 |
+
<div class="gc-sub">Turn rough ideas into cinematic video prompts</div>
|
| 325 |
+
</div>
|
| 326 |
+
</div>
|
| 327 |
+
</div>
|
| 328 |
+
"""
|
| 329 |
+
|
| 330 |
+
|
| 331 |
+
def build_app(*, model_path: str | Path | None = None):
|
| 332 |
+
import gradio as gr
|
| 333 |
+
|
| 334 |
+
session = InferenceSession(default_model=model_path or CHAMPION_DIR)
|
| 335 |
+
default_model = str(session.default_model)
|
| 336 |
+
|
| 337 |
+
theme = gr.themes.Base(
|
| 338 |
+
font=["system-ui", "-apple-system", "Segoe UI", "Roboto", "sans-serif"],
|
| 339 |
+
font_mono=["ui-monospace", "JetBrains Mono", "Cascadia Code", "Consolas", "monospace"],
|
| 340 |
+
)
|
| 341 |
+
|
| 342 |
+
with gr.Blocks(title="MiniMax H3 Prompt Enhancer", theme=theme, css=_CSS, head=_HEAD) as demo:
|
| 343 |
+
gr.HTML(_BRAND_HTML)
|
| 344 |
+
gr.Markdown(
|
| 345 |
+
"""
|
| 346 |
+
**Not a template expander** — a **trained LLM** that writes **MiniMax H3–style**
|
| 347 |
+
video prompts: shots, camera, soundscape, and score, following the official
|
| 348 |
+
structured format (not “cinematic 8k masterpiece” fluff).
|
| 349 |
+
|
| 350 |
+
Drop a rough idea (and optional references), set duration, hit **Generate**.
|
| 351 |
+
"""
|
| 352 |
+
)
|
| 353 |
+
|
| 354 |
+
with gr.Row(equal_height=True):
|
| 355 |
+
with gr.Column(scale=1, elem_classes=["gc-col-left"]):
|
| 356 |
+
with gr.Accordion("Prompt", open=True, elem_classes=["gc-plain"]):
|
| 357 |
+
duration = gr.Slider(
|
| 358 |
+
DURATION["min"], DURATION["max"], value=DURATION["default"], step=DURATION["step"],
|
| 359 |
+
label="Duration (seconds)",
|
| 360 |
+
)
|
| 361 |
+
prompt = gr.Textbox(
|
| 362 |
+
label="User prompt",
|
| 363 |
+
lines=6,
|
| 364 |
+
max_length=PROMPT_MAX,
|
| 365 |
+
placeholder="Rough idea, dialogue, shot notes…",
|
| 366 |
+
)
|
| 367 |
+
# Role slots — the task is derived from which of these are filled,
|
| 368 |
+
# so the user never picks a task code. Empty → plain text-to-video.
|
| 369 |
+
# Counts are capped to the H3 Ref2VA media limits (≤9 images,
|
| 370 |
+
# ≤2 videos, ≤2 audio, ≤12 total).
|
| 371 |
+
with gr.Accordion("References & frames · optional", open=False, elem_classes=["gc-decoding"]):
|
| 372 |
+
detected = gr.Markdown(
|
| 373 |
+
"**Detected:** Text → video · `T2VA`", elem_classes=["gc-detected"]
|
| 374 |
+
)
|
| 375 |
+
first_frame = gr.Textbox(
|
| 376 |
+
label="First frame",
|
| 377 |
+
placeholder="young woman by a rainy train window at night",
|
| 378 |
+
)
|
| 379 |
+
last_frame = gr.Textbox(
|
| 380 |
+
label="Last frame",
|
| 381 |
+
placeholder="cyclist paused at a stone bridge, mist clearing (optional)",
|
| 382 |
+
)
|
| 383 |
+
subject_refs = gr.Textbox(
|
| 384 |
+
label="Subject / look references",
|
| 385 |
+
lines=3,
|
| 386 |
+
info=f"A subject or look kept across the video — one per line, up to {REF_LIMITS['image']} images.",
|
| 387 |
+
placeholder=(
|
| 388 |
+
"man in a red-and-blue spider-themed suit\n"
|
| 389 |
+
"woman with a silver bob and a leather jacket\n"
|
| 390 |
+
"rain-slick neon alley at night\n"
|
| 391 |
+
"…"
|
| 392 |
+
),
|
| 393 |
+
)
|
| 394 |
+
keyframe = gr.Textbox(
|
| 395 |
+
label="Keyframe to hit",
|
| 396 |
+
placeholder="a steam train on an oil-painted cobblestone street",
|
| 397 |
+
)
|
| 398 |
+
continue_video = gr.Textbox(
|
| 399 |
+
label="Video to continue",
|
| 400 |
+
placeholder="panoramic golden beach at sunset, calm blue tide",
|
| 401 |
+
)
|
| 402 |
+
edit_footage = gr.Textbox(
|
| 403 |
+
label="Footage to edit",
|
| 404 |
+
lines=2,
|
| 405 |
+
info=f"Clips / scene elements to edit — one per line, up to {REF_LIMITS['video']} clips.",
|
| 406 |
+
placeholder=(
|
| 407 |
+
"coffee-shop interior — brick wall, orange sofa, neon sign\n"
|
| 408 |
+
"young man in a dark-grey hoodie with a Samoyed"
|
| 409 |
+
),
|
| 410 |
+
)
|
| 411 |
+
audio_ref = gr.Textbox(
|
| 412 |
+
label="Audio reference",
|
| 413 |
+
lines=2,
|
| 414 |
+
info=f"Sound or voice to match — adds audio, one per line, up to {REF_LIMITS['audio']}.",
|
| 415 |
+
placeholder=(
|
| 416 |
+
"gentle waves and wind continuity (not a 1:1 copy)\n"
|
| 417 |
+
"distant seabird calls"
|
| 418 |
+
),
|
| 419 |
+
)
|
| 420 |
+
with gr.Accordion("Decoding", open=False, elem_classes=["gc-decoding"]):
|
| 421 |
+
temperature = gr.Slider(0.0, 1.2, value=0.0, step=0.05, label="Temperature (0 = greedy)")
|
| 422 |
+
top_k = gr.Slider(0, 100, value=40, step=1, label="Top-k (sampling only)")
|
| 423 |
+
repetition_penalty = gr.Slider(
|
| 424 |
+
1.0, 1.5, value=1.0, step=0.01, label="Repetition penalty"
|
| 425 |
+
)
|
| 426 |
+
max_new_tokens = gr.Number(
|
| 427 |
+
value=0,
|
| 428 |
+
precision=0,
|
| 429 |
+
label="Max new tokens (0 = task default)",
|
| 430 |
+
)
|
| 431 |
+
# Created interactive so Gradio renders a real editable input;
|
| 432 |
+
# the initial disabled state (randomize defaults on) is applied
|
| 433 |
+
# via demo.load below. A component born interactive=False cannot
|
| 434 |
+
# be reliably re-enabled at runtime.
|
| 435 |
+
seed = gr.Number(value=42, precision=0, label="Seed")
|
| 436 |
+
randomize_seed = gr.Checkbox(
|
| 437 |
+
value=True,
|
| 438 |
+
label="Randomize seed each run",
|
| 439 |
+
)
|
| 440 |
+
|
| 441 |
+
with gr.Column(scale=1, elem_classes=["gc-col-right"]):
|
| 442 |
+
with gr.Accordion("Enhanced prompt", open=True, elem_classes=["gc-plain"]):
|
| 443 |
+
output = gr.Textbox(
|
| 444 |
+
label="Enhanced prompt",
|
| 445 |
+
show_label=False,
|
| 446 |
+
lines=20,
|
| 447 |
+
interactive=True,
|
| 448 |
+
elem_id="gc-output",
|
| 449 |
+
)
|
| 450 |
+
generate_btn = gr.Button("Generate", variant="primary")
|
| 451 |
+
report = gr.Markdown(label="Format score")
|
| 452 |
+
|
| 453 |
+
role_inputs = [
|
| 454 |
+
first_frame, last_frame, subject_refs, keyframe,
|
| 455 |
+
continue_video, edit_footage, audio_ref,
|
| 456 |
+
]
|
| 457 |
+
|
| 458 |
+
def _derive_from(first_v, last_v, subj_v, key_v, cont_v, edit_v, audio_v):
|
| 459 |
+
return derive(
|
| 460 |
+
first_frame=first_v,
|
| 461 |
+
last_frame=last_v,
|
| 462 |
+
subject_refs=subj_v,
|
| 463 |
+
keyframe=key_v,
|
| 464 |
+
continue_video=cont_v,
|
| 465 |
+
edit_footage=edit_v,
|
| 466 |
+
audio_ref=audio_v,
|
| 467 |
+
)
|
| 468 |
+
|
| 469 |
+
def _detect(first_v, last_v, subj_v, key_v, cont_v, edit_v, audio_v):
|
| 470 |
+
d = _derive_from(first_v, last_v, subj_v, key_v, cont_v, edit_v, audio_v)
|
| 471 |
+
human = _TASK_HUMAN.get(d.task, d.task)
|
| 472 |
+
line = f"**Detected:** {human} · `{d.task}`"
|
| 473 |
+
if d.warnings:
|
| 474 |
+
line += " \n⚠ " + "; ".join(d.warnings)
|
| 475 |
+
return line
|
| 476 |
+
|
| 477 |
+
# Live badge: recompute the derived task whenever any role slot changes.
|
| 478 |
+
for field in role_inputs:
|
| 479 |
+
field.change(_detect, inputs=role_inputs, outputs=detected)
|
| 480 |
+
|
| 481 |
+
def _generate(
|
| 482 |
+
duration_v, prompt_v,
|
| 483 |
+
first_v, last_v, subj_v, key_v, cont_v, edit_v, audio_v,
|
| 484 |
+
temperature_v, top_k_v, rep_v, max_tok_v, seed_v, randomize_v,
|
| 485 |
+
):
|
| 486 |
+
# When randomization is on, draw a fresh seed per run and surface it
|
| 487 |
+
# back to the Seed field so the exact run stays reproducible.
|
| 488 |
+
if randomize_v:
|
| 489 |
+
seed_v = random.randint(0, 2**31 - 1)
|
| 490 |
+
used_seed = int(seed_v) if seed_v is not None else None
|
| 491 |
+
d = _derive_from(first_v, last_v, subj_v, key_v, cont_v, edit_v, audio_v)
|
| 492 |
+
text, checks = session.generate(
|
| 493 |
+
prompt=prompt_v or "",
|
| 494 |
+
task=d.task,
|
| 495 |
+
duration=float(duration_v),
|
| 496 |
+
assets_text=d.assets_text,
|
| 497 |
+
model_path=default_model,
|
| 498 |
+
max_new_tokens=int(max_tok_v) if max_tok_v else None,
|
| 499 |
+
temperature=float(temperature_v),
|
| 500 |
+
top_k=int(top_k_v),
|
| 501 |
+
repetition_penalty=float(rep_v),
|
| 502 |
+
seed=used_seed,
|
| 503 |
+
)
|
| 504 |
+
return text, checks, used_seed
|
| 505 |
+
|
| 506 |
+
# Grey out the Seed field while randomization owns it; re-enable for
|
| 507 |
+
# manual entry when the checkbox is cleared.
|
| 508 |
+
def _toggle_seed(on):
|
| 509 |
+
return gr.update(interactive=not on)
|
| 510 |
+
|
| 511 |
+
randomize_seed.change(_toggle_seed, inputs=randomize_seed, outputs=seed)
|
| 512 |
+
# Apply the initial disabled state (checkbox defaults on) after render.
|
| 513 |
+
demo.load(_toggle_seed, inputs=randomize_seed, outputs=seed)
|
| 514 |
+
|
| 515 |
+
generate_btn.click(
|
| 516 |
+
_generate,
|
| 517 |
+
inputs=[
|
| 518 |
+
duration, prompt,
|
| 519 |
+
*role_inputs,
|
| 520 |
+
temperature, top_k, repetition_penalty, max_new_tokens, seed,
|
| 521 |
+
randomize_seed,
|
| 522 |
+
],
|
| 523 |
+
outputs=[output, report, seed],
|
| 524 |
+
)
|
| 525 |
+
|
| 526 |
+
demo.queue(default_concurrency_limit=1)
|
| 527 |
+
demo._minimax_session = session # type: ignore[attr-defined]
|
| 528 |
+
return demo
|
| 529 |
+
|
| 530 |
+
|
| 531 |
+
def launch(
|
| 532 |
+
*,
|
| 533 |
+
model_path: str | Path | None = None,
|
| 534 |
+
host: str = "127.0.0.1",
|
| 535 |
+
port: int = 7860,
|
| 536 |
+
share: bool = False,
|
| 537 |
+
require_gpu: bool = True,
|
| 538 |
+
):
|
| 539 |
+
import torch
|
| 540 |
+
|
| 541 |
+
if require_gpu and not torch.cuda.is_available():
|
| 542 |
+
raise SystemExit(
|
| 543 |
+
"CUDA is not available — refusing to launch Gradio on CPU. "
|
| 544 |
+
"Use the minimax-lfm env with a GPU, or pass require_gpu=False."
|
| 545 |
+
)
|
| 546 |
+
if torch.cuda.is_available():
|
| 547 |
+
print(
|
| 548 |
+
f"GPU: {torch.cuda.get_device_name(0)} (cuda:{torch.cuda.current_device()})",
|
| 549 |
+
flush=True,
|
| 550 |
+
)
|
| 551 |
+
else:
|
| 552 |
+
print("GPU: none (CPU)", flush=True)
|
| 553 |
+
|
| 554 |
+
demo = build_app(model_path=model_path)
|
| 555 |
+
session: InferenceSession = demo._minimax_session # type: ignore[attr-defined]
|
| 556 |
+
bundle = session.ensure_loaded()
|
| 557 |
+
device = next(bundle.model.parameters()).device
|
| 558 |
+
print(f"Model loaded: {bundle.path} device={device}", flush=True)
|
| 559 |
+
if require_gpu and device.type != "cuda":
|
| 560 |
+
raise SystemExit(f"Model landed on {device}, expected CUDA.")
|
| 561 |
+
|
| 562 |
+
demo.launch(server_name=host, server_port=port, share=share)
|
minimax/formatting/__init__.py
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from .envelope import ( # noqa: F401
|
| 2 |
+
BASE_TASKS,
|
| 3 |
+
REF_TASKS,
|
| 4 |
+
build_user_message,
|
| 5 |
+
load_system,
|
| 6 |
+
parse_user_blob,
|
| 7 |
+
task_label,
|
| 8 |
+
)
|
| 9 |
+
from .postprocess import postprocess_generation # noqa: F401
|
minimax/formatting/envelope.py
ADDED
|
@@ -0,0 +1,122 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""User/system message construction and parsing.
|
| 2 |
+
|
| 3 |
+
The user envelope is the contract between training data, inference, and eval:
|
| 4 |
+
|
| 5 |
+
Task: <label>
|
| 6 |
+
Duration: <D.DD>s
|
| 7 |
+
Assets:
|
| 8 |
+
- <asset or "(none)">
|
| 9 |
+
|
| 10 |
+
User prompt:
|
| 11 |
+
<original rough prompt>
|
| 12 |
+
|
| 13 |
+
Task labels:
|
| 14 |
+
- base tasks use the task name directly (T2VA / I2VA / FL2VA / L2VA)
|
| 15 |
+
- ref tasks use the fine-grained form "video_editing (full-reference rewrite)"
|
| 16 |
+
so the 350M student can condition on the subtype. The old corpus used the
|
| 17 |
+
generic "full-reference rewrite" label; ``legacy=True`` reproduces it,
|
| 18 |
+
which is needed when evaluating checkpoints trained on the old corpus.
|
| 19 |
+
"""
|
| 20 |
+
|
| 21 |
+
from __future__ import annotations
|
| 22 |
+
|
| 23 |
+
import html
|
| 24 |
+
import re
|
| 25 |
+
|
| 26 |
+
from ..paths import SYSTEM_BASE_FILE, SYSTEM_BASE_TASK_FILES, SYSTEM_REF_FILE
|
| 27 |
+
|
| 28 |
+
BASE_TASKS = {"T2VA", "I2VA", "FL2VA", "L2VA"}
|
| 29 |
+
REF_TASKS = {
|
| 30 |
+
"reference_generation",
|
| 31 |
+
"reference_generation+audio_reference",
|
| 32 |
+
"keyframe_completion",
|
| 33 |
+
"video_editing",
|
| 34 |
+
"video_editing+audio_reuse",
|
| 35 |
+
"video_continuation",
|
| 36 |
+
"video_continuation+audio_reference",
|
| 37 |
+
}
|
| 38 |
+
|
| 39 |
+
LEGACY_REF_LABEL = "full-reference rewrite"
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
def is_base_task(task: str) -> bool:
|
| 43 |
+
return task in BASE_TASKS
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
def task_label(task: str, *, legacy: bool = False) -> str:
|
| 47 |
+
if task in BASE_TASKS:
|
| 48 |
+
return task
|
| 49 |
+
if legacy:
|
| 50 |
+
return LEGACY_REF_LABEL
|
| 51 |
+
return f"{task} ({LEGACY_REF_LABEL})"
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
def build_user_message(
|
| 55 |
+
task: str,
|
| 56 |
+
duration: float,
|
| 57 |
+
prompt: str,
|
| 58 |
+
assets: list[str] | None = None,
|
| 59 |
+
*,
|
| 60 |
+
legacy: bool = False,
|
| 61 |
+
) -> str:
|
| 62 |
+
asset_block = "\n".join(f"- {a}" for a in assets) if assets else "- (none)"
|
| 63 |
+
return (
|
| 64 |
+
f"Task: {task_label(task, legacy=legacy)}\n"
|
| 65 |
+
f"Duration: {float(duration):.2f}s\n"
|
| 66 |
+
f"Assets:\n{asset_block}\n\n"
|
| 67 |
+
f"User prompt:\n{html.unescape(prompt).strip()}"
|
| 68 |
+
)
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
def load_system(task: str) -> str:
|
| 72 |
+
"""Load the system prompt for a task.
|
| 73 |
+
|
| 74 |
+
Base tasks use a task-specific file when present (T2VA has no alignment
|
| 75 |
+
rules; I2VA/FL2VA/L2VA only their own line) to cut cross-task instruction
|
| 76 |
+
leakage. Falls back to ``system_base.txt`` / ``system_ref.txt``.
|
| 77 |
+
"""
|
| 78 |
+
if task in BASE_TASKS:
|
| 79 |
+
path = SYSTEM_BASE_TASK_FILES.get(task, SYSTEM_BASE_FILE)
|
| 80 |
+
if not path.is_file():
|
| 81 |
+
path = SYSTEM_BASE_FILE
|
| 82 |
+
else:
|
| 83 |
+
path = SYSTEM_REF_FILE
|
| 84 |
+
return path.read_text(encoding="utf-8").strip()
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
def parse_user_blob(user_content: str) -> dict:
|
| 88 |
+
"""Extract task, duration, assets, original prompt from a packaged user message.
|
| 89 |
+
|
| 90 |
+
Handles both the fine-grained label ("video_editing (full-reference rewrite)")
|
| 91 |
+
and the legacy generic label ("full-reference rewrite").
|
| 92 |
+
"""
|
| 93 |
+
task = "T2VA"
|
| 94 |
+
duration = 6.0
|
| 95 |
+
assets: list[str] = []
|
| 96 |
+
prompt = user_content
|
| 97 |
+
|
| 98 |
+
m = re.search(r"Task:\s*(.+)", user_content)
|
| 99 |
+
if m:
|
| 100 |
+
raw = m.group(1).strip()
|
| 101 |
+
if raw == LEGACY_REF_LABEL:
|
| 102 |
+
task = "full-reference"
|
| 103 |
+
else:
|
| 104 |
+
fm = re.match(r"(.+?)\s*\(full-reference rewrite\)$", raw)
|
| 105 |
+
task = fm.group(1).strip() if fm else raw
|
| 106 |
+
m = re.search(r"Duration:\s*([0-9.]+)", user_content)
|
| 107 |
+
if m:
|
| 108 |
+
duration = float(m.group(1))
|
| 109 |
+
if "Assets:" in user_content and "User prompt:" in user_content:
|
| 110 |
+
asset_block = user_content.split("Assets:", 1)[1].split("User prompt:", 1)[0]
|
| 111 |
+
for line in asset_block.splitlines():
|
| 112 |
+
line = line.strip()
|
| 113 |
+
if line.startswith("- ") and line not in {"- (none)", "- none"}:
|
| 114 |
+
assets.append(line[2:].strip())
|
| 115 |
+
prompt = user_content.split("User prompt:", 1)[1].strip()
|
| 116 |
+
|
| 117 |
+
return {
|
| 118 |
+
"task": task,
|
| 119 |
+
"duration": duration,
|
| 120 |
+
"assets": assets or None,
|
| 121 |
+
"prompt": prompt,
|
| 122 |
+
}
|
minimax/formatting/fields.py
ADDED
|
@@ -0,0 +1,46 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Shared field lists and task normalization for base/ref output formats.
|
| 2 |
+
|
| 3 |
+
Lives in ``formatting`` (not ``scoring``) so both scoring and postprocessing
|
| 4 |
+
can depend on it without import cycles: scoring -> formatting is the allowed
|
| 5 |
+
direction.
|
| 6 |
+
"""
|
| 7 |
+
|
| 8 |
+
from __future__ import annotations
|
| 9 |
+
|
| 10 |
+
BASE_FIELDS = [
|
| 11 |
+
"integrated_multimodal_description:",
|
| 12 |
+
"overall_soundscape:",
|
| 13 |
+
"non_diegetic_music:",
|
| 14 |
+
]
|
| 15 |
+
REF_FIELDS = [
|
| 16 |
+
"subject_definitions:",
|
| 17 |
+
"summary:",
|
| 18 |
+
"retention_analysis:",
|
| 19 |
+
"detailed_description:",
|
| 20 |
+
"overall_soundscape:",
|
| 21 |
+
"non_diegetic_music:",
|
| 22 |
+
]
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
def normalize_task(task: str) -> str:
|
| 26 |
+
t = (task or "T2VA").strip()
|
| 27 |
+
if t in {"T2VA", "I2VA", "FL2VA", "L2VA"}:
|
| 28 |
+
return t
|
| 29 |
+
if t.startswith("full-reference"):
|
| 30 |
+
return "ref"
|
| 31 |
+
if any(
|
| 32 |
+
x in t
|
| 33 |
+
for x in (
|
| 34 |
+
"reference_generation",
|
| 35 |
+
"keyframe",
|
| 36 |
+
"video_editing",
|
| 37 |
+
"video_continuation",
|
| 38 |
+
"audio_",
|
| 39 |
+
)
|
| 40 |
+
):
|
| 41 |
+
return "ref"
|
| 42 |
+
return t
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
def is_base(task: str) -> bool:
|
| 46 |
+
return normalize_task(task) in {"T2VA", "I2VA", "FL2VA", "L2VA"}
|
minimax/formatting/instructions.py
ADDED
|
@@ -0,0 +1,126 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Canonical alignment-instruction lines from the official writing guides.
|
| 2 |
+
|
| 3 |
+
VIDEO_PROMPT_WRITING_GUIDE_base_en.md fixes these lines verbatim (only the
|
| 4 |
+
final shot number N and the duration mark S.SS vary). Teacher batches that
|
| 5 |
+
paraphrase them create mixed supervision and destabilize the student's first
|
| 6 |
+
line, so packaging/validation must enforce the exact forms:
|
| 7 |
+
|
| 8 |
+
I2VA : For the target video, at 0.00 seconds into the target video,
|
| 9 |
+
<Picture 1> (from [Shot 1]) is fully referenced.
|
| 10 |
+
FL2VA: How the reference pictures align with the target video — Picture 1
|
| 11 |
+
(from Shot 1) aligns with the 0.00-second mark of the target video;
|
| 12 |
+
Picture 2 (from Shot N) aligns with the S.SS-second mark of the
|
| 13 |
+
target video.
|
| 14 |
+
L2VA : How the reference pictures align with the target video —
|
| 15 |
+
<Picture 1> (from [Shot N]) aligns with the S.SS-second mark of the
|
| 16 |
+
target video.
|
| 17 |
+
|
| 18 |
+
Note the asymmetry from the guide: FL2VA uses bare "Picture 1 (from Shot 1)"
|
| 19 |
+
while L2VA uses bracketed "<Picture 1> (from [Shot N])".
|
| 20 |
+
"""
|
| 21 |
+
|
| 22 |
+
from __future__ import annotations
|
| 23 |
+
|
| 24 |
+
import re
|
| 25 |
+
|
| 26 |
+
I2VA_INSTRUCTION = (
|
| 27 |
+
"For the target video, at 0.00 seconds into the target video, "
|
| 28 |
+
"<Picture 1> (from [Shot 1]) is fully referenced."
|
| 29 |
+
)
|
| 30 |
+
|
| 31 |
+
FL2VA_RE = re.compile(
|
| 32 |
+
r"^How the reference pictures align with the target video — "
|
| 33 |
+
r"Picture 1 \(from Shot 1\) aligns with the 0\.00-second mark of the target video; "
|
| 34 |
+
r"Picture 2 \(from Shot \d+\) aligns with the \d+\.\d{2}-second mark of the target video\.$"
|
| 35 |
+
)
|
| 36 |
+
L2VA_RE = re.compile(
|
| 37 |
+
r"^How the reference pictures align with the target video — "
|
| 38 |
+
r"<Picture 1> \(from \[Shot \d+\]\) aligns with the \d+\.\d{2}-second mark of the target video\.$"
|
| 39 |
+
)
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
def fl2va_instruction(final_shot: int, duration: float) -> str:
|
| 43 |
+
return (
|
| 44 |
+
"How the reference pictures align with the target video — "
|
| 45 |
+
"Picture 1 (from Shot 1) aligns with the 0.00-second mark of the target video; "
|
| 46 |
+
f"Picture 2 (from Shot {final_shot}) aligns with the {duration:.2f}-second mark "
|
| 47 |
+
"of the target video."
|
| 48 |
+
)
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def l2va_instruction(final_shot: int, duration: float) -> str:
|
| 52 |
+
return (
|
| 53 |
+
"How the reference pictures align with the target video — "
|
| 54 |
+
f"<Picture 1> (from [Shot {final_shot}]) aligns with the {duration:.2f}-second "
|
| 55 |
+
"mark of the target video."
|
| 56 |
+
)
|
| 57 |
+
|
| 58 |
+
|
| 59 |
+
def is_canonical_instruction(task: str, first_line: str) -> bool:
|
| 60 |
+
line = first_line.strip()
|
| 61 |
+
if task == "T2VA":
|
| 62 |
+
return line.startswith("integrated_multimodal_description:")
|
| 63 |
+
if task == "I2VA":
|
| 64 |
+
return line == I2VA_INSTRUCTION
|
| 65 |
+
if task == "FL2VA":
|
| 66 |
+
return bool(FL2VA_RE.match(line))
|
| 67 |
+
if task == "L2VA":
|
| 68 |
+
return bool(L2VA_RE.match(line))
|
| 69 |
+
return True
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
def final_shot_number(body: str) -> int:
|
| 73 |
+
shots = [int(n) for n in re.findall(r"\[Shot (\d+)\]", body)]
|
| 74 |
+
return max(shots) if shots else 1
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
def canonical_instruction(task: str, body: str, duration: float) -> str | None:
|
| 78 |
+
"""The exact guide line for a base task, or None for T2VA."""
|
| 79 |
+
if task == "I2VA":
|
| 80 |
+
return I2VA_INSTRUCTION
|
| 81 |
+
if task == "FL2VA":
|
| 82 |
+
return fl2va_instruction(final_shot_number(body), duration)
|
| 83 |
+
if task == "L2VA":
|
| 84 |
+
return l2va_instruction(final_shot_number(body), duration)
|
| 85 |
+
return None
|
| 86 |
+
|
| 87 |
+
|
| 88 |
+
def enforce_instruction_line(task: str, text: str, duration: float) -> str:
|
| 89 |
+
"""Deterministically (re)build the alignment instruction on a generation.
|
| 90 |
+
|
| 91 |
+
A 350M student is unreliable at reproducing long fixed boilerplate (it may
|
| 92 |
+
omit the line, blend two task lines, or invent a Picture 2). Since the
|
| 93 |
+
line is fully determined by task + duration + final shot number, the
|
| 94 |
+
runtime constructs it: keep the body from the first core field onward and
|
| 95 |
+
prepend the canonical line.
|
| 96 |
+
"""
|
| 97 |
+
if task not in {"I2VA", "FL2VA", "L2VA"}:
|
| 98 |
+
return text
|
| 99 |
+
idx = text.find("integrated_multimodal_description:")
|
| 100 |
+
if idx < 0:
|
| 101 |
+
return text # no recognizable body; nothing safe to repair
|
| 102 |
+
body = text[idx:].strip()
|
| 103 |
+
line = canonical_instruction(task, body, duration)
|
| 104 |
+
return f"{line}\n\n{body}"
|
| 105 |
+
|
| 106 |
+
|
| 107 |
+
def repair_base_instruction(task: str, output: str, duration: float) -> str:
|
| 108 |
+
"""Replace a paraphrased/missing alignment instruction with the guide line.
|
| 109 |
+
|
| 110 |
+
The instruction is fully deterministic given task, duration, and the final
|
| 111 |
+
shot number, so this is a safe mechanical repair (no content invented).
|
| 112 |
+
"""
|
| 113 |
+
if task not in {"I2VA", "FL2VA", "L2VA"}:
|
| 114 |
+
return output
|
| 115 |
+
|
| 116 |
+
text = output.strip()
|
| 117 |
+
if text.startswith("integrated_multimodal_description:"):
|
| 118 |
+
body = text # instruction missing entirely
|
| 119 |
+
else:
|
| 120 |
+
parts = text.split("\n", 1)
|
| 121 |
+
body = parts[1].strip() if len(parts) > 1 else ""
|
| 122 |
+
if is_canonical_instruction(task, parts[0]):
|
| 123 |
+
return output
|
| 124 |
+
|
| 125 |
+
line = canonical_instruction(task, body, duration)
|
| 126 |
+
return f"{line}\n\n{body}"
|
minimax/formatting/meta_leak.py
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Detect system-prompt instruction leakage in generations.
|
| 2 |
+
|
| 3 |
+
A small student can paraphrase system rules into the scene body. Detection is
|
| 4 |
+
used by the format scorer (and teacher validation) so those outputs fail
|
| 5 |
+
honestly. Content is *not* auto-rewritten here — fix via training + systems.
|
| 6 |
+
"""
|
| 7 |
+
|
| 8 |
+
from __future__ import annotations
|
| 9 |
+
|
| 10 |
+
import re
|
| 11 |
+
|
| 12 |
+
# Phrases that only appear when the model is narrating the *format rules*
|
| 13 |
+
# rather than describing the video.
|
| 14 |
+
_LEAK_RES: list[re.Pattern[str]] = [
|
| 15 |
+
re.compile(r"(?i)\brequired alignment instruction\b"),
|
| 16 |
+
re.compile(r"(?i)\bbegin(?:s|ning)? with the required\b"),
|
| 17 |
+
re.compile(r"(?i)\bintegrated_multimodal_description begins\b"),
|
| 18 |
+
re.compile(r"(?i)\bdetailed_description begins\b"),
|
| 19 |
+
re.compile(r"(?i)\bfills in a full story\b"),
|
| 20 |
+
re.compile(r"(?i)\bfrom the referenced picture\b"),
|
| 21 |
+
re.compile(r"(?i)\bthree core fields\b"),
|
| 22 |
+
re.compile(r"(?i)\bas the first line,? then one blank line\b"),
|
| 23 |
+
re.compile(r"(?i)\bwrites? the body in english\b"),
|
| 24 |
+
re.compile(r"(?i)\bdo not invent timestamps\b"),
|
| 25 |
+
re.compile(r"(?i)\boutput rules?\b"),
|
| 26 |
+
]
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
def has_instruction_leak(text: str) -> bool:
|
| 30 |
+
"""True if any known instruction-leak phrase appears in the generation."""
|
| 31 |
+
t = text or ""
|
| 32 |
+
return any(rx.search(t) for rx in _LEAK_RES)
|
minimax/formatting/postprocess.py
ADDED
|
@@ -0,0 +1,168 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Generation postprocessing: trim chat markers, section loops, structural fixes.
|
| 2 |
+
|
| 3 |
+
Structural repairs only (deterministic guide rules) — no invented scene prose.
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
from __future__ import annotations
|
| 7 |
+
|
| 8 |
+
import re
|
| 9 |
+
|
| 10 |
+
from .fields import BASE_FIELDS, REF_FIELDS, is_base, normalize_task
|
| 11 |
+
from .instructions import enforce_instruction_line
|
| 12 |
+
from .timestamps import enforce_shot_timestamps
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
_CITED_SHOTS_RE = re.compile(r"\s*\(appears in ([^)]*\[Shot \d+\][^)]*)\)")
|
| 16 |
+
_SHOT_TOKEN_RE = re.compile(r"\[Shot (\d+)\]")
|
| 17 |
+
_DESC_HEADERS = ("detailed_description:", "integrated_multimodal_description:")
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
def _description_body_span(text: str, header: str) -> tuple[int, int] | None:
|
| 21 |
+
h = text.find(header)
|
| 22 |
+
if h < 0:
|
| 23 |
+
return None
|
| 24 |
+
start = h + len(header)
|
| 25 |
+
end = text.find("overall_soundscape:", start)
|
| 26 |
+
if end < 0:
|
| 27 |
+
end = len(text)
|
| 28 |
+
return start, end
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
def ensure_trailing_audio_fields(text: str, *, base: bool) -> str:
|
| 32 |
+
"""Append missing overall_soundscape / non_diegetic_music with N/A.
|
| 33 |
+
|
| 34 |
+
Models sometimes EOS right after the description (seen on long FL2VA).
|
| 35 |
+
Headers are required by the guide; N/A is the guide-legal empty value
|
| 36 |
+
for music and total silence, so this is structural — not scene invention.
|
| 37 |
+
"""
|
| 38 |
+
t = text.rstrip()
|
| 39 |
+
# Only repair when a description section exists (otherwise empty/garbage).
|
| 40 |
+
if "integrated_multimodal_description:" not in t and "detailed_description:" not in t:
|
| 41 |
+
return t
|
| 42 |
+
sep = " " if base else "\n"
|
| 43 |
+
# Append missing fields in guide order after whatever we already have.
|
| 44 |
+
if "overall_soundscape:" not in t:
|
| 45 |
+
t = t + "\n\noverall_soundscape:" + sep + "N/A"
|
| 46 |
+
if "non_diegetic_music:" not in t:
|
| 47 |
+
t = t + "\n\nnon_diegetic_music:" + sep + "N/A"
|
| 48 |
+
return t
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def ensure_shot1_header(text: str) -> str:
|
| 52 |
+
"""If a description body has content but no [Shot 1], prepend the marker.
|
| 53 |
+
|
| 54 |
+
Guide requires an opening shot section; 350M models sometimes write only
|
| 55 |
+
prose under detailed_description. Injecting the marker is structural —
|
| 56 |
+
it does not invent visual content.
|
| 57 |
+
"""
|
| 58 |
+
t = text
|
| 59 |
+
for header in _DESC_HEADERS:
|
| 60 |
+
span = _description_body_span(t, header)
|
| 61 |
+
if span is None:
|
| 62 |
+
continue
|
| 63 |
+
start, end = span
|
| 64 |
+
body = t[start:end]
|
| 65 |
+
if not body.strip() or "[Shot 1]" in body:
|
| 66 |
+
continue
|
| 67 |
+
t = t[:start] + " [Shot 1] " + body.lstrip() + t[end:]
|
| 68 |
+
break
|
| 69 |
+
return t
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
def _align_retention_citations(t: str) -> str:
|
| 73 |
+
"""Drop retention citations of shots that have no section in the description.
|
| 74 |
+
|
| 75 |
+
The guide only allows retention_analysis to cite shot numbers that exist
|
| 76 |
+
in detailed_description. Phantom [Shot 2] citations trip the whole-text
|
| 77 |
+
timestamp scorer even when the description is legitimately single-shot.
|
| 78 |
+
"""
|
| 79 |
+
if "retention_analysis:" not in t or "detailed_description:" not in t:
|
| 80 |
+
return t
|
| 81 |
+
desc = t.split("detailed_description:", 1)[1]
|
| 82 |
+
real = set(
|
| 83 |
+
re.findall(r"\[Shot (\d+)\]", desc.split("overall_soundscape:", 1)[0])
|
| 84 |
+
)
|
| 85 |
+
|
| 86 |
+
head, rest = t.split("retention_analysis:", 1)
|
| 87 |
+
ret, desc_part = rest.split("detailed_description:", 1)
|
| 88 |
+
|
| 89 |
+
if not real:
|
| 90 |
+
# No shot sections in DD — strip all shot tokens from retention so
|
| 91 |
+
# format scoring is not poisoned by phantom multi-shot claims.
|
| 92 |
+
ret2 = _CITED_SHOTS_RE.sub("", ret)
|
| 93 |
+
ret2 = _SHOT_TOKEN_RE.sub("", ret2)
|
| 94 |
+
return head + "retention_analysis:" + ret2 + "detailed_description:" + desc_part
|
| 95 |
+
|
| 96 |
+
def fix_appears(m: re.Match) -> str:
|
| 97 |
+
cited = re.findall(r"\[Shot (\d+)\]", m.group(1))
|
| 98 |
+
kept = [n for n in cited if n in real]
|
| 99 |
+
if not kept:
|
| 100 |
+
return ""
|
| 101 |
+
if kept == cited:
|
| 102 |
+
return m.group(0)
|
| 103 |
+
return " (appears in " + ", ".join(f"[Shot {n}]" for n in kept) + ")"
|
| 104 |
+
|
| 105 |
+
ret2 = _CITED_SHOTS_RE.sub(fix_appears, ret)
|
| 106 |
+
|
| 107 |
+
def bare_shot(m: re.Match) -> str:
|
| 108 |
+
return m.group(0) if m.group(1) in real else ""
|
| 109 |
+
|
| 110 |
+
ret2 = _SHOT_TOKEN_RE.sub(bare_shot, ret2)
|
| 111 |
+
return head + "retention_analysis:" + ret2 + "detailed_description:" + desc_part
|
| 112 |
+
|
| 113 |
+
|
| 114 |
+
def postprocess_generation(text: str, task: str, duration: float | None = None) -> str:
|
| 115 |
+
t = (text or "").strip()
|
| 116 |
+
for stop in ("<|im_end|>", "<|endoftext|>", "<|im_start|>"):
|
| 117 |
+
if stop in t:
|
| 118 |
+
t = t.split(stop, 1)[0].strip()
|
| 119 |
+
|
| 120 |
+
task_n = normalize_task(task)
|
| 121 |
+
fields = BASE_FIELDS if is_base(task_n) else REF_FIELDS
|
| 122 |
+
|
| 123 |
+
# Field headers are fixed lowercase tokens; models occasionally emit
|
| 124 |
+
# sentence-cased variants ("Overall_soundscape:") after a paragraph break.
|
| 125 |
+
for f in fields:
|
| 126 |
+
t = re.sub(rf"(?im)^[ \t]*{re.escape(f)}", f, t)
|
| 127 |
+
|
| 128 |
+
# If a field header appears twice, keep only the first complete document.
|
| 129 |
+
for f in fields:
|
| 130 |
+
first = t.find(f)
|
| 131 |
+
if first < 0:
|
| 132 |
+
continue
|
| 133 |
+
second = t.find(f, first + len(f))
|
| 134 |
+
if second > 0:
|
| 135 |
+
t = t[:second].rstrip()
|
| 136 |
+
break
|
| 137 |
+
|
| 138 |
+
# Trim anything after the first paragraph of non_diegetic_music.
|
| 139 |
+
if "non_diegetic_music:" in t:
|
| 140 |
+
head, tail = t.split("non_diegetic_music:", 1)
|
| 141 |
+
music_body = tail.strip().split("\n\n")[0].strip()
|
| 142 |
+
for f in fields:
|
| 143 |
+
if f in music_body:
|
| 144 |
+
music_body = music_body.split(f, 1)[0].strip()
|
| 145 |
+
sep = " " if is_base(task_n) else "\n"
|
| 146 |
+
t = head + "non_diegetic_music:" + sep + music_body
|
| 147 |
+
|
| 148 |
+
# The alignment instruction is deterministic boilerplate; rebuild it
|
| 149 |
+
# rather than trusting the model to reproduce it verbatim.
|
| 150 |
+
if duration is not None and is_base(task_n):
|
| 151 |
+
t = enforce_instruction_line(task_n, t.strip(), duration)
|
| 152 |
+
|
| 153 |
+
# Structural shot markers before timestamp repair / retention cleanup.
|
| 154 |
+
t = ensure_shot1_header(t)
|
| 155 |
+
|
| 156 |
+
# Timestamp grammar is fully specified by the guide; repair missing or
|
| 157 |
+
# non-increasing "At MM:SS.mmm" stamps by interpolation.
|
| 158 |
+
if duration is not None:
|
| 159 |
+
t = enforce_shot_timestamps(t, duration)
|
| 160 |
+
|
| 161 |
+
if not is_base(task_n):
|
| 162 |
+
t = _align_retention_citations(t)
|
| 163 |
+
|
| 164 |
+
# Fill trailing sound/music headers when the model stops after the body
|
| 165 |
+
# (common on long keyframe FL2VA). Uses N/A — not invented ambience.
|
| 166 |
+
t = ensure_trailing_audio_fields(t, base=is_base(task_n))
|
| 167 |
+
|
| 168 |
+
return t.strip()
|
minimax/formatting/ref_repair.py
ADDED
|
@@ -0,0 +1,106 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Mechanical repairs for full-reference gold that drifted from the ref guide.
|
| 2 |
+
|
| 3 |
+
Two known teacher drifts (gold_04..06 era):
|
| 4 |
+
|
| 5 |
+
1. retention_analysis lines shaped "<Subject 1> fully_preserved" — missing the
|
| 6 |
+
guide's "<Label> (appears in [Shot ...]): marker - explanation" form. The
|
| 7 |
+
repair keeps the label + marker and derives the explanation from that
|
| 8 |
+
label's own subject_definitions entry (content-derived, not templated).
|
| 9 |
+
|
| 10 |
+
2. video_editing summaries missing the mandated opener sentence
|
| 11 |
+
"The target video is an edited version of <Video 1>." right after the
|
| 12 |
+
[task type] prefix.
|
| 13 |
+
"""
|
| 14 |
+
|
| 15 |
+
from __future__ import annotations
|
| 16 |
+
|
| 17 |
+
import re
|
| 18 |
+
|
| 19 |
+
BARE_RET_LINE = re.compile(
|
| 20 |
+
r"^(?P<label><(?:Subject|Picture|Video|Audio) \d+>)\s+"
|
| 21 |
+
r"(?P<marker>fully_preserved|partially_preserved|attribute_transfer|"
|
| 22 |
+
r"weak_reference|fully_copy|partially_copy|reference)\s*$"
|
| 23 |
+
)
|
| 24 |
+
|
| 25 |
+
_MARKER_PHRASE = {
|
| 26 |
+
"fully_preserved": "{desc} is retained in the target video",
|
| 27 |
+
"partially_preserved": "{desc} is used with some defined characteristics changed",
|
| 28 |
+
"attribute_transfer": "characteristics of {desc} are transferred to the target subject",
|
| 29 |
+
"weak_reference": "only broad similarity to {desc} is retained",
|
| 30 |
+
"fully_copy": "the audio of {desc} is reused as the target video's final audio track",
|
| 31 |
+
"partially_copy": "part of the audio timeline of {desc} is copied into the target video",
|
| 32 |
+
"reference": "only the audible characteristics of {desc} are referenced without copying the signal",
|
| 33 |
+
}
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def _label_description(label: str, subject_definitions: str) -> str:
|
| 37 |
+
"""First clause of the label's definition line, e.g. 'the young woman ...'."""
|
| 38 |
+
for line in subject_definitions.splitlines():
|
| 39 |
+
line = line.strip()
|
| 40 |
+
if line.startswith(label):
|
| 41 |
+
rest = line[len(label):].lstrip()
|
| 42 |
+
rest = re.sub(r"^(is|are)\s+", "", rest)
|
| 43 |
+
clause = re.split(r"[.;]", rest, 1)[0].strip().rstrip(",")
|
| 44 |
+
if clause:
|
| 45 |
+
return clause
|
| 46 |
+
return "this reference"
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
def _shots_for_label(label: str, detailed_description: str) -> list[int]:
|
| 50 |
+
"""Shot numbers whose section mentions the label."""
|
| 51 |
+
parts = re.split(r"(\[Shot (\d+)\])", detailed_description)
|
| 52 |
+
shots: list[int] = []
|
| 53 |
+
current = None
|
| 54 |
+
for chunk in parts:
|
| 55 |
+
m = re.match(r"\[Shot (\d+)\]$", chunk or "")
|
| 56 |
+
if m:
|
| 57 |
+
current = int(m.group(1))
|
| 58 |
+
elif current is not None and label in (chunk or ""):
|
| 59 |
+
if current not in shots:
|
| 60 |
+
shots.append(current)
|
| 61 |
+
return shots
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
def repair_retention_lines(output: str) -> str:
|
| 65 |
+
if "retention_analysis:" not in output:
|
| 66 |
+
return output
|
| 67 |
+
head, rest = output.split("retention_analysis:", 1)
|
| 68 |
+
if "detailed_description:" in rest:
|
| 69 |
+
ret_block, tail = rest.split("detailed_description:", 1)
|
| 70 |
+
tail = "detailed_description:" + tail
|
| 71 |
+
else:
|
| 72 |
+
ret_block, tail = rest, ""
|
| 73 |
+
subj_defs = head.split("subject_definitions:", 1)[-1]
|
| 74 |
+
detailed = tail
|
| 75 |
+
|
| 76 |
+
fixed_lines = []
|
| 77 |
+
for line in ret_block.split("\n"):
|
| 78 |
+
m = BARE_RET_LINE.match(line.strip())
|
| 79 |
+
if not m:
|
| 80 |
+
fixed_lines.append(line)
|
| 81 |
+
continue
|
| 82 |
+
label, marker = m.group("label"), m.group("marker")
|
| 83 |
+
desc = _label_description(label, subj_defs)
|
| 84 |
+
phrase = _MARKER_PHRASE[marker].format(desc=desc)
|
| 85 |
+
appears = ""
|
| 86 |
+
if label.startswith("<Subject"):
|
| 87 |
+
shots = _shots_for_label(label, detailed)
|
| 88 |
+
if shots:
|
| 89 |
+
appears = " (appears in " + ", ".join(f"[Shot {n}]" for n in shots) + ")"
|
| 90 |
+
fixed_lines.append(f"{label}{appears}: {marker} - {phrase}.")
|
| 91 |
+
|
| 92 |
+
return head + "retention_analysis:" + "\n".join(fixed_lines) + tail
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
VIDEO_EDIT_OPENER = "The target video is an edited version of <Video 1>."
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
def repair_video_editing_opener(output: str, task: str) -> str:
|
| 99 |
+
if not task.startswith("video_editing"):
|
| 100 |
+
return output
|
| 101 |
+
if "an edited version of <Video 1>" in output:
|
| 102 |
+
return output
|
| 103 |
+
m = re.search(r"(summary:\s*\n?\s*\[[^\]]+\])\s*", output)
|
| 104 |
+
if not m:
|
| 105 |
+
return output
|
| 106 |
+
return output[: m.end(1)] + f" {VIDEO_EDIT_OPENER}" + output[m.end(1):]
|
minimax/formatting/timestamps.py
ADDED
|
@@ -0,0 +1,108 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Deterministic repair of shot timestamps in generated descriptions.
|
| 2 |
+
|
| 3 |
+
The guide fixes the timestamp grammar completely: [Shot 1] carries no
|
| 4 |
+
timestamp, every later shot must open with "At MM:SS.mmm," and times must
|
| 5 |
+
increase strictly inside the clip duration. When a generation drops or
|
| 6 |
+
garbles a timestamp, the product layer can rebuild it mechanically —
|
| 7 |
+
existing valid stamps are kept, missing ones are interpolated evenly
|
| 8 |
+
between their known neighbours (0.0 at the start, the duration at the end).
|
| 9 |
+
|
| 10 |
+
Only the description section is touched, and only its *sequential* shot
|
| 11 |
+
markers ([Shot 1] then [Shot 2] then ...) count as section starts; shot
|
| 12 |
+
references elsewhere (e.g. retention_analysis "appears in [Shot 2]") are
|
| 13 |
+
left alone.
|
| 14 |
+
"""
|
| 15 |
+
|
| 16 |
+
from __future__ import annotations
|
| 17 |
+
|
| 18 |
+
import re
|
| 19 |
+
|
| 20 |
+
SHOT_RE = re.compile(r"\[Shot (\d+)\]\s*")
|
| 21 |
+
STAMP_RE = re.compile(r"^At (\d{2}):(\d{2}\.\d{3}),?\s*")
|
| 22 |
+
|
| 23 |
+
_DESC_HEADERS = ("detailed_description:", "integrated_multimodal_description:")
|
| 24 |
+
_NEXT_HEADER = "overall_soundscape:"
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def _fmt(seconds: float) -> str:
|
| 28 |
+
m = int(seconds // 60)
|
| 29 |
+
return f"{m:02d}:{seconds - m * 60:06.3f}"
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def _repair_section(section: str, duration: float) -> str:
|
| 33 |
+
# Sequential shot-section markers only: [Shot 1], then [Shot 2], ...
|
| 34 |
+
marks: list[re.Match] = []
|
| 35 |
+
expected = 1
|
| 36 |
+
for m in SHOT_RE.finditer(section):
|
| 37 |
+
if int(m.group(1)) == expected:
|
| 38 |
+
marks.append(m)
|
| 39 |
+
expected += 1
|
| 40 |
+
if len(marks) < 2:
|
| 41 |
+
return section
|
| 42 |
+
|
| 43 |
+
stamps: list[float | None] = []
|
| 44 |
+
spans: list[tuple[int, int]] = []
|
| 45 |
+
for m in marks:
|
| 46 |
+
sm = STAMP_RE.match(section[m.end():])
|
| 47 |
+
if sm:
|
| 48 |
+
stamps.append(int(sm.group(1)) * 60 + float(sm.group(2)))
|
| 49 |
+
spans.append((m.end(), m.end() + sm.end()))
|
| 50 |
+
else:
|
| 51 |
+
stamps.append(None)
|
| 52 |
+
spans.append((m.end(), m.end()))
|
| 53 |
+
|
| 54 |
+
# Shot 1 implicitly owns 0.0; keep valid increasing stamps, interpolate
|
| 55 |
+
# anything missing or out of order toward the next trusted value.
|
| 56 |
+
times: list[float] = [0.0] * len(stamps)
|
| 57 |
+
prev = 0.0
|
| 58 |
+
i = 1
|
| 59 |
+
while i < len(stamps):
|
| 60 |
+
s = stamps[i]
|
| 61 |
+
if s is not None and prev < s < duration:
|
| 62 |
+
times[i] = s
|
| 63 |
+
prev = s
|
| 64 |
+
i += 1
|
| 65 |
+
continue
|
| 66 |
+
j = i + 1
|
| 67 |
+
nxt = duration
|
| 68 |
+
while j < len(stamps):
|
| 69 |
+
sj = stamps[j]
|
| 70 |
+
if sj is not None and prev < sj < duration:
|
| 71 |
+
nxt = sj
|
| 72 |
+
break
|
| 73 |
+
j += 1
|
| 74 |
+
gaps = (j - i) + 1
|
| 75 |
+
for k in range(i, j):
|
| 76 |
+
times[k] = prev + (nxt - prev) * (k - i + 1) / gaps
|
| 77 |
+
prev = times[j - 1]
|
| 78 |
+
i = j
|
| 79 |
+
|
| 80 |
+
out = section
|
| 81 |
+
for idx in range(len(marks) - 1, 0, -1):
|
| 82 |
+
start, end = spans[idx]
|
| 83 |
+
if stamps[idx] is not None and abs(stamps[idx] - times[idx]) < 0.0005:
|
| 84 |
+
continue
|
| 85 |
+
out = out[:start] + f"At {_fmt(times[idx])}, " + out[end:]
|
| 86 |
+
# Shot 1 must not carry a timestamp.
|
| 87 |
+
s0, e0 = spans[0]
|
| 88 |
+
if stamps[0] is not None:
|
| 89 |
+
out = out[:s0] + out[e0:]
|
| 90 |
+
return out
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
def enforce_shot_timestamps(text: str, duration: float) -> str:
|
| 94 |
+
"""Ensure every sequential [Shot N>=2] opens with a valid increasing stamp."""
|
| 95 |
+
if duration <= 0:
|
| 96 |
+
return text
|
| 97 |
+
|
| 98 |
+
for header in _DESC_HEADERS:
|
| 99 |
+
h = text.find(header)
|
| 100 |
+
if h < 0:
|
| 101 |
+
continue
|
| 102 |
+
start = h + len(header)
|
| 103 |
+
end = text.find(_NEXT_HEADER, start)
|
| 104 |
+
if end < 0:
|
| 105 |
+
end = len(text)
|
| 106 |
+
section = text[start:end]
|
| 107 |
+
return text[:start] + _repair_section(section, duration) + text[end:]
|
| 108 |
+
return text
|
minimax/modeling/__init__.py
ADDED
|
File without changes
|
minimax/modeling/generate.py
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Single-item generation with stop tokens, postprocess, and format scoring."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from ..formatting.envelope import build_user_message, load_system
|
| 6 |
+
from ..formatting.postprocess import postprocess_generation
|
| 7 |
+
from ..scoring.format_score import default_max_new_tokens, score_format
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
def generate_one(
|
| 11 |
+
model,
|
| 12 |
+
tokenizer,
|
| 13 |
+
*,
|
| 14 |
+
task: str,
|
| 15 |
+
duration: float,
|
| 16 |
+
prompt: str,
|
| 17 |
+
assets: list[str] | None,
|
| 18 |
+
stop_ids: list[int],
|
| 19 |
+
max_new_tokens: int | None = None,
|
| 20 |
+
temperature: float = 0.0,
|
| 21 |
+
top_k: int = 40,
|
| 22 |
+
repetition_penalty: float = 1.0,
|
| 23 |
+
legacy_labels: bool = False,
|
| 24 |
+
) -> tuple[str, dict]:
|
| 25 |
+
import torch
|
| 26 |
+
|
| 27 |
+
messages = [
|
| 28 |
+
{"role": "system", "content": load_system(task)},
|
| 29 |
+
{
|
| 30 |
+
"role": "user",
|
| 31 |
+
"content": build_user_message(task, duration, prompt, assets, legacy=legacy_labels),
|
| 32 |
+
},
|
| 33 |
+
]
|
| 34 |
+
prompt_text = tokenizer.apply_chat_template(
|
| 35 |
+
messages, tokenize=False, add_generation_prompt=True
|
| 36 |
+
)
|
| 37 |
+
inputs = tokenizer(prompt_text, return_tensors="pt")
|
| 38 |
+
inputs = {k: v.to(model.device) for k, v in inputs.items()}
|
| 39 |
+
|
| 40 |
+
mnt = max_new_tokens if max_new_tokens is not None else default_max_new_tokens(task)
|
| 41 |
+
gen_kwargs = dict(
|
| 42 |
+
max_new_tokens=mnt,
|
| 43 |
+
do_sample=temperature > 0,
|
| 44 |
+
temperature=max(temperature, 1e-5) if temperature > 0 else None,
|
| 45 |
+
top_k=top_k if temperature > 0 else None,
|
| 46 |
+
repetition_penalty=repetition_penalty,
|
| 47 |
+
pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id,
|
| 48 |
+
)
|
| 49 |
+
gen_kwargs = {k: v for k, v in gen_kwargs.items() if v is not None}
|
| 50 |
+
if stop_ids:
|
| 51 |
+
gen_kwargs["eos_token_id"] = stop_ids
|
| 52 |
+
|
| 53 |
+
with torch.inference_mode():
|
| 54 |
+
out = model.generate(**inputs, **gen_kwargs)
|
| 55 |
+
|
| 56 |
+
gen = out[0][inputs["input_ids"].shape[-1] :]
|
| 57 |
+
raw = tokenizer.decode(gen, skip_special_tokens=True).strip()
|
| 58 |
+
raw = raw.replace("<|im_end|>", "").replace("<|endoftext|>", "").strip()
|
| 59 |
+
text = postprocess_generation(raw, task, duration)
|
| 60 |
+
report = score_format(text, task)
|
| 61 |
+
report["raw_char_len"] = len(raw)
|
| 62 |
+
report["max_new_tokens"] = mnt
|
| 63 |
+
return text, report
|
minimax/modeling/loader.py
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Model/tokenizer loading — ZeroGPU-friendly (place on cuda at load time)."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
|
| 7 |
+
from ..paths import BASE_MODEL
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
def resolve_model_dir(model: str | Path) -> tuple[str, str | None]:
|
| 11 |
+
"""Return (model_path, adapter_path). Prefers merged weights when present."""
|
| 12 |
+
p = Path(model)
|
| 13 |
+
if p.is_dir():
|
| 14 |
+
if (p / "merged").is_dir() and (p / "merged" / "config.json").is_file():
|
| 15 |
+
return str(p / "merged"), None
|
| 16 |
+
if (p / "config.json").is_file():
|
| 17 |
+
return str(p), None
|
| 18 |
+
if (p / "adapter_config.json").is_file():
|
| 19 |
+
return BASE_MODEL, str(p)
|
| 20 |
+
return str(model), None
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
def load_model_and_tokenizer(model: str | Path):
|
| 24 |
+
"""Load weights onto CUDA.
|
| 25 |
+
|
| 26 |
+
ZeroGPU emulates CUDA at import/startup so ``.to("cuda")`` is required at
|
| 27 |
+
the root level; a real GPU is only attached inside ``@spaces.GPU`` calls.
|
| 28 |
+
"""
|
| 29 |
+
import torch
|
| 30 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 31 |
+
|
| 32 |
+
model_path, adapter_path = resolve_model_dir(model)
|
| 33 |
+
tokenizer = AutoTokenizer.from_pretrained(
|
| 34 |
+
adapter_path or model_path, trust_remote_code=True
|
| 35 |
+
)
|
| 36 |
+
# bf16 on GPU; ZeroGPU always exposes a cuda device (real or emulated).
|
| 37 |
+
dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32
|
| 38 |
+
lm = AutoModelForCausalLM.from_pretrained(
|
| 39 |
+
model_path,
|
| 40 |
+
trust_remote_code=True,
|
| 41 |
+
torch_dtype=dtype,
|
| 42 |
+
low_cpu_mem_usage=True,
|
| 43 |
+
)
|
| 44 |
+
if adapter_path:
|
| 45 |
+
from peft import PeftModel
|
| 46 |
+
|
| 47 |
+
lm = PeftModel.from_pretrained(lm, adapter_path).merge_and_unload()
|
| 48 |
+
# ZeroGPU docs: place models on cuda at root, not lazily inside @spaces.GPU
|
| 49 |
+
if torch.cuda.is_available():
|
| 50 |
+
lm = lm.to("cuda")
|
| 51 |
+
lm.eval()
|
| 52 |
+
return lm, tokenizer
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
def build_stop_ids(tokenizer) -> list[int]:
|
| 56 |
+
"""ChatML / LFM end tokens for clean stopping."""
|
| 57 |
+
candidates = ["<|im_end|>", "<|endoftext|>", tokenizer.eos_token]
|
| 58 |
+
ids: list[int] = []
|
| 59 |
+
for s in candidates:
|
| 60 |
+
if not s:
|
| 61 |
+
continue
|
| 62 |
+
try:
|
| 63 |
+
tid = tokenizer.convert_tokens_to_ids(s)
|
| 64 |
+
if tid is not None and tid != tokenizer.unk_token_id and int(tid) >= 0:
|
| 65 |
+
ids.append(int(tid))
|
| 66 |
+
except Exception:
|
| 67 |
+
pass
|
| 68 |
+
try:
|
| 69 |
+
enc = tokenizer.encode(s, add_special_tokens=False)
|
| 70 |
+
if len(enc) == 1:
|
| 71 |
+
ids.append(int(enc[0]))
|
| 72 |
+
except Exception:
|
| 73 |
+
pass
|
| 74 |
+
out: list[int] = []
|
| 75 |
+
for i in ids:
|
| 76 |
+
if i not in out:
|
| 77 |
+
out.append(i)
|
| 78 |
+
if tokenizer.eos_token_id is not None and tokenizer.eos_token_id not in out:
|
| 79 |
+
out.append(int(tokenizer.eos_token_id))
|
| 80 |
+
return out
|
minimax/paths.py
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Paths for the Hugging Face Space package (self-contained)."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
|
| 7 |
+
# Space root = parent of the minimax package
|
| 8 |
+
ROOT = Path(__file__).resolve().parent.parent
|
| 9 |
+
PROMPTS = ROOT / "prompts"
|
| 10 |
+
|
| 11 |
+
SYSTEM_BASE_FILE = PROMPTS / "system_base.txt"
|
| 12 |
+
SYSTEM_REF_FILE = PROMPTS / "system_ref.txt"
|
| 13 |
+
SYSTEM_BASE_TASK_FILES = {
|
| 14 |
+
"T2VA": PROMPTS / "system_base_t2va.txt",
|
| 15 |
+
"I2VA": PROMPTS / "system_base_i2va.txt",
|
| 16 |
+
"FL2VA": PROMPTS / "system_base_fl2va.txt",
|
| 17 |
+
"L2VA": PROMPTS / "system_base_l2va.txt",
|
| 18 |
+
}
|
| 19 |
+
|
| 20 |
+
# Default Hub model for this Space
|
| 21 |
+
DEFAULT_MODEL = "geocine/minimax-video-prompt-enhancer-350m"
|
| 22 |
+
BASE_MODEL = "LiquidAI/LFM2.5-350M"
|
| 23 |
+
# Alias used by local app code
|
| 24 |
+
CHAMPION_DIR = DEFAULT_MODEL
|
minimax/scoring/__init__.py
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from .format_score import ( # noqa: F401
|
| 2 |
+
BASE_FIELDS,
|
| 3 |
+
REF_FIELDS,
|
| 4 |
+
default_max_new_tokens,
|
| 5 |
+
is_base,
|
| 6 |
+
normalize_task,
|
| 7 |
+
score_format,
|
| 8 |
+
)
|
minimax/scoring/format_score.py
ADDED
|
@@ -0,0 +1,157 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""MiniMax structure scoring for base + full-reference rewrites.
|
| 2 |
+
|
| 3 |
+
pass = every critical check true. ``score`` is the fraction of critical checks
|
| 4 |
+
passed (soft signal; a high mean score with low pass rate means outputs are
|
| 5 |
+
one brittle check away from passing).
|
| 6 |
+
|
| 7 |
+
Changes vs the v1 scorer:
|
| 8 |
+
- ``has_reference_label`` accepts any of <Subject N> / <Picture N> /
|
| 9 |
+
<Video N> / <Audio N>. The v1 ``has_subject_label`` required <Subject N>
|
| 10 |
+
and wrongly failed e.g. video continuations that only track <Video 1>.
|
| 11 |
+
- every [Shot N] with N >= 2 must carry an "At MM:SS.mmm" timestamp
|
| 12 |
+
(v1 only checked Shot 2).
|
| 13 |
+
"""
|
| 14 |
+
|
| 15 |
+
from __future__ import annotations
|
| 16 |
+
|
| 17 |
+
import re
|
| 18 |
+
from typing import Any
|
| 19 |
+
|
| 20 |
+
from ..formatting.fields import ( # noqa: F401 (re-exported for compat)
|
| 21 |
+
BASE_FIELDS,
|
| 22 |
+
REF_FIELDS,
|
| 23 |
+
is_base,
|
| 24 |
+
normalize_task,
|
| 25 |
+
)
|
| 26 |
+
from ..formatting.instructions import FL2VA_RE, I2VA_INSTRUCTION, L2VA_RE
|
| 27 |
+
from ..formatting.meta_leak import has_instruction_leak
|
| 28 |
+
|
| 29 |
+
REFERENCE_LABEL_RE = re.compile(r"<(?:Subject|Picture|Video|Audio)\s+\d+>")
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def field_order_ok(text: str, fields: list[str]) -> bool:
|
| 33 |
+
positions = []
|
| 34 |
+
for f in fields:
|
| 35 |
+
p = text.find(f)
|
| 36 |
+
if p < 0:
|
| 37 |
+
return False
|
| 38 |
+
positions.append(p)
|
| 39 |
+
return positions == sorted(positions)
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
def count_section_repeats(text: str, fields: list[str]) -> int:
|
| 43 |
+
return sum(1 for f in fields if text.count(f) > 1)
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
def shots_have_timestamps(text: str) -> bool:
|
| 47 |
+
"""Every shot number N >= 2 needs a timestamped header "[Shot N] At MM:SS.mmm".
|
| 48 |
+
|
| 49 |
+
Checked per shot number, not per occurrence: sections like
|
| 50 |
+
retention_analysis legitimately cross-reference shots as "([Shot 1], [Shot 2])"
|
| 51 |
+
without timestamps.
|
| 52 |
+
"""
|
| 53 |
+
shot_numbers = {int(n) for n in re.findall(r"\[Shot (\d+)\]", text)}
|
| 54 |
+
for n in shot_numbers:
|
| 55 |
+
if n < 2:
|
| 56 |
+
continue
|
| 57 |
+
if not re.search(rf"\[Shot {n}\]\s+At\s+\d{{2}}:\d{{2}}\.\d{{3}}", text):
|
| 58 |
+
return False
|
| 59 |
+
return True
|
| 60 |
+
|
| 61 |
+
|
| 62 |
+
def score_format(text: str, task: str) -> dict[str, Any]:
|
| 63 |
+
"""Return pass/fail + checklist for one generation."""
|
| 64 |
+
text = (text or "").strip()
|
| 65 |
+
task_n = normalize_task(task)
|
| 66 |
+
checks: dict[str, bool] = {}
|
| 67 |
+
notes: list[str] = []
|
| 68 |
+
|
| 69 |
+
if not text:
|
| 70 |
+
return {
|
| 71 |
+
"pass": False,
|
| 72 |
+
"score": 0.0,
|
| 73 |
+
"task": task_n,
|
| 74 |
+
"checks": {"nonempty": False},
|
| 75 |
+
"notes": ["empty generation"],
|
| 76 |
+
}
|
| 77 |
+
|
| 78 |
+
checks["nonempty"] = True
|
| 79 |
+
checks["has_shot1"] = "[Shot 1]" in text
|
| 80 |
+
# MiniMax H3 prompt field hard limit: 7,000 characters incl. whitespace.
|
| 81 |
+
checks["within_h3_char_limit"] = len(text) <= 7000
|
| 82 |
+
# Reject system-rule paraphrases dumped into the scene body. Structural
|
| 83 |
+
# field checks alone green-lit this failure mode (T2VA pass with meta text).
|
| 84 |
+
checks["no_instruction_leak"] = not has_instruction_leak(text)
|
| 85 |
+
|
| 86 |
+
if is_base(task_n):
|
| 87 |
+
fields = BASE_FIELDS
|
| 88 |
+
for f in fields:
|
| 89 |
+
checks[f"field:{f}"] = f in text
|
| 90 |
+
checks["field_order"] = field_order_ok(text, fields)
|
| 91 |
+
checks["no_heavy_loop"] = count_section_repeats(text, fields) == 0
|
| 92 |
+
|
| 93 |
+
# Alignment instructions are fixed verbatim by the writing guide
|
| 94 |
+
# (only shot number / duration mark vary), so check the exact line.
|
| 95 |
+
first_line = text.split("\n", 1)[0].strip()
|
| 96 |
+
if task_n == "I2VA":
|
| 97 |
+
checks["i2va_instruction"] = first_line == I2VA_INSTRUCTION
|
| 98 |
+
if task_n == "FL2VA":
|
| 99 |
+
checks["fl2va_alignment"] = bool(FL2VA_RE.match(first_line))
|
| 100 |
+
if task_n == "L2VA":
|
| 101 |
+
checks["l2va_alignment"] = bool(L2VA_RE.match(first_line))
|
| 102 |
+
|
| 103 |
+
checks["has_shot2_optional"] = "[Shot 2]" in text
|
| 104 |
+
checks["shots_have_timestamps"] = shots_have_timestamps(text)
|
| 105 |
+
# Shot 1 must not carry a timestamp in base style
|
| 106 |
+
checks["shot1_no_bogus_timestamp"] = not bool(
|
| 107 |
+
re.search(r"\[Shot 1\]\s+At\s+\d{2}:", text)
|
| 108 |
+
)
|
| 109 |
+
|
| 110 |
+
else:
|
| 111 |
+
fields = REF_FIELDS
|
| 112 |
+
for f in fields:
|
| 113 |
+
checks[f"field:{f}"] = f in text
|
| 114 |
+
checks["field_order"] = field_order_ok(text, fields)
|
| 115 |
+
checks["no_heavy_loop"] = count_section_repeats(text, fields) == 0
|
| 116 |
+
|
| 117 |
+
if "summary:" in text:
|
| 118 |
+
sum_part = text.split("summary:", 1)[1]
|
| 119 |
+
if "retention_analysis:" in sum_part:
|
| 120 |
+
sum_part = sum_part.split("retention_analysis:", 1)[0]
|
| 121 |
+
checks["summary_task_prefix"] = bool(re.search(r"\[[^\]]+\]", sum_part))
|
| 122 |
+
else:
|
| 123 |
+
checks["summary_task_prefix"] = False
|
| 124 |
+
|
| 125 |
+
checks["has_reference_label"] = bool(REFERENCE_LABEL_RE.search(text))
|
| 126 |
+
checks["has_shot2_optional"] = "[Shot 2]" in text
|
| 127 |
+
checks["shots_have_timestamps"] = shots_have_timestamps(text)
|
| 128 |
+
|
| 129 |
+
critical = [k for k in checks if not k.endswith("_optional")]
|
| 130 |
+
n_ok = sum(1 for k in critical if checks.get(k))
|
| 131 |
+
score = n_ok / max(len(critical), 1)
|
| 132 |
+
passed = all(checks.get(k, False) for k in critical)
|
| 133 |
+
|
| 134 |
+
if not checks.get("no_heavy_loop", True):
|
| 135 |
+
notes.append("section headers repeated (possible loop)")
|
| 136 |
+
if not checks.get("has_shot1", True):
|
| 137 |
+
notes.append("missing [Shot 1]")
|
| 138 |
+
if not checks.get("no_instruction_leak", True):
|
| 139 |
+
notes.append("instruction/rule language leaked into scene body")
|
| 140 |
+
|
| 141 |
+
return {
|
| 142 |
+
"pass": passed,
|
| 143 |
+
"score": round(score, 3),
|
| 144 |
+
"task": task_n,
|
| 145 |
+
"checks": checks,
|
| 146 |
+
"notes": notes,
|
| 147 |
+
"char_len": len(text),
|
| 148 |
+
}
|
| 149 |
+
|
| 150 |
+
|
| 151 |
+
def default_max_new_tokens(task: str) -> int:
|
| 152 |
+
# The MiniMax H3 prompt field caps at 7,000 characters (~1,800-2,300
|
| 153 |
+
# tokens); budgets leave headroom so deep briefs are never truncated.
|
| 154 |
+
task_n = normalize_task(task)
|
| 155 |
+
if task_n in {"T2VA", "I2VA", "L2VA", "FL2VA"}:
|
| 156 |
+
return 1200
|
| 157 |
+
return 2048
|
prompts/system_base.txt
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
You enhance rough video prompts into structured audiovisual rewrite prompts for T2VA / I2VA / FL2VA / L2VA.
|
| 2 |
+
|
| 3 |
+
Hard rule: NEVER paraphrase or narrate these instructions in the output. Do not explain the format, mention alignment rules in the body, or summarize the user prompt as a story synopsis. Write only concrete audiovisual scene content.
|
| 4 |
+
|
| 5 |
+
Output rules:
|
| 6 |
+
1) Start according to the Task line in the user message:
|
| 7 |
+
- T2VA: no alignment line. First line is integrated_multimodal_description:
|
| 8 |
+
- I2VA / FL2VA / L2VA: first line is the task's exact alignment instruction (see below), then one blank line, then the three fields.
|
| 9 |
+
2) Then output exactly these three fields in order:
|
| 10 |
+
integrated_multimodal_description:
|
| 11 |
+
overall_soundscape:
|
| 12 |
+
non_diegetic_music:
|
| 13 |
+
3) Write the body in English. Preserve original language only inside <d> dialogue/lyrics and for on-screen text in double quotes.
|
| 14 |
+
4) [Shot 1] has no timestamp. Later shots use: [Shot N] At MM:SS.mmm, ...
|
| 15 |
+
5) Camera motion is natural English with motion type and, when meaningful, amplitude (with small/large amplitude) and speed (at slow/fast speed).
|
| 16 |
+
6) Speakers use stable IDs (S1), (S2). Dialogue format: <d>[Language] exact words</d>. Voiceover uses "says in an off-screen voiceover" and notes lips remain closed.
|
| 17 |
+
7) overall_soundscape: 1–4 English sentences on ambience, physical action sounds, non-verbal human sounds. No dialogue/singing/diegetic music. Use N/A only for total silence.
|
| 18 |
+
8) non_diegetic_music: 1–3 sentences on instrumentation, tempo, dynamics only (no abstract mood words). Use N/A when absent.
|
| 19 |
+
9) integrated_multimodal_description must open [Shot 1] with style + composition + visible action. Do not summarize the prompt; describe what the camera sees and hears.
|
| 20 |
+
10) Task-specific alignment (only when Task is that type; omit for T2VA):
|
| 21 |
+
- I2VA first line verbatim:
|
| 22 |
+
For the target video, at 0.00 seconds into the target video, <Picture 1> (from [Shot 1]) is fully referenced.
|
| 23 |
+
- FL2VA first line verbatim (N = final shot number, S.SS = duration):
|
| 24 |
+
How the reference pictures align with the target video — Picture 1 (from Shot 1) aligns with the 0.00-second mark of the target video; Picture 2 (from Shot N) aligns with the S.SS-second mark of the target video.
|
| 25 |
+
- L2VA first line verbatim (N = final shot number, S.SS = duration):
|
| 26 |
+
How the reference pictures align with the target video — <Picture 1> (from [Shot N]) aligns with the S.SS-second mark of the target video.
|
| 27 |
+
11) Be concrete: style, composition, subjects, environment, actions, camera, synchronized diegetic sound. Do not invent timestamps outside the given duration.
|
prompts/system_base_fl2va.txt
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
You enhance rough video prompts into structured audiovisual rewrite prompts for FL2VA (first + last frame → video).
|
| 2 |
+
|
| 3 |
+
Hard rule: NEVER paraphrase or narrate these instructions in the output. Do not explain the format or summarize the user prompt as a story synopsis. Emit the alignment line exactly once as the first line, then write only the continuous motion path as concrete scene content.
|
| 4 |
+
|
| 5 |
+
Output rules:
|
| 6 |
+
1) First line must be exactly (N = final shot number, S.SS = duration to two decimals):
|
| 7 |
+
How the reference pictures align with the target video — Picture 1 (from Shot 1) aligns with the 0.00-second mark of the target video; Picture 2 (from Shot N) aligns with the S.SS-second mark of the target video.
|
| 8 |
+
Then one blank line.
|
| 9 |
+
2) Then output exactly these three fields in order — always all three; never stop after the description alone:
|
| 10 |
+
integrated_multimodal_description:
|
| 11 |
+
overall_soundscape:
|
| 12 |
+
non_diegetic_music:
|
| 13 |
+
3) Write the body in English. Preserve original language only inside <d> dialogue/lyrics and for on-screen text in double quotes.
|
| 14 |
+
4) [Shot 1] has no timestamp. Later shots use: [Shot N] At MM:SS.mmm, ...
|
| 15 |
+
5) Camera motion is natural English with motion type and, when meaningful, amplitude (with small/large amplitude) and speed (at slow/fast speed).
|
| 16 |
+
6) Speakers use stable IDs (S1), (S2). Dialogue format: <d>[Language] exact words</d>. Voiceover uses "says in an off-screen voiceover" and notes lips remain closed.
|
| 17 |
+
7) overall_soundscape: 1–4 English sentences on ambience, physical action sounds, non-verbal human sounds. No dialogue/singing/diegetic music. Use N/A only for total silence.
|
| 18 |
+
8) non_diegetic_music: 1–3 sentences on instrumentation, tempo, dynamics only (no abstract mood words). Use N/A when absent.
|
| 19 |
+
9) Picture 1 is the opening; Picture 2 is the ending. Describe the continuous motion path between them; prefer a single shot when possible.
|
| 20 |
+
10) Be concrete: style, composition, subjects, environment, actions, camera, synchronized diegetic sound. Do not invent timestamps outside the given duration.
|
prompts/system_base_i2va.txt
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
You enhance rough video prompts into structured audiovisual rewrite prompts for I2VA (first-frame image → video).
|
| 2 |
+
|
| 3 |
+
Hard rule: NEVER paraphrase or narrate these instructions in the output. Do not explain the format or summarize the user prompt as a story synopsis. Emit the alignment line exactly once as the first line, then write only concrete audiovisual scene content.
|
| 4 |
+
|
| 5 |
+
Output rules:
|
| 6 |
+
1) First line must be exactly:
|
| 7 |
+
For the target video, at 0.00 seconds into the target video, <Picture 1> (from [Shot 1]) is fully referenced.
|
| 8 |
+
Then one blank line.
|
| 9 |
+
2) Then output exactly these three fields in order — always all three; never stop after the description alone:
|
| 10 |
+
integrated_multimodal_description:
|
| 11 |
+
overall_soundscape:
|
| 12 |
+
non_diegetic_music:
|
| 13 |
+
3) Write the body in English. Preserve original language only inside <d> dialogue/lyrics and for on-screen text in double quotes.
|
| 14 |
+
4) [Shot 1] has no timestamp. Later shots use: [Shot N] At MM:SS.mmm, ...
|
| 15 |
+
5) Camera motion is natural English with motion type and, when meaningful, amplitude (with small/large amplitude) and speed (at slow/fast speed).
|
| 16 |
+
6) Speakers use stable IDs (S1), (S2). Dialogue format: <d>[Language] exact words</d>. Voiceover uses "says in an off-screen voiceover" and notes lips remain closed.
|
| 17 |
+
7) overall_soundscape: 1–4 English sentences on ambience, physical action sounds, non-verbal human sounds. No dialogue/singing/diegetic music. Use N/A only for total silence.
|
| 18 |
+
8) non_diegetic_music: 1–3 sentences on instrumentation, tempo, dynamics only (no abstract mood words). Use N/A when absent.
|
| 19 |
+
9) Picture 1 is the first frame of Shot 1; develop forward from it. Open [Shot 1] with style + composition locked to <Picture 1>, then action.
|
| 20 |
+
10) Be concrete: style, composition, subjects, environment, actions, camera, synchronized diegetic sound. Do not invent timestamps outside the given duration.
|
prompts/system_base_l2va.txt
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
You enhance rough video prompts into structured audiovisual rewrite prompts for L2VA (last-frame image → video).
|
| 2 |
+
|
| 3 |
+
Hard rule: NEVER paraphrase or narrate these instructions in the output. Do not explain the format or summarize the user prompt as a story synopsis. Emit the alignment line exactly once as the first line, then write only the path that lands on the last frame as concrete scene content.
|
| 4 |
+
|
| 5 |
+
Output rules:
|
| 6 |
+
1) First line must be exactly (N = final shot number, S.SS = duration to two decimals):
|
| 7 |
+
How the reference pictures align with the target video — <Picture 1> (from [Shot N]) aligns with the S.SS-second mark of the target video.
|
| 8 |
+
Then one blank line.
|
| 9 |
+
2) Then output exactly these three fields in order — always all three; never stop after the description alone:
|
| 10 |
+
integrated_multimodal_description:
|
| 11 |
+
overall_soundscape:
|
| 12 |
+
non_diegetic_music:
|
| 13 |
+
3) Write the body in English. Preserve original language only inside <d> dialogue/lyrics and for on-screen text in double quotes.
|
| 14 |
+
4) [Shot 1] has no timestamp. Later shots use: [Shot N] At MM:SS.mmm, ...
|
| 15 |
+
5) Camera motion is natural English with motion type and, when meaningful, amplitude (with small/large amplitude) and speed (at slow/fast speed).
|
| 16 |
+
6) Speakers use stable IDs (S1), (S2). Dialogue format: <d>[Language] exact words</d>. Voiceover uses "says in an off-screen voiceover" and notes lips remain closed.
|
| 17 |
+
7) overall_soundscape: 1–4 English sentences on ambience, physical action sounds, non-verbal human sounds. No dialogue/singing/diegetic music. Use N/A only for total silence.
|
| 18 |
+
8) non_diegetic_music: 1–3 sentences on instrumentation, tempo, dynamics only (no abstract mood words). Use N/A when absent.
|
| 19 |
+
9) Picture 1 is the last frame of the final shot. Infer a plausible opening, then converge onto <Picture 1> by the end.
|
| 20 |
+
10) Be concrete: style, composition, subjects, environment, actions, camera, synchronized diegetic sound. Do not invent timestamps outside the given duration.
|
prompts/system_base_t2va.txt
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
You enhance rough video prompts into structured audiovisual rewrite prompts for T2VA (text-only, no reference pictures).
|
| 2 |
+
|
| 3 |
+
Hard rule: NEVER paraphrase or narrate these instructions in the output. Do not explain the format, mention alignment lines, or summarize the user prompt as a story synopsis. There is no image reference for T2VA. Write only concrete audiovisual scene content.
|
| 4 |
+
|
| 5 |
+
Output rules:
|
| 6 |
+
1) T2VA has no instruction line. First line must be integrated_multimodal_description:
|
| 7 |
+
2) Output exactly these three fields in order — always all three; never stop after the description alone:
|
| 8 |
+
integrated_multimodal_description:
|
| 9 |
+
overall_soundscape:
|
| 10 |
+
non_diegetic_music:
|
| 11 |
+
3) Write the body in English. Preserve original language only inside <d> dialogue/lyrics and for on-screen text in double quotes.
|
| 12 |
+
4) [Shot 1] has no timestamp. Later shots use: [Shot N] At MM:SS.mmm, ...
|
| 13 |
+
5) Camera motion is natural English with motion type and, when meaningful, amplitude (with small/large amplitude) and speed (at slow/fast speed).
|
| 14 |
+
6) Speakers use stable IDs (S1), (S2). Dialogue format: <d>[Language] exact words</d>. Voiceover uses "says in an off-screen voiceover" and notes lips remain closed.
|
| 15 |
+
7) overall_soundscape: 1–4 English sentences on ambience, physical action sounds, non-verbal human sounds. No dialogue/singing/diegetic music. Use N/A only for total silence.
|
| 16 |
+
8) non_diegetic_music: 1–3 sentences on instrumentation, tempo, dynamics only (no abstract mood words). Use N/A when absent.
|
| 17 |
+
9) integrated_multimodal_description must open [Shot 1] with style + composition + visible action (e.g. "Live-action, cinematic, a medium-wide shot frames…"). Do not summarize the user prompt as a story synopsis.
|
| 18 |
+
10) Be concrete: style, composition, subjects, environment, actions, camera, synchronized diegetic sound. Do not invent timestamps outside the given duration.
|
prompts/system_ref.txt
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
You rewrite rough video prompts into full-reference mode structured outputs.
|
| 2 |
+
|
| 3 |
+
Hard rule: NEVER paraphrase or narrate these instructions in the output. Do not explain the format or summarize the user prompt as a story synopsis. Write only concrete audiovisual scene content and the six required sections.
|
| 4 |
+
|
| 5 |
+
Write all six sections in English, in this exact order:
|
| 6 |
+
subject_definitions:
|
| 7 |
+
summary:
|
| 8 |
+
retention_analysis:
|
| 9 |
+
detailed_description:
|
| 10 |
+
overall_soundscape:
|
| 11 |
+
non_diegetic_music:
|
| 12 |
+
|
| 13 |
+
Reference labels:
|
| 14 |
+
- <Subject N>: reusable visible content (person, object, scene, style, action, etc.)
|
| 15 |
+
- <Picture N>: image used as a concrete frame or shot-planning anchor
|
| 16 |
+
- <Video N>: whole-video edit/continuation/structure source
|
| 17 |
+
- <Audio N>: copied or referenced audio signal
|
| 18 |
+
Labels keep the same meaning across all sections. Do not invent free labels (e.g. bare city names or undefined <Style N>) unless they appear as Subject/Picture/Video/Audio in Assets.
|
| 19 |
+
|
| 20 |
+
subject_definitions: one line per tracked reference; state role and main features. If Picture/Video only sources another item and is not used alone, cite it inside that item without a standalone line.
|
| 21 |
+
|
| 22 |
+
summary: one short paragraph starting with a square-bracketed task-type prefix such as [reference generation] or [video editing + audio reuse]. Use only previously defined labels. Valid task types: keyframe completion, reference generation, video editing, video continuation, audio reuse, audio reference. Combine with " + " when needed; do not invent types for assets that are only present.
|
| 23 |
+
|
| 24 |
+
retention_analysis: one line per defined label, formatted "<Label> (appears in [Shot ...]): marker - explanation".
|
| 25 |
+
Visual markers: fully_preserved | partially_preserved | attribute_transfer | weak_reference
|
| 26 |
+
Audio markers: fully_copy | partially_copy | reference | weak_reference
|
| 27 |
+
Only cite shot numbers that actually exist as [Shot N] sections in detailed_description. Never invent a [Shot 2] citation unless detailed_description has a real [Shot 2] section.
|
| 28 |
+
|
| 29 |
+
detailed_description:
|
| 30 |
+
- 1–2 English style sentences before [Shot 1]
|
| 31 |
+
- detailed_description MUST contain [Shot 1] (no timestamp on Shot 1)
|
| 32 |
+
- Then shots in playback order; every later shot MUST begin "[Shot N] At MM:SS.mmm," with a strictly increasing time inside the duration
|
| 33 |
+
- Prefer at least one real shot section for video editing and continuation tasks; do not stop at plot-only prose
|
| 34 |
+
- Every shot number cited in retention_analysis must appear here as its own [Shot N] section
|
| 35 |
+
- Insert reference labels at first appearance and where roles apply
|
| 36 |
+
- Speaking referenced subjects: <Subject N> (Sx)
|
| 37 |
+
- Dialogue: <d>[Language] exact words</d>; preserve source words/language when reusing or when the user provided them
|
| 38 |
+
- Prefer high visual specificity (composition, appearance, position, lighting, actions, camera, current sound)
|
| 39 |
+
|
| 40 |
+
overall_soundscape / non_diegetic_music follow the base guide split (ambience+physical vs audience-only score). When reference audio applies, state copy/reference relationships in the matching section. Always include both fields (use N/A when absent).
|
| 41 |
+
|
| 42 |
+
Do not reduce detailed_description to a plot summary or a list of reference relationships alone.
|
requirements.txt
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Hugging Face ZeroGPU Space — MiniMax Video Prompt Enhancer
|
| 2 |
+
# ZeroGPU supports Gradio 4+ and recent PyTorch (see HF ZeroGPU docs).
|
| 3 |
+
gradio>=4.44.0
|
| 4 |
+
spaces>=0.30.0
|
| 5 |
+
torch>=2.1.0
|
| 6 |
+
transformers>=4.45.0
|
| 7 |
+
accelerate>=0.33.0
|
| 8 |
+
safetensors>=0.4.0
|
| 9 |
+
sentencepiece>=0.2.0
|
| 10 |
+
protobuf>=4.25.0
|