H3-ScriptGen: MiniMax-H3 FL2VA scriptwriting LoRA (Qwen3.5-0.8B base, continue-trained from final adapter)

7dbeac1 verified 3 days ago

6.23 kB

language:
  - en
library_name: peft
pipeline_tag: text-generation
license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B
tags:
  - lora
  - sft
  - trl
  - script-generation
  - minimax-h3
  - video-generation

H3-ScriptGen — MiniMax-H3 FL2VA scriptwriting LoRA

A PEFT LoRA adapter on Qwen/Qwen3.5-0.8B that writes stage/camera directions for MiniMax H3 — one FL2VA scene beat per request, in the exact field structure the H3 video pipeline consumes (ACTION, SHOT, STORYBOARD_PROMPT, H3_MODE: FL2VA, H3_VIDEO_PROMPT, overall_soundscape, non_diegetic_music, DURATION). Each beat maps 1:1 to a storyboard still + one ~5 s H3 FL2V clip.

This is the merged adapter: it was continue-trained from the previous final/ adapter (story craft + TVTropes) on 836 H3-format SFT rows, so it keeps the old screenplay/trope knowledge and adds the MiniMax-H3 prompt format on top. Per the project's own guidance, that continue-train is the "practical merge of skills" (see docs/H3_FORMAT_README.md) — it is not a weighted merge of two separately-trained LoRAs.

Artifact	Description
`adapter_model.safetensors` + `adapter_config.json`	The merged adapter (final, epoch 2) — load with PEFT
`tokenizer_config.json`, `tokenizer.json`, `chat_template.jinja`	Qwen3.5 tokenizer + chat template (from base)
`training_config.json`	Training metadata (`init_from: …/final`, base, hyperparams)
`checkpoint-105/`	Epoch-1 checkpoint (full trainer state, resumable)
`checkpoint-210/`	Epoch-2 checkpoint (== root adapter; full trainer state)
`scripts/`	`train_script_lora_h3.py`, `build_sft_from_scriptlib.py`, SFT dataset (`train_dataset.full.jsonl`, 836 rows) + seed examples
`docs/`	MiniMax H3 prompt guides (`VIDEO_PROMPT_base-en.txt`, `VIDEO_PROMPT_ref-en.txt`), `h3_prompt_format.py` (runtime field builders), `H3_FORMAT_README.md`

Base model

Qwen/Qwen3.5-0.8B (Apache-2.0), 0.8B params, causal LM.
LoRA: r=16, alpha=32, dropout=0.1, target modules q/k/v/o_proj + gate/up/down_proj (193 tensors, 193 = standard PEFT layout).

Training

Setting	Value
Format	`minimax-h3-fl2va-v1` (SFT, chat template)
Init	Continue-train from `models/script-lora/final` adapter
Data	`train_dataset.full.jsonl` — 836 rows from 102 screenplays (`scriptlib`) + TVTropes seeds + hand-written H3 examples
Epochs / steps	2 / 210
Learning rate	1e-4 (cosine decay)
Max seq len	1536
Optimizer	AdamW (non-fused, XPU)
Device	Intel Arc A770 (XPU)

Final metrics (from training log): train_loss 0.7535, final-step mean_token_accuracy 0.8932; token accuracy ranged ~0.86–0.91 over the run. checkpoint-105 (epoch 1) and checkpoint-210 (epoch 2) are both included; the root adapter_model.safetensors is identical to checkpoint-210.

Output format (one beat)

## SCENE {N} — {SLUGLINE}
ACTION: <1–2 sentences of visual action for ~5 s>
DIALOGUE — {NAME}: <line>            (at most 1 line, or omit if silent)
SHOT: <camera type + optional amplitude + speed, natural English>
STORYBOARD_PROMPT: <self-contained still-image prompt; no camera timeline>
H3_MODE: FL2VA
H3_VIDEO_PROMPT:
How the reference pictures align with the target video — Picture 1 (from Shot 1) aligns with the 0.00-second mark of the target video; Picture 2 (from Shot 1) aligns with the 5.00-second mark of the target video.

integrated_multimodal_description: [Shot 1] Live-action, cinematic, <opening composition matching the storyboard>. <continuous motion path Picture 1 → Picture 2; camera motion as natural English>. <dialogue as: the {name} (S1) says: <d>[English] line here</d>>

overall_soundscape: <ambience / physical sounds, or N/A>

non_diegetic_music: <audience-only score, or N/A>
LORA: <image-lora:strength, or "none">
AUDIO: <post-process sfx/music note, or "none">
DURATION: 5

The h3_video_prompt and storyboard_prompt fields feed directly into the MiniMax H3 FL2V pipeline (storyboard panel N → panel N+1, zvideo_h3_storyboard_fl2v).

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3.5-0.8B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16")
model = PeftModel.from_pretrained(model, "woodfireind/H3-ScriptGen")

messages = [
    {"role": "system", "content": "You write ONE MiniMax-H3 FL2VA scene beat for Backlot."},
    {"role": "user", "content": "Premise: A courier delivers a package through a neon alley in the rain.\nWrite SCENE 1 now."},
]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
out = tok.generate(tok(text, return_tensors="pt").input_ids, max_new_tokens=512)
print(tok.decode(out[0]))

The adapter also loads on llama.cpp / vLLM servers that support PEFT LoRA adapters on the same base model.

Limitations

Small base (0.8B) — strong on structure/format adherence; weaker than larger models on nuance, and token accuracy is ~0.9, so occasional malformed fields are expected. Validate output with docs/h3_prompt_format.py (parse_scene_h3_fields).
Text-only. This adapter only produces prompt text. Producing video still requires the MiniMax H3 stack (GGUF DiT + VAE + text encoder); on this project's local stack the H3 pipeline has no audio (audio VAE dropped) and cut timing can drift ±2 s.
H3 prompt rules are exacting. H3_VIDEO_PROMPT must keep the FL2VA alignment line, <d>[Language] …</d> dialogue tags, and speaker (S1) IDs. See docs/VIDEO_PROMPT_base-en.txt.
Training-data provenance. The SFT set was built from an internal 102-screenplay corpus + TVTropes metadata + hand-written examples. Review rights before commercial redistribution of generated content.
The H3 prompt-field structure follows MiniMax's public H3 prompt guides; using it to generate videos is subject to the MiniMax H3 Community License Agreement.

License

Adapter weights are Apache-2.0 (matching the Qwen3.5-0.8B base). Training data is from an internal screenplay corpus — see provenance note above.

woodfireind
/

H3-ScriptGen