Add Aurora editor checkpoint + agent LoRA adapter

README.md

@@ -0,0 +1,85 @@
+---
+license: mit
+tags:
+  - video-editing
+  - video-generation
+  - diffusion
+  - wan
+  - lora
+  - agent
+pipeline_tag: video-to-video
+---
+
+# Aurora — Model Weights
+
+Pretrained weights for *"Aurora: Unified Video Editing with a Tool-Using Agent"*
+([arXiv:2605.18748](https://arxiv.org/abs/2605.18748)).
+Code: [github.com/yeates/Aurora](https://github.com/yeates/Aurora) ·
+Project page: [yeates.github.io/Aurora-Page](https://yeates.github.io/Aurora-Page)
+
+This repository bundles the two trained Aurora components, laid out to drop
+straight into the code repository's `models/` directory:
+
+```bash
+huggingface-cli download yeates/aurora-weights --local-dir models
+```
+
+## Contents
+
+| Path | Component | Notes |
+|---|---|---|
+| `aurora_editor.safetensors` | Video editor | trained `dit` + `mllm.context_projector` + `ref_vae_condition` (~9.4 GB, bf16) |
+| `aurora_agent_vlm/` | Agent planner adapter | PEFT LoRA (`r=32`, `alpha=64`) on `Qwen/Qwen3-VL-8B-Instruct` |
+
+### Editor — `aurora_editor.safetensors`
+
+A **partial checkpoint** containing only the trained Aurora modules:
+
+- `dit.*` — the WAN2.2-TI2V-5B diffusion transformer (fine-tuned)
+- `mllm.context_projector.*` — projects frozen Qwen3.5-4B hidden states into DiT width
+- `ref_vae_condition.*` — multi-reference conditioning with per-reference index embedding
+
+It is loaded **on top of the frozen backbones** (WAN2.2-TI2V-5B + WAN2.2 VAE +
+Qwen3.5-4B), not standalone. One checkpoint covers source-conditioned (s2v),
+video-to-video (v2v), and reference-conditioned (sv2v) editing.
+
+### Agent adapter — `aurora_agent_vlm/`
+
+PEFT LoRA adapter for the tool-using planner: base `Qwen/Qwen3-VL-8B-Instruct`,
+`r=32`, `lora_alpha=64`, on the attention + MLP projections. `adapter_config.json`
+records `base_model_name_or_path = Qwen/Qwen3-VL-8B-Instruct`.
+
+## Usage
+
+After downloading into `models/` and installing the code repository:
+
+```python
+# Editor
+from evaluation.pipeline_loader import load_v2_pipeline
+pipe = load_v2_pipeline("models/aurora_editor.safetensors", device="cuda:0", ref_max_items=8)
+
+# Agent planner (LoRA merged at load)
+import aurora.agent
+agent = aurora.agent.AgentVLM("models/Qwen3-VL-8B-Instruct", "models/aurora_agent_vlm", device="cuda:0")
+```
+
+You also need the frozen backbones under `models/` (WAN2.2-TI2V-5B,
+`Wan2.2_VAE.pth`, Qwen3.5-4B, Qwen3-VL-8B-Instruct) — see the code repository's
+Model Zoo. The full inference recipe (3-pass CFG defaults, per-benchmark
+commands) is in the repository README.
+
+## License
+
+MIT (Aurora weights). The WAN2.2-TI2V-5B / WAN2.2 VAE / Qwen3.5-4B /
+Qwen3-VL-8B-Instruct backbones carry their own respective licenses.
+
+## Citation
+
+```bibtex
+@article{yu2026aurora,
+  title={Aurora: Unified Video Editing with a Tool-Using Agent},
+  author={Yu, Yongsheng and Zeng, Ziyun and Xiao, Zhiyuan and Zhou, Zhenghong and Hua, Hang and Xiong, Wei and Luo, Jiebo},
+  journal={arXiv preprint arXiv:2605.18748},
+  year={2026}
+}
+```

aurora_agent_vlm/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-VL-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "k_proj",
+    "gate_proj",
+    "q_proj",
+    "up_proj",
+    "v_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

aurora_agent_vlm/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4517e62b5bedf6944f25dfc24fe942c0446d0d427a553c5d9001785eb4b76f01
+size 349251312

aurora_editor.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e5b6577c3d65d3b5527beabf910113058a91aa4eef0110b3a5305dc4325caf5
+size 10035514032