README.md

---
license: apache-2.0
base_model: Qwen/Qwen3.5-4B
library_name: transformers
pipeline_tag: video-text-to-text
tags:
  - streaming-video
  - video-understanding
  - on-policy-distillation
  - knowledge-distillation
  - qwen3_5
  - arxiv:2608.16320
language:
  - en
---

# StreamOPD-4B-ST-CueGate

A 4B streaming video-understanding model, post-trained from **Qwen3.5-4B** with on-policy
distillation from a **Qwen3.5-9B** teacher under **ST-CueGate**, the spatio-temporal cue
gating method from *StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for
Streaming Video Understanding*.

- **Paper:** https://arxiv.org/abs/2608.16320
- **Code:** https://github.com/UniX-AI-Lab/StreamOPD
- **Project page:** https://unix-ai-lab.github.io/StreamOPD/

## What is different about this model

The model is trained for a deliberately austere streaming setting: at answer time it sees
only the **4 most recent frames at 1 fps** — no memory bank, no retrieval, no KV-cache
compression, and no generated reasoning trace. All of the capability lives in the weights
rather than in test-time machinery.

Training uses on-policy distillation in *thinking* mode, while deployment is in *instruct*
mode. During training the frozen teacher scores the student's own response twice — once
conditioned on a grounded spatio-temporal cue and once without it — and the resulting
per-token likelihood ratio is aggregated into a response-level weight that gates the
distillation advantage. The cue is a training-time signal only: it is never present at
inference, so the deployment path is identical to a plain Qwen3.5-4B.

## Results

Evaluated in instruct mode with greedy decoding under the recent-4-frame protocol.
StreamingBench and OVO-Bench use recent-4 frames at 1 fps; Video-MME and LongVideoBench use
their standard protocols with at most 32 frames.

| Model | StreamingBench | OVO-Bench (excl. HLD) | Video-MME | LongVideoBench |
|-------|:---:|:---:|:---:|:---:|
| Qwen3.5-4B (untrained) | 77.87 | 59.94 | 64.22 | 57.74 |
| Qwen3.5-9B (teacher) | 84.15 | — | — | — |
| **This checkpoint** | **84.19** | **70.48** | **64.85** | **60.36** |

The full comparison, including per-subtask OVO-Bench breakdowns, the teacher-conditioning
ablations, and the self-distillation variant, is in the paper.

## Usage

```python
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model_id = "UniX-Lab/StreamOPD-4B-ST-CueGate"
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id, torch_dtype="bfloat16", device_map="auto"
).eval()
processor = AutoProcessor.from_pretrained(model_id)

messages = [{
    "role": "user",
    "content": [
        {"type": "video", "video": "clip.mp4"},
        {"type": "text", "text": "What did the person pick up?\nA. book  B. cup  C. apple\n"
                                 "Only give the best option's letter directly."},
    ],
}]

inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to(model.device)

out = model.generate(**inputs, max_new_tokens=32, do_sample=False)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

Run with **thinking disabled** — the model is trained to answer directly, and the reported
scores use instruct-mode greedy decoding. Set `FORCE_QWENVL_VIDEO_READER=decord` for video
decoding.

To reproduce the streaming evaluation exactly, use the evaluators in the code repository,
which implement the recent-window protocol and the OVO-Bench B+R macro:

```bash
bash scripts/eval/run_all.sh <path-to-this-model> my_run 0,1,2,3
bash scripts/eval/score_all.sh my_run
```

## Training

| | |
|---|---|
| Student | Qwen3.5-4B |
| Teacher | Qwen3.5-9B (frozen) |
| Objective | on-policy distillation, sampled-token k1 reverse KL via policy gradient |
| Gating | ST-CueGate, α=0.5, gate range [0, 2], UID sibling normalization, response-level |
| Rollouts | n=4 |
| Data | 25,118 verifiable video QA items (multiple choice / binary / counting) |
| Mode | trained in thinking mode, deployed in instruct mode |

Training data is derived from public sources (LLaVA-Video-178K and Kinetics-700 clips); the
data pipeline, the parquets, and the cue-generation tooling are in the code repository.

## Limitations

- Answers come from a 4-frame window, so questions needing evidence outside that window are
  out of scope by construction.
- Distillation from a larger teacher reduces the willingness to abstain on unanswerable
  queries (OVO-Bench's HLD subtask) relative to the untrained base model. The paper reports
  a self-distillation variant that recovers this.
- Trained and evaluated on English data.

## Citation

```bibtex
@article{wu2026streamopd,
  title   = {StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating
             for Streaming Video Understanding},
  author  = {Wu, Keming and Wang, Baoyi and Zhang, Kaichen and An, Xiang and
             Yang, Zuhao and Wang, Sudong and Zhu, Haowei and Huang, Tingxuan and
             Gao, Hongcheng and Wang, Bin},
  journal = {arXiv preprint arXiv:2608.16320},
  year    = {2026}
}
```