--- library_name: transformers license: apache-2.0 pipeline_tag: image-text-to-text base_model: - Qwen/Qwen3.5-9B tags: - transformers - vllm - video - multimodal - reinforcement-learning - temporal-grounding - object-tracking - video-segmentation - visual-question-answering - spatial-reasoning - qwen3.5 --- # Video-ORA-9B [[🏠 Homepage](https://orarl.github.io/)] [[📖 Arxiv Paper](https://arxiv.org/abs/2608.20492)] [[🤗 Evaluation Data](https://huggingface.co/datasets/OraRL/OraRL-Data/tree/main/OraRL-eval-data)] [[💻 Code](https://github.com/HVision-NKU/OraRL)] We release **Video-ORA-9B**, the 9B model trained with **OraRL (Annotations as Rollouts)** for **unified video understanding**. Starting from [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B), OraRL post-training equips one model to handle seven task families with direct, task-native answers and no chain-of-thought decoding. ## Supported task families - Temporal grounding - Visual tracking - Image and video segmentation - Spatial grounding - Spatial-temporal grounding - Video question answering - Spatial intelligence ## Model summary | Property | Value | | --- | --- | | Architecture | `Qwen3_5ForConditionalGeneration` | | Parameters | 9B | | Weight dtype | BF16 | | Native context length | 262,144 tokens | | Base model | [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) | | Post-training | OraRL annotation-augmented on-policy reinforcement learning | | Tested serving | Transformers 5.5.4 and vLLM 0.19.1 | ## Results ### Dataset-level comparison Dataset-level benchmark matrix comparing Video-ORA with multimodal baselines Video-ORA-9B leads the matched seven-family comparison without chain-of-thought decoding. Best and second-best values are highlighted per row; `†` denotes an original-report value whose frame, prompt, split, or decoding settings may differ. Averages require complete family coverage. See the [OraRL repository](https://github.com/HVision-NKU/OraRL), [project page](https://orarl.github.io/), and [paper](https://arxiv.org/abs/2608.20492) for complete benchmark protocols and source attribution. ## Quick start ### vLLM serving ```bash pip install "vllm==0.19.1" openai vllm serve OraRL/Video-ORA-9B \ --served-model-name Video-ORA-9B \ --port 8000 \ --tensor-parallel-size 1 \ --max-model-len 262144 \ --reasoning-parser qwen3 \ --media-io-kwargs '{"video": {"num_frames": -1}}' \ --limit-mm-per-prompt '{"image": 1, "video": 1}' ``` Reduce `--max-model-len` if KV-cache memory is limited. Increase `--tensor-parallel-size` for multi-GPU serving. Model-weight loading occupies approximately **17.6 GiB** in the tested BF16 vLLM environment; this is not a full peak-memory measurement. Send an OpenAI-compatible video request: ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") response = client.chat.completions.create( model="Video-ORA-9B", messages=[ { "role": "user", "content": [ { "type": "video_url", "video_url": { "url": "https://orarl.github.io/assets/orarl-teaser.mp4" }, }, { "type": "text", "text": "Describe the video and answer the question directly.", }, ], } ], max_tokens=128, temperature=0.0, extra_body={ "top_k": 20, "chat_template_kwargs": {"enable_thinking": False}, "mm_processor_kwargs": {"fps": 2, "do_sample_frames": True}, }, ) print(response.choices[0].message.content) ``` Replace the demo URL with your own accessible video URL. For local files, launch vLLM with an appropriate `--allowed-local-media-path`. ### Transformers server Qwen3.5 requires a recent Transformers version: ```bash pip install "transformers[serving] @ git+https://github.com/huggingface/transformers.git@main" pip install accelerate torchvision pillow transformers serve \ --force-model OraRL/Video-ORA-9B \ --port 8000 \ --continuous-batching ``` The checkpoint also contains the tokenizer, processor configuration, generation configuration, and chat template required by compatible Transformers and vLLM releases. Task-specific prompts and output schemas are documented on the [project page](https://orarl.github.io/). ## Intended use Video-ORA-9B is intended for research on structured image/video perception, benchmark evaluation, and task-specific adaptation. Use direct answer prompts with `enable_thinking=False` to match the reported evaluation protocol. Out-of-scope uses include safety-critical decisions, identity inference, surveillance deployment, or use that violates the licenses or consent requirements of upstream media. ## Training data Training uses public training splits from the task families described in the paper. Evaluation identities, questions, and media anchors are excluded during mixture construction. Dataset and media licenses remain governed by their original sources; no training or benchmark media are distributed with this checkpoint. ## Limitations Video-ORA-9B is a research checkpoint optimized for structured video and spatial-understanding tasks. It may produce malformed task-specific outputs, hallucinate visual details, or inherit limitations and biases from its base model and training data. It has not been validated for safety-critical or high-stakes use. ## License The checkpoint is released under the Apache License 2.0. It is derived from [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B), which is also distributed under Apache 2.0. ## Citation ```bibtex @article{li2026orarl, title = {Annotations as Rollouts: Efficient and Scalable Reinforcement Learning for Video MLLMs}, author = {Li, Yunheng and Mu, Guohong and Li, Hao and Qian, Shengsheng and Zhang, Dingwen and Hou, Qibin and Cheng, Ming-Ming}, journal = {arXiv preprint arXiv:2608.20492}, year = {2026}, url = {https://arxiv.org/abs/2608.20492} } ```