Update README

README.md

@@ -4,7 +4,7 @@ language:
 - en
 library_name: transformers
 pipeline_tag: video-text-to-text
-base_model: OpenMOSS-Team/MOSS-VL-Base-0708
+base_model: OpenMOSS-Team/MOSS-VL-Base
 tags:
 - moss_vl
 - feature-extraction
@@ -24,15 +24,15 @@ tags:
   <img src="assets/logo.png" width="300" alt="MOSS-VL"/>
 </p>
 
-# MOSS-VL-Realtime-0708
+# MOSS-VL-Realtime
 
 ## Introduction
 
-MOSS-VL-Realtime-0708 is the realtime streaming checkpoint of the MOSS-VL 0708 release, part of the OpenMOSS ecosystem for open visual understanding.
+MOSS-VL-Realtime is the realtime streaming checkpoint of the MOSS-VL release, part of the OpenMOSS ecosystem for open visual understanding.
 
-Unlike offline video-language models that first read a complete video and then answer, MOSS-VL-Realtime-0708 is designed for continuous video streams. It can perceive incoming frames and generate text in parallel, support multi-turn realtime interaction, answer questions at arbitrary moments in the stream, and decide whether to speak or keep observing when the visual evidence is not yet sufficient.
+Unlike offline video-language models that first read a complete video and then answer, MOSS-VL-Realtime is designed for continuous video streams. It can perceive incoming frames and generate text in parallel, support multi-turn realtime interaction, answer questions at arbitrary moments in the stream, and decide whether to speak or keep observing when the visual evidence is not yet sufficient.
 
-The 0708 release keeps the MOSS-VL cross-attention design and a 256K text context window while adding realtime streaming data and an inference interface for timestamped frame-by-frame input.
+This release keeps the MOSS-VL cross-attention design and a 256K text context window while adding realtime streaming data and an inference interface for timestamped frame-by-frame input.
 
 ## Highlights
 
@@ -40,11 +40,11 @@ The 0708 release keeps the MOSS-VL cross-attention design and a 256K text contex
 - Proactive silence: the model can emit `<|silence|>` and continue observing when no important visual event has happened or the context is insufficient.
 - Dynamic correction: as new frames arrive, the model can update or correct earlier responses instead of being locked to an initial interpretation.
 - Native timestamped frames: each streamed frame is associated with an absolute timestamp, helping the model reason about event order, duration, pacing, and fine-grained temporal localization.
-- Open MOSS-VL family: released together with MOSS-VL-Instruct-0708 and MOSS-VL-Base-0708 for offline use, continued pretraining, fine-tuning, and applied research.
+- Open MOSS-VL family: released together with MOSS-VL-Instruct and MOSS-VL-Base for offline use, continued pretraining, fine-tuning, and applied research.
 
 ## Realtime Interaction
 
-MOSS-VL-Realtime-0708 exposes a model-owned streaming loop through `create_realtime_session(...)`, `online_generate(...)`, and `real_time_generate(...)`.
+MOSS-VL-Realtime exposes a model-owned streaming loop through `create_realtime_session(...)`, `online_generate(...)`, and `real_time_generate(...)`. Applications should normally use `create_realtime_session(...)` for direct session control or `online_generate(...)` for queue-based backends.
 
 The recommended deployment API is `create_realtime_session(...)`. A service or application owns the video capture pipeline, converts camera, screen, or video-file input into PIL-compatible frames, and pushes each frame with a non-decreasing timestamp:
 
@@ -53,11 +53,13 @@ The recommended deployment API is `create_realtime_session(...)`. A service or a
 - `session.push_prompt_frame(prompt, image, timestamp=...)` aligns a prompt with a specific frame.
 - `session.poll_output(...)` or `session.stream_outputs(...)` returns incremental text chunks.
 
+`system_prompt` and `initial_prompt` are tokenized as the initial system/user turns before the first frame arrives. Subsequent user turns can be appended with `push_prompt(...)` while the same session continues observing frames.
+
 For backend systems that already use queues, `online_generate(...)` accepts dictionaries containing frames, prompts, events, reset controls, and stop controls.
 
 ## Model Architecture
 
-MOSS-VL-Realtime-0708 adopts a cross-attention-based vision-language architecture that decouples visual encoding from language reasoning. This design is important for realtime usage because incoming visual content can be integrated into the running generation context without forcing the model into a strictly offline "load all frames, then answer" workflow.
+MOSS-VL-Realtime adopts a cross-attention-based vision-language architecture that decouples visual encoding from language reasoning. This design is important for realtime usage because incoming visual content can be integrated into the running generation context without forcing the model into a strictly offline "load all frames, then answer" workflow.
 
 <p align="center">
   <img src="assets/architecture.png" alt="MOSS-VL Architecture" width="100%"/>
@@ -87,13 +89,13 @@ MOSS-VL uses Cross-attention Rotary Position Embedding (XRoPE), which maps text
 
 ## Model Performance
 
-MOSS-VL-Realtime-0708 is designed for streaming video understanding benchmarks where questions can arrive before a full video has been observed and correct answers may change as the scene evolves. It targets realtime interaction quality, proactive silence, and dynamic response updates in addition to standard video understanding accuracy.
+MOSS-VL-Realtime is designed for streaming video understanding benchmarks where questions can arrive before a full video has been observed and correct answers may change as the scene evolves. It targets realtime interaction quality, proactive silence, and dynamic response updates in addition to standard video understanding accuracy.
 
 <p align="center">
   <img src="assets/benchmark-streaming.png" alt="MOSS-VL Streaming Benchmark" width="100%"/>
 </p>
 
-Detailed benchmark tables and comparisons for the 0708 release will be maintained in the MOSS-VL project resources.
+Detailed benchmark tables and comparisons for this release will be maintained in the MOSS-VL project resources.
 
 ## Quickstart
 
@@ -115,7 +117,7 @@ pip install -i https://pypi.org/simple --no-build-isolation -r requirements.txt
 import torch
 from transformers import AutoModelForCausalLM, AutoProcessor
 
-checkpoint = "OpenMOSS-Team/MOSS-VL-Realtime-0708"
+checkpoint = "OpenMOSS-Team/MOSS-VL-Realtime"
 
 processor = AutoProcessor.from_pretrained(
     checkpoint,
@@ -177,8 +179,13 @@ try:
 
     session.push_prompt("What changed in the latest frames?")
 
-    for chunk in session.stream_outputs(poll_interval=0.1):
-        print(chunk, end="", flush=True)
+    # Realtime sessions stay alive waiting for future input, so use a bounded
+    # drain window and close the session explicitly when the producer is done.
+    drain_deadline = time.monotonic() + 5.0
+    while time.monotonic() < drain_deadline:
+        chunk = session.poll_output(timeout=0.1)
+        if chunk is not None:
+            print(chunk, end="", flush=True)
 finally:
     session.close()
 ```
@@ -231,9 +238,13 @@ worker.join()
 
 Each queue item can contain `frame` or `image`, `timestamp`, `prompt`, `frames`, `event`, `events`, `initial_prompt`, `system_prompt`, `generate_kwargs`, `reset_session`, or stop controls such as `stop_online_generate`.
 
+### Realtime Inference Package
+
+The release bundle includes `../realtime_inference/run_online_inference.py`. It provides command-line video replay, camera and screen sources, training-format JSONL input, and a FastAPI WebSocket service built on the model APIs above. See `../realtime_inference/README.md` for deployment commands and the WebSocket protocol.
+
 ### Offline Inference Compatibility
 
-MOSS-VL-Realtime-0708 also keeps the offline helper APIs for image and video prompts. For purely offline use, MOSS-VL-Instruct-0708 is usually the preferred checkpoint, but the realtime checkpoint can still process complete image and video inputs.
+MOSS-VL-Realtime also keeps the offline helper APIs for image and video prompts. For purely offline use, MOSS-VL-Instruct is usually the preferred checkpoint, but the realtime checkpoint can still process complete image and video inputs.
 
 <details>
 <summary><b>Single-video Offline Inference</b></summary>
@@ -334,15 +345,15 @@ print(texts)
 
 | Model | Parameters | Context | Usage | Hugging Face |
 | --- | ---: | ---: | --- | --- |
-| MOSS-VL-Realtime-0708 | 11B | 256K | Realtime streaming video interaction | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Realtime-0708 |
-| MOSS-VL-Instruct-0708 | 11B | 256K | Offline multimodal instruction following | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct-0708 |
-| MOSS-VL-Base-0708 | 11B | 256K | Continued pretraining and fine-tuning | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Base-0708 |
+| MOSS-VL-Realtime | 11B | 256K | Realtime streaming video interaction | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Realtime |
+| MOSS-VL-Instruct | 11B | 256K | Offline multimodal instruction following | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct |
+| MOSS-VL-Base | 11B | 256K | Continued pretraining and fine-tuning | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Base |
 | MOSS-VL-Instruct-0408 | 11B | 256K | Previous instruction-tuned checkpoint | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct-0408 |
 | MOSS-VL-Base-0408 | 11B | 256K | Previous base checkpoint | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Base-0408 |
 
 ## Limitations and Future Work
 
-MOSS-VL-Realtime-0708 is optimized for timestamped frame-by-frame streaming, but production latency depends on GPU hardware, frame sampling rate, transport overhead, and decoding speed. One model instance supports one active realtime session. The default frame queue bounds latency by dropping older pending frames when needed.
+MOSS-VL-Realtime is optimized for timestamped frame-by-frame streaming, but production latency depends on GPU hardware, frame sampling rate, transport overhead, and decoding speed. One model instance supports one active realtime session. The default frame queue bounds latency by dropping older pending frames when needed.
 
 The model may emit realtime control tokens such as `<|silence|>`, `<|round_start|>`, and `<|round_end|>` depending on the application protocol. Downstream services should filter or render these tokens according to their UI needs.