Video-Text-to-Text
Transformers
Safetensors
English
Chinese
moss_vl
feature-extraction
Realtime
Streaming
Video-Understanding
Image-Understanding
MOSS-VL
OpenMOSS
multimodal
video
vision-language
custom_code
Instructions to use OpenMOSS-Team/MOSS-VL-Realtime with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenMOSS-Team/MOSS-VL-Realtime with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("OpenMOSS-Team/MOSS-VL-Realtime", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
Ascend NPU support: device-aware cache release, GIL-friendly idle wait, <=8-D vision processing, BICUBIC declaration
Verified signatureAscend NPU (910B, CANN 9.0.0) support for the realtime checkpoint. CUDA behavior unchanged — every NPU path is guarded. - modeling_moss_vl.py - empty_cache: release NPU HBM after realtime sessions (torch_npu import guarded; the CUDA branch is unchanged) - realtime idle wait: a bare `while True: continue` busy-spin monopolizes the CPython GIL and starves co-resident threads (measured: an in-process ASR decode inflates ~150ms -> 40-80s while a session idles in <|silence|>). Replaced with a 20ms sleep poll. Platform-neutral — CUDA deployments embedding this checkpoint benefit equally. - processing / video_processing: the 10-D vision permute+reshape is routed through CPU on NPU (NPU ops support at most 8-D tensors); CUDA keeps the on-device path untouched. - preprocessor_config.json: declare interpolation=BICUBIC explicitly (NPU has no implicit resize kernel; explicit declaration keeps CUDA outputs identical — pixel_values verified bitwise-equal on identica…5c67b2b
-
Upload MOSS-VL-Realtime-0708 release
Verified signaturef55d7c8