Add files using upload-large-folder tool

.gitattributes

@@ -34,3 +34,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text
+assets/benchmark-offline.png filter=lfs diff=lfs merge=lfs -text
+assets/mossvl_quantization_benchmark_comparison_zh_4k.png filter=lfs diff=lfs merge=lfs -text
+assets/mossvl_quantization_benchmark_comparison_en_4k.png filter=lfs diff=lfs merge=lfs -text
+assets/architecture.png filter=lfs diff=lfs merge=lfs -text
+assets/logo.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -8,61 +8,174 @@ pipeline_tag: video-text-to-text
 base_model: OpenMOSS-Team/MOSS-VL-Instruct-0708
 tags:
 - MOSS-VL
-- offline
-- quantized
-- nf4
+- image-understanding
+- video-understanding
 - bitsandbytes
+- NF4
+- quantized
 - custom_code
 ---
 
-# OpenMOSS-Team/MOSS-VL-Instruct-0708-NF4
+<p align="center">
+  <img src="assets/logo.png" width="300" alt="MOSS-VL"/>
+</p>
+
+<p align="center">
+  English | <a href="https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct-0708-NF4/blob/main/README_zh.md">中文</a>
+</p>
+
+# MOSS-VL-Instruct-0708 W4A16 NF4
 
-This is the Transformers offline release of MOSS-VL Instruct 0708 with
-bitsandbytes NF4 weights. It supports image and video inference through the
-standard MOSS-VL `offline_generate` path.
+This is the Transformers NF4 release of
+[MOSS-VL-Instruct-0708](https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct-0708).
+It supports image and video inference through the standard MOSS-VL offline
+inference path. This checkpoint is not an SGLang release.
 
-## Quantization
+## Architecture
 
-| Component | Precision |
+<p align="center">
+  <img src="assets/architecture.png" alt="MOSS-VL architecture" width="100%"/>
+</p>
+
+## Quantization profile
+
+| Component | Format |
 | --- | --- |
-| 240 eligible Linear layers in language layers 4-43 | bitsandbytes NF4 weight-only with double quantization; BF16 compute |
+| 240 eligible Linear layers in language layers 4-43 | bitsandbytes NF4 weight-only quantization with double quantization and BF16 compute |
 | First four and last four language layers | BF16 |
 | Cross-attention projection modules | BF16 |
 | Vision encoder and merger | BF16 |
 | Embeddings, norms and `lm_head` | BF16 |
-| Transformers KV cache | BF16 (`KV16`) |
+| Transformers KV cache | BF16 |
+| Attention backend | FlashAttention 2 |
 
-This checkpoint does **not** enable HQQ KV8: `generation_config.json` has no
-quantized-cache setting. The canonical name therefore ends in `KV16`.
+The checkpoint carries its bitsandbytes configuration. Load it directly and
+do not add a second runtime quantization configuration. This variant does not
+enable HQQ KV8; `generation_config.json` uses the standard BF16 KV cache.
 
-## Validated Environment
+## Base model benchmark
 
-```text
-Python:        3.12.8
-torch:         2.8.0+cu128
-transformers:  4.57.1
-bitsandbytes:  0.49.2
-accelerate:    1.12.0
-flash-attn:    2.8.1
-```
+The following figure reports the benchmark results of the original
+MOSS-VL-Instruct-0708 release. It provides base-model context; it is not
+presented as a separate quality evaluation of this NF4 checkpoint.
+
+<p align="center">
+  <img src="assets/benchmark-offline.png" alt="MOSS-VL-Instruct-0708 benchmark" width="100%"/>
+</p>
+
+## Hardware requirements
+
+The validated image test peaked at 12,494 MiB of process VRAM. The 1 FPS,
+maximum-32-frame video test peaked at 16,708 MiB. A single NVIDIA GPU with
+24 GB of VRAM is sufficient for the validated profile.
 
-Use the existing environment without upgrading packages:
+## Environment
+
+### Installation
 
 ```bash
-source /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/venv/bin/activate
+git clone https://github.com/OpenMOSS/MOSS-VL.git
+cd MOSS-VL
+
+conda create -n moss_vl_quant python=3.12 pip -y
+conda activate moss_vl_quant
+pip install -i https://pypi.org/simple --no-build-isolation -r requirements.txt
+pip install -i https://pypi.org/simple bitsandbytes==0.49.2
+python -m pip check
 ```
 
-## Transformers Offline Validation
+Validated core versions:
 
-The official runner is:
+| Package | Version |
+| --- | --- |
+| Python | 3.12.8 |
+| PyTorch | 2.8.0 + CUDA 12.8 |
+| Transformers | 4.57.1 |
+| Accelerate | 1.12.0 |
+| FlashAttention | 2.8.1 |
+| bitsandbytes | 0.49.2 |
+
+Video decoding also requires FFmpeg in `PATH`.
+
+### Load the model
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoProcessor
+
+checkpoint = "OpenMOSS-Team/MOSS-VL-Instruct-0708-NF4"
+
+processor = AutoProcessor.from_pretrained(
+    checkpoint,
+    trust_remote_code=True,
+    frame_extract_num_threads=1,
+)
+model = AutoModelForCausalLM.from_pretrained(
+    checkpoint,
+    trust_remote_code=True,
+    device_map="auto",
+    torch_dtype=torch.bfloat16,
+    attn_implementation="flash_attention_2",
+)
+model.eval()
+```
 
-```text
-/inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/mossvl-github/MOSS-VL/inference/run_inference.py
+## Image inference
+
+```python
+text = model.offline_image_generate(
+    processor,
+    prompt="Describe this image.",
+    image="data/example_image.jpg",
+    shortest_edge=4096,
+    longest_edge=16777216,
+    multi_image_max_pixels=201326592,
+    patch_size=16,
+    temporal_patch_size=1,
+    merge_size=2,
+    image_mean=[0.5, 0.5, 0.5],
+    image_std=[0.5, 0.5, 0.5],
+    max_new_tokens=256,
+    do_sample=False,
+    vision_chunked_length=64,
+)
+print(text)
+```
+
+## Video inference
+
+```python
+text = model.offline_video_generate(
+    processor,
+    prompt="Describe this video.",
+    video="data/example_video.mp4",
+    shortest_edge=4096,
+    longest_edge=16777216,
+    video_max_pixels=201326592,
+    patch_size=16,
+    temporal_patch_size=1,
+    merge_size=2,
+    video_fps=1.0,
+    min_frames=1,
+    max_frames=32,
+    num_extract_threads=4,
+    image_mean=[0.5, 0.5, 0.5],
+    image_std=[0.5, 0.5, 0.5],
+    max_new_tokens=256,
+    do_sample=False,
+    vision_chunked_length=64,
+)
+print(text)
 ```
 
-Direct image validation:
+## Validated reproduction
+
+The official runner passed both the receipt image and the 1 FPS Starbucks
+video tests:
 
 ```bash
+source /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/venv/bin/activate
+
 /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/venv/bin/python \
   /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/mossvl-github/MOSS-VL/inference/run_inference.py \
   --checkpoint /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/quant/MOSS-VL-0708-Offline-NF4-Keep4-KV16 \
@@ -70,11 +183,7 @@ Direct image validation:
   --input /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811/inputs/offline_image.json \
   --output /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811/results/offline_nf4_image_output.json \
   --timeout-seconds 300
-```
-
-Direct video validation:
 
-```bash
 /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/venv/bin/python \
   /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/mossvl-github/MOSS-VL/inference/run_inference.py \
   --checkpoint /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/quant/MOSS-VL-0708-Offline-NF4-Keep4-KV16 \
@@ -84,12 +193,14 @@ Direct video validation:
   --timeout-seconds 300
 ```
 
-The latest validation passed both cases. Peak process memory was 12,494 MiB
-for the image and 16,708 MiB for the 1 FPS, maximum-32-frame video. The outputs
-correctly described the receipt and the Starbucks scene.
-
-Full inputs, memory-monitor commands and raw results:
+Full inputs, commands and raw results:
 
 ```text
 /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811
 ```
+
+## Configuration files
+
+- `config.json`: model and bitsandbytes NF4 configuration.
+- `generation_config.json`: standard generation settings with BF16 KV cache.
+- `modeling_moss_vl.py`: checkpoint-local offline MOSS-VL code.

README_zh.md

@@ -0,0 +1,176 @@
+---
+license: apache-2.0
+language:
+- en
+- zh
+library_name: transformers
+pipeline_tag: video-text-to-text
+base_model: OpenMOSS-Team/MOSS-VL-Instruct-0708
+tags:
+- MOSS-VL
+- image-understanding
+- video-understanding
+- bitsandbytes
+- NF4
+- quantized
+- custom_code
+---
+
+<p align="center">
+  <img src="assets/logo.png" width="300" alt="MOSS-VL"/>
+</p>
+
+<p align="center">
+  <a href="https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct-0708-NF4/blob/main/README.md">English</a> | 中文
+</p>
+
+# MOSS-VL-Instruct-0708 W4A16 NF4
+
+这是 [MOSS-VL-Instruct-0708](https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct-0708)
+的 Transformers NF4 发布版本，支持 MOSS-VL 标准离线图片和视频推理。
+该 checkpoint 不是 SGLang 发布版本。
+
+## 模型架构
+
+<p align="center">
+  <img src="assets/architecture.png" alt="MOSS-VL 模型架构" width="100%"/>
+</p>
+
+## 量化方法
+
+| 模块 | 格式 |
+| --- | --- |
+| 第 4-43 层中的 240 个可量化 Linear | bitsandbytes NF4 weight-only，启用 double quantization，BF16 计算 |
+| 前四层和后四层语言模型层 | BF16 |
+| Cross-attention projection | BF16 |
+| 视觉编码器和 merger | BF16 |
+| Embedding、norm 和 `lm_head` | BF16 |
+| Transformers KV Cache | BF16 |
+| Attention 后端 | FlashAttention 2 |
+
+checkpoint 已包含 bitsandbytes 配置，加载时不要再传入第二份运行时量化配置。
+该版本没有启用 HQQ KV8，`generation_config.json` 使用标准 BF16 KV Cache。
+
+## 基础模型 Benchmark
+
+下图是原始 MOSS-VL-Instruct-0708 的 benchmark，用于说明基础模型能力；
+它不作为该 NF4 checkpoint 的独立精度测评结果。
+
+<p align="center">
+  <img src="assets/benchmark-offline.png" alt="MOSS-VL-Instruct-0708 benchmark" width="100%"/>
+</p>
+
+## 硬件要求
+
+图片实测进程峰值显存为 12,494 MiB；1 FPS、最多 32 帧的视频实测峰值为
+16,708 MiB。单张 24 GB 显存的 NVIDIA GPU 可以运行已验证配置。
+
+## 环境安装
+
+```bash
+git clone https://github.com/OpenMOSS/MOSS-VL.git
+cd MOSS-VL
+
+conda create -n moss_vl_quant python=3.12 pip -y
+conda activate moss_vl_quant
+pip install -i https://pypi.org/simple --no-build-isolation -r requirements.txt
+pip install -i https://pypi.org/simple bitsandbytes==0.49.2
+python -m pip check
+```
+
+已验证的主要环境版本：
+
+| 依赖 | 版本 |
+| --- | --- |
+| Python | 3.12.8 |
+| PyTorch | 2.8.0 + CUDA 12.8 |
+| Transformers | 4.57.1 |
+| Accelerate | 1.12.0 |
+| FlashAttention | 2.8.1 |
+| bitsandbytes | 0.49.2 |
+
+视频解码还需要确保 FFmpeg 已加入 `PATH`。
+
+## 加载模型
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoProcessor
+
+checkpoint = "OpenMOSS-Team/MOSS-VL-Instruct-0708-NF4"
+
+processor = AutoProcessor.from_pretrained(
+    checkpoint,
+    trust_remote_code=True,
+    frame_extract_num_threads=1,
+)
+model = AutoModelForCausalLM.from_pretrained(
+    checkpoint,
+    trust_remote_code=True,
+    device_map="auto",
+    torch_dtype=torch.bfloat16,
+    attn_implementation="flash_attention_2",
+)
+model.eval()
+```
+
+## 图片推理
+
+```python
+text = model.offline_image_generate(
+    processor,
+    prompt="请描述这张图片。",
+    image="data/example_image.jpg",
+    max_new_tokens=256,
+    do_sample=False,
+    vision_chunked_length=64,
+)
+print(text)
+```
+
+## 视频推理
+
+```python
+text = model.offline_video_generate(
+    processor,
+    prompt="请描述这段视频。",
+    video="data/example_video.mp4",
+    video_fps=1.0,
+    min_frames=1,
+    max_frames=32,
+    max_new_tokens=256,
+    do_sample=False,
+    vision_chunked_length=64,
+)
+print(text)
+```
+
+## 完整复测命令
+
+官方 runner 已通过收据图片和 1 FPS 星巴克视频测试：
+
+```bash
+source /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/venv/bin/activate
+
+/inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/venv/bin/python \
+  /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/mossvl-github/MOSS-VL/inference/run_inference.py \
+  --checkpoint /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/quant/MOSS-VL-0708-Offline-NF4-Keep4-KV16 \
+  --mode image \
+  --input /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811/inputs/offline_image.json \
+  --output /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811/results/offline_nf4_image_output.json \
+  --timeout-seconds 300
+
+/inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/venv/bin/python \
+  /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/mossvl-github/MOSS-VL/inference/run_inference.py \
+  --checkpoint /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/quant/MOSS-VL-0708-Offline-NF4-Keep4-KV16 \
+  --mode video \
+  --input /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811/inputs/offline_video.json \
+  --output /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811/results/offline_nf4_video_output.json \
+  --timeout-seconds 300
+```
+
+完整输入、命令和原始结果：
+
+```text
+/inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811
+```