Add metadata and link to paper/code

README.md

@@ -1,5 +1,15 @@
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: image-feature-extraction
+---
+
 # OneVision-Encoder
 
+OneVision-Encoder is an LLM-aligned vision transformer specifically optimized for Large Multimodal Models (LMMs). It is a core component of the [LLaVA-OneVision-2](https://huggingface.co/papers/2605.25979) series and is further detailed in the technical report [OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligence](https://arxiv.org/abs/2602.08683).
+
+[**Project Page**](https://evolvinglmms-lab.github.io/LLaVA-OneVision-2/) | [**GitHub**](https://github.com/EvolvingLMMs-Lab/LLaVA-OneVision-2)
+
 ### Key Features
 
 - **LLM-Aligned Architecture**: Unlike standard vision backbones, this model is specifically optimized for **Large Multimodal Models (LMMs)**, ensuring seamless feature alignment and superior performance when connected to language models.
@@ -75,57 +85,12 @@ w_positions = torch.arange(grid_w, device="cuda").repeat(grid_h)
 w_positions = w_positions.repeat(num_frames)  # [T * frame_tokens]
 
 patch_positions = torch.stack([t_positions, h_positions, w_positions], dim=-1).unsqueeze(0)
-# patch_positions example (256 tokens per frame, 16x16 patch grid):
-#   Each row is [t, h, w].
-#   First 4 patches of frame 0 (t=0):
-#     patch_positions[0, 0:4, :] -> [[0, 0, 0], [0, 0, 1], [0, 0, 2], [0, 0, 3]]
-#   First 4 patches of frame 1 (t=4):
-#     patch_positions[0, 256:260, :] -> [[4, 0, 0], [4, 0, 1], [4, 0, 2], [4, 0, 3]]
 
 with torch.no_grad():
   outputs = model(video, patch_positions=patch_positions)
-
 ```
 
-### Loading from Source Code
-
-```bash
-git clone [https://github.com/EvolvingLMMs-Lab/OneVision-Encoder.git](https://github.com/EvolvingLMMs-Lab/OneVision-Encoder.git)
-cd OneVision-Encoder
-pip install -e .
-
-```
-
-```python
-from onevision_encoder import OneVisionEncoderModel, OneVisionEncoderConfig
-from transformers import AutoImageProcessor
-model = OneVisionEncoderModel.from_pretrained(
-    "lmms-lab-encoder/onevision-encoder-large-lang",
-    trust_remote_code=True,
-    attn_implementation="flash_attention_2"
-).to("cuda").eval()
-preprocessor = AutoImageProcessor.from_pretrained(
-    "lmms-lab-encoder/onevision-encoder-large-lang",
-    trust_remote_code=True
-)
-
-```
-
-### LMM Probe Results
-
-Training on a mixed dataset of 740K samples from LLaVA-OneVision and 800K samples from LLaVA-Video SFT. The training pipeline proceeds directly to Stage 2 fine-tuning.
-
-We adopt a streamlined **native-resolution strategy** inspired by LLaVA-OneVision: when the input frame resolution matches the model's native input size, it is fed **directly**—without tiling or cropping—to evaluate the ViT's capability to handle **true native resolution** and **arbitrary frame sequences**.
-
-<p align="center">
-<picture>
-<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/anxiangsir/asset/main/OneVision/probe_lmm_github_dark_fixed.png">
-<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/anxiangsir/asset/main/OneVision/probe_lmm_github_light.png">
-<img alt="LMM Probe Results" src="https://raw.githubusercontent.com/anxiangsir/asset/main/OneVision/probe_lmm_github_light.png" width="800" style="max-width: 100%;">
-</picture>
-</p>
-
-### Model Card
+### Model Properties
 
 | Property | Value |
 | --- | --- |
@@ -143,3 +108,21 @@ We adopt a streamlined **native-resolution strategy** inspired by LLaVA-OneVisio
 | **Normalization** | Layer Normalization |
 | **Activation Function** | GELU |
 | **License** | Apache 2.0 |
+
+### Citation
+
+```bibtex
+@inproceedings{LLaVA-OneVision-2,
+  title={LLaVA-OneVision-2},
+  author={llava-onevision contributors},
+  booktitle={arXiv},
+  year={2026}
+}
+
+@article{tang2026onevisionencoder,
+  title={OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligence},
+  author={Tang, Feilong and An, Xiang and Yan, Yunyao and Xie, Yin and Qin, Bin and Yang, Kaicheng and Shen, Yifei and Zhang, Yuanhan and Li, Chunyuan and Feng, Shikun and Chen, Changrui and Tan, Huajie and Hu, Ming and Zhang, Manyuan and Li, Bo and Feng, Ziyong and Liu, Ziwei and Ge, Zongyuan and Deng, Jiankang},
+  journal={arXiv preprint arXiv:2602.08683},
+  year={2026}
+}
+```