Improve model card: Update to RICE paper details, add usage, and refine metadata

This PR significantly updates the model card for the `rice-vit-large-patch14-560` model to accurately reflect its association with the paper **"Region-based Cluster Discrimination for Visual Representation Learning" (RICE)**.

Key changes include:
* Updating the primary paper link to the correct RICE paper ([https://huggingface.co/papers/2507.20025](https://huggingface.co/papers/2507.20025)).
* Adding `library_name: transformers` to enable the "How to use" widget, as the model is compatible with the Transformers library.
* Refining the `pipeline_tag` from `feature-extraction` to `image-feature-extraction` for better discoverability.
* Incorporating comprehensive sections from the official GitHub repository's RICE section, including "Highlights", "Experiments" (with the relevant image, replacing outdated MLCD tables), "How to use" (sample code), "Visualize Semantic Features", "ModelZoo", and "Citation".
* Adding the paper abstract for a detailed overview.

This ensures the model card provides precise and up-to-date information for users.

README.md

@@ -1,94 +1,161 @@
 ---
-license: apache-2.0
 datasets:
 - laion/laion400m
 - kakaobrain/coyo-700m
-pipeline_tag: feature-extraction
+license: apache-2.0
+pipeline_tag: image-feature-extraction
 tags:
 - Vision
 - LLaVA
+library_name: transformers
 ---
 
+# Region-based Cluster Discrimination for Visual Representation Learning (RICE)
 
+[[Paper]](https://huggingface.co/papers/2507.20025) [[GitHub]](https://github.com/deepglint/unicom)
 
+## Abstract
+Region-Aware Cluster Discrimination (RICE) is a novel method that enhances region-level visual and OCR capabilities. We first construct a billion-scale candidate region dataset and propose a Region Transformer layer to extract rich regional semantics. We further design a unified region cluster discrimination loss that jointly supports object and OCR learning within a single classification framework, enabling efficient and scalable distributed training on large-scale data. Extensive experiments show that RICE consistently outperforms previous methods on tasks, including segmentation, dense detection, and visual perception for Multimodal Large Language Models (MLLMs).
 
-[[Paper]](https://arxiv.org/abs/2407.17331) [[GitHub]](https://github.com/deepglint/unicom)  
-## Model
-We used the same Vision Transformer architecture  [ViT-L/14@336px as CLIP](https://huggingface.co/openai/clip-vit-large-patch14-336).
+## Model Overview
+We used the Vision Transformer architecture [ViT-L/14@336px as CLIP](https://huggingface.co/openai/clip-vit-large-patch14-336).
 
 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6478679d7b370854241b2ad8/8n_jBobanaLNAQjM5eZeg.png)
 
+## Highlights
+![470695215-38e89eea-8a73-4e3f-b43a-fa1ea6e32f0f](https://github.com/user-attachments/assets/e0de38b3-b20a-491e-9382-1839e9968481)
+
+RICE efficiently processes diverse semantic regions within the image using a single forward pass. The model jointly captures both general visual semantics (objects) and OCR semantics (texts), seamlessly integrating them into a unified representation.
 
 ## Data
-Our model was trained on publicly available image-caption data from the [LAION400M](https://arxiv.org/abs/2111.02114) and [COYO700M](https://github.com/kakaobrain/coyo-dataset) datasets. 
+Our model was trained on publicly available image-caption data from the [LAION400M](https://arxiv.org/abs/2111.02114) and [COYO700M](https://github.com/kakaobrain/coyo-dataset) datasets.
 
 ## Performance and Limitations
 
-### A. MLLMs Evaluation Results
-In our experiments, we replaced the CLIP model in [LLaVA-NeXT](https://github.com/LLaVA-VL/LLaVA-NeXT) with the MLCD model to demonstrate the performance of the MLCD model in Multimodal Large Language Models (MLLMs). For the language model, we used [Qwen2.5-7B](https://huggingface.co/Qwen/Qwen2.5-7B). The evaluation results show that the modified model performs exceptionally well across multiple benchmarks, validating the effectiveness of the MLCD model within MLLMs.
-
-| Vision Tower    | MLCD (ViT_L_14_336px) | CLIP (ViT_L_14_336px) |
-|:----------------|:----------------------|:----------------------|
-| LLM             | Qwen2.5-7B            | Qwen2.5-7B            |
-| AI2D            | <span style="color:red">76.98</span>           | 73.15                 |
-| ScienceQA_img   | <span style="color:red">78.09</span>           | 76.35                 |
-| GQA             | <span style="color:red">64.17</span>           | 63.31                 |
-| InfoVQA_val     | <span style="color:red">43.48</span>           | 38.88                 |
-| MMBench_cn_dev  | <span style="color:red">74.83</span>           | 72.51                 |
-| MMBench_en_dev  | <span style="color:red">76.37</span>           | 74.57                 |
-| MME(cognition)  | <span style="color:red">432</span>             | 384                   |
-| MME(perception) | <span style="color:red">1598</span>            | 1512                  |
-| SeedBench       | <span style="color:red">68.20</span>           | 66.80                 |
-| SeedBench_img   | <span style="color:red">73.75</span>           | 72.72                 |
-| MMStar          | <span style="color:red">50.98</span>           | 48.98                 |
-| MMMU            | <span style="color:red">44.30</span>           | 44.20                 |
-| OCRBench        | <span style="color:red">531.00</span>          | 525.00                |
-| ChartQA         | <span style="color:red">67.84</span>           | 66.52                 |
-| DocVQA_val      | <span style="color:red">76.46</span>           | 75.21                 |
-| POPE            | 88.69                 | <span style="color:red">88.83</span>  |
-| TextVQA_val     | 61.69                 | <span style="color:red">62.47</span>  |
-
-
-
-
-### B. Linear Probe Evaluation Results
-This table presents the results of linear probe evaluations comparing CLIP and MLCD models on the ViT_L_14_336px architecture across various datasets. The linear probe test freezes the pre-trained model's weights and trains a linear classifier on top to assess how well the model's representations generalize to different tasks.
-
-| Dataset        | MLCD (ViT_L_14_336px) | CLIP (ViT_L_14_336px) |
-|:---------------|:----------------------|:----------------------|
-| AVG            | <span style="color:red">87.15</span>             | 85.35                 |
-| Food101        | <span style="color:red">96.21</span>             | 95.90                 |
-| CIFAR-10       | <span style="color:red">99.36</span>             | 97.90                 |
-| CIFAR-100      | <span style="color:red">93.69</span>             | 87.40                 |
-| Birdsnap       | <span style="color:red">88.18</span>             | 79.90                 |
-| SUN397         | <span style="color:red">87.96</span>             | 82.20                 |
-| Stanford Cars  | <span style="color:red">95.16</span>             | 91.50                 |
-| FGVC Aircraft  | <span style="color:red">86.38</span>             | 71.60                 |
-| Describable Textures Dataset | <span style="color:red">86.70</span> | 83.00                 |
-| Oxford-IIIT Pets | <span style="color:red">96.27</span>          | 95.10                 |
-| Caltech-101    | <span style="color:red">97.92</span>             | 96.00                 |
-| Flowers102     | <span style="color:red">99.58</span>             | 99.20                 |
-| MNIST          | 98.67                 | <span style="color:red">99.20</span>             |
-| STL-10         | 99.28                 | <span style="color:red">99.70</span>             |
-| EuroSAT        | <span style="color:red">99.06</span>             | 98.10                 |
-| RESISC45       | <span style="color:red">95.48</span>             | 94.90                 |
-| GTSRB          | 92.32                 | <span style="color:red">92.40</span>             |
-| KITTI          | <span style="color:red">75.39</span>             | 69.20                 |
-| Country211     | 38.12                 | <span style="color:red">46.40</span>             |
-| PatchCamelyon  | <span style="color:red">88.00</span>             | 85.60                 |
-| UCF101         | <span style="color:red">92.86</span>             | 92.00                 |
-| Kinetics-700   | <span style="color:red">73.35</span>             | 73.00                 |
-| CLEVR          | <span style="color:red">64.40</span>             | 60.30                 |
-| Hateful Memes  | 72.00                 | <span style="color:red">77.30</span>             |
-| SST-2          | 76.33                 | <span style="color:red">80.50</span>             |
-| ImageNet       | <span style="color:red">86.30</span>             | 85.40                 |
-
-
-### C. Limitations
+### Experiments
+This table presents a comprehensive performance comparison of RICE with state-of-the-art vision encoders. For all experiments within the LLaVA-NeXT framework, we adopt a high-resolution tiling strategy: each input image is divided into a 2×2+1 grid of crops, where each crop matches the pre-training resolution of the backbone model (e.g., 336px, 378px, or 560px).
 
+![470696193-65b351ac-9399-4dac-8999-b4412286731a](https://github.com/user-attachments/assets/cd66223f-1757-4ff4-859c-19dd25f1246d)
+
+### Limitations
 Models with higher resolution are more friendly to OCR results. We are currently training such models and will soon make them available.
 
+## How to use
+
+#### 1. Standard Usage
+
+```python
+# Install dependencies
+# pip install torch transformers
+# git clone https://github.com/deepglint/unicom
+# cd unicom/mlcd
+
+from vit_rope2d_hf import MLCDVisionModel
+from transformers import CLIPImageProcessor
+from PIL import Image
+import requests
+import torch
+
+# Load model and processor
+model = MLCDVisionModel.from_pretrained("DeepGlint-AI/rice-vit-large-patch14-560")
+processor = CLIPImageProcessor.from_pretrained("DeepGlint-AI/rice-vit-large-patch14-560")
+
+# Load and process an image
+url = "http://images.cocodataset.org/val2017/000000039769.jpg"
+image = Image.open(requests.get(url, stream=True).raw)
+inputs = processor(images=image, return_tensors="pt")
+
+# Extract visual features
+with torch.no_grad():
+    outputs = model(**inputs)
+features = outputs.last_hidden_state
+
+print(f"Extracted features shape: {features.shape}")
+```
+
+#### 2. Using HuggingFace Transformers >= 4.51.3
+
+```python
+# pip install torch transformers>=4.51.3
+
+from transformers import AutoProcessor, AutoModel
+from PIL import Image
+import requests
+import torch
+
+# Load model and processor
+model = AutoModel.from_pretrained("DeepGlint-AI/rice-vit-large-patch14-560")
+processor = AutoProcessor.from_pretrained("DeepGlint-AI/rice-vit-large-patch14-560")
+
+# Load and process an image
+url = "http://images.cocodataset.org/val2017/000000039769.jpg"
+image = Image.open(requests.get(url, stream=True).raw)
+inputs = processor(images=image, return_tensors="pt")
+
+# Extract visual features
+with torch.no_grad():
+    outputs = model(**inputs)
+features = outputs.last_hidden_state[0]
+
+print(f"Extracted features shape: {features.shape}")
+```
+
+### Visualize Semantic Features
+
+![screenshot-20250725-232729](https://github.com/user-attachments/assets/0ff3b764-c5b6-4a10-a63c-89ccbc99d06b)
+
+Using 2048-resolution images as input to a ViT-B/16 model, we project token features onto RGB channels via
+PCA to visualize the semantic structure. Sequential frames (arranged vertically) illustrate the evolution of model attention, consistently
+highlighting salient objects across time. The visualization reveals stable color patterns for tracked entities such as ice skaters, deers,
+motorcyclists, and cyclists, demonstrating the model’s ability to maintain semantic focus throughout the sequence.
+
+## ModelZoo
+
+| Model | Download |
+|-------|-------------|
+| RICE-ViT-L-14-560px | [huggingface](https://huggingface.co/DeepGlint-AI/rice-vit-large-patch14-560) |
+| MLCD-ViT-bigG-14-448px | [huggingface](https://huggingface.co/DeepGlint-AI/mlcd-vit-bigG-patch14-448) |
+| MLCD-ViT-L-14-336px | [huggingface](https://huggingface.co/DeepGlint-AI/mlcd-vit-large-patch14-336) |
+| MLCD-ViT-B-32-224px | [huggingface](https://huggingface.co/DeepGlint-AI/mlcd-vit-base-patch32-224) |
 
 ## Acknowledgments
 
-We would like to express our gratitude to [Xie Yin](https://huggingface.co/Yin-Xie) and [Yumeng Wang](https://huggingface.co/devymex) for their significant contributions to the experimental validation in MLLMs.
+We would like to express our gratitude to [Xie Yin](https://huggingface.co/Yin-Xie) and [Yumeng Wang](https://huggingface.co/devymex) for their significant contributions to the experimental validation in MLLMs.
+The authors are from DeepGlint team and Huawei London Research Institute.
+
+## Citation
+
+If you find our work helpful or inspiring, please feel free to cite it.
+
+```latex
+@inproceedings{yinxie_2025_rice,
+  title={Region-based Cluster Discrimination for Visual Representation Learning},
+  author={Xie, Yin and Yang, Kaicheng and An, Xiang and Wu, Kun and Zhao, Yongle and Deng, Weimo and Ran, Zimin and Wang, Yumeng and Feng, Ziyong And Roy, Miles And Ismail, Elezi And Deng, Jiankang},
+  booktitle={ICCV},
+  year={2025}
+}
+@inproceedings{anxiang_2024_mlcd,
+  title={Multi-label Cluster Discrimination for Visual Representation Learning},
+  author={An, Xiang and Yang, Kaicheng and Dai, Xiangzi and Feng, Ziyong and Deng, Jiankang},
+  booktitle={ECCV},
+  year={2024}
+}
+@inproceedings{anxiang_2023_unicom,
+  title={Unicom: Universal and Compact Representation Learning for Image Retrieval},
+  author={An, Xiang and Deng, Jiankang and Yang, Kaicheng and Li, Jiawei and Feng, Ziyong and Guo, Jia and Yang, Jing and Liu, Tongliang},
+  booktitle={ICLR},
+  year={2023}
+}
+@inproceedings{anxiang_2022_partialfc,
+    author={An, Xiang and Deng, Jiankang and Guo, Jia and Feng, Ziyong and Zhu, XuHan and Yang, Jing and Liu, Tongliang},
+    title={Killing Two Birds With One Stone: Efficient and Robust Training of Face Recognition CNNs by Partial FC},
+    booktitle={CVPR},
+    year={2022},
+}
+@inproceedings{deng_2019_arcface,
+  title={Arcface: Additive angular margin loss for deep face recognition},
+  author={Deng, Jiankang and Guo, Jia and Xue, Niannan and Zafeiriou, Stefanos},
+  booktitle={CVPR},
+  year={2019}
+}
+```