Update README.md

README.md

@@ -1,3 +1,66 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+---
+
+
+## Overview
+
+[MMDocIR/MMDocIR_Retrievers](https://huggingface.co/MMDocIR/MMDocIR_Retrievers) huggingface repository  contains all retriever checkpoints needed for [MMDocIR](https://github.com/MMDocRAG/MMDocIR), specifically to [Download Retriever Checkpoints](https://github.com/MMDocRAG/MMDocIR?tab=readme-ov-file#2-download-retriever-checkpoints).
+
+
+
+## 🛠️Retriever Checkpoints
+
+The list of available retrievers are as follows:
+
+- **BGE**: [bge-large-en-v1.5](https://huggingface.co/MMDocIR/MMDocIR_Retrievers/tree/main/bge-large-en-v1.5) which is cloned from [BAAI](https://huggingface.co/BAAI)/[bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5).
+- **ColBERT**: [colbertv2.0](https://huggingface.co/MMDocIR/MMDocIR_Retrievers/tree/main/colbertv2.0) which is cloned from [colbert-ir](https://huggingface.co/colbert-ir)/[colbertv2.0](https://huggingface.co/colbert-ir/colbertv2.0).
+- **E5**: [e5-large-v2](https://huggingface.co/MMDocIR/MMDocIR_Retrievers/tree/main/e5-large-v2) which is cloned from [ intfloat](https://huggingface.co/intfloat)/[e5-large-v2](https://huggingface.co/intfloat/e5-large-v2).
+- **GTE**: [gte-large](https://huggingface.co/MMDocIR/MMDocIR_Retrievers/tree/main/gte-large) which is cloned from [thenlper](https://huggingface.co/thenlper)/[gte-large](https://huggingface.co/thenlper/gte-large).
+- **Contriever**: [contriever-msmarco](https://huggingface.co/MMDocIR/MMDocIR_Retrievers/tree/main/contriever-msmarco) which is cloned from [facebook](https://huggingface.co/facebook)/[contriever-msmarco](https://huggingface.co/facebook/contriever-msmarco).
+- **DPR**:
+  - question encoder: [dpr-question_encoder-multiset-base](https://huggingface.co/MMDocIR/MMDocIR_Retrievers/tree/main/dpr-question_encoder-multiset-base) which is cloned from [facebook](https://huggingface.co/facebook)/[dpr-question_encoder-multiset-base](https://huggingface.co/facebook/dpr-question_encoder-multiset-base).
+  - passage encoder: [dpr-ctx_encoder-multiset-base](https://huggingface.co/MMDocIR/MMDocIR_Retrievers/tree/main/dpr-ctx_encoder-multiset-base) which is cloned from [facebook](https://huggingface.co/facebook)/[dpr-ctx_encoder-multiset-base](https://huggingface.co/facebook/dpr-ctx_encoder-multiset-base).
+- **ColPali**:
+  - retriever adapter: [colpali-v1.1](https://huggingface.co/MMDocIR/MMDocIR_Retrievers/tree/main/colpali-v1.1) which is cloned from [vidore](https://huggingface.co/vidore)/[colpali-v1.1](https://huggingface.co/vidore/colpali-v1.1).
+  - retriever base VLM: [colpaligemma-3b-mix-448-base](https://huggingface.co/MMDocIR/MMDocIR_Retrievers/tree/main/colpaligemma-3b-mix-448-base) which is cloned from [vidore](https://huggingface.co/vidore)/[colpaligemma-3b-mix-448-base](https://huggingface.co/vidore/colpaligemma-3b-mix-448-base).
+- **ColQwen**:
+  - retriever adapter: [colqwen2-v1.0](https://huggingface.co/MMDocIR/MMDocIR_Retrievers/tree/main/colqwen2-v1.0) which is cloned from [vidore](https://huggingface.co/vidore)/[colqwen2-v1.0](https://huggingface.co/vidore/colqwen2-v1.0).
+  - retriever base VLM: [colqwen2-base](https://huggingface.co/MMDocIR/MMDocIR_Retrievers/tree/main/colqwen2-base) which is cloned from [vidore](https://huggingface.co/vidore)/[colqwen2-base](https://huggingface.co/vidore/colqwen2-base).
+- **DSE-wikiss**: [dse-phi3-v1](https://huggingface.co/MMDocIR/MMDocIR_Retrievers/tree/main/dse-phi3-v1) which is processed as follows:
+  -  clone from [Tevatron](https://huggingface.co/Tevatron)/[dse-phi3-v1.0](https://huggingface.co/Tevatron/dse-phi3-v1.0).
+  -  fix  batch processing issue based on: https://huggingface.co/microsoft/Phi-3-vision-128k-instruct/discussions/32/files
+  -  change `config.json` and `preprocessor_config.json` to point to .py files in checkpoint.
+
+- **DSE-docmatix**: [dse-phi3-docmatix-v2](https://huggingface.co/MMDocIR/MMDocIR_Retrievers/tree/main/dse-phi3-docmatix-v2) which is cloned from [Tevatron](https://huggingface.co/Tevatron)/[dse-phi3-docmatix-v2](https://huggingface.co/Tevatron/dse-phi3-docmatix-v2).
+
+
+
+
+## Environment
+
+```bash
+python 3.9
+torch2.4.0+cu121
+transformers==4.45.0
+sentence-transformers==2.2.2  # for BGE, GTE, E5 retrievers
+colbert-ai==0.2.21 # for colbert retriever
+flash-attn==2.7.4.post1  # for DSE retrievers to run with flash attention
+```
+
+
+## 💾Citation
+If you use any datasets or models from this organization in your research, please cite our work as follows:
+
+```
+@misc{dong2025mmdocirbenchmarkingmultimodalretrieval,
+      title={MMDocIR: Benchmarking Multi-Modal Retrieval for Long Documents}, 
+      author={Kuicai Dong and Yujing Chang and Xin Deik Goh and Dexun Li and Ruiming Tang and Yong Liu},
+      year={2025},
+      eprint={2501.08828},
+      archivePrefix={arXiv},
+      primaryClass={cs.IR},
+      url={https://arxiv.org/abs/2501.08828}, 
+}
+```
+