HugSib commited on
Commit
4d1e895
·
verified ·
1 Parent(s): 826c353

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -8
README.md CHANGED
@@ -31,13 +31,15 @@ Combined with a late interaction matching mechanism, *ColPali* largely outperfor
31
  ## Organisation
32
 
33
 
34
- ### Models [add description of released model]
35
- - [*ColPali*](https://huggingface.co/vidore/colpali): TODO
 
36
 
37
- - [*BiPali*](https://huggingface.co/vidore/bipali): TODO
 
 
 
38
 
39
-
40
- - [*BiSigLip*](https://huggingface.co/vidore/bisiglip): TODO
41
 
42
 
43
  ### Datasets
@@ -61,7 +63,7 @@ You can either load a specific dataset using the standard `load_dataset` functio
61
  dataset = load_dataset(dataset_item.item_id)
62
  ```
63
 
64
- To use the whole benchmark you can list the datasets in the collection using the following snippet.
65
 
66
  ```python
67
  from datasets import load_dataset
@@ -81,12 +83,18 @@ To use the whole benchmark you can list the datasets in the collection using the
81
  ```
82
  ## Autorship + Citation
83
 
84
- TODO : Contact
85
 
 
86
 
87
- If you use any datasets or models from this organisation in your research, please cite the original dataset as follows:
 
 
88
 
89
  **BibTeX Citation**
 
 
 
90
  ```latex
91
  [include BibTeX]
92
  ```
 
31
  ## Organisation
32
 
33
 
34
+ ### Models
35
+ - [*ColPali*](https://huggingface.co/vidore/colpali): *ColPali* is our main contribution, it is a model based on a novel model architecture and training strategy based on Vision Language Models (VLMs), to efficiently index documents from their visual features.
36
+ It is a [PaliGemma-3B](https://huggingface.co/google/paligemma-3b-mix-448) extension that generates [ColBERT](https://arxiv.org/abs/2004.12832)- style multi-vector representations of text and images.
37
 
38
+ - [*BiPali*](https://huggingface.co/vidore/bipali): It is an extension of original SigLip architecture, the SigLIP-generated patch embeddings are fed to a text language model, PaliGemma-3B, to obtain LLM contextualized output patch embeddings.
39
+ These representations are pool-averaged to get a single vector representation and create a PaliGemma bi-encoder, *BiPali*.
40
+
41
+ - [*BiSigLip*](https://huggingface.co/vidore/bisiglip): Finetuned version of original [SigLip](https://huggingface.co/google/siglip-so400m-patch14-384), a strong vision-language bi-encoder model.
42
 
 
 
43
 
44
 
45
  ### Datasets
 
63
  dataset = load_dataset(dataset_item.item_id)
64
  ```
65
 
66
+ To use the whole benchmark, you can list the datasets in the collection using the following snippet.
67
 
68
  ```python
69
  from datasets import load_dataset
 
83
  ```
84
  ## Autorship + Citation
85
 
86
+ **Contact**
87
 
88
+ Please report any issues with the models or the benchmark or contact us:
89
 
90
+ - Manuel Faysse : [email?]()
91
+ - Hugues Sibille : [email?]()
92
+ - Tony Wu : [email?]()
93
 
94
  **BibTeX Citation**
95
+
96
+ If you use any datasets or models from this organisation in your research, please cite the original dataset as follows:
97
+
98
  ```latex
99
  [include BibTeX]
100
  ```