Idan0405
/

ClipMD

@@ -14,7 +14,7 @@ ClipMD is a medical image-text matching model based on OpenAI's CLIP model with
 The model uses a ViT-B/32 Transformer architecture as an image encoder and uses a masked sliding window elf-attention Transformer as a text encoder. These encoders are trained to maximize the similarity of (image, text) pairs via a contrastive loss.
-The model was fine-tuned on the ROCO dataset.
 ## Use with Transformers
 ```
@@ -32,4 +32,6 @@ inputs = processor(text=["chest x-ray", "head MRI"], images=image, return_tensor
 outputs = model(**inputs)
 logits_per_image = outputs[0] # this is the image-text similarity score
 probs = logits_per_image.softmax(dim=1) # we can take the softmax to get the label probabilities
-```

 The model uses a ViT-B/32 Transformer architecture as an image encoder and uses a masked sliding window elf-attention Transformer as a text encoder. These encoders are trained to maximize the similarity of (image, text) pairs via a contrastive loss.
+The model was fine-tuned on the [ROCO dataset](https://github.com/razorx89/roco-dataset).
 ## Use with Transformers
 ```
 outputs = model(**inputs)
 logits_per_image = outputs[0] # this is the image-text similarity score
 probs = logits_per_image.softmax(dim=1) # we can take the softmax to get the label probabilities
+```
+# See also
+* [ClipMD repository on github.](https://github.cs.huji.ac.il/tomhope-lab/ClipMD)