| --- |
| language: |
| - en |
| library_name: open_clip |
| pipeline_tag: zero-shot-image-classification |
| tags: |
| - biomedical |
| - vision-language |
| - image-text-retrieval |
| - zero-shot-classification |
| license: other |
| --- |
| |
| # BioLoom |
|
|
| BioLoom is a biomedical vision-language model for image-text representation learning. This repository provides two pretrained checkpoints with a ViT-L/14 vision encoder and a BioClinical ModernBERT text encoder. The image input size is 224 pixels and the text context length is 512 tokens. |
|
|
| ## Released checkpoints |
|
|
| - **BioLoom (Scratch):** random vision initialization, trained for 10 epochs. Download [random_vision_10epoch.pt](checkpoints/random_vision_10epoch.pt). |
| - **BioLoom (OpenCLIP initialization):** pretrained vision initialization, trained for 10 epochs. Download [openclip_init_10epoch_best.pt](checkpoints/openclip_init_10epoch_best.pt). |
|
|
| The architecture configuration, checkpoint loader, and ModernBERT compatibility helper are included. Repository access is gated; request access and authenticate with Hugging Face before downloading. |
|
|
| ## Installation |
|
|
| Download the repository's Python files, configuration and requirements file into the same working directory, then run: |
|
|
| ```bash |
| pip install -r requirements.txt |
| hf auth login |
| ``` |
|
|
| ## Load a model |
|
|
| ```bash |
| python load_model.py --checkpoint random_vision_10epoch.pt --device cuda |
| ``` |
|
|
| For the pretrained vision variant: |
|
|
| ```bash |
| python load_model.py --checkpoint openclip_init_10epoch_best.pt --device cuda |
| ``` |
|
|
| The Python interface returns the model, image preprocessing transform, and tokenizer: |
|
|
| ```python |
| from load_model import load_model |
| |
| model, preprocess, tokenizer = load_model( |
| "random_vision_10epoch.pt", device="cuda" |
| ) |
| ``` |
|
|
| To use a checkpoint already stored locally, set `BIOLOOM_CHECKPOINT` to its file path. Model configuration and tokenizer assets may still require downloading. |
|
|
| ## Repository contents |
|
|
| - `checkpoints/`: the two released model weights. |
| - `MODEL_INFO.json`: architecture and checkpoint metadata. |
| - `BioClinicalModernBERT-large-ViT-L-14-pretrained.json`: OpenCLIP architecture configuration. |
| - `load_model.py`: checkpoint loader. |
| - `openclip_compat.py`: ModernBERT compatibility helper. |
| - `requirements.txt`: inference dependencies. |
|
|
| The model produces image and text embeddings that can be compared using cosine similarity for retrieval or zero-shot classification. |
|
|
