| --- |
| language: |
| - en |
| library_name: open_clip |
| pipeline_tag: zero-shot-image-classification |
| tags: |
| - cell-senescence |
| - microscopy |
| - vision-language |
| - zero-shot-classification |
| license: other |
| --- |
| |
| # Cell Senescence CLIP |
|
|
| This gated repository releases two BioCLIP-family checkpoints for research evaluation of cellular senescence images: |
|
|
| - **OpenCLIP initialization, 10 epochs:** [`checkpoints/openclip_init_10epoch_best.pt`](checkpoints/openclip_init_10epoch_best.pt) |
| - **Random vision initialization, 10 epochs:** [`checkpoints/random_vision_10epoch.pt`](checkpoints/random_vision_10epoch.pt) |
|
|
| Both checkpoints use a ViT-L/14 image tower and a BioClinical ModernBERT text tower with a 512-token context. The OpenCLIP model configuration and ModernBERT compatibility shim are included in this repository. Access is gated: request access on Hugging Face and wait for approval before downloading. |
|
|
| ## Zero-shot evaluation |
|
|
| The results use the lab-provided P4, P12, and DOX image groups. The class meanings follow the accompanying group descriptions: P4 is the non-senescent control, P12 is replicative senescence, and DOX is chemically induced senescence. No model is fine-tuned on these evaluation images. Each image is encoded once; normalized image features are compared with normalized text features for the fixed class descriptions, and the highest cosine similarity determines the prediction. |
|
|
| Two tasks are reported: |
|
|
| 1. **P4 vs P12:** binary classification on 100 images, 50 from each group. |
| 2. **P4 vs P12 vs DOX:** three-class classification on all 150 images, 50 from each group. |
|
|
| The same class descriptions are used across models: |
|
|
| | Class | Text description | |
| |---|---| |
| | P4 | Fluorescence microscopy of passage-4 adipose-derived mesenchymal stem cells, non-senescent control with elongated spindle-shaped cells and small nuclei. | |
| | P12 | Fluorescence microscopy of passage-12 adipose-derived mesenchymal stem cells after serial passaging, replicative senescence with enlarged flattened cells and nuclei. | |
| | DOX | Fluorescence microscopy of adipose-derived mesenchymal stem cells treated with 0.05 micromolar doxorubicin for 7 days, chemically induced senescence with enlarged flattened cells. | |
|
|
| We report accuracy, macro-F1, and recall for each class. The complete two-table comparison, including the evaluated public models, is in [`evaluation/results/cell_senescence_zero_shot.md`](evaluation/results/cell_senescence_zero_shot.md); machine-readable values are in the adjacent JSON file. |
|
|
|
|
| ## Results |
|
|
| All 10 listed methods use the same fixed class descriptions. The P4 vs P12 comparison uses 50 images from each group; the three-way comparison uses 50 images from each of P4, P12, and DOX. Metrics are percentages. Bold marks the column maximum. The image-level results describe the supplied cohort; multiple fields from the same well may not be independent biological replicates. |
|
|
| ### P4 vs P12 (binary) |
|
|
| | Model | Accuracy (%) | Macro-F1 (%) | Recall · P4 control (%) | Recall · P12 senescent (%) | |
| | --- | --- | --- | --- | --- | |
| | MedSigLIP | 50.00 | 33.33 | **100.00** | 0.00 | |
| | CoCa | 50.00 | 33.33 | **100.00** | 0.00 | |
| | BioMedCLIP | 55.00 | 48.72 | 20.00 | 90.00 | |
| | OpenCLIP | 50.00 | 33.33 | 0.00 | **100.00** | |
| | BMC-LongCLIP | 53.00 | 41.07 | 98.00 | 8.00 | |
| | PMC-CLIP | 50.00 | 33.33 | **100.00** | 0.00 | |
| | BMC-CLIP-CF | 50.00 | 48.69 | 66.00 | 34.00 | |
| | MedPMC-CLIP | 51.00 | 38.56 | 96.00 | 6.00 | |
| | Ours · random vision · 10 ep | **69.00** | **68.75** | 60.00 | 78.00 | |
| | Ours · OpenCLIP init · 10 ep | 50.00 | 33.33 | **100.00** | 0.00 | |
|
|
| ### P4 vs P12 vs DOX (three-class) |
|
|
| | Model | Accuracy (%) | Macro-F1 (%) | Recall · P4 control (%) | Recall · P12 senescent (%) | Recall · DOX senescent (%) | |
| | --- | --- | --- | --- | --- | --- | |
| | MedSigLIP | 33.33 | 16.67 | **100.00** | 0.00 | 0.00 | |
| | CoCa | 40.00 | 31.99 | 46.00 | 0.00 | **74.00** | |
| | BioMedCLIP | 36.67 | 26.15 | 20.00 | **90.00** | 0.00 | |
| | OpenCLIP | **46.00** | **37.27** | 0.00 | 70.00 | 68.00 | |
| | BMC-LongCLIP | 35.33 | 21.60 | 98.00 | 8.00 | 0.00 | |
| | PMC-CLIP | 31.33 | 17.68 | 92.00 | 0.00 | 2.00 | |
| | BMC-CLIP-CF | 33.33 | 25.97 | 66.00 | 34.00 | 0.00 | |
| | MedPMC-CLIP | 34.00 | 20.05 | 96.00 | 6.00 | 0.00 | |
| | Ours · random vision · 10 ep | **46.00** | 36.72 | 60.00 | 78.00 | 0.00 | |
| | Ours · OpenCLIP init · 10 ep | 33.33 | 16.67 | **100.00** | 0.00 | 0.00 | |
|
|
|
|
| ## Run the released checkpoints |
|
|
| To run the evaluation, prepare a local data directory with the three image groups shown below. Install the inference dependencies and authenticate to Hugging Face to download a gated checkpoint: |
|
|
| ```bash |
| pip install -r evaluation/requirements.txt |
| hf auth login |
| ``` |
|
|
| The local data root should contain these three folders, each with 50 images: |
|
|
| ```text |
| <data-root>/ |
| ├── P4 _ 50 Images/ |
| ├── P12 _ 50 Images/ |
| └── DOX _ 50 Images/ |
| ``` |
|
|
| Run either released checkpoint; the script reports both requested tasks: |
|
|
| ```bash |
| python evaluation/code/evaluate_cell_senescence.py \ |
| --data-root "/path/to/Smaller data sets" \ |
| --checkpoint openclip_init_10epoch_best.pt \ |
| --output openclip_init_results.json |
| |
| python evaluation/code/evaluate_cell_senescence.py \ |
| --data-root "/path/to/Smaller data sets" \ |
| --checkpoint random_vision_10epoch.pt \ |
| --output random_vision_results.json |
| ``` |
|
|
| The script expects TIFF, PNG, JPEG, or BMP files. It checks that each group contains 50 images and does not crop or resize files beyond the model's standard image preprocessing. |
|
|
| ## Fine-tuning |
|
|
| This repository provides pretrained checkpoints and evaluation utilities; it does not include training code. For a general reference implementation of CLIP training, see [OpenCLIP](https://github.com/mlfoundations/open_clip). |
|
|
| ## Intended use |
|
|
| For research use only. This model is not a clinical diagnostic device. |
|
|
