import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("amrosero0203/pbcell-sd21-caption-models", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]PBCell SD2.1 Caption Models
Fine-tuned Stable Diffusion 2.1 checkpoints for synthetic peripheral blood cell image generation using different semantic conditioning strategies.
These models were trained to study how caption semantic richness influences biomedical image synthesis quality and downstream hematological classification performance.
Included Models
| Model | Caption Strategy |
|---|---|
label_caption |
Minimal class-only prompts |
vlm_caption |
Automatically generated morphological captions |
expert_caption |
Expert-designed morphology captions |
Base Model
All checkpoints were fine-tuned from:
sd2-community/stable-diffusion-2-1
Resolution:
768 × 768
Repository Structure
pbcell-sd21-caption-models/
│
├── README.md
│
├── label_caption/
│ ├── checkpoint-2500/
│ ├── checkpoint-5000/
│ ├── final/
│ ├── validation/
│ ├── run_config.yaml
│ └── training_stats.json
│
├── vlm_caption/
│ ├── checkpoint-2500/
│ ├── checkpoint-5000/
│ ├── final/
│ ├── validation/
│ ├── run_config.yaml
│ └── training_stats.json
│
└── hematologist_caption/
├── checkpoint-2500/
│ ├── checkpoint-5000/
├── final/
├── validation/
├── run_config.yaml
└── training_stats.json
Loading a Model
Example using Diffusers:
from diffusers import StableDiffusionPipeline
pipe = StableDiffusionPipeline.from_pretrained(
"anmrr/pbcell-sd21-caption-models",
torch_dtype=torch.float16,
)
pipe = pipe.to("cuda")
image = pipe(
"A neutrophil cell image."
).images[0]
image.save("sample.png")
Example Prompts
Label Caption
A neutrophil cell image.
VLM Caption
A neutrophil cell with segmented nucleus and moderate azurophilic granulation.
Expert Caption
A Neutrophil cell with intermediate size, low N/C ratio, segmented nucleus, condensed chromatin, abundant azurophilic cytoplasm and neutrophil granulation.
Validation Samples
The validation/ directories contain example generations produced during training for qualitative inspection of:
- morphology consistency,
- caption adherence,
- and visual realism.
Notes
- These checkpoints are provided for research and educational purposes only.
- The original peripheral blood cell dataset is not included.
- Internal experimental workflows and full evaluation pipelines are intentionally omitted from this public release.
Related Repository
Code and minimal reproducible framework:
https://github.com/anmrr/pbcell-caption-strategies
Dataset Source
These models were trained using peripheral blood smear images derived from the public dataset introduced by:
Acevedo, A. et al. A dataset of microscopic peripheral blood cell images for development of automatic recognition systems. Data in Brief, 30, 105474 (2020). https://doi.org/10.1016/j.dib.2020.105474
The original dataset contains annotated MGG-stained peripheral blood cell images acquired using a Cellavision DM96 microscope system.
Citation
@misc{rosero2026pbcellmodels,
title={PBCell SD2.1 Caption Models},
author={Rosero, Angie M},
year={2026},
howpublished={Hugging Face repository}
}
License
Apache-2.0
- Downloads last month
- -