---
license: mit
library_name: cell-fm
tags:
  - biology
  - microscopy
  - protein
  - condensate
  - flow-matching
---

# CELL-FM weights

Checkpoints of CELL-FM.

| File | Model |
|---|---|
| `condenseq/cellfm_seq2img.bin` | CELL-FM CS sequence-to-image generator, trained with IDPs held out (includes the ESM-C 600M encoder) |
| `condenseq/vae.bin` | Image VAE, 160x160, 3 down blocks, 4 latent channels |
| `condenseq/vit_cls.bin` | ViT condensed/diffuse classifier, 2-channel 160x160 input |
| `condenseq/reference_nucleus.npy` | The DAPI channel every CondenSeq generation is conditioned on: `(1, 160, 160)` float32 in [-1, 1]. CondenSeq protein index 12626, image 0 — the same one the offline seq2img runs used |
| `hpa/cellfm_seq2img.bin` | CELL-FM sequence-to-image generator for HPA, trained with test proteins held out; 256x256, 3-channel conditioning (includes the ESM-C 600M encoder) |
| `hpa/cellfm_vs.bin` | CELL-FM virtual-staining generator for HPA, trained on all HPA proteins; 256x256, 3-channel conditioning (includes the ESM-C 600M encoder) |
| `hpa/vae.bin` | Image VAE at 256x256, the one both `hpa/cellfm_seq2img.bin` and `hpa/cellfm_vs.bin` were trained against |
| `hpa/vit.bin` | Image-embedding ViT for HPA: 12 layers, 512 hidden, 2048 MLP, 8 heads, patch 4, 256x256 input. Trained to identify the protein in an image over 13,908 classes; the embedding is the representation, not the prediction. Four input channels — protein, nucleus, ER, microtubules |
| `hpa/cellfm_img2seq.bin` | CELL-FM image-to-sequence model for HPA, 512x512, 3-channel conditioning (includes the ESM-C 600M encoder) |
| `hpa/vae_512.bin` | Image VAE at 512x512, the one `hpa/cellfm_img2seq.bin` was trained against — a different model from `hpa/vae.bin`, not a rename |
| `opencell/cellfm_vs.bin` | CELL-FM virtual-staining generator for OpenCell, fine-tuned on all OpenCell proteins and meant for those proteins only; 256x256, single nucleus conditioning channel (includes the ESM-C 600M encoder) |
| `opencell/vae.bin` | Image VAE at 256x256, the one `opencell/cellfm_vs.bin` was trained against — OpenCell-finetuned, so not interchangeable with `hpa/vae.bin` despite the matching shape |
| `opencell/vit.bin` | The same ViT fine-tuned on OpenCell, 1,311 classes. Identical backbone to `hpa/vit.bin`, but its input stem takes **two** channels — protein and nucleus — where the HPA one takes four, so the two cannot be swapped: the count is fixed in `conv_proj` and a mismatch fails there |
| `opencell/vs_anchor_nucleus.npy` | The nucleus every OpenCell generation is conditioned on: `(1, 256, 256)` float32 in [-1, 1]. Gene ATG7, crop `CID001813_FID00035838_proj_11` — bit-for-bit the conditioning channel of the published offline run |
| `opencell/vs_genes.csv` | OpenCell's 1,311 genes: name, protein name, UniProt accession, Ensembl id, localization annotation and sequence. The metadata table minus its image paths |
| `opencell/vs_reference_cells.npz` | Four proteins in two matched pairs (POLR1A/SNRPF nuclear, LSM14A/DDX6 both P-body), each with one real OpenCell crop as a `(nucleus, protein)` float16 pair — the image shown beside the generated one |
| `hpa/anchor_cell.npy` | The fixed cell every NLS-screening image is conditioned on: `(3, 256, 256)` float32 in [-1, 1], channels nucleus, ER, microtubules. HPA gene H3C13, cell crop `1194_B2_2_4` |
| `hpa/anchor_masks.npz` | Two 256x256 boolean masks over that cell, `nucleus` and `cell`; cytoplasm is `cell & ~nucleus` |
| `hpa/pls_anchor_nls.npz` | The cell PLS generation conditions on for nuclear signals: `cell` `(3, 512, 512)` nucleus/ER/microtubules and `protein` `(1, 512, 512)`, float32 in [-1, 1]. HPA gene PPM1G (Nucleoplasm), crop `392_B9_1_11` |
| `hpa/pls_anchor_nes.npz` | The same for export signals. HPA gene DIAPH1 (Cytosol, Plasma membrane), crop `1608_B3_1_1` |
| `hpa/proteome_aa_counts.json` | Residue counts over the 12,894 HPA proteins (7,940,784 residues), the proteome baseline the frequency analysis compares against |
| `hpa/pls_reference_nls.csv` | The 320 published NLS signals: 20 independent draws at each of 16 tail lengths, 10-25 aa |
| `hpa/pls_reference_nes.csv` | The same 320 for export signals |

Hyperparameters for the CondenSeq models are set in `pipeline.py` in the Space and mirror
`scripts/cell_fm_cs/evaluate_seq2img.sh` and
`scripts/vit_cls_condenseq_img/pretrain.sh` in the CELL-FM repository. The HPA
hyperparameters are spelled out in `notebooks/nls_screening.ipynb` and mirror
`scripts/cell_fm/evaluate_virtual_staining_hpa_dict.sh`. The img2seq pair mirrors
`scripts_local/cell_fm/evaluate_img2seq_hpa_v2.sh`: 512 px, `sample_size` 128,
`encoder_patch_size` 8, `img_generator_patch_size` 4, 8 attention heads. The OpenCell pair
mirrors `scripts/cell_fm/evaluate_virtual_staining_opencell.sh`: 256 px, `sample_size` 64,
`encoder_patch_size` 4, `img_generator_patch_size` 2, 18 attention heads, `cell_image` `nucl`
— spelled out in `notebooks/opencell_vs.ipynb`. That single conditioning channel is the one
architectural difference a caller can see: the HPA generator takes three.

Each generator must be loaded with the VAE it was trained against — pairing
`cellfm_img2seq.bin` with the 256 px `vae.bin` gives a latent-size mismatch.