README.md

metadata

license: mit
title: CELL-FM
emoji: 🧬
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 5.9.1
app_file: app.py
pinned: false
short_description: Cell microscopy images from protein sequence

CELL-FM

A virtual microscopy model that bridges microscopy images and protein sequences.

Each tab in the app is one application of the model. More will be added; the sections below document the ones that exist.

---

Condensate Titration

Predict how an Intrinsically Disordered Peptide (IDP) behaves as its concentration rises, directly from sequence.

The only input is the sequence, and it must be exactly 66 amino acids — the entire CondenSeq library the model was trained on is 66-mers (all 14,578 of them), so other lengths are off-distribution. Sampling is fixed at 512 concentration levels and 100 ODE steps, seed 6, matching the offline runs.

Enter a sequence and this section runs the three stages of the CELL-FM condensate pipeline end to end:

1. Generate. CELL-FM CS samples one microscopy image per protein-intensity level, over a log-spaced concentration ladder from 20 to 2048 a.u. Every image is conditioned on the same fixed reference nucleus, so curves are comparable across sequences.
2. Classify. A ViT calls each generated image condensed or diffuse.
3. Summarise. A centred moving average over those binary calls is the titration curve; integrating it against log-intensity gives the two numbers the demo reports.

What AUC and AAC mean

AUC — the area under the titration curve. How much of the scanned concentration range the sequence spends in the condensed state. Higher means it condenses more readily, over a wider range.

AAC — the area above the curve. The curve is compared against the one it would have followed without reentrant dissolution: everything past the curve's maximum is held flat at that maximum, and AAC is the area between the two. It is zero when the sequence condenses and stays condensed, and grows the more the sequence re-dissolves at high concentration.

Both are normalised by the width of the log-intensity range, so both lie in [0, 1] and are directly comparable between sequences.

The curve also marks c_sat, the first intensity at which it reaches probability 0.8, and the raw per-image calls download as a CSV.

Layout

The tab is laid out like the Virtual OpenCell one, in the same OpenCell style:

• Left — the input alone, any 66-aa IDR sequence and Run analysis; then the run's AUC and AAC, and the titration curve.
• Right — the generated images in OpenCell's colours, nucleus blue and the IDP grey, with the same channel switch and per-channel intensity range and gamma. The slider steps through all 512 images along the ladder and the strip below holds 64 evenly spaced ones; the viewer opens on the image where condensation peaks, captioned with its expression level and call. Before a run it shows the reference nucleus alone.

Only generation and classification run on the GPU; the curve, viewer and strip are drawn afterwards on the CPU, so none of that spends the ZeroGPU budget.

Default sequence

The box is pre-filled with the NUP98 IDP, wild type, unnamed in the UI. Its F→S panel — the wild type and 91 random F→S mutants, ten draws for each of 1 to 9 substitutions and the one way to replace all ten F — ships in assets/nup98_mutation_seqs.csv; pasting a mutant in and comparing its AUC against WT is the quickest way to see the model respond to sequence grammar, as in the paper's Fig. 5b.

Amino acid substitution scan

Below the titration, the tab shows the paper's two substitution matrices side by side: the mean change in AUC (Fig. 5d) and in AAC (Supplementary Fig. 5c) when every residue of one type (row, Mutate FROM) is replaced by another (column, Mutate TO). They come from an in silico scan of 100 held-out CondenSeq IDPs — 380 substitutions, 512 images per sequence on the same ladder as above — and each cell averages over the IDPs that contain the FROM residue (58–95 of them, median 89). An IDP without it is left out, not counted as a zero change.

Click any cell to see the distribution behind its mean: a histogram of the per-IDP change in AUC and in AAC, with mean and median, and a table of every IDP's wild-type and mutant values. The scan is far past a demo's GPU budget, so it is precomputed: condenseq/substitution_scan.npz in the weights repo, built by notebooks/tools/build_substitution_scan_assets.py, which recomputes every value from the stored per-image calls and checks the means against the matrices ana_all_mutation_log_scale.py wrote for the paper (they agree to 1e-16). This section runs on CPU.

---

Virtual OpenCell

Browse CELL-FM's virtual staining of 1,276 OpenCell proteins, laid out like an OpenCell target page. Search for a protein:

• Left is OpenCell's annotation of the real cell line: localization grades (bar height is the grade, 3 the most prominent), UniProt and Ensembl IDs, sequence length, HEK293 abundance, and a link to the real images on OpenCell. None of it is predicted.
• Right are CELL-FM's images, in OpenCell's colours: the nucleus blue and the generated protein (the target) gray. OpenCell's channel switch and per-channel intensity range and gamma adjust the view; the strip below holds every sample of the protein, and Download TIFF saves the selected one as the raw 2-channel image.

Below the annotations, the virtual staining map places every protein's samples by UMAP of their image embeddings (opencell/vit.bin), coloured by OpenCell localization in the palette of the paper figures; the selected protein's samples are outlined and labelled. Zoom, pan and hover work as in any Plotly chart, and clicking a legend entry hides that category. The map is opencell/vs_umap_embedding.npz in the weights repo; the backdrop draws 12 evenly spaced samples per protein to keep each update light.

Every image is generated from the protein sequence in one shared cell: the nucleus channel is the same real nucleus (ATG7, crop 1) for every protein, so differences between proteins come from the sequence, and a protein's samples are independent draws for that cell.

The images are the dataset BoHuangLab/CELL-FM: OpenCell/virtual_staining/<GENE>/<NNNN>.tif, 93,510 samples, each (2, 256, 256) uint16 with the nucleus first. A protein's samples are fetched the first time it is viewed (VIRTUAL_OPENCELL_LOCAL_DIR reads a local copy instead); while that repo is private, the Space needs an HF_TOKEN secret that can read it. The protein table, assets/virtual_opencell_proteins.csv, is built by notebooks/tools/build_virtual_opencell_assets.py. This section runs on CPU.

---

Hardware

A GPU is required in practice. The code itself runs on CPU — flash-attn is deliberately left out of requirements.txt, so ESM-C uses its pure-torch rotary fallback (verified bit-identical to the Triton path: same AUC, same AAC, same 256/256 per-image calls) — but a CPU run of the defaults would take hours and time out. Use ZeroGPU or a GPU hardware tier.

The fixed settings — 512 images at 100 ODE steps — take ~85 s on an A40 (measured: 11.5 s per batch of 64, plus ~9 s of one-off model loading). Runtime scales linearly in both, so a section that needs to be faster should lower them in pipeline.py (NUM_IMAGES, NUM_STEPS).

On ZeroGPU the per-call budget is set by ZEROGPU_DURATION (default 75 s). Keep it close to what a run needs: spaces requests 1.5× it on the default GPU size, and ZeroGPU admits a run only when the visitor's remaining daily quota covers the whole request, while charging only the time used. A titration takes ~56 s on ZeroGPU (measured 2026-10-16), so 75 s requests ~113 s — twice the run, and two runs a day on an anonymous visitor's ~180 s; signed-in visitors get more. At 300 s (450 s requested) every visitor without PRO was turned away with "The requested GPU duration (450s) is larger than the maximum allowed".

---

Running it without a Space

Creating a Gradio Space under an organization now needs a Team or Enterprise plan. Until HuangLab has one, run_local.sh serves the identical app from a cluster GPU over a temporary public URL:

sbatch run_local.sh      # prints a *.gradio.live URL, valid 72 h

Weights

The Space downloads ~2.1 GB of checkpoints at startup from BoHuangLab/CELL-FM, overridable with CELLFM_MODEL_REPO. That repo keeps one subfolder per model family:

File	Model
condenseq/cellfm_seq2img.bin	CELL-FM CS generator, ESM-C 600M encoder included
condenseq/vae.bin	160×160 image VAE
condenseq/vit_cls.bin	Condensed/diffuse ViT classifier
condenseq/substitution_scan.npz	The precomputed substitution scan: per-IDP AUC and AAC, wild type and every mutant

To run against a local copy instead, set CELLFM_LOCAL_WEIGHTS to a flat directory holding the three .bin files and substitution_scan.npz — the prefix is stripped for local paths. upload_weights.py pushes them from the cluster.

Provenance

Ported from the CELL-FM repository, one module per stage, with behaviour preserved:

Stage	Origin
Generation	cell_fm/tasks_local/cell_fm_cs/evaluate_seq2img_dict.py
Classification	cell_fm/tasks_local/vit_cls_condenseq_img/evaluate_single_img.py
Moving average	cell_fm/tasks_local/vit_cls_condenseq_img/ma_plot.py
AUC / AAC	cell_diff/tasks/analysis/{utils,ana_all_mutation_log_scale}.py
Substitution scan	cell_diff/tasks/analysis/ana_all_mutation_log_scale.py, precomputed by notebooks/tools/build_substitution_scan_assets.py

Model hyperparameters mirror scripts/cell_fm_cs/evaluate_seq2img.sh and scripts/vit_cls_condenseq_img/pretrain.sh. The metric port in metrics.py was checked against the original on 40 stored curves and agrees to 0.0 in both log and linear mode. Generated images make the same round-trip through uint16 that the offline pipeline's TIFF write imposed, so demo numbers stay comparable to published ones.

Reference values, and how noisy they are

Measured at the fixed settings (512 images, 100 steps, seed 6) on an A40:

	AUC	AAC	c_sat
NUP98 WT	0.4912	0.0034	360
1F→S mutant	0.4794	0.0035	412

These are single draws, and how much that matters depends on whether you pair.

Taken alone, the numbers move a lot. Over seeds 6–9 the WT AUC ranges 0.4660–0.5087 (sd 0.019) and c_sat ranges 289–531 (sd 102). Treat any individual c_sat or AAC as indicative rather than precise.

Compared at a matched seed, small differences hold up. The 1F→S effect is −0.0123 ± 0.0042 across four seeds and negative in every one, even though it is smaller than the marginal spread above — driving both sequences from the same seed cancels most of the sampling noise, so the comparison is far tighter than either number is on its own. Compare sequences at the same seed and a single-residue change is resolvable; the demo does this by construction, with the seed fixed at 6.

Measured on the cluster stack (torch 2.4.1); the Space runs torch 2.8.0, which is not expected to change these but has not been compared directly.

---

Citation

If you use CELL-FM, please cite:

@article{zheng2026virtual,
  title   = {Virtual experiments bridge sequence and microscopy with generative models},
  author  = {Zheng, Dihan and Hong, Kibeom and Huang, Bo},
  journal = {bioRxiv},
  year    = {2026},
  doi     = {10.64898/2026.09.13.751243},
  url     = {https://www.biorxiv.org/content/10.64898/2026.09.13.751243v1}
}