README.md

---
library_name: transformers
license: mit
---

# Model Card for PULSAR-pbmc 

<!-- Provide a quick summary of what the model is/does. -->

**PULSAR** (Patient Understanding Leveraging Single-cell universAl Representation) is a multi-scale, multi-cellular foundation model for human peripheral blood mononuclear cells (PBMCs). It transforms a set of single-cell transcriptomes into an interpretable **donor-level embedding** that preserves single-cell resolution while capturing multicellular composition and coordination.

This repo hosts the **zero-shot PBMC model** (`PULSAR-pbmc`) used to produce donor embeddings without task-specific fine-tuning. A disease-aligned variant is also available (see **Model Sources**).


## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->
PULSAR (Patient Understanding Leveraging Single-cell universAl Representation) is a hierarchical, multi-scale foundation model for PBMC scRNA-seq that converts unordered sets of single cells into a 512-d donor embedding while preserving single-cell resolution. It integrates molecular priors from ESM2 protein embeddings, cellular representations via Universal Cell Embeddings (UCE, 1,280-d), and a Multicellular Transformer encoder–decoder trained with a high-masking, Masked Cell Modeling objective. Pretraining proceeds in two stages: a pan-tissue CELLxGENE corpus (≈36.2M cells; 6,807 samples) followed by continual pretraining on blood (≈8.74M cells; 2,588 samples). The resulting donor embeddings support zero-shot and lightweight-head downstream tasks, including large-scale reference mapping for disease classification (state-of-the-art accuracy with strong external generalization), regression of plasma proteomics from transcriptomes, forecasting of future outcomes (e.g., RA conversion in ACPA+ individuals and influenza vaccine responsiveness), and individualized cytokine perturbation modeling across donor, cellular, and gene levels. A  “virtual instrument” conditions on cytokine protein embeddings to transform baseline donor states and, with the decoder and an optional UCE→expression head, generates perturbed cell distributions and gene programs. Attention over cells provides mechanistic interpretability, highlighting disease- and severity-relevant subsets and enriching for antigen-specific clonotypes in viral infection. PULSAR thus operationalizes the AI Virtual Cell vision by linking molecular, cellular, and multicellular organization into a unified, transferable representation for precision immunology.

- **Developed by:** Kuan Pang (Stanford University, kuanpang@stanford.edu)
- **Model type:** Transformer
- **License:** MIT

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:**: https://github.com/snap-stanford/PULSAR
- **Paper:** https://www.biorxiv.org/content/10.1101/2025.11.24.685470v1
- **Aligned version:** https://huggingface.co/KuanP/PULSAR-aligned

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use


- Generate 512-d **donor embeddings** from PBMC scRNA-seq to:
  - Perform **reference mapping/retrieval** (kNN) for disease phenotypes
  - Build **lightweight predictors** for clinical variables (e.g., plasma proteomics, vaccine response)
  - Support **in-silico perturbation** pipelines (with the provided virtual-instrument and decoders)
  - Enable **interpretability** via attention over single cells and cell types

### Downstream Use [optional]


- Fine-tune/align the embedding space for a labeled task (e.g., contrastive alignment by disease label).
- Integrate with perturbation modules to predict donor-, cell-, and gene-level responses to cytokines.


### Out-of-Scope Use

The model might not work for tissue types other than PBMC, that also includes cell sorting samples.


## How to Get Started with the Model

Use the code below to get started with the model.


## Training Details

### Training Data

Stage-1 pretraining corpus: CZ CELLxGENE Census (LTS 2023-07-25), 36.2M cells, 6,807 samples across 53 tissues and 69 conditions.

Stage-2 continual pretraining (blood focus): 8.736M cells, 2,588 blood/PBMC samples (balanced sexes; broad ages).


More details can be found in the Paper and GitHub.

## Citation

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

```
@article{pang2025pulsar,
  title={PULSAR: a Foundation Model for Multi-scale and Multicellular Biology},
  author={Pang, Kuan and Rosen, Yanay and Kedzierska, Kasia and He, Ziyuan and Rajagopal, Abhe and Gustafson, Claire E and Huynh, Grace and Leskovec, Jure},
  journal={bioRxiv},
  pages={2025--11},
  year={2025},
  publisher={Cold Spring Harbor Laboratory}
}
```