---
license: apache-2.0
pipeline_tag: feature-extraction
library_name: pytorch
base_model:
  - facebook/esm2_t12_35M_UR50D
  - EvolutionaryScale/esm3-sm-open-v1
tags:
  - biology
  - bioinformatics
  - protein
  - protein-embeddings
  - contrastive-learning
  - multimodal
  - structure
  - sequence
  - sequence-segments
  - pytorch
---

# CLSS (Contrastive Learning Sequence–Structure)

CLSS is a **self-supervised, two-tower contrastive model** that **co-embeds protein sequences and protein structures into a shared latent space**, enabling unified analysis of protein space across modalities.

**Links**
- Hugging Face model repo: https://huggingface.co/guyyanai/CLSS
- Code + examples (`clss-model`): https://github.com/guyyanai/CLSS
- Paper (bioRxiv): https://doi.org/10.1101/2025.09.05.674454
- Interactive CLSS viewer: https://gabiaxel.github.io/clss-viewer/

---

## Model description

### Architecture (high level)

CLSS follows a **two-tower architecture**:

- **Sequence tower:** a trainable ESM2-like sequence encoder
- **Structure tower:** a frozen ESM3 structure encoder
- Each tower is followed by a lightweight **linear projection head** mapping into a shared embedding space, with **L2-normalized outputs**

The result is a pair of embeddings (sequence and structure) that live in the **same latent space**, making cosine similarity directly comparable across modalities.

The paper’s primary configuration uses **32-dimensional embeddings**, but multiple embedding sizes are provided in this repository.

### Training objective

CLSS is trained with a **CLIP-style contrastive objective**, aligning:
- **Random sequence segments**
- With their corresponding **full-domain protein structures**

**No** hierarchical labels (e.g. ECOD or CATH) are used during training; structural and evolutionary organization emerges implicitly.

---

## Files in this repository

This Hugging Face repository contains multiple PyTorch Lightning checkpoints, differing only in **embedding dimensionality**:

- `h8_r10.lckpt`   → 8-dimensional embeddings  
- `h16_r10.lckpt`  → 16-dimensional embeddings  
- `h32_r10.lckpt`  → 32-dimensional embeddings (paper default)  
- `h64_r10.lckpt`  → 64-dimensional embeddings  
- `h128_r10.lckpt` → 128-dimensional embeddings  

---

## How to use CLSS

CLSS is intended to be used via the **`clss-model` Python library**, which provides:

- Model loading from Lightning checkpoints
- End-to-end inference examples
- Scripts used for generating interactive protein space maps

---

## License

The CLSS codebase is released under the **Apache 2.0 License**.  
Please consult the repository for details on third-party model dependencies.

---

## Citation

If you use CLSS, please cite:

```bibtex
@article{Yanai2025CLSS,
  title   = {Contrastive learning unites sequence and structure in a global representation of protein space},
  author  = {Yanai, Guy and Axel, Gabriel and Longo, Liam M. and Ben-Tal, Nir and Kolodny, Rachel},
  journal = {bioRxiv},
  year    = {2025},
  doi     = {10.1101/2025.09.05.674454},
  url     = {https://doi.org/10.1101/2025.09.05.674454}
}