base_model: meta-llama/Llama-3.1-8B-Instruct
license: llama3
pipeline_tag: text-generation
tags:
- biology
- protein
- molecule
- dna
- rna
- multimodal
- structure-grounded
Cuttlefish
Cuttlefish is a unified all-atom multimodal LLM that grounds language reasoning in geometric cues while scaling structural tokens with structural complexity. Built on Llama-3.1-8B-Instruct, it extends the base LLM with a graph encoder and a Scaling-Aware Patching connector for processing proteins, molecules, DNA, and RNA structures.
The model was introduced in the paper Scaling-Aware Adapter for Structure-Grounded LLM Reasoning.
Code: https://github.com/zihao-jing/Cuttlefish
Quick start
To use the model, you can download the weights using huggingface_hub. Running inference requires the original codebase.
from huggingface_hub import snapshot_download
# Download model
local_dir = snapshot_download("zihaojing/Cuttlefish")
# Run inference (requires cuttlefish codebase)
# python src/runner/inference.py --config configs/inference/octopus_8B_s3_v1_5.yaml
Input format
Cuttlefish accepts a unified parquet schema with structural graph columns:
| Field | Description |
|---|---|
modality |
"molecule", "protein", "dna", or "rna" |
node_feat |
Atom/node features (N × d) |
pos |
3D coordinates in Å (N × 3) |
edge_index |
Spatial graph edges in COO (2 × E) |
messages |
Chat-style instruction with <STRUCTURE> token |
The <STRUCTURE> placeholder in the user message is replaced by the encoded structural tokens at inference time.
Training details
- Base model: Llama-3.1-8B-Instruct
- Encoder: Cuttlefish-Encoder (pretrained on all-atom graph data)
- SFT data: Cuttlefish-SFT-Data
- Training stages: 2-stage SFT — connector training then full LLM fine-tuning with LoRA
Related resources
| Resource | Link |
|---|---|
| Cuttlefish-Encoder | zihaojing/Cuttlefish-Encoder |
| SFT instruction data | zihaojing/Cuttlefish-SFT-Data |
| Encoder pretraining data | zihaojing/Cuttlefish-Encoder-Data |
Citation
@article{jing2026cuttlefish,
title = {Cuttlefish: Scaling-Aware Adapter for Structure-Grounded LLM Reasoning},
author = {Jing, Zihao and Zeng, Qiuhao and Fang, Ruiyi and Li, Yan Yi and Sun, Yan and Wang, Boyu and Hu, Pingzhao},
booktitle = {Proceedings of the 43rd International Conference on Machine Learning (ICML)},
year = {2026},
url = {https://arxiv.org/abs/2602.02780}
}