| --- |
| base_model: meta-llama/Llama-3.1-8B-Instruct |
| license: llama3 |
| pipeline_tag: text-generation |
| tags: |
| - biology |
| - protein |
| - molecule |
| - dna |
| - rna |
| - multimodal |
| - structure-grounded |
| --- |
| |
| # Cuttlefish |
|
|
| **Cuttlefish** is a unified all-atom multimodal LLM that grounds language reasoning in geometric cues while scaling structural tokens with structural complexity. Built on Llama-3.1-8B-Instruct, it extends the base LLM with a graph encoder and a Scaling-Aware Patching connector for processing proteins, molecules, DNA, and RNA structures. |
|
|
| The model was introduced in the paper [Scaling-Aware Adapter for Structure-Grounded LLM Reasoning](https://huggingface.co/papers/2602.02780). |
|
|
| **Code**: [https://github.com/zihao-jing/Cuttlefish](https://github.com/zihao-jing/Cuttlefish) |
|
|
| ## Quick start |
|
|
| To use the model, you can download the weights using `huggingface_hub`. Running inference requires the original codebase. |
|
|
| ```python |
| from huggingface_hub import snapshot_download |
| |
| # Download model |
| local_dir = snapshot_download("zihaojing/Cuttlefish") |
| |
| # Run inference (requires cuttlefish codebase) |
| # python src/runner/inference.py --config configs/inference/octopus_8B_s3_v1_5.yaml |
| ``` |
|
|
| ## Input format |
|
|
| Cuttlefish accepts a unified parquet schema with structural graph columns: |
|
|
| | Field | Description | |
| |---|---| |
| | `modality` | `"molecule"`, `"protein"`, `"dna"`, or `"rna"` | |
| | `node_feat` | Atom/node features (N × d) | |
| | `pos` | 3D coordinates in Å (N × 3) | |
| | `edge_index` | Spatial graph edges in COO (2 × E) | |
| | `messages` | Chat-style instruction with `<STRUCTURE>` token | |
|
|
| The `<STRUCTURE>` placeholder in the user message is replaced by the encoded structural tokens at inference time. |
|
|
| ## Training details |
|
|
| - **Base model**: Llama-3.1-8B-Instruct |
| - **Encoder**: [Cuttlefish-Encoder](https://huggingface.co/zihaojing/Cuttlefish-Encoder) (pretrained on all-atom graph data) |
| - **SFT data**: [Cuttlefish-SFT-Data](https://huggingface.co/datasets/zihaojing/Cuttlefish-SFT-Data) |
| - **Training stages**: 2-stage SFT — connector training then full LLM fine-tuning with LoRA |
|
|
| ## Related resources |
|
|
| | Resource | Link | |
| |---|---| |
| | Cuttlefish-Encoder | [zihaojing/Cuttlefish-Encoder](https://huggingface.co/zihaojing/Cuttlefish-Encoder) | |
| | SFT instruction data | [zihaojing/Cuttlefish-SFT-Data](https://huggingface.co/datasets/zihaojing/Cuttlefish-SFT-Data) | |
| | Encoder pretraining data | [zihaojing/Cuttlefish-Encoder-Data](https://huggingface.co/datasets/zihaojing/Cuttlefish-Encoder-Data) | |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{jing2026cuttlefish, |
| title = {Cuttlefish: Scaling-Aware Adapter for Structure-Grounded LLM Reasoning}, |
| author = {Jing, Zihao and Zeng, Qiuhao and Fang, Ruiyi and Li, Yan Yi and Sun, Yan and Wang, Boyu and Hu, Pingzhao}, |
| booktitle = {Proceedings of the 43rd International Conference on Machine Learning (ICML)}, |
| year = {2026}, |
| url = {https://arxiv.org/abs/2602.02780} |
| } |
| ``` |
