Papers
arxiv:2611.20986

Calpha: Toward Native Biomolecular Intelligence in a General-Purpose Language Model

Published on Nov 16
ยท Submitted by
Yong Liu
on Nov 17
Authors:
,
,
,
,
,
,

Abstract

A large language model can output single-chain protein structures from its internal residue states by invoking a pretrained structure head, with reduced long-chain accuracy and MMLU-Pro accuracy in the reported tests.

Biomolecular foundation models predict the structures of proteins and their complexes accurately, and large language models (LLMs) reason in natural language, but the two remain separate: the abilities of the former are not part of a general-purpose LLM, and the reasoning of the latter does not enter a structure prediction. Calpha aims at one general model that folds proteins and complexes, understands protein sequence and structure, and lets its language reasoning influence the structures it predicts. This report covers the first step, in which an LLM outputs single-chain protein structures. Qwen3.5-9B reads a sequence enriched with ESM2 residue features. It can respond in text or emit a token that invokes a pretrained SimpleFold structure head, which receives information about the protein only through Qwen's residue states. Under a fixed folding request, the head achieves mean TM-scores of 0.808 on the FoldBench-334 development set, 0.635 on CASP16 domains and 0.900 on a CAMEO-style PDB set. We compare these predictions with a baseline in which ESM2-3B encodes the sequence and a fine-tuned SimpleFold-3B head predicts the coordinates. The two systems also differ in trainable weights, training duration and crop length. For chains of 100--299 residues, all three paired confidence intervals include zero. At 300--600 residues, the mean differences are -0.044, -0.160 and -0.025, respectively. In a separate test, four English request types triggered folding on every evaluated example. Protein training reduced accuracy on a 350-item MMLU-Pro subset from 77.4% to 70.3%. An LLM can therefore output single-chain structures from its own states, with lower long-chain scores in the current comparisons and reduced accuracy on the evaluated MMLU-Pro subset. Complex prediction, structure understanding and the effect of language reasoning on the predicted coordinates remain future work.

Community

Paper author Paper submitter

Excited to share Calpha, our open-source effort to bring native protein structure prediction into general-purpose LLMs!

Instead of calling an external folding tool, Calpha equips Qwen3.5-9B with a flow-matching structure head that generates all-atom coordinates directly from the model's hidden states. Our initial results reach TM-scores of 0.807 on FoldBench-334, 0.635 on CASP16, and 0.900 on CAMEO-2025.

We're still at an early stage, and the model currently relies on pretrained ESM2 and SimpleFold components. But we believe this is a promising step toward LLMs that can not only understand biological sequences, but also reason about and generate molecular structures.

We're open-sourcing the research process itself, including experiments, failures, and lessons learned. We'd love to hear your thoughts, critiques, and ideas!

๐ŸŒ https://calpha-ai.com

Hi authors, thanks for the work. A quick arithmetic question: Sec. 4.4 says BioMatrix-4B scores 0.96 on the 15 verbatim-contaminated CAMEO-2025 sequences and 0.28 on the other 185, which averages 0.331, but Table 4 gives 0.311 on all 200. Details in this GitHub issue: https://github.com/yliust/Calpha/issues/1

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.20986 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.20986 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.20986 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.