Add MetaboLLM organization overview

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -7,4 +7,250 @@ sdk: static
 pinned: false
 ---
 
-Edit this `README.md` markdown file to author your organization card.
+# MetaboLLM
+
+**MetaboLLM** is a family of metabolomics-specialized large language models designed to integrate biochemical knowledge across heterogeneous resources and support metabolite-, pathway-, reaction-, and enzyme-centered reasoning and description generation.
+
+MetaboLLM was developed through continual pretraining, supervised fine-tuning, and structured retrieval using harmonized biochemical knowledge from KEGG, HMDB, PubChem, and SMPDB.
+
+<p align="center">
+  <img src="https://huggingface.co/spaces/MetaboLLM/README/resolve/main/assets/Figure1.png" alt="Overview of the MetaboLLM framework" width="950">
+</p>
+
+## Highlights
+
+- Integrated knowledge covering **237,243 metabolites**, **2,359 pathways**, **12,323 reactions**, and **5,993 enzymes**
+- Released **four MetaboLLM model variants** across Qwen, Gemma, and Llama backbones
+- Constructed a metabolomics benchmark containing **17 tasks**, **6,000 training examples**, and **4,200 test examples**
+- Evaluated factual knowledge, class identification, biochemical relationships, and description generation
+- Compared MetaboLLM against corresponding base models and five publicly available medical language models
+- Evaluated transfer on the independently developed MetaBench benchmark
+
+## Resources
+
+| Resource | Description |
+|---|---|
+| [MetaboLLM-Qwen3-4B](https://huggingface.co/MetaboLLM/MetaboLLM-Qwen3-4B) | Primary MetaboLLM model based on Qwen3-4B |
+| [MetaboLLM-Qwen3-8B](https://huggingface.co/MetaboLLM/MetaboLLM-Qwen3-8B) | MetaboLLM model based on Qwen3-8B |
+| [MetaboLLM-Gemma-3-4B](https://huggingface.co/MetaboLLM/MetaboLLM-Gemma-3-4B) | MetaboLLM model based on Gemma-3-4B |
+| [MetaboLLM-Llama-3.2-3B](https://huggingface.co/MetaboLLM/MetaboLLM-Llama-3.2-3B) | MetaboLLM model based on Llama-3.2-3B |
+| [MetaboLLM-Benchmark](https://huggingface.co/datasets/MetaboLLM/MetaboLLM-Benchmark) | Training and evaluation benchmark across 17 metabolomics tasks |
+
+## Model Family
+
+| Model | Backbone | Release Format |
+|---|---|---|
+| MetaboLLM-Qwen3-4B | Qwen3-4B | PEFT LoRA adapter |
+| MetaboLLM-Qwen3-8B | Qwen3-8B | PEFT LoRA adapter |
+| MetaboLLM-Gemma-3-4B | Gemma-3-4B | PEFT LoRA adapter |
+| MetaboLLM-Llama-3.2-3B | Llama-3.2-3B | PEFT LoRA adapter |
+
+MetaboLLM-Qwen3-4B served as the primary model in the associated study. All released repositories contain adapter weights and require the corresponding base model.
+
+## Integrated Biochemical Knowledge
+
+MetaboLLM was developed from a unified resource integrating complementary information from:
+
+- **KEGG** for metabolites, reactions, enzymes, and pathways
+- **HMDB** for human metabolites, biological roles, chemical taxonomy, and compound descriptions
+- **PubChem** for chemical structures, identifiers, molecular properties, and compound descriptions
+- **SMPDB** for curated pathways and physiological descriptions
+
+The harmonized resource contains:
+
+| Entity type | Count |
+|---|---:|
+| Metabolites | 237,243 |
+| Pathways | 2,359 |
+| Reactions | 12,323 |
+| Enzymes | 5,993 |
+
+## MetaboLLM Benchmark
+
+The benchmark contains 17 tasks organized into four complementary categories.
+
+| Category | Tasks | Train | Test |
+|---|---:|---:|---:|
+| Knowledge recall | 5 | 1,000 | 1,000 |
+| Class identification | 4 | 1,000 | 1,000 |
+| Relation identification | 3 | 1,000 | 1,000 |
+| Description generation | 5 | 3,000 | 1,200 |
+| **Total** | **17** | **6,000** | **4,200** |
+
+The benchmark evaluates:
+
+- molecular identity and formula knowledge
+- metabolite and reaction class recognition
+- metabolite–pathway relationships
+- metabolite–reaction relationships
+- reaction–enzyme relationships
+- metabolite, pathway, and enzyme description generation
+- structure-rich and structure-poor metabolite description generation
+
+## Benchmark Results
+
+### Biochemical Knowledge and Structured Relationships
+
+Mean accuracy across 12 multiple-choice and short-answer tasks.
+
+| Category | Model | Mean Accuracy |
+|---|---|---:|
+| Medical LLM | MedGemma-1.5-4B | 49.0 |
+| Medical LLM | FineMedLM-O1 | 57.9 |
+| Medical LLM | II-Medical-8B-1706 | 63.0 |
+| Medical LLM | Qwen2.5-Aloe-Beta-7B | 63.9 |
+| Medical LLM | Meditron3-Qwen2.5-7B | 65.7 |
+| Base model | Llama-3.2-3B | 51.5 |
+| Base model | Gemma-3-4B | 52.2 |
+| Base model | Qwen3-4B | 66.5 |
+| Base model | Qwen3-8B | 66.8 |
+| MetaboLLM | MetaboLLM-Llama-3.2-3B | 70.0 |
+| MetaboLLM | MetaboLLM-Gemma-3-4B | 75.3 |
+| MetaboLLM | MetaboLLM-Qwen3-8B | 75.9 |
+| MetaboLLM | **MetaboLLM-Qwen3-4B** | **79.1** |
+
+All four MetaboLLM variants outperformed their corresponding unadapted backbones and all evaluated medical language models. MetaboLLM-Qwen3-4B achieved the highest mean accuracy at 79.1%, compared with 65.7% for the strongest evaluated medical model and 66.5% for its corresponding base model.
+
+### Biochemical Description Generation
+
+BERTScore-F1 on a 0–100 scale.
+
+| Category | Model | Metabolite Description | Pathway Description | Enzyme Description | Structure-Rich Metabolite Description | Structure-Poor Metabolite Description |
+|---|---|---:|---:|---:|---:|---:|
+| Medical LLM | MedGemma-1.5-4B | 81.05 | 80.97 | 79.33 | 84.89 | 84.73 |
+| Medical LLM | FineMedLM-O1 | 83.05 | 82.87 | 81.22 | 84.10 | 83.96 |
+| Medical LLM | II-Medical-8B-1706 | 82.71 | 82.94 | 81.60 | 86.37 | 85.84 |
+| Medical LLM | Qwen2.5-Aloe-Beta-7B | 83.34 | 83.39 | 81.68 | 85.47 | 84.52 |
+| Medical LLM | Meditron3-Qwen2.5-7B | 83.69 | 83.14 | 81.99 | 84.47 | 83.76 |
+| Base model | Llama-3.2-3B | 82.81 | 82.56 | 81.40 | 84.56 | 84.50 |
+| Base model | Gemma-3-4B | 81.74 | 81.54 | 80.37 | 85.48 | 85.07 |
+| Base model | Qwen3-4B | 82.39 | 81.85 | 80.68 | 86.48 | 85.80 |
+| Base model | Qwen3-8B | 82.30 | 82.28 | 80.79 | 86.20 | 85.61 |
+| MetaboLLM | MetaboLLM-Llama-3.2-3B | 90.93 | 86.74 | 83.78 | 87.13 | 87.65 |
+| MetaboLLM | MetaboLLM-Gemma-3-4B | 87.39 | 83.83 | 80.69 | 87.03 | 87.74 |
+| MetaboLLM | MetaboLLM-Qwen3-8B | 90.94 | 87.26 | 83.70 | 87.66 | 88.02 |
+| MetaboLLM | **MetaboLLM-Qwen3-4B** | **91.71** | **88.04** | **84.12** | **87.79** | **88.06** |
+
+MetaboLLM-Qwen3-4B achieved the highest BERTScore-F1 across all five description-generation tasks.
+
+## External Benchmark Transfer
+
+MetaboLLM was also evaluated on MetaBench, an independently developed public metabolomics benchmark.
+
+- **MetaboLLM-Qwen3-8B** achieved the highest Knowledge MCQA accuracy at **56.42%**
+- **MetaboLLM-Qwen3-4B** achieved the highest pathway-description scores, including **85.19 BERTScore-F1**, **25.83 ROUGE-L-F1**, and **17.61 BLEU-2**
+
+These results demonstrate transfer beyond the internally constructed MetaboLLM benchmark.
+
+## Quick Start
+
+Install the required packages:
+
+```bash
+pip install -U transformers peft accelerate torch
+```
+
+Example using MetaboLLM-Qwen3-4B:
+
+```python
+import torch
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+base_model_id = "Qwen/Qwen3-4B-Instruct-2507"
+adapter_id = "MetaboLLM/MetaboLLM-Qwen3-4B"
+
+tokenizer = AutoTokenizer.from_pretrained(adapter_id)
+
+base_model = AutoModelForCausalLM.from_pretrained(
+    base_model_id,
+    torch_dtype="auto",
+    device_map="auto",
+)
+
+model = PeftModel.from_pretrained(
+    base_model,
+    adapter_id,
+)
+
+messages = [
+    {
+        "role": "user",
+        "content": "What is the biochemical role of pyruvate?"
+    }
+]
+
+inputs = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt",
+).to(model.device)
+
+with torch.no_grad():
+    outputs = model.generate(
+        inputs,
+        max_new_tokens=256,
+        do_sample=False,
+    )
+
+response = tokenizer.decode(
+    outputs[0][inputs.shape[-1]:],
+    skip_special_tokens=True,
+)
+
+print(response)
+```
+
+Please check the corresponding model card for the exact base-model identifier and model-specific loading instructions.
+
+## Loading the Benchmark
+
+```python
+from datasets import load_dataset
+
+dataset = load_dataset(
+    "MetaboLLM/MetaboLLM-Benchmark",
+    "Knowledge_recall",
+)
+
+print(dataset["test"][0])
+```
+
+Available configurations:
+
+- `Knowledge_recall`
+- `Class_identification`
+- `Relation_identification`
+- `Description_generation`
+
+## Intended Uses
+
+MetaboLLM is intended for research involving:
+
+- metabolomics-specific question answering
+- biochemical knowledge recall
+- metabolite and reaction class identification
+- metabolite–pathway and reaction–enzyme relationship identification
+- metabolite, pathway, and enzyme description generation
+- evaluation of metabolomics-specialized language models
+
+## Limitations
+
+- MetaboLLM may generate incorrect or unsupported biochemical statements.
+- Performance outside metabolomics and biochemical knowledge tasks has not been comprehensively evaluated.
+- Outputs should be independently verified and should not be used for clinical decision-making.
+
+## Licenses
+
+Each MetaboLLM repository is distributed according to its model card and license notice.
+
+- Use of each adapter remains subject to the license and terms of its corresponding base model.
+- Benchmark components created by the MetaboLLM authors are distributed under the terms described in the benchmark repository.
+- Third-party and source-derived biochemical content remains subject to the licenses and terms of the original providers.
+
+Please review the license files in the relevant model and dataset repositories before use.
+
+## Resource Maintainers
+
+- Dohyun Ku
+- Min Gu Kwak