Cyanex/BioGPT-X

🧬 DTI-LLM: Drug–Target Interaction Prediction with LLaMA-3

A LoRA fine-tuned LLaMA-3 model for predicting drug–target interactions from molecular and protein information.

Current Status: 🚧 Active Development / Research Prototype

---

Overview

DTI-LLM is a biomedical Large Language Model fine-tuned for Drug–Target Interaction (DTI) prediction. The model was trained on curated drug activity data and is designed to perform two related tasks simultaneously:

1. Classification

Predict whether a drug is likely to be biologically active against a target protein.

2. Regression

Estimate the interaction potency (pXC50) of the drug–target pair.

This makes DTI-LLM a dual-task model, capable of both activity prediction and potency estimation within a single inference pipeline.

The project is currently in an experimental stage and is being actively improved through continued dataset refinement, prompt engineering, architecture experimentation, and evaluation on larger benchmark datasets.

---

Research Goal

The primary objective of this project is to explore how modern Large Language Models can be adapted for computational drug discovery tasks.

The long-term goals include:

• Improving virtual screening workflows
• Assisting early-stage lead prioritization
• Exploring LLM-based molecular reasoning
• Investigating structured biomedical prediction with generative models
• Building lightweight domain-specific AI systems deployable on consumer hardware

This repository represents an ongoing research effort rather than a finished production model.

---

Model Details

Component	Value
Base Model	unsloth/llama-3-8b-bnb-4bit
Fine-Tuning Method	LoRA
LoRA Rank (r)	32
LoRA Alpha	64
LoRA Dropout	0.05
Trainable Parameters	~83.9M
Context Length	1024
Training Samples	~50,000
Training Epochs	1
Training Hardware	NVIDIA T4 16GB
Framework	Unsloth

---

Input Format

The model expects information about:

• Drug molecule (SMILES)
• Protein target (UniProt ID)
• Optional assay metadata

Example:

Drug:
SMILES: NC1=NC(=S)C2=C(N1)N=CN2

Target:
UniProt ID: Q13043

Output Format

The model generates structured JSON:

{
  "is_active": true,
  "pxc50": 6.2,
  "confidence": "high",
  "reasoning": "Structural similarity suggests moderate binding affinity."
}

Field	Description
is_active	Binary activity prediction
pxc50	Predicted potency value
confidence	Model confidence estimate
reasoning	Brief generated explanation

---

Performance

Classification Task (Activity Prediction)

Metric	Score
Accuracy	0.946
Precision	1.000
Recall	0.512
F1 Score	0.658
ROC-AUC	0.765
PR-AUC	0.610

Interpretation

The model currently exhibits extremely high precision.

When the model predicts that a compound is active, it is rarely incorrect. This behavior makes it useful for reducing false positives during early-stage virtual screening.

However, recall remains moderate, meaning some genuinely active compounds may not be identified.

Current development efforts are focused on improving recall while maintaining strong precision.

---

Regression Task (Potency Prediction)

Metric	Score
RMSE	1.099
MAE	0.723
R²	-0.235
Pearson r	0.404
Spearman ρ	0.578

Interpretation

The regression component remains experimental.

While the model demonstrates moderate ranking capability (Spearman correlation 0.578), absolute potency prediction is currently unreliable.

The model can often distinguish stronger interactions from weaker ones, but exact pXC50 values should not be interpreted as experimentally accurate measurements.

For the current release:

✅ Suitable for relative ranking

⚠️ Not suitable for precise potency estimation

Future work will focus heavily on improving regression performance through larger datasets, improved loss functions, and multi-task optimization.

---

Current Development Status

This model is actively being developed.

Planned improvements include:

• Larger and more diverse training datasets
• Additional target protein coverage
• Improved regression accuracy
• Better calibration of confidence scores
• Multi-stage fine-tuning strategies
• Retrieval-augmented biomedical context
• Expanded benchmark evaluation

Performance metrics and model behavior may change significantly between releases.

---

Example Usage

Installation

pip install unsloth transformers accelerate bitsandbytes peft

Loading the Adapter

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/llama-3-8b-bnb-4bit"
)

model = PeftModel.from_pretrained(
    base_model,
    "YOUR_USERNAME/DTI-LLM"
)

tokenizer = AutoTokenizer.from_pretrained(
    "YOUR_USERNAME/DTI-LLM"
)

CLI Inference (Recommended)

The repository includes a ready-to-use inference script for generating Drug–Target Interaction predictions.

Example:

python inference.py \
  --model_path ./lora_adapter \
  --smiles "CCO" \
  --uniprot "P04637" \
  --target_name "p53" \
  --mechanism "binding" \
  --technology "IC50 assay"

Supported Arguments

Argument	Description
--model_path	Path to the LoRA adapter
--smiles	Drug SMILES string
--uniprot	UniProt protein identifier
--target_name	Optional target name
--mechanism	Optional assay mechanism
--technology	Optional assay technology

The CLI script is the recommended way to run inference and reproduce the results reported in this repository.

Repository Contents

lora_adapter/
├── adapter_config.json
├── adapter_model.safetensors
├── tokenizer.json
├── tokenizer_config.json
├── special_tokens_map.json
├── chat_template.jinja
├── training_config.json
└── README.md

---

Limitations

This model has several important limitations.

Regression Performance

Potency prediction remains the weakest component of the system and should be considered experimental.

Dataset Bias

Training data originates from public biological assays and may not represent all protein families, assay conditions, or chemical spaces.

Hallucinated Reasoning

Generated explanations are model-generated text and should not be interpreted as mechanistic biological evidence.

Not for Clinical Use

This model is intended solely for research, education, and experimentation.

Predictions must never be used for:

• Clinical decision making
• Medical diagnosis
• Drug prescription
• Regulatory submissions

All predictions require experimental validation.

---

Intended Use

Appropriate uses include:

• Academic research
• Educational projects
• Drug discovery experimentation
• Virtual screening exploration
• Biomedical AI benchmarking
• LLM fine-tuning demonstrations

---

Acknowledgements

Special thanks to:

• Meta for LLaMA-3
• Unsloth for efficient fine-tuning tools
• The creators of the eve-bio/drug-target-activity dataset
• The open-source biomedical AI community

---

License

Research Only.

This repository is provided for educational and research purposes.

Commercial use may be subject to the license terms of the underlying LLaMA-3 model.

---

Disclaimer

DTI-LLM is an experimental research project under active development.

The model demonstrates promising classification performance and moderate ranking capability, but it is not a substitute for laboratory validation, computational chemistry pipelines, or expert scientific review.