README.md

---
license: llama3

language:
  - en

tags:
  - drug-discovery
  - drug-target-interaction
  - dti
  - biomedical-ai
  - cheminformatics
  - llm
  - llama-3
  - lora
  - unsloth
  - biology
  - regression
  - classification
  - virtual-screening
  - molecular-modeling
  - bioinformatics
  - structure-activity-relationship
  - qsar
  - protein-ligand-interaction

pipeline_tag: text-generation

base_model: unsloth/llama-3-8b-bnb-4bit

datasets:
  - eve-bio/drug-target-activity

library_name: transformers

task_categories:
  - text-generation

widget:
  - text: |
      Drug:
      SMILES: CC(=O)OC1=CC=CC=C1C(=O)O

      Target:
      UniProt ID: P35354
---

# 🧬 DTI-LLM: Drug–Target Interaction Prediction with LLaMA-3

> A LoRA fine-tuned LLaMA-3 model for predicting drug–target interactions from molecular and protein information.
>
> **Current Status:** 🚧 Active Development / Research Prototype

---

# Overview

DTI-LLM is a biomedical Large Language Model fine-tuned for **Drug–Target Interaction (DTI) prediction**. The model was trained on curated drug activity data and is designed to perform two related tasks simultaneously:

### 1. Classification

Predict whether a drug is likely to be biologically active against a target protein.

### 2. Regression

Estimate the interaction potency (pXC50) of the drug–target pair.

This makes DTI-LLM a **dual-task model**, capable of both activity prediction and potency estimation within a single inference pipeline.

The project is currently in an experimental stage and is being actively improved through continued dataset refinement, prompt engineering, architecture experimentation, and evaluation on larger benchmark datasets.

---

# Research Goal

The primary objective of this project is to explore how modern Large Language Models can be adapted for computational drug discovery tasks.

The long-term goals include:

* Improving virtual screening workflows
* Assisting early-stage lead prioritization
* Exploring LLM-based molecular reasoning
* Investigating structured biomedical prediction with generative models
* Building lightweight domain-specific AI systems deployable on consumer hardware

This repository represents an ongoing research effort rather than a finished production model.

---

# Model Details

| Component            | Value                       |
| -------------------- | --------------------------- |
| Base Model           | unsloth/llama-3-8b-bnb-4bit |
| Fine-Tuning Method   | LoRA                        |
| LoRA Rank (r)        | 32                          |
| LoRA Alpha           | 64                          |
| LoRA Dropout         | 0.05                        |
| Trainable Parameters | ~83.9M                      |
| Context Length       | 1024                        |
| Training Samples     | ~50,000                     |
| Training Epochs      | 1                           |
| Training Hardware    | NVIDIA T4 16GB              |
| Framework            | Unsloth                     |

---

# Input Format

The model expects information about:

* Drug molecule (SMILES)
* Protein target (UniProt ID)
* Optional assay metadata

Example:

```text
Drug:
SMILES: NC1=NC(=S)C2=C(N1)N=CN2

Target:
UniProt ID: Q13043
```

# Output Format

The model generates structured JSON:

```json
{
  "is_active": true,
  "pxc50": 6.2,
  "confidence": "high",
  "reasoning": "Structural similarity suggests moderate binding affinity."
}
```

| Field      | Description                 |
| ---------- | --------------------------- |
| is_active  | Binary activity prediction  |
| pxc50      | Predicted potency value     |
| confidence | Model confidence estimate   |
| reasoning  | Brief generated explanation |

---

# Performance

## Classification Task (Activity Prediction)

| Metric    | Score |
| --------- | ----- |
| Accuracy  | 0.946 |
| Precision | 1.000 |
| Recall    | 0.512 |
| F1 Score  | 0.658 |
| ROC-AUC   | 0.765 |
| PR-AUC    | 0.610 |

### Interpretation

The model currently exhibits extremely high precision.

When the model predicts that a compound is active, it is rarely incorrect. This behavior makes it useful for reducing false positives during early-stage virtual screening.

However, recall remains moderate, meaning some genuinely active compounds may not be identified.

Current development efforts are focused on improving recall while maintaining strong precision.

---

## Regression Task (Potency Prediction)

| Metric     | Score  |
| ---------- | ------ |
| RMSE       | 1.099  |
| MAE        | 0.723  |
| R²         | -0.235 |
| Pearson r  | 0.404  |
| Spearman ρ | 0.578  |

### Interpretation

The regression component remains experimental.

While the model demonstrates moderate ranking capability (Spearman correlation 0.578), absolute potency prediction is currently unreliable.

The model can often distinguish stronger interactions from weaker ones, but exact pXC50 values should not be interpreted as experimentally accurate measurements.

For the current release:

✅ Suitable for relative ranking

⚠️ Not suitable for precise potency estimation

Future work will focus heavily on improving regression performance through larger datasets, improved loss functions, and multi-task optimization.

---

# Current Development Status

This model is actively being developed.

Planned improvements include:

* Larger and more diverse training datasets
* Additional target protein coverage
* Improved regression accuracy
* Better calibration of confidence scores
* Multi-stage fine-tuning strategies
* Retrieval-augmented biomedical context
* Expanded benchmark evaluation

Performance metrics and model behavior may change significantly between releases.

---

# Example Usage

## Installation

```bash
pip install unsloth transformers accelerate bitsandbytes peft
```

## Loading the Adapter

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/llama-3-8b-bnb-4bit"
)

model = PeftModel.from_pretrained(
    base_model,
    "YOUR_USERNAME/DTI-LLM"
)

tokenizer = AutoTokenizer.from_pretrained(
    "YOUR_USERNAME/DTI-LLM"
)
```
# CLI Inference (Recommended)

The repository includes a ready-to-use inference script for generating Drug–Target Interaction predictions.

Example:

```bash
python inference.py \
  --model_path ./lora_adapter \
  --smiles "CCO" \
  --uniprot "P04637" \
  --target_name "p53" \
  --mechanism "binding" \
  --technology "IC50 assay"
```

### Supported Arguments

| Argument | Description |
|-----------|-------------|
| `--model_path` | Path to the LoRA adapter |
| `--smiles` | Drug SMILES string |
| `--uniprot` | UniProt protein identifier |
| `--target_name` | Optional target name |
| `--mechanism` | Optional assay mechanism |
| `--technology` | Optional assay technology |

The CLI script is the recommended way to run inference and reproduce the results reported in this repository.
---

# Repository Contents

```text
lora_adapter/
├── adapter_config.json
├── adapter_model.safetensors
├── tokenizer.json
├── tokenizer_config.json
├── special_tokens_map.json
├── chat_template.jinja
├── training_config.json
└── README.md
```

---

# Limitations

This model has several important limitations.

### Regression Performance

Potency prediction remains the weakest component of the system and should be considered experimental.

### Dataset Bias

Training data originates from public biological assays and may not represent all protein families, assay conditions, or chemical spaces.

### Hallucinated Reasoning

Generated explanations are model-generated text and should not be interpreted as mechanistic biological evidence.

### Not for Clinical Use

This model is intended solely for research, education, and experimentation.

Predictions must never be used for:

* Clinical decision making
* Medical diagnosis
* Drug prescription
* Regulatory submissions

All predictions require experimental validation.

---

# Intended Use

Appropriate uses include:

* Academic research
* Educational projects
* Drug discovery experimentation
* Virtual screening exploration
* Biomedical AI benchmarking
* LLM fine-tuning demonstrations

---

# Acknowledgements

Special thanks to:

* Meta for LLaMA-3
* Unsloth for efficient fine-tuning tools
* The creators of the eve-bio/drug-target-activity dataset
* The open-source biomedical AI community

---

# License

Research Only.

This repository is provided for educational and research purposes.

Commercial use may be subject to the license terms of the underlying LLaMA-3 model.

---

## Disclaimer

DTI-LLM is an experimental research project under active development.

The model demonstrates promising classification performance and moderate ranking capability, but it is not a substitute for laboratory validation, computational chemistry pipelines, or expert scientific review.