README.md

---
license: other
library_name: pytorch
tags:
- biology
- bioinformatics
- biosynthetic-gene-clusters
- retrieval
- protein-language-models
- pfam
- pytorch
pipeline_tag: feature-extraction
---

# BGC SetNet and Weighted Pfam Retrieval Models

This repository contains the final five-seed model artifacts for the manuscript **"Sequence-Derived Representations versus Pfam-Domain Content for Biosynthetic Gene Cluster Retrieval"**.

The release includes two model families:

- **Pfam-augmented BGC-SetNet** (`pfam_setnet`): a Set Transformer encoder over frozen ESM-2 gene embeddings, relative gene positions, padding masks, and a BGC-level Pfam inventory embedding.
- **Weighted Pfam Jaccard** (`weighted_pfam_jaccard`): learned non-negative Pfam-domain weights used in a weighted set-Jaccard retrieval baseline.

The manuscript's main result is conservative: Pfam-domain content remains the strongest signal for this silver-label retrieval benchmark, while the ESM + BGC-SetNet + Pfam ensemble gives small, statistically unsupported gains on some secondary metrics.

## Repository Contents

```text
checkpoints/
  setnet/seed-20260810..20260814/
    model.safetensors      # verified model_state checkpoint
    config.json
    pfam_vocab.json
    phase2_history.json
  weighted_pfam/seed-20260810..20260814/
    model.safetensors
    config.json
    pfam_vocab.json
    phase2_history.json
evaluation/                # per-seed metadata, summaries, group/query outputs, alpha searches
results/dgx_final/          # aggregate manuscript result tables
source/                     # source snapshot used for training/evaluation/provenance
provenance/checkpoint_manifest.json
```

## Evaluation Summary

Family means over five optimization seeds and 16 held-out MIBiG-reference groups:

| Method | Recall@50 | MRR | MAP | nDCG@50 |
|---|---:|---:|---:|---:|
| Raw ESM mean | 0.7946 | 0.2550 | 0.7251 | 0.8078 |
| BGC-SetNet + Pfam | 0.8472 | 0.2786 | 0.7771 | 0.8502 |
| Pfam Jaccard | 0.8788 | 0.3071 | 0.8480 | 0.9042 |
| ESM + BGC-SetNet + Pfam | 0.8769 | 0.3096 | 0.8503 | 0.9058 |
| Weighted Pfam Jaccard | 0.8789 | 0.3069 | 0.8477 | 0.9040 |

The weighted Pfam Recall@50 advantage over unweighted Pfam is numerically tiny (+0.00003) and should not be interpreted as a meaningful improvement.

## Checkpoint Verification

All ten source checkpoints were verified by SHA-256 before conversion to clean `model.safetensors` files. The split file hash used by the final run is:

```text
dc26fae17e54fd2ad41a9e10353b3da3e0aacf3144b64f6ee62e8341b4360555
```

See `provenance/checkpoint_manifest.json` for per-seed original checkpoint hashes, safetensors hashes, parameter counts, selected validation ensemble alpha, model config hashes, and split hashes.

## Minimal Loading Example

```python
import json
import torch
from safetensors.torch import load_file

from source.src.bgc_retrieval.model import ModelConfig, build_model

seed_dir = "checkpoints/setnet/seed-20260810"
config = json.load(open(f"{seed_dir}/config.json"))["model"]
model = build_model(ModelConfig.from_dict(config))
state = load_file(f"{seed_dir}/model.safetensors")
model.load_state_dict(state)
model.eval()
```

For `pfam_setnet`, inference expects:

- `gene_embeddings`: `[batch, genes, 1280]` frozen ESM-2 gene embeddings
- `relative_positions`: `[batch, genes]` normalized gene positions
- `padding_mask`: `[batch, genes]` boolean padding mask
- `pfam_tokens`: `[batch, domains]` Pfam IDs encoded with `pfam_vocab.json` (`0` = padding, `1` = unknown)

For `weighted_pfam_jaccard`, inference expects only `pfam_tokens` and returns learned domain weights; pairwise retrieval scores are computed with the model's `pairwise_jaccard` method.

## Intended Use

These models are research artifacts for reproducing and auditing BGC retrieval experiments from the associated manuscript. They are not validated for clinical, diagnostic, regulatory, or production biological decision-making.

## Limitations

- The benchmark labels are silver labels derived from shared MIBiG references, not a manually curated gold retrieval set.
- The split is group-disjoint by MIBiG-reference group, but it is not a complete sequence-homology separation benchmark.
- The strongest primary-metric methods are Pfam-based; learned sequence-derived representations did not show statistically supported superiority in this study.
- The large input atlas, frozen ESM-2 embeddings, and raw gene annotation tables are not bundled in this model repository.
- License terms are pending; contact the authors before reuse beyond inspection and reproducibility work.

## Citation

Citation metadata will be updated after the preprint is posted. For now, cite the manuscript title above and this Hugging Face repository.