HaploLM and HaploLM-VL

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,116 @@
+---
+license: mit
+library_name: pytorch
+tags:
+  - biology
+  - genomics
+  - dna
+  - mitochondrial-dna
+  - haplogroups
+  - dna-language-model
+---
+
+# HaploLM
+
+HaploLM and HaploLM-VL are the two DNA language models of the paper **Human Maternal Family Tree
+from DNA Language Models** by Faias Satter and Aminul Islam.
+They are trained only to predict human mitochondrial DNA and never see a haplogroup label, yet
+they learn the human maternal family tree (Mitotree).
+
+Code, data preparation and every test: [github.com/FaiasPromit/HaploLM](https://github.com/FaiasPromit/HaploLM)
+
+| Folder | Model | Reads | Parameters | Training |
+|---|---|---|---|---|
+| `SC30/` | **HaploLM** | the whole mitochondrial sequence, base by base | 20.6M | causal (each base predicted from the bases before it), 30 epochs over 49,310 sequences |
+| `VC/` | **HaploLM-VL** | only the person's variants (differences from the reference sequence, rCRS), about 37 per person | 11.2M | causal, 20,000 steps of 128 people (about 52 epochs over 49,259 variant lists) |
+
+Each folder holds `checkpoint.pt`, a PyTorch file with the parameters (`model`), the settings
+(`args`) and the training step (`step`). For HaploLM-VL, `vocab` maps every distinct variant
+seen in training to its token. The folder names are the run names used in the GitHub repository.
+
+## Results
+
+| Model | Marker test | Frequency-matched | Carrier vs. relative | Held-out branches |
+|---|---|---|---|---|
+| Counting (no model) | .890 | .533 | .500 | .475 |
+| HaploLM | .947 | .872 | .808 | .467 |
+| HaploLM-VL | .955 | .939 | .965 | .502 |
+
+- **Marker test**: AUROC for telling a person's branch markers from their private variants, by
+  the probability the model gives each variant.
+- **Frequency-matched**: the same, with markers compared only with private variants carried by
+  about as many training people.
+- **Carrier vs. relative** (the main score): the share of pairs in which the model finds a
+  branch marker more probable in a person who carries it than in a close relative who does not;
+  0.5 is random guessing.
+- **Held-out branches** (negative control): the marker test on six branches excluded from
+  training. A model that relies on relationships learned during training should score near 0.5
+  here, and all models do.
+
+Counting scores a variant by how many training people carry it. The published Evo 2, NTv3 and
+mtDNA-FM score close to random guessing on all four tests.
+
+## How to use
+
+The model code is in the GitHub repository. To score the paper's tests with these checkpoints,
+first build the data (step 1 of the repository's README), then:
+
+```bash
+python -c "from huggingface_hub import snapshot_download; snapshot_download('FaiasPromit/HaploLM', local_dir='runs', allow_patterns=['SC30/*', 'VC/*'])"
+torchrun --standalone --nproc_per_node=4 scripts/score_causal_sequence_model.py --run runs/SC30
+python scripts/train_causal_variant_list_model.py --name VC --score-only
+```
+
+To load HaploLM in Python and score a variant in a sequence:
+
+```python
+import sys
+
+import numpy as np
+import torch
+from huggingface_hub import hf_hub_download
+
+sys.path.insert(0, "HaploLM/scripts")       # the scripts folder of the GitHub repository
+from train_causal_sequence_model import CausalSequenceModel, next_base_logp
+from train_masked_sequence_model import BASE_CODE
+
+ck = torch.load(hf_hub_download("FaiasPromit/HaploLM", "SC30/checkpoint.pt"),
+                map_location="cpu", weights_only=False)
+a = ck["args"]
+model = CausalSequenceModel(a["d"], a["layers"], a["heads"], dropout=0.0)
+model.load_state_dict(ck["model"])
+model.eval()
+
+
+def log_prob(seq):
+    """log P(seq): the sum over bases of log P(base | the bases before it)."""
+    ids = torch.from_numpy(BASE_CODE[np.frombuffer(seq.encode("ascii"), np.uint8)]).long()[None]
+    with torch.no_grad():
+        lp, _ = next_base_logp(model(ids), ids)
+    return lp.double().sum().item()
+
+# score of a variant, as in the paper:
+# log_prob(sequence with the variant) - log_prob(the same sequence with the rCRS base there)
+```
+
+The sequence must be in upper case, start at position 1 of the rCRS (the repository's
+`clean_sequences.py` rotates sequences this way) and be at most 16,768 bases long. A GPU is much
+faster: move `model` and `ids` to `"cuda"`.
+
+## Training data
+
+49,310 human mitochondrial genomes from GenBank, on MITOMAP's list of full-length sequences:
+the training set of our split. Identical sequences are always in the same set, and six
+branches (L0a1, L3f, M9a, D4b, H13 and U2e), with everything below them, are held out of
+training. The haplogroup labels, from HaploGrep 3 with Mitotree 2026.05.06, are used only to
+evaluate the models.
+
+## Limitations
+
+- The models are trained on human mitochondrial DNA only.
+- The answer key depends on HaploGrep and Mitotree. The tests limit the effects of errors by
+  excluding poorly assigned sequences, hotspots and the control region.
+- Public mitochondrial data overrepresent some populations, leaving lineages common in
+  undersampled populations with fewer training examples.
+- They are research models for studying what DNA language models learn. They are not
+  haplogroup classifiers and are not validated for medical or forensic use.

SC30/checkpoint.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af819268c3f8e14ad668240423b51ca6fe4239d920343a88dc64b7a1b689ad98
+size 82597303

VC/checkpoint.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d988d0071d64a52525cfd0751822b98b5e67acf10b03a0d1b79b3c5e13661e6
+size 45275125