MiCoFormer
MiCoFormer is a pretrained encoder for genus-level microbial community profiles. It takes a sample's genus-level relative abundances aligned to Greengenes2 and returns a 768-dimensional sample embedding. This release provides the epoch-96 checkpoint and uses the MiCoFormer v0.5 Python code.
| Item | Value |
|---|---|
| Checkpoint | micoformer_epoch096.pt |
| Architecture | 12 Transformer layers, width 768, 16 attention heads, approximately 67.5M trainable parameters |
| Vocabulary | 8,114 genera aligned to Greengenes2 2024.09 |
| Training corpus | MCFCorpus V4.3, 1,978,718 training observations and 9,943 validation observations |
| Checkpoint selection | Epoch 96, step 659,616, with the lowest weighted pretraining validation loss (0.472478314) in this run |
The checkpoint contains the model tensors, genus vocabulary and fixed genus prior table. The prior combines 288 phylogenetic features and 480 protein features. See MCFCorpus sources and citations for the taxonomy, assemblies and representation methods underlying those features.
Quickstart
Use Python 3.12 to install the v0.5 release, then run the example from the cloned repository. The example downloads the checkpoint and the public MCFCorpus V20261119 AnnData file (about 2.7 GB), then embeds its first four representative observations.
git clone --depth 1 --branch v0.5 https://github.com/kiancai/MiCoFormer.git
cd MiCoFormer
python -m pip install . huggingface_hub
from pathlib import Path
import h5py
import numpy as np
from huggingface_hub import hf_hub_download
from micoformer.workflows.extract import extract_representations
weights = hf_hub_download(
repo_id="kiancai/MiCoFormer",
filename="micoformer_epoch096.pt",
)
corpus = hf_hub_download(
repo_id="kiancai/MCFCorpus",
repo_type="dataset",
revision="5d04e5acba60cc93d39f35cff9ce62f18dc0d5ba",
filename="MCFCorpusV20261119.gg2.h5ad",
)
rows_path = Path("example_rows.npy")
with h5py.File(corpus, "r") as source:
rows = np.flatnonzero(source["obs/analysis_primary"][:])[:4]
np.save(rows_path, rows, allow_pickle=False)
output = extract_representations(
checkpoint=Path(weights),
corpus=Path(corpus),
rows_path=rows_path,
output=Path("micoformer_example"),
device="cpu",
batch_size=4,
precision="float32",
)
embeddings = np.load(output / "embeddings.npy", allow_pickle=False)
print(embeddings.shape) # (4, 768)
For your own data, supply an AnnData .h5ad with a sparse CSR relative-abundance matrix in X and the checkpoint's genus taxonomy paths in var_names, in the order given by the genus list. The extraction code takes up to 512 observed genera ranked by abundance, centers log abundances within those retained genera, and averages the final-layer representations of valid tokens without masking.
Training and evaluation scope
Training used weighted objectives for masked genus identities, continuous abundance and presence, with weak UniFrac supervision. The validation set was sampled at the observation level with seed 42. See MCFCorpus V20261119 for the public inference example and corpus documentation. This release is intended for microbiome representation research. Performance on independent studies and downstream tasks requires task-specific evaluation.
License and citation
The released weights are labeled CC BY-NC 4.0. The MiCoFormer v0.5 source code is MIT-licensed. Cite this Hugging Face repository and the epoch-96 checkpoint, together with the relevant upstream resources for your use. File integrity can be checked with SHA256SUMS.