YoshimuraHiroto/bfm-finetuned-large

BFM Finetuned Large (GeoLifeCLEF 500 Species)

Finetuned version of the BioAnalyst Foundation Model (BFM) Large (702M params) on GeoLifeCLEF 2024 species distribution data (500 species).

Model Details

• Base model: BFM Large (embed_dim=512, depth=10, Swin 3D backbone)
• Parameters: 702M total, 320M trainable (BFMRaw wrapper)
• Finetuning data: GeoLifeCLEF 2024 (500 species, 0.25 deg grid, Europe)
• Training: 100 epochs, L1 loss, AdamW (lr=3e-4), CosineAnnealingLR
• Best validation loss: 0.007513466756790876

Architecture

The finetuning uses the BFMRaw wrapper from the BFM paper (arXiv:2507.09080v2):

1. Base BFM encoder/decoder replaced with nn.Identity()
2. Custom TemporalSpatialEncoder projects species tensor to backbone token space
3. Swin 3D backbone processes species tokens
4. Custom TemporalSpatialDecoder reconstructs species predictions

Evaluation on BioCube 28 Species

Metric	Finetuned	Paper Reference
F1 (Eq.20)	0.8950	0.9964
Mean Species MAE (norm)	0.0216	0.0836

Usage

import torch
from bfm_model.bfm.model import BFM
from bfm_finetune.bfm_mod import BFMRaw

# 1. Build base model
base_model = BFM(
    embed_dim=512, depth=10, patch_size=8,
    swin_backbone_size="large", perceiver_latents=16100,
    # ... (see finetune_large_geolifeclef.py for full config)
)

# 2. Load pretrained weights
from safetensors.torch import load_file
state = load_file("bfm-pretrain-large.safetensors", device="cpu")
base_model.load_state_dict(state, strict=False)

# 3. Wrap with BFMRaw and load finetuned weights
model = BFMRaw(base_model=base_model, n_species=500, mode="eval")
ckpt = torch.load("best_checkpoint.pth", map_location="cpu")
model.load_state_dict(ckpt["model_state_dict"], strict=False)
model.eval()

Files

• best_checkpoint.pth - Finetuned model checkpoint (epoch 100)
• bfm_finetuned_eval_large_results.json - Full evaluation results
• finetune_large_geolifeclef.py - Finetuning script
• eval_finetuned_large.py - Evaluation script

Citation

@article{foerster2025bfm,
  title={A Foundation Model for Forecasting Biodiversity Dynamics},
  author={Foerster et al.},
  journal={arXiv preprint arXiv:2507.09080v2},
  year={2025}
}