README.md

metadata

language:
  - en
tags:
  - ecology
  - multimodal
  - missing-modality
  - masking
  - species-distribution-modeling
  - species-classification
library_name: pytorch
pipeline_tag: image-classification
datasets:
  - MVRL/TaxaBench-8k
metrics:
  - accuracy
  - auc

MIAM: Modality Imbalance-Aware Masking for Multimodal Ecological Applications

This repository hosts released checkpoints for MIAM from the ICLR 2026 paper:

• Paper: https://openreview.net/forum?id=oljjAkgZN4
• Project page: https://zbirobin.github.io/publications/miam/
• Source code: https://github.com/zbirobin/MIAM

MIAM is a dynamic masking strategy for multimodal ecological learning. During training, it adapts masking probabilities using modality-specific performance and learning-speed signals to reduce modality imbalance and improve robustness to missing inputs.

Model details

• Model family: Multimodal ecological models trained with MIAM and masking baselines
• Modalities covered:
  • GeoPlant (MaskSDM): satellite + environmental tabular + climate time series
  • TaxaBench-8k: location + environmental tabular + natural image + audio + satellite
  • SatBird: satellite + environmental tabular
• Framework: PyTorch

Available files

GeoPlant (MaskSDM)

• geoplant_miam.pt
• geoplant_opm.pt
• geoplant_dropout.pt
• geoplant_constant.pt
• geoplant_dirichlet.pt
• geoplant_uniform.pt
• geoplant_pretraining_miam.pt
• geoplant_pretraining_opm.pt
• geoplant_pretraining_dropout.pt
• geoplant_pretraining_constant.pt
• geoplant_pretraining_dirichlet.pt
• geoplant_pretraining_uniform.pt

TaxaBench-8k

• taxabench_miam.pt
• taxabench_dirichlet.pt
• taxabench_dropout.pt
• taxabench_opm.pt
• taxabench_uniform.pt
• taxabench_embeds_loc_env_img_aud_sat.pt
• taxabench_num_species_10.csv

SatBird

• satbird_miam.pth
• satbird_opm.pth
• satbird_dropout.pth
• satbird_dirichlet.pth

Quick start

Download with hf CLI

hf download zbirobin/MIAM geoplant_miam.pt
hf download zbirobin/MIAM taxabench_miam.pt
hf download zbirobin/MIAM satbird_miam.pth

Download in Python

from huggingface_hub import hf_hub_download
import torch

ckpt_path = hf_hub_download(repo_id="zbirobin/MIAM", filename="geoplant_miam.pt")
state = torch.load(ckpt_path, map_location="cpu")

Intended use

• Research on multimodal ecological modeling
• Robustness to missing-modality settings
• Benchmark comparison of masking strategies
• Modality contribution analysis and interpretability studies

Out-of-scope use

• Safety-critical ecological decisions without domain validation
• Deployment outside the data distributions represented in GeoPlant, TaxaBench-8k, and SatBird
• Applications requiring calibrated uncertainty without additional validation

Training and evaluation data

• GeoPlant: species distribution modeling benchmark used in the paper
• TaxaBench-8k: multimodal species classification benchmark
• SatBird-USA-summer: bird species distribution benchmark

Please follow dataset licenses, terms of use, and any access restrictions from the original providers.

Limitations

• Transferability across regions/taxa/modalities may be limited

Reproducibility

Use the benchmark READMEs in the source repository for exact folder structure, commands, and evaluation scripts:

• maskSDM/README.md
• taxabench/README.md
• satbird/README.md

Citation

@inproceedings{
  zbinden2026miam,
  title={{MIAM}: Modality Imbalance-Aware Masking for Multimodal Ecological Applications},
  author={Robin Zbinden and Wesley Monteith-Finas and Gencer Sumbul and Nina van Tiel and Chiara Vanalli and Devis Tuia},
  booktitle={International Conference on Learning Representations (ICLR)},
  year={2026},
  url={https://openreview.net/forum?id=oljjAkgZN4}
}

Contact

For issues and questions, please open a ticket in the source repository.