README.md

---
language:
- en
tags:
- ecology
- multimodal
- missing-modality
- masking
- species-distribution-modeling
- species-classification
library_name: pytorch
pipeline_tag: image-classification
datasets:
- MVRL/TaxaBench-8k
metrics:
- accuracy
- auc
---

# MIAM: Modality Imbalance-Aware Masking for Multimodal Ecological Applications

This repository hosts released checkpoints for **MIAM** from the ICLR 2026 paper:

- Paper: https://openreview.net/forum?id=oljjAkgZN4
- Project page: https://zbirobin.github.io/publications/miam/
- Source code: https://github.com/zbirobin/MIAM

MIAM is a dynamic masking strategy for multimodal ecological learning. During training, it adapts masking probabilities using modality-specific performance and learning-speed signals to reduce modality imbalance and improve robustness to missing inputs.

## Model details

- **Model family**: Multimodal ecological models trained with MIAM and masking baselines
- **Modalities covered**:
  - GeoPlant (MaskSDM): satellite + environmental tabular + climate time series
  - TaxaBench-8k: location + environmental tabular + natural image + audio + satellite
  - SatBird: satellite + environmental tabular
- **Framework**: PyTorch

## Available files

### GeoPlant (MaskSDM)

- `geoplant_miam.pt`
- `geoplant_opm.pt`
- `geoplant_dropout.pt`
- `geoplant_constant.pt`
- `geoplant_dirichlet.pt`
- `geoplant_uniform.pt`
- `geoplant_pretraining_miam.pt`
- `geoplant_pretraining_opm.pt`
- `geoplant_pretraining_dropout.pt`
- `geoplant_pretraining_constant.pt`
- `geoplant_pretraining_dirichlet.pt`
- `geoplant_pretraining_uniform.pt`

### TaxaBench-8k

- `taxabench_miam.pt`
- `taxabench_dirichlet.pt`
- `taxabench_dropout.pt`
- `taxabench_opm.pt`
- `taxabench_uniform.pt`
- `taxabench_embeds_loc_env_img_aud_sat.pt`
- `taxabench_num_species_10.csv`

### SatBird

- `satbird_miam.pth`
- `satbird_opm.pth`
- `satbird_dropout.pth`
- `satbird_dirichlet.pth`

## Quick start

### Download with `hf` CLI

```bash
hf download zbirobin/MIAM geoplant_miam.pt
hf download zbirobin/MIAM taxabench_miam.pt
hf download zbirobin/MIAM satbird_miam.pth
```

### Download in Python

```python
from huggingface_hub import hf_hub_download
import torch

ckpt_path = hf_hub_download(repo_id="zbirobin/MIAM", filename="geoplant_miam.pt")
state = torch.load(ckpt_path, map_location="cpu")
```

## Intended use

- Research on multimodal ecological modeling
- Robustness to missing-modality settings
- Benchmark comparison of masking strategies
- Modality contribution analysis and interpretability studies

## Out-of-scope use

- Safety-critical ecological decisions without domain validation
- Deployment outside the data distributions represented in GeoPlant, TaxaBench-8k, and SatBird
- Applications requiring calibrated uncertainty without additional validation

## Training and evaluation data

- **GeoPlant**: species distribution modeling benchmark used in the paper
- **TaxaBench-8k**: multimodal species classification benchmark
- **SatBird-USA-summer**: bird species distribution benchmark

Please follow dataset licenses, terms of use, and any access restrictions from the original providers.

## Limitations

- Transferability across regions/taxa/modalities may be limited

## Reproducibility

Use the benchmark READMEs in the source repository for exact folder structure, commands, and evaluation scripts:

- `maskSDM/README.md`
- `taxabench/README.md`
- `satbird/README.md`

## Citation

```bibtex
@inproceedings{
  zbinden2026miam,
  title={{MIAM}: Modality Imbalance-Aware Masking for Multimodal Ecological Applications},
  author={Robin Zbinden and Wesley Monteith-Finas and Gencer Sumbul and Nina van Tiel and Chiara Vanalli and Devis Tuia},
  booktitle={International Conference on Learning Representations (ICLR)},
  year={2026},
  url={https://openreview.net/forum?id=oljjAkgZN4}
}
```

## Contact

For issues and questions, please open a ticket in the source repository.