File size: 3,938 Bytes
45fc6ba b8b5b6e 45fc6ba b8b5b6e 45fc6ba b8b5b6e 45fc6ba b8b5b6e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 | ---
language:
- en
tags:
- ecology
- multimodal
- missing-modality
- masking
- species-distribution-modeling
- species-classification
library_name: pytorch
pipeline_tag: image-classification
datasets:
- MVRL/TaxaBench-8k
metrics:
- accuracy
- auc
---
# MIAM: Modality Imbalance-Aware Masking for Multimodal Ecological Applications
This repository hosts released checkpoints for **MIAM** from the ICLR 2026 paper:
- Paper: https://openreview.net/forum?id=oljjAkgZN4
- Project page: https://zbirobin.github.io/publications/miam/
- Source code: https://github.com/zbirobin/MIAM
MIAM is a dynamic masking strategy for multimodal ecological learning. During training, it adapts masking probabilities using modality-specific performance and learning-speed signals to reduce modality imbalance and improve robustness to missing inputs.
## Model details
- **Model family**: Multimodal ecological models trained with MIAM and masking baselines
- **Modalities covered**:
- GeoPlant (MaskSDM): satellite + environmental tabular + climate time series
- TaxaBench-8k: location + environmental tabular + natural image + audio + satellite
- SatBird: satellite + environmental tabular
- **Framework**: PyTorch
## Available files
### GeoPlant (MaskSDM)
- `geoplant_miam.pt`
- `geoplant_opm.pt`
- `geoplant_dropout.pt`
- `geoplant_constant.pt`
- `geoplant_dirichlet.pt`
- `geoplant_uniform.pt`
- `geoplant_pretraining_miam.pt`
- `geoplant_pretraining_opm.pt`
- `geoplant_pretraining_dropout.pt`
- `geoplant_pretraining_constant.pt`
- `geoplant_pretraining_dirichlet.pt`
- `geoplant_pretraining_uniform.pt`
### TaxaBench-8k
- `taxabench_miam.pt`
- `taxabench_dirichlet.pt`
- `taxabench_dropout.pt`
- `taxabench_opm.pt`
- `taxabench_uniform.pt`
- `taxabench_embeds_loc_env_img_aud_sat.pt`
- `taxabench_num_species_10.csv`
### SatBird
- `satbird_miam.pth`
- `satbird_opm.pth`
- `satbird_dropout.pth`
- `satbird_dirichlet.pth`
## Quick start
### Download with `hf` CLI
```bash
hf download zbirobin/MIAM geoplant_miam.pt
hf download zbirobin/MIAM taxabench_miam.pt
hf download zbirobin/MIAM satbird_miam.pth
```
### Download in Python
```python
from huggingface_hub import hf_hub_download
import torch
ckpt_path = hf_hub_download(repo_id="zbirobin/MIAM", filename="geoplant_miam.pt")
state = torch.load(ckpt_path, map_location="cpu")
```
## Intended use
- Research on multimodal ecological modeling
- Robustness to missing-modality settings
- Benchmark comparison of masking strategies
- Modality contribution analysis and interpretability studies
## Out-of-scope use
- Safety-critical ecological decisions without domain validation
- Deployment outside the data distributions represented in GeoPlant, TaxaBench-8k, and SatBird
- Applications requiring calibrated uncertainty without additional validation
## Training and evaluation data
- **GeoPlant**: species distribution modeling benchmark used in the paper
- **TaxaBench-8k**: multimodal species classification benchmark
- **SatBird-USA-summer**: bird species distribution benchmark
Please follow dataset licenses, terms of use, and any access restrictions from the original providers.
## Limitations
- Transferability across regions/taxa/modalities may be limited
## Reproducibility
Use the benchmark READMEs in the source repository for exact folder structure, commands, and evaluation scripts:
- `maskSDM/README.md`
- `taxabench/README.md`
- `satbird/README.md`
## Citation
```bibtex
@inproceedings{
zbinden2026miam,
title={{MIAM}: Modality Imbalance-Aware Masking for Multimodal Ecological Applications},
author={Robin Zbinden and Wesley Monteith-Finas and Gencer Sumbul and Nina van Tiel and Chiara Vanalli and Devis Tuia},
booktitle={International Conference on Learning Representations (ICLR)},
year={2026},
url={https://openreview.net/forum?id=oljjAkgZN4}
}
```
## Contact
For issues and questions, please open a ticket in the source repository. |