README.md

---
license: cc-by-4.0
library_name: pytorch
pipeline_tag: image-classification
datasets:
- ILSVRC/imagenet-1k
- uoft-cs/cifar10
- uoft-cs/cifar100
metrics:
- accuracy
arxiv: 2601.12137
tags:
- image-classification
- vision-transformer
- mixture-of-experts
- pytorch
- model_hub_mixin
---

# EMoE: Eigenbasis-Guided Routing for Mixture-of-Experts

This repository hosts pretrained checkpoints for **EMoE** and a Hub-compatible loading path.

Paper: https://arxiv.org/abs/2601.12137  or https://huggingface.co/papers/2601.12137

Code: https://github.com/Belis0811/EMoE

## Available checkpoints

- `model.safetensors`: EMoE ViT-Base in standard Hub format (`vit_base_patch16_224`, ImageNet-1k)
- `eigen_moe_vit_base_patch16_224_imagenet1k.pth`
- `eigen_moe_vit_large_patch16_224.augreg_in21k_ft_in1k_imagenet1k.pth`
- `eigen_moe_vit_huge_patch14_224_in21k_imagenet1k.pth`

## Usage

Install dependencies:

```bash
pip install -U torch timm huggingface_hub safetensors
```

Load the Hub-formatted checkpoint:

```python
import torch
from eigen_moe import HFEigenMoE

model = HFEigenMoE.from_pretrained(
    "anzheCheng/EMoE",
    vit_model_name="vit_base_patch16_224",
    num_classes=1000,
    strict=False,
)
model.eval()

x = torch.randn(1, 3, 224, 224)
with torch.no_grad():
    logits = model(x)
print(logits.shape)
```

Load one of the original `.pth` files explicitly:

```python
model = HFEigenMoE.from_pretrained(
    "anzheCheng/EMoE",
    vit_model_name="vit_large_patch16_224.augreg_in21k_ft_in1k",
    num_classes=1000,
    checkpoint_filename="eigen_moe_vit_large_patch16_224.augreg_in21k_ft_in1k_imagenet1k.pth",
    strict=False,
)
```

## Citation

```bibtex
@article{cheng2026emoe,
  title={EMoE: Eigenbasis-Guided Routing for Mixture-of-Experts},
  author={Cheng, Anzhe and Duan, Shukai and Li, Shixuan and Yin, Chenzhong and Cheng, Mingxi and Nazarian, Shahin and Thompson, Paul and Bogdan, Paul},
  journal={arXiv preprint arXiv:2601.12137},
  year={2026}
}
```