anzheCheng commited on
Commit
1f1ec1c
·
verified ·
1 Parent(s): de6781f

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +77 -5
README.md CHANGED
@@ -1,10 +1,82 @@
1
  ---
 
 
 
 
 
 
 
 
 
2
  tags:
 
 
 
 
3
  - model_hub_mixin
4
- - pytorch_model_hub_mixin
5
  ---
6
 
7
- This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
8
- - Code: [More Information Needed]
9
- - Paper: [More Information Needed]
10
- - Docs: [More Information Needed]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: cc-by-4.0
3
+ library_name: pytorch
4
+ pipeline_tag: image-classification
5
+ datasets:
6
+ - ILSVRC/imagenet-1k
7
+ - uoft-cs/cifar10
8
+ - uoft-cs/cifar100
9
+ metrics:
10
+ - accuracy
11
  tags:
12
+ - image-classification
13
+ - vision-transformer
14
+ - mixture-of-experts
15
+ - pytorch
16
  - model_hub_mixin
 
17
  ---
18
 
19
+ # EMoE: Eigenbasis-Guided Routing for Mixture-of-Experts
20
+
21
+ This repository hosts pretrained checkpoints for **EMoE** and a Hub-compatible loading path.
22
+
23
+ Paper: https://arxiv.org/abs/2601.12137
24
+ Code: https://github.com/Belis0811/EMoE
25
+
26
+ ## Available checkpoints
27
+
28
+ - `model.safetensors`: EMoE ViT-Base in standard Hub format (`vit_base_patch16_224`, ImageNet-1k)
29
+ - `eigen_moe_vit_base_patch16_224_imagenet1k.pth`
30
+ - `eigen_moe_vit_large_patch16_224.augreg_in21k_ft_in1k_imagenet1k.pth`
31
+ - `eigen_moe_vit_huge_patch14_224_in21k_imagenet1k.pth`
32
+
33
+ ## Usage
34
+
35
+ Install dependencies:
36
+
37
+ ```bash
38
+ pip install -U torch timm huggingface_hub safetensors
39
+ ```
40
+
41
+ Load the Hub-formatted checkpoint:
42
+
43
+ ```python
44
+ import torch
45
+ from eigen_moe import HFEigenMoE
46
+
47
+ model = HFEigenMoE.from_pretrained(
48
+ "anzheCheng/EMoE",
49
+ vit_model_name="vit_base_patch16_224",
50
+ num_classes=1000,
51
+ strict=False,
52
+ )
53
+ model.eval()
54
+
55
+ x = torch.randn(1, 3, 224, 224)
56
+ with torch.no_grad():
57
+ logits = model(x)
58
+ print(logits.shape)
59
+ ```
60
+
61
+ Load one of the original `.pth` files explicitly:
62
+
63
+ ```python
64
+ model = HFEigenMoE.from_pretrained(
65
+ "anzheCheng/EMoE",
66
+ vit_model_name="vit_large_patch16_224.augreg_in21k_ft_in1k",
67
+ num_classes=1000,
68
+ checkpoint_filename="eigen_moe_vit_large_patch16_224.augreg_in21k_ft_in1k_imagenet1k.pth",
69
+ strict=False,
70
+ )
71
+ ```
72
+
73
+ ## Citation
74
+
75
+ ```bibtex
76
+ @article{cheng2026emoe,
77
+ title={EMoE: Eigenbasis-Guided Routing for Mixture-of-Experts},
78
+ author={Cheng, Anzhe and Duan, Shukai and Li, Shixuan and Yin, Chenzhong and Cheng, Mingxi and Nazarian, Shahin and Thompson, Paul and Bogdan, Paul},
79
+ journal={arXiv preprint arXiv:2601.12137},
80
+ year={2026}
81
+ }
82
+ ```