throsturx
/

bihmoe-poc

mixture-of-experts

interpretability

Model card Files Files and versions

bihmoe-poc / src /bihmoe /models

19.8 kB

Ctrl+K

Ctrl+K

1 contributor

History: 11 commits

Throstur

arch: add chiasm_mode=bias to avoid global-branch starvation

74790c5 5 months ago

__init__.py

0 Bytes
head: make models query-aware (last-token readout) + trainer update 5 months ago
blocks.py

4.68 kB
arch-v2: asymmetry (local vs global), competitive callosum, optic-chiasm bias; default stem=0 5 months ago
dense.py

2.14 kB
head: make models query-aware (last-token readout) + trainer update 5 months ago
structured.py

12.9 kB
arch: add chiasm_mode=bias to avoid global-branch starvation 5 months ago