Instructions to use throsturx/bihmoe-poc with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use throsturx/bihmoe-poc with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("throsturx/bihmoe-poc", dtype="auto") - Notebooks
- Google Colab
- Kaggle
Architecture notes (high level)
This repo compares a dense baseline vs a structured "dual-stream" model on synthetic compositional tasks.
Key stabilized regime (what produced the replicated ood_long gains):
- chiasm_mode = bias (no information starvation)
- curriculum mixing that includes long+gap samples during training
- Schedule-2 with ramp; braid rehearsal prevents late-phase regression
See docs/preliminary_results.md for the replication panel and ablations.