Upload MoE router and config

README.md

@@ -0,0 +1,9 @@
+# fast-code-moe
+
+Mixture of Experts (MoE) with:
+- Experts: ['mistralai/Mistral-7B-Instruct-v0.2', 'Qwen/Qwen2.5-7B-Instruct']
+- Router: distilbert-base-uncased + MLP
+- Top‑k: 1
+- Quantization: 4‑bit (bitsandbytes)
+
+Trained on a subset of FLAN/IMDb to route instructions to the most suitable expert.

config.json

@@ -0,0 +1,10 @@
+{
+  "experts": [
+    "mistralai/Mistral-7B-Instruct-v0.2",
+    "Qwen/Qwen2.5-7B-Instruct"
+  ],
+  "top_k": 1,
+  "router_encoder": "distilbert-base-uncased",
+  "max_new_tokens": 256,
+  "description": "Claude-style MoE with lazy-loaded 4-bit experts"
+}

router.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d48864ec5198121793e02acc06d657a4fc48aba711a4d776662f80d70a643ed8
+size 791891