Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,14 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tinymoe2m.BF16.gguf filter=lfs diff=lfs merge=lfs -text
+tinymoe2m.F16.gguf filter=lfs diff=lfs merge=lfs -text
+tinymoe2m.F32.gguf filter=lfs diff=lfs merge=lfs -text
+tinymoe2m.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+tinymoe2m.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+tinymoe2m.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+tinymoe2m.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
+tinymoe2m.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+tinymoe2m.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+tinymoe2m.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+tinymoe2m.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,135 @@
+---
+license: mit
+base_model: mistralai/Mixtral-8x7B-v0.1
+tags:
+- mixtral
+- moe
+- gguf
+- safetensors
+- transformers
+- validation
+- test-suite
+---
+
+# TinyStories Mixtral 2M Top-2 MoE (tinymoe2m) GGUF & HF Validation Suite
+
+This repository provides an ultra-lightweight Mixtral model variant (a Mixture-of-Experts architecture utilizing the Llama 2 compute topology) scaled down to a **1.95M total parameter footprint** and a **1.14M active parameter execution frame**. It is trained on the TinyStories dataset and optimized as a precise validation asset.
+
+It is designed specifically for debugging custom inference engines, and native tensor compilers against MoE-specific runtime features. These include Gating network weight allocation, token distribution/gathering (Scatter/Gather loops), and the weighted addition combining multiple independent expert outputs.
+
+---
+
+## 📊 Comparison: `tinymoe2m` vs Other 1M Variants
+
+To help track feature coverage across the 1M/2M verification suite, the core structural layouts are outlined below:
+
+| Feature / Metric | `tiny1m` (Standard) | `tinybpe1m` (BPE Variant) | `tinygemma1m` (Gemma 2 Variant) | `tinymoe2m` (This Repository) |
+| :--- | :--- | :--- | :--- | :--- |
+| **Base Architecture** | Llama 2 | Llama 2 | Gemma 2 | **Llama 2 (Mixtral Format)** |
+| **FFN Structure** | Single FFN (Dense) | Single FFN (Dense) | Single FFN (Dense) | **Mixture-of-Experts (MoE)** |
+| **Attention Mechanism** | MHA (Multi-Head) | MHA (Multi-Head) | GQA (Grouped-Query) | **MHA (Multi-Head)** |
+| **Total Experts** | 1 (Non-MoE) | 1 (Non-MoE) | 1 (Non-MoE) | **4 Experts** |
+| **Selected Experts** | - | - | - | **Top-2 Experts** |
+| **Expert FFN Dim (`intermediate_size`)** | 564 | 352 | 352 | **352** (Shared across all experts) |
+| **Total Parameters** | ~1.2M | ~1.0M | ~1.0M | **~1.95M (1.95M Total)** |
+| **Active Parameters** | ~1.2M | ~1.0M | ~1.0M | **~1.14M (1.14M Active)** |
+| **Primary Debug Target** | Core matrix mult & layout | `byte_fallback` decode | Gemma 2 advanced graph | **Dynamic Routing & Scatter/Gather** |
+
+### 💡 Compute Cost vs Capacity Optimization
+With a total parameter count of approximately 1.95M, this model retains roughly twice the absolute capacity of standard 1M dense variants, allowing it to maintain a stable command of grammar rules and coherent phrasings from the TinyStories corpus. Crucially, because only the top-2 experts fire per token, the active parameter execution count is capped at ~1.14M. 
+This layout perfectly replicates the fundamental benefit of MoE architectures: expanding a model's total internal capacity by 2x while restricting the added floating-point operation (FLOPs) overhead to just a 1.1x–1.2x increase compared to a 1M dense counterpart.
+
+---
+
+## 📂 Repository Structure & File Descriptions
+
+### 1. GGUF Formats (Root Directory `./`)
+Binary files optimized for execution via `llama.cpp` or compatible lower-level inference engines. Upstream parsers will automatically recognize this architecture under the `mixed` (Mixtral) type descriptor.
+
+| Filename | Type | Size | Target / Validation Focus |
+| :--- | :--- | :--- | :--- |
+| **`tinymoe2m.F32.gguf`** | `F32` | ~8.0 MB | **Baseline Test.** Eliminates quantization noise to isolate and verify the raw probability mathematics of the Gating network and expert tensor synthesis. |
+| **`tinymoe2m.F16.gguf`**<br>**`tinymoe2m.BF16.gguf`** | `F16`<br>`BF16` | ~4.0 MB | **Half-Precision Test.** Evaluates 16-bit floating-point unpacking routines and stability under parallelized accumulation layers. |
+| **`tinymoe2m.Q8_0.gguf`** | `Q8_0` | ~2.2 MB | **Standard Quantization.** Verifies block-based uniform scaling (32-element blocks) across decentralized MoE structures. |
+| **`tinymoe2m.Q4_0.gguf`**<br>**`tinymoe2m.Q4_1.gguf`** | `Q4_0`<br>`Q4_1` | ~1.4 MB | **Classic Quantization.** Tests 4-bit linear scaling and unpacking logic across multiple discontinuous expert weight matrices. |
+| **`tinymoe2m.Q2_K.gguf`** | `Q2_K` | ~1.1 MB | **Standard K-Quant (2-bit).** Evaluates mixed super-block dequantization loops feeding sparse FFN routines. |
+| **`tinymoe2m.Q3_K_M.gguf`** | `Q3_K_M` | ~1.2 MB | **Standard K-Quant (3-bit).** Tests sub-variant multi-block layouts handling dynamic routing vectors. |
+| **`tinymoe2m.Q4_K_M.gguf`** | `Q4_K_M` | ~1.4 MB | **Standard K-Quant (4-bit).** The baseline testing target for modern 4-bit super-block logic coupled with MoE paths. |
+| **`tinymoe2m.Q5_K_M.gguf`** | `Q5_K_M` | ~1.5 MB | **Standard K-Quant (5-bit).** Validates high-fidelity mixed 5-bit precision layouts. |
+| **`tinymoe2m.Q6_K.gguf`** | `Q6_K` | ~1.7 MB | **Standard K-Quant (6-bit).** Validates 6-bit high-fidelity super-block dequantization. |
+
+### 2. Hugging Face Native Format (`./hf/`)
+Unquantized components formatted for direct instantiation inside the PyTorch `transformers` library ecosystem:
+* **`hf/model.safetensors`**: Raw unquantized matrix parameters containing all 4 expert sub-networks alongside the master router tensor.
+* **`hf/config.json`**: Architectural specifications built around `MixtralConfig` criteria (layer depth, head maps, absolute expert counts, and top-k selection targets).
+* **`hf/generation_config.json`**: Standard generation defaults.
+* **`hf/tokenizer.model`**: The custom 512-vocabulary size SentencePiece BPE master binary.
+* **`hf/tokenizer_config.json`**: Metadata linking `LlamaTokenizer` classes to guarantee correct handling of prefix spacing and manage automatic `<s>` (BOS) injection properly on the Hugging Face backend.
+* **`hf/special_tokens_map.json`**: Structural map linking token strings (`<s>`=1, `</s>`=2) back to internal index bounds.
+
+---
+
+## 🚀 Usage Examples
+
+### A. Running GGUF via llama.cpp
+To process the MoE execution graph and evaluate dynamic expert routing directly on your shell:
+```bash
+./llama-cli -m tinymoe2m.Q4_K_M.gguf -p "Tom and Jerry are " -n 64 --temp 0.0
+
+```
+
+### B. Loading Hugging Face Formats via Python
+
+Because the configuration parameters are seamlessly matched with the custom vocabulary schema, you can invoke the classes using standard automated loaders without building proprietary wrapper systems.
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+repo_id = "shibatch/tinymoe2m"
+
+print("Loading MoE configuration and tokenizer layers...")
+tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="hf")
+model = AutoModelForCausalLM.from_pretrained(repo_id, subfolder="hf")
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+model = model.to(device)
+model.eval()
+
+prompt = "Tom and Jerry are "
+inputs = tokenizer(prompt, return_tensors="pt").to(device)
+
+print("Running inference loop (Validating Top-2 sparse routing matrices)...")
+with torch.no_grad():
+    outputs = model.generate(
+        **inputs, 
+        max_length=64, 
+        do_sample=False
+    )
+    
+generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+
+print("\n--- Inference Test Result ---")
+print("Prompt   :", prompt)
+print("Generated:", generated_text)
+
+```
+
+---
+
+## 📝 Model Specifications
+
+* **Architecture:** Mixtral (`MixtralForCausalLM`)
+* **Dataset:** TinyStories
+* **Total Parameters (`num_local_experts` = 4):** ~1.95M
+* **Active Parameters (`num_experts_per_tok` = 2):** ~1.14M
+* **Vocabulary Size (`vocab_size`):** 512 (Custom SentencePiece BPE with `byte_fallback` enabled)
+* **Hidden Size (`hidden_size`):** 128
+* **Number of Hidden Layers (`num_hidden_layers`):** 3
+* **Number of Attention Heads (`num_heads` / `num_kv_heads`):** 2 / 2 *(MHA layout)*
+* **Individual Expert Internal Dimension (`intermediate_size`):** 352 *(SwiGLU structure)*
+* **Max Position Embeddings (`max_position_embeddings`):** 256
+
+## 📜 License
+
+* **License:** **MIT License**. You are completely free to duplicate, modify, distribute, and utilize these assets across any commercial, personal, or educational environments.

hf/config.json

@@ -0,0 +1,35 @@
+{
+  "architectures": [
+    "MixtralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "head_dim": null,
+  "hidden_act": "silu",
+  "hidden_size": 128,
+  "initializer_range": 0.02,
+  "intermediate_size": 352,
+  "max_position_embeddings": 256,
+  "model_type": "mixtral",
+  "num_attention_heads": 2,
+  "num_experts_per_tok": 2,
+  "num_hidden_layers": 3,
+  "num_key_value_heads": 2,
+  "num_local_experts": 4,
+  "output_router_logits": false,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 1000000.0,
+    "rope_type": "default"
+  },
+  "router_aux_loss_coef": 0.001,
+  "router_jitter_noise": 0.0,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.9.0",
+  "use_cache": false,
+  "vocab_size": 512
+}

hf/generation_config.json

@@ -0,0 +1,10 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 2,
+  "transformers_version": "5.9.0",
+  "use_cache": true
+}

hf/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:639f23e32fe1d96d3c3608b707446efec64487051b806b761d09dc3950896134
+size 7815432

hf/special_tokens_map.json

@@ -0,0 +1,6 @@
+{
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "pad_token": "</s>",
+  "unk_token": "<unk>"
+}

hf/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35a43faf8c59184ca3109ef09ed7b6336ac047baea19032a669d037a793eeb6e
+size 247523

hf/tokenizer_config.json

@@ -0,0 +1,10 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "model_max_length": 256,
+  "pad_token": "</s>",
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>"
+}

tinymoe2m.BF16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b18ff85f62de21fc0c2eb68bb31378d3663d5f662eda1f1989349a06b322144
+size 3922752

tinymoe2m.F16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b329df06e638d7c996df9f287d1f7d9903327623efc531c6a78e6486073188f1
+size 3922752

tinymoe2m.F32.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f2ccb48ce78dde41d14dca96712c3df606ba75a44b05e70e3b032ff425ef49a
+size 7822144

tinymoe2m.Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c11b1b881ead0c2f7fdaf0a191ce4cc3d0159f2c809624408a2ac9b7a94c2ed
+size 1152832

tinymoe2m.Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b431a6e950a4db6218955d1872f3aaf9abf23bd8c0466288328f8b9f23dfd6a
+size 1234752

tinymoe2m.Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89803d18cdbafe4432fbdc136b1adacedea5708e8f6381b2bcd9c026b08d8c4f
+size 1152832

tinymoe2m.Q4_1.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7881d326d3649e087ec56b4de9505c57c2be085e6ac0270027d915770f8bde1
+size 1270592

tinymoe2m.Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb6c40f54d12edf5b6ec546d2023dff3d232a09d81e5c1540acc5467d8f9f608
+size 1462080

tinymoe2m.Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45ac88cee0b4cf9ad26859f480457d4e30cf06eae337ea75d7f3ef0eecec7301
+size 1567552

tinymoe2m.Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d195dda47ae604ffd13e9c88112e90b5273951e65b256e482a277dcf89a25bcb
+size 2094912

tinymoe2m.Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9eaeba40359bb03252a478076217e93fd55ef0ca7460ae179b135fcd4474b307
+size 2094912