| base_model: anthonym21/Eve-2-MoE-NanoSQL-272M | |
| tags: | |
| - gguf | |
| - quantized | |
| - moe | |
| - eve-2 | |
| license: apache-2.0 | |
| # Eve-2-MoE-NanoSQL-272M - GGUF | |
| GGUF quantizations of [anthonym21/Eve-2-MoE-NanoSQL-272M](https://huggingface.co/anthonym21/Eve-2-MoE-NanoSQL-272M). | |
| ## Quantization Variants | |
| | Quantization | Filename | Size | | |
| |---|---|---| | |
| | Q8_0 | Eve-2-MoE-NanoSQL-272M-Q8_0.gguf | 290.9 MB | | |
| | Q4_K_M | Eve-2-MoE-NanoSQL-272M-Q4_K_M.gguf | 189.5 MB | | |
| ## Usage with Ollama | |
| ```bash | |
| ollama run anthonym21/eve-2-moe-nanosql-272m | |
| ``` | |
| ## Usage with llama.cpp | |
| ```bash | |
| llama-cli -m Eve-2-MoE-NanoSQL-272M-Q4_K_M.gguf -p "Your prompt here" | |
| ``` | |
| ## Architecture | |
| - **Type**: DeepSeek-style Mixture of Experts (MoE) | |
| - **Parameters**: 272M total | |
| - **Layers**: 12 | |
| - **Hidden dim**: 512 | |
| - **Experts**: 8 routed (top-2) + 1 shared per layer | |
| - **Context**: 2048 tokens | |
| - **Tokenizer**: GPT-2 | |
| ## Parent Model | |
| This is a quantized version of [anthonym21/Eve-2-MoE-NanoSQL-272M](https://huggingface.co/anthonym21/Eve-2-MoE-NanoSQL-272M). | |