File size: 1,081 Bytes
0441b84 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | ---
base_model: anthonym21/Eve-2-MoE-NanoSQL-272M
tags:
- gguf
- quantized
- moe
- eve-2
license: apache-2.0
---
# Eve-2-MoE-NanoSQL-272M - GGUF
GGUF quantizations of [anthonym21/Eve-2-MoE-NanoSQL-272M](https://huggingface.co/anthonym21/Eve-2-MoE-NanoSQL-272M).
## Quantization Variants
| Quantization | Filename | Size |
|---|---|---|
| Q8_0 | Eve-2-MoE-NanoSQL-272M-Q8_0.gguf | 290.9 MB |
| Q4_K_M | Eve-2-MoE-NanoSQL-272M-Q4_K_M.gguf | 189.5 MB |
## Usage with Ollama
```bash
ollama run anthonym21/eve-2-moe-nanosql-272m
```
## Usage with llama.cpp
```bash
llama-cli -m Eve-2-MoE-NanoSQL-272M-Q4_K_M.gguf -p "Your prompt here"
```
## Architecture
- **Type**: DeepSeek-style Mixture of Experts (MoE)
- **Parameters**: 272M total
- **Layers**: 12
- **Hidden dim**: 512
- **Experts**: 8 routed (top-2) + 1 shared per layer
- **Context**: 2048 tokens
- **Tokenizer**: GPT-2
## Parent Model
This is a quantized version of [anthonym21/Eve-2-MoE-NanoSQL-272M](https://huggingface.co/anthonym21/Eve-2-MoE-NanoSQL-272M).
|