yakkshit/nde-25m

NDE (Nano-Decision Engine)

NDE is a ~25M parameter, from-scratch, "System One" decision routing model. Inspired by the Jev architecture, it is designed to replace brittle if/else logic and expensive LLM API calls with ultra-fast, deterministic, structured JSON outputs.

🚀 Why NDE?

• Blazing Fast: Runs at 100+ tokens/second on a standard 2-core CPU.
• Zero Hallucination: Designed exclusively for strict JSON tool-calling, compatible with grammar-constrained decoding (e.g., llama.cpp / Ollama).
• From Scratch: Built with pure PyTorch (no HuggingFace transformers bloat), featuring modern efficiency blocks: RMSNorm, RoPE, SwiGLU, and Weight Tying.
• Edge-Ready: The final quantized .gguf model is ~40MB, making it deployable on Raspberry Pi, local servers, or lightweight cloud instances.

🏗️ Architecture

• Parameters: ~25.17 Million
• Layers: 8
• Hidden Dimension: 512
• Attention Heads: 8
• Vocabulary Size: 8,192 (Optimized for fast CPU output projection)
• Context Length: 512 tokens

📂 Project Structure

nde/
├── src/nde/          # Core PyTorch architecture and utilities
├── data/             # Synthetic training datasets (ignored in git)
├── configs/          # YAML configurations for training and inference
├── scripts/          # Setup, data generation, and GGUF export scripts
├── tests/            # Unit and integration tests
└── docs/             # Architecture and deployment documentation