| --- |
| license: mit |
| language: |
| - en |
| library_name: transformers |
| pipeline_tag: text-generation |
| widget: |
| - text: "Once upon a time" |
| - text: "Write a short poem about the ocean" |
| tags: |
| - text-generation |
| - causal-lm |
| - tiny-transformer |
| - edge-ai |
| - onnx |
| - int8 |
| - distillation |
| - tinybert-style |
| model_name: Krull-Micro |
| base_model: Krull-Micro |
| datasets: |
| - custom |
| metrics: |
| - perplexity |
| inference: true |
| --- |
| |
| # KRULL-Nano Simple |
|
|
| **KRULL** means **Knowledge Running Under Lightweight Language**. |
|
|
| KRULL-Nano is a lightweight decoder-only Small Language Model (SLM) architecture designed for embedded devices, edge AI, offline inference, and privacy-preserving applications. |
|
|
| Unlike cloud-oriented LLMs optimized for massive datacenters, KRULL-Nano is designed from the ground up for: |
|
|
| - low latency |
| - low memory usage |
| - deterministic inference |
| - offline execution |
| - edge sovereignty |
| - efficient deployment on constrained hardware |
|
|
| ## Architecture |
|
|
| KRULL-Nano uses a compact decoder-only transformer architecture with multiple optimizations for edge execution. |
|
|
| ## Core Components |
|
|
| ### Decoder-Only Transformer |
| - autoregressive causal language modeling |
| - GPT-style token prediction |
| - sreaming-friendly generation |
|
|
| ### RMSNorm |
|
|
| KRULL replaces LayerNorm with RMSNorm to: |
|
|
| - reduce computational overhead |
| - improve low-precision stability |
| - minimize memory bandwidth |
|
|
| ### Multi-Query Attention (MQA) |
|
|
| Instead of full multi-head attention: |
|
|
| - multiple query heads |
| - shared key/value heads |
|
|
| Benefits: |
|
|
| - reduced KV cache size |
| - faster inference |
| - lower RAM usage |
|
|
| ### Gated Feed Forward Network |
|
|
| KRULL uses a gated FFN inspired by modern efficient transformer architectures. |
|
|
| Benefits: |
|
|
| - improved parameter efficiency |
| - lower compute cost |
| - better expressivity per parameter |
|
|
| ### What is included |
|
|
| - Decoder-only GPT-style model |
| - RMSNorm |
| - Multi-query attention |
| - Gated feed-forward block |
| - Simple character tokenizer |
| - CPU training script |
| - Text generation script |
| - ONNX export script |
| - Windows-friendly imports |
|
|
| ### Project structure |
|
|
| ```text |
| krull_nano_simple/ |
| ├── krull/ |
| │ ├── __init__.py |
| │ ├── model.py |
| │ └── tokenizer.py |
| ├── scripts/ |
| │ ├── train_tokenizer.py |
| │ ├── train_lm.py |
| │ ├── generate.py |
| │ └── export_onnx.py |
| ├── configs/ |
| │ └── krull_nano.json |
| ├── data/ |
| │ └── tiny_corpus.txt |
| ├── artifacts/ |
| ├── requirements.txt |
| ├── LICENSE |
| └── README.md |
| ``` |
|
|
| ### Setup on Windows |
|
|
| Open PowerShell or CMD: |
|
|
| ```bash |
| cd C:\workspace\krull_nano_simple |
| python -m venv .venv |
| .venv\Scripts\activate |
| pip install -r requirements.txt |
| ``` |
|
|
| ### Setup on Linux/macOS |
|
|
| ```bash |
| cd krull_nano_simple |
| python -m venv .venv |
| source .venv/bin/activate |
| pip install -r requirements.txt |
| ``` |
|
|
| ## 1. Train tokenizer |
|
|
| ```bash |
| python scripts/train_tokenizer.py --input data/tiny_corpus.txt --out artifacts/tokenizer.json |
| ``` |
|
|
| ## 2. Train model |
|
|
| ```bash |
| python scripts/train_lm.py --config configs/krull_nano.json --tokenizer artifacts/tokenizer.json --data data/tiny_corpus.txt --out artifacts/krull_nano.pt --epochs 10 --device cpu |
| ``` |
|
|
| ## 3. Generate text |
|
|
| ```bash |
| python scripts/generate.py --model artifacts/krull_nano.pt --tokenizer artifacts/tokenizer.json --prompt "KRULL is" --device cpu |
| ``` |
|
|
| ## 4. Export ONNX |
|
|
| ```bash |
| python scripts/export_onnx.py --model artifacts/krull_nano.pt --out artifacts/krull_nano.onnx |
| ``` |
|
|
| ## Notes |
|
|
| This repo is for learning and experimentation. The default dataset is tiny, so the generated text will not be intelligent. Replace `data/tiny_corpus.txt` with a larger corpus to train a better model. |
|
|
|
|
| ## License |
|
|
| MIT |
|
|
