File size: 1,144 Bytes
3fd41f7 c9f415d 3fd41f7 c9f415d a225e9c c9f415d a225e9c c9f415d a225e9c c9f415d a225e9c c9f415d a225e9c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | ---
license: mit
language:
- en
- ru
tags:
- rust
- custom-architecture
- text-generation
---
# DeepCode-Rust-2M
This is a lightweight, custom language model built entirely from scratch in **Rust**. It does not rely on Python, PyTorch, or standard Hugging Face `transformers` libraries.
## Model Details
* **Parameters:** ~2 Million
* **Architecture:** Decoder-only Transformer (6 layers, 256 embedding dim)
* **Format:** Serialized via `bincode` (requires the custom Rust inference engine)
## Benchmark Results (Napkin Benchmark)
Tested on 6 basic prompts:
* **Python Addition:** 4/5 (Successfully generated `def add(a, b): return a + b`)
* **Python Multiplication:** 2/5 (Recognized `multiply`, but broken syntax)
* **Python Loop:** 0/5 (Stuck in a repetition loop)
* **HTML Structure:** 1/5 (Recognized tags, but no hierarchical structure)
* **HTML Button:** 1/5
* **Overall Score:** ~2 out of 5
## Limitations
Due to its extremely small size, the model often suffers from repetition loops and fails to understand nested structures (like HTML trees). It is an educational project demonstrating how language models work under the hood.
|