Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,32 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
- ru
|
| 6 |
+
tags:
|
| 7 |
+
- rust
|
| 8 |
+
- custom-architecture
|
| 9 |
+
- text-generation
|
| 10 |
---
|
| 11 |
+
|
| 12 |
+
# DeepCode-Rust-2M
|
| 13 |
+
|
| 14 |
+
This is a lightweight, custom language model built entirely from scratch in **Rust**. It does not rely on Python, PyTorch, or standard Hugging Face `transformers` libraries. The entire forward pass, backpropagation, and Adam optimizer are implemented in pure Rust.
|
| 15 |
+
|
| 16 |
+
## Model Details
|
| 17 |
+
* **Parameters:** ~2 Million (2,000,000)
|
| 18 |
+
* **Architecture:** Decoder-only Transformer (6 layers, 256 embedding dim)
|
| 19 |
+
* **Language:** Rust (trained on a multilingual dataset)
|
| 20 |
+
* **Format:** Serialized via `bincode` (requires the custom Rust inference engine to run)
|
| 21 |
+
* **Capabilities:** Generates basic Python functions and simple HTML structures.
|
| 22 |
+
|
| 23 |
+
## Limitations
|
| 24 |
+
Due to its extremely small size (2M parameters), the model can sometimes hallucinate, lose track of syntax, or fail on complex prompts. It is an educational project demonstrating how language models work under the hood.
|
| 25 |
+
|
| 26 |
+
## How to Use
|
| 27 |
+
Because this is a custom Rust implementation, you cannot load it using `AutoModel.from_pretrained()`.
|
| 28 |
+
To run this model, you need the Rust source code and the `model.bin` file.
|
| 29 |
+
|
| 30 |
+
```bash
|
| 31 |
+
# Clone the repo (example)
|
| 32 |
+
cargo run --release
|