| --- |
| license: mit |
| language: |
| - ru |
| - en |
| pipeline_tag: text-generation |
| tags: |
| - eblan |
| model-index: |
| - name: eblan-2.0-flash-lite |
| results: |
| - task: |
| type: text-generation |
| name: Text Generation |
| dataset: |
| name: Synthetic Benchmark |
| type: synthetic |
| metrics: |
| - name: Accuracy |
| type: accuracy |
| value: 314 |
| --- |
| |
| # 🚀 Eblan 2.0 Flash Lite (`EblanForCausalLM`) |
|
|
| **Eblan 2.0 Flash Lite** is a next-generation lightweight language model built on the custom `EblanForCausalLM` architecture. |
|
|
| By leveraging an innovative **O(1)** polynomial recurrent state kernel, the model delivers ultra-fast inference with virtually zero memory overhead, running natively via NumPy and Tiktoken. |
|
|
| --- |
|
|
| ## 🏗️ Architecture (`EblanForCausalLM`) |
|
|
| Unlike traditional Transformer-based models that rely on heavy Attention blocks (**Q**, **K**, **V**), `EblanForCausalLM` utilizes a single scalar weight vector **W** (in **R**¹) with a direct scalar projection layer. |
|
|
| ### Forward Pass Formulation: |
|
|
| $$y = W \cdot (x^2 + x) \cdot \text{vocab\_size}$$ |
| |
| Where: |
| |
| * **x** = `current_id` / `vocab_size` — normalized input token index. |
| * **W** — trained scalar weight (`eblan-2.0-flash-lite.npy`). |
| * **Logits** formula for distance-based logit calculation prior to Softmax sampling: |
| |
| $$\text{Logits}_i = -\frac{|i - y|}{\tau}$$ |
|
|
| --- |
|
|
| ## 📜 License |
|
|
| This project is licensed under the **MIT License**. |
