File size: 1,414 Bytes
97c56ca 8cd4182 6867e61 97c56ca 8cd4182 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | ---
license: mit
language:
- ru
- en
pipeline_tag: text-generation
tags:
- eblan
model-index:
- name: eblan-2.0-flash-lite
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: Synthetic Benchmark
type: synthetic
metrics:
- name: Accuracy
type: accuracy
value: 314
---
# 🚀 Eblan 2.0 Flash Lite (`EblanForCausalLM`)
**Eblan 2.0 Flash Lite** is a next-generation lightweight language model built on the custom `EblanForCausalLM` architecture.
By leveraging an innovative **O(1)** polynomial recurrent state kernel, the model delivers ultra-fast inference with virtually zero memory overhead, running natively via NumPy and Tiktoken.
---
## 🏗️ Architecture (`EblanForCausalLM`)
Unlike traditional Transformer-based models that rely on heavy Attention blocks (**Q**, **K**, **V**), `EblanForCausalLM` utilizes a single scalar weight vector **W** (in **R**¹) with a direct scalar projection layer.
### Forward Pass Formulation:
$$y = W \cdot (x^2 + x) \cdot \text{vocab\_size}$$
Where:
* **x** = `current_id` / `vocab_size` — normalized input token index.
* **W** — trained scalar weight (`eblan-2.0-flash-lite.npy`).
* **Logits** formula for distance-based logit calculation prior to Softmax sampling:
$$\text{Logits}_i = -\frac{|i - y|}{\tau}$$
---
## 📜 License
This project is licensed under the **MIT License**. |