| --- |
| title: README |
| emoji: 🏆 |
| colorFrom: gray |
| colorTo: purple |
| sdk: static |
| pinned: false |
| --- |
| |
| <div align="center"> |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/6615494716917dfdc645c44e/k839qGKNniqOM_4PdpvoG.png" alt="FWKV Banner" style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em; reading-order: 20px; border-radius: 20px;"/> |
| In partnership with <a href="https://huggingface.co/bench-labs"><b>Bench Labs</b></a> |
| </div> |
| |
| **What is "FWKV"???**<br> |
| FWKV (call it Feed-forward Weighted Key Value, or Floored Weighted Key Value, its up to you) is a novel efficient language model architecture proposed on May 13, 2026 by [me](https://huggingface.co/FlameF0X). |
|
|
| It started as a fun side project inspired by RWKV. Then I (We, [FlameF0X](https://huggingface.co/FlameF0X)) realised that for a 50 million parameter language model is fast, like, faster than any **Transformers** base *Dense* or *MoE* 50m model |
|
|
| **Why should *I* use FWKV?**<br> |
| You dont use it since its still in its early stages and its very unstable. But its fast. |
|
|
| # Models |
| **How do *we* name the models?**<br> |
| Very simple actually!<br> |
| `FWKV-[model size or type or something uniquie]`<br> |
| They **must** stay simple so you can search easily. |
|
|
| **Models:** |
| - Language models: |
| - FWKV-100M (being planned to enter in development in 2026-2027) |
| - FWKV-Translation (being planned to enter in development in late 2026) |
| - [FWKV-ROSA](https://huggingface.co/FWKV/FWKV-ROSA) |
| - [FWKV-50M](https://huggingface.co/FWKV/FWKV-50M) |
| - [FWKV-29M](https://huggingface.co/FWKV/FWKV-29M) |
| - [FKWV-Tinystories](https://huggingface.co/FWKV/FWKV-Tinystories) |
| - Vision models: |
| - [FWKV-Image](https://huggingface.co/FWKV/FWKV-Image) |
| - FWKV-CLIP (**NAP** - Not A Prioriy, you will see this very frequently) |
| - Other modalities: |
| - FWKV-TTS (NAP) |
| - FWKV-Bert (NAP) |
| |
| |
|
|
