---
title: README
emoji: 🏆
colorFrom: gray
colorTo: purple
sdk: static
pinned: false
---

<div align="center">
    <img src="https://cdn-uploads.huggingface.co/production/uploads/6615494716917dfdc645c44e/k839qGKNniqOM_4PdpvoG.png" alt="FWKV Banner" style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em; reading-order: 20px; border-radius: 20px;"/>
    In partnership with <a href="https://huggingface.co/bench-labs"><b>Bench Labs</b></a>
</div>

**What is "FWKV"???**<br>
FWKV (call it Feed-forward Weighted Key Value, or Floored Weighted Key Value, its up to you) is a novel efficient language model architecture proposed on May 13, 2026 by [me](https://huggingface.co/FlameF0X).

It started as a fun side project inspired by RWKV. Then I (We, [FlameF0X](https://huggingface.co/FlameF0X)) realised that for a 50 million parameter language model is fast, like, faster than any **Transformers** base *Dense* or *MoE* 50m model

**Why should *I* use FWKV?**<br>
You dont use it since its still in its early stages and its very unstable. But its fast.

# Models
**How do *we* name the models?**<br>
Very simple actually!<br>
`FWKV-[model size or type or something uniquie]`<br>
They **must** stay simple so you can search easily.

**Models:**
- Language models:
  - FWKV-100M (being planned to enter in development in 2026-2027)
  - FWKV-Translation (being planned to enter in  development in late 2026)
  - [FWKV-ROSA](https://huggingface.co/FWKV/FWKV-ROSA)
  - [FWKV-50M](https://huggingface.co/FWKV/FWKV-50M)
  - [FWKV-29M](https://huggingface.co/FWKV/FWKV-29M)
  - [FKWV-Tinystories](https://huggingface.co/FWKV/FWKV-Tinystories)
- Vision models:
  - [FWKV-Image](https://huggingface.co/FWKV/FWKV-Image) 
  - FWKV-CLIP (**NAP** - Not A Prioriy, you will see this very frequently)
- Other modalities:
  - FWKV-TTS (NAP)
  - FWKV-Bert (NAP)