Spaces:

thinletter
/

README

Running

App Files Files Community

README / README.md

honza-rosecky

Org card: ThinCtx-Qwen3.5-4B released, thinctx 0.2.0; the GPU context memory grows 30 to 40 times more slowly

22713f7 verified about 2 hours ago

preview code

Raw History Blame Contribute Delete

1.85 kB

metadata

title: thinletter
emoji: 🪶
colorFrom: pink
colorTo: yellow
sdk: static
pinned: false

thinletter is a small research group. We make existing models lighter to run and measure them against the originals. thinletter.io · info@thinletter.io

ThinCtx

Qwen3.5 models converted so that the GPU memory for the context stays small: it grows 30 to 40 times more slowly than the original's cache, with accuracy on par with the original model. The full context is kept in system memory, so long documents and long agent sessions fit on one consumer GPU.

ThinCtx-Qwen3.5-4B
ThinCtx-Qwen3.5-2B
ThinCtx-Qwen3.5-0.8B (not recommended for tool calling)
Runtime: pip install thinctx · results: thinletter.io/thinctx

ThinEmbed

The query side of an embedding model, compressed into a small file that runs in the browser on WebGPU, against your unchanged document index. Open source, Apache-2.0.

harrier-0.6b (English): WebGPU clients · GGUF clients
Qwen3-Embedding-0.6B (Czech): WebGPU client · GGUF client
bge-m3 (Czech, multilingual): GGUF clients
Demo · code · results: thinletter.io/query-encoders