MiniCPM5-2B for webyIQ (int4 GPTQ)

openbmb/MiniCPM5-2B converted for webyIQ, a model-specialized LLM engine that runs in the browser on WebGPU (Rust + hand-written WGSL kernels).

Weights: asymmetric int4, group 32, f16 scale/min, GPTQ (calibrated on WikiText-2 train); the embedding is int4 RTN with clip search. 1.47 GiB in ≤128 MB shards + manifest.json.
Quality (8 × 512 tokens, webyIQ engine with int8 activations): perplexity WikiText-2 21.83 (bf16: 21.23), C4 26.70 (bf16: 25.76).
Speed in Chrome on an RTX 5080: first token 31 ms (128-token prompt), ~280 tok/s generation.

This is not a transformers / llama.cpp checkpoint; it is loaded by webyIQ's web/lib/webyiq.js. Try it in the online demo.

License: Apache-2.0, as the original model (© OpenBMB).

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for tixiren/MiniCPM5-2B-webyIQ

Base model

Finetuned

(87)

this model