Flint-ONNX-WebGPU

Recursive's Flint (39.8B parameters, about 2.5B active per token) converted to ONNX for running in a web browser with ONNX Runtime Web's WebGPU backend, as used by Quick Inference.

Changes from the original

• Converted to an ONNX decoder in ONNX Runtime GenAI's layout (input_ids, attention_mask, past_key_values.* → logits, present.*) by Quick Inference's own exporter (tools/export/export_flint.py in Quick Inference), following Flint's SGLang model code.
• Attention is GroupQueryAttention with Flint's learned sinks (head_sink), its 513-token sliding window on the sliding layers, and rotary embeddings only where Flint uses them. The routed experts run as one QMoE per layer: sigmoid scores, the top 6 renormalised; the shared expert is added to their output.
• Linear layers and experts are quantised to 4 bits (round to nearest, symmetric, block size 32), the output layer to 8 bits. Embeddings, activations and the KV cache are float16; the residual stream and the RMSNorms are float32, and the last five layers' experts run on an input scaled down by 8 (with the activation and output compensated exactly), because Flint's massive activations overflow float16 otherwise. Rotary tables cover 65,536 positions.

These are modified files: their outputs are close to, but not identical to, the original model's.

Licence

Apache 2.0, as the original (LICENSE).