Michionlion/Nanbeige4.2-3B-ONNX-WebGPU

Nanbeige 4.2 3B ONNX WebGPU

Browser-focused ONNX export of Nanbeige/Nanbeige4.2-3B for Transformers.js and ONNX Runtime Web.

The model:

• preserves Nanbeige's two passes over 22 shared physical layers as 44 logical KV-cache slots;
• stores projection weights as symmetric block-32 q4;
• uses ONNX Runtime Web's fused rotary/attention and quantized SwiGLU MLP operators; and
• supports one graph for prefill and cached decoding.

Transformers.js

This export currently requires the ONNX Runtime Web development build used by Shlvl because MatMulNBitsMlp has not reached a stable release.

import { pipeline } from "@huggingface/transformers";

const generator = await pipeline(
  "text-generation",
  "Michionlion/Nanbeige4.2-3B-ONNX-WebGPU",
  {
    device: "webgpu",
    dtype: "q4f16",
    model_file_name: "model_webgpu_mlp",
    use_external_data_format: 2,
  },
);

The two external-data shards are approximately 1.89 GB and 1.19 GB. Shlvl pins the Hub revision used in production and adds a guarded Apple subgroup reduction for the generated q4 decode shaders; other devices use ONNX Runtime's default kernels.

Validation

The tiny fp32 export matched the source PyTorch model for both prefill and cached decode. Full-model paired browser tests retained exact token fingerprints for a 512-token prompt followed by 128 greedy tokens and for a separate chat-like prompt.

This is an experimental browser export. Reduction-order changes in optimized GPU kernels can change close greedy decisions, so evaluate it for your use case before relying on exact reproducibility.