com.microsoft.FusedGemm

com.microsoft · ONNX Runtime contrib operator · contrib since_version 1

Description

Gemm with a fused activation: Y = act(alpha * A' * B' + beta * C), where A' and B' are optionally transposed and C is broadcastable to (M, N). The activation runs in the f32 accumulator before the single output narrowing. This package supports Relu, LeakyRelu, Sigmoid, Tanh and HardSigmoid; the other activation strings and numeric types admitted by the open schema are not implemented. Omitting activation gives plain Gemm.

See the ONNX Runtime FusedGemm contrib-operator spec for the reference semantics.

Inputs

Name	Bind key	Logical dtype	Rank	Shape	Description	Presence
`A`	`A`	`T`	`2`	—	Left operand, `(M, K)` when `transA` is 0 and `(K, M)` otherwise.	required
`B`	`B`	`T`	`2`	—	Right operand, `(K, N)` when `transB` is 0 and `(N, K)` otherwise.	required
`C`	`C`	`T`	—	—	Optional additive term, unidirectionally broadcastable to `(M, N)`: a scalar, a row `(N)`, a column `(M, 1)`, or the full matrix.	optional

Outputs

Name	Bind key	Logical dtype	Rank	Shape	Description	Presence
`Y`	`Y`	`T`	`2`	derived; see description	`act(alpha * A' * B' + beta * C)` with shape `(M, N)`.	required

Attributes

Attributes and default values (overridable per request):

Attribute	Default	Description
`alpha`	`1`	Scalar multiplier for the product `A' * B'`; the standard default is 1.
`beta`	`1`	Scalar multiplier for `C`; the standard default is 1.
`transA`	`0`	Whether `A` is stored transposed. The standard default is 0.
`transB`	`0`	Whether `B` is stored transposed. The standard default is 0.
`activation`	—	Optional fused activation name. Supported modes are `Relu`, `LeakyRelu`, `Sigmoid`, `Tanh` and `HardSigmoid`; omission applies none.
`activation_alpha`	—	First activation parameter: the slope for `LeakyRelu` or `alpha` for `HardSigmoid`.
`activation_beta`	—	Second activation parameter: `beta` for `HardSigmoid`.

Type constraints

Variable	Allowed dtypes
`T`	`float32`, `float16`

Device requirements

Some implementation variants require subgroup-matrix and subgroups. These are route-specific capabilities, not package-wide requirements; availability also depends on the request shape and dtype.

Files

metadata.json — kernel metadata (id, digests, provenance)
manifest.json — the op contract (source of truth)
test.json — correctness cases
bench.json — benchmark + tuning cases
fused-gemm.wgsl.jinja
gemm-subgroup-matrix.wgsl.jinja

Use with `@huggingface/kernels`

The loader derives every required output's shape and logical dtype from the manifest contract and this call. It then allocates the result tensors automatically.

The version: 1 option selects the published kernel contract; it is independent of any operator opset, contrib since_version, or model version.

Replace each *Data placeholder with a typed array containing the corresponding input data.

import { getKernel } from "@huggingface/kernels";

const kernel = await getKernel("webgpu-kernels/com.microsoft.FusedGemm", { version: 1 });
const { Y } = await kernel({ A: { data: AData, shape: [7, 13] }, B: { data: BData, shape: [13, 11] } }, {
  attrs: { activation: "Relu" },
});

Downloads last month: -

kernel

webgpu

wgsl

apache-2.0

WebGPU

Requires WebGPU support. See the compatibility table.