litert-community/BiSeNet-Face-Parsing-LiteRT

BiSeNet Face Parsing — LiteRT (GPU)

On-device real-time face parsing running fully on the LiteRT CompiledModel GPU delegate (no CPU fallback). BiSeNet (zllrunning/face-parsing.PyTorch) segments a face into the 19 CelebAMask-HQ classes (skin, brows, eyes, nose, lips, ears, hair, hat, glasses, neck, cloth, …) — for AR / beauty / makeup. ~22 ms/frame on a Pixel 8a.

• Architecture: BiSeNet (ResNet18 backbone + context path + feature-fusion) — pure CNN.
• Weights: zllrunning/face-parsing.PyTorch · MIT · ~13.3 M params.
• Size: 53 MB.

I/O

• Input: [1, 3, 512, 512] NCHW, RGB, ImageNet-normalized (mean [0.485,0.456,0.406], std [0.229,0.224,0.225]).
• Output: [1, 19, 512, 512] class logits — argmax over the 19 classes per pixel.

Classes: background, skin, l_brow, r_brow, l_eye, r_eye, eyeglass, l_ear, r_ear, earring, nose, mouth, u_lip, l_lip, neck, necklace, cloth, hair, hat.

GPU conversion

BiSeNet is a pure CNN; three re-authoring patches make it a fully GPU-compatible graph — 74/74 nodes on the delegate, 1 partition (device corr 0.99999, argmax 99.96% vs PyTorch):

1. align_corners=True → False — the output upsamples use align_corners=True, which the GPU delegate rejects (1.6% argmax change vs original).
2. global avg_pool2d(x, x.size()[2:]) → mean([2,3]) — the context/attention modules pool with a full-spatial kernel, which the Mali delegate rejects as an AVERAGE_POOL_2D; a MEAN reduce is supported.
3. zero-pad maxpool — the ResNet stem MaxPool2d(padding=1) lowers to a PADV2 with -inf padding (PADV2: src has wrong size on Mali); an explicit 0-pad + unpadded maxpool is exact (input is post-ReLU ≥ 0).

CPU-exact vs PyTorch (corr 0.99999999999).

Minimal usage

Kotlin (Android, LiteRT CompiledModel GPU)

val options = CompiledModel.Options(Accelerator.GPU)
val model = CompiledModel.create(context.assets, "faceparsing.tflite", options, null)
val inBufs = model.createInputBuffers()
val outBufs = model.createOutputBuffers()

inBufs[0].writeFloat(inputNCHW)          // [1,3,512,512], RGB, ImageNet-norm
model.run(inBufs, outBufs)
val logits = outBufs[0].readFloat()      // [19,512,512] (NCHW, batch dropped)

val hw = 512 * 512
val label = IntArray(hw) { i ->
    var best = 0; var bv = logits[i]
    for (c in 1 until 19) { val v = logits[c * hw + i]; if (v > bv) { bv = v; best = c } }
    best
}

Python (LiteRT / ai-edge-litert)

from ai_edge_litert.interpreter import Interpreter
import numpy as np

it = Interpreter(model_path="faceparsing.tflite"); it.allocate_tensors()
inp, out = it.get_input_details(), it.get_output_details()
it.set_tensor(inp[0]["index"], x)        # [1,3,512,512] float32, ImageNet-norm
it.invoke()
logits = it.get_tensor(out[0]["index"])[0]   # [19,512,512]
label = logits.argmax(0)                      # [512,512] class ids

Conversion

Converted with litert-torch (build_faceparsing.py): loads the trained BiSeNet weights, applies the three patches, and exports.

Performance

Measured on a Pixel 8a (Tensor G3, Android 16) with the standard TFLite benchmark_model tool — 10 warm-up runs then 50 timed runs, reported as the tool's mean.

Runtime	Backend	Graph on GPU	Latency
LiteRT CompiledModel (LITERT_CL)	GPU	74 / 74	~22 ms
TFLite benchmark_model (TfLiteGpuDelegateV2)	GPU (OpenCL)	74 / 74	52.9 ms
TFLite benchmark_model	CPU (XNNPACK, 4 threads)	—	229.4 ms

The two GPU rows are different runtimes, not a contradiction. The LITERT_CL figure is the one recorded when this model shipped, taken through LiteRT's own CompiledModel accelerator — the path the Kotlin sample app and the LiteRT API use. The TfLiteGpuDelegateV2 figure is the classic TFLite OpenCL delegate, measured with a tool anyone can download and re-run. They agree on how much of the graph the GPU takes; they disagree on speed, and the classic delegate is the slower of the two here. Read the TfLiteGpuDelegateV2 row as a reproducible floor, not as this model's speed on LiteRT.

Snapdragon NPU (Hexagon)

This file runs on the Qualcomm Hexagon NPU as published — no conversion and no pre-compiled artifact. LiteRT compiles it on the device and caches the result.

Measured on a physical Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81) with LiteRT CompiledModel 2.2.0 — 5 warm-up runs then 50 timed runs, one accelerator per process, every row taken at device thermal status NONE.

Compute unit	Inference (median / min)	Load	Start headroom
NPU (Hexagon) — first launch	7.89 ms / 7.11 ms	1808 ms	0.59
NPU (Hexagon) — cached	7.88 ms / 7.41 ms	116 ms	0.59
GPU (Adreno)	26.02 ms / 19.32 ms	816 ms	0.59

The NPU is 3.3x faster on inference here (7.88 ms against 26.02 ms). The first launch pays once for on-device compilation; every launch after that loads in 116 ms against 816 ms for the GPU (7.0x), because the GPU rebuilds its shaders each time. The file is fp16 and needs no int8 quantization to reach the NPU.

Running it on the NPU

Put these in jniLibs/arm64-v8a/. None of them are distributed from this repository — the first two come from Google, the rest from Qualcomm's own SDK:

Library	Source
libLiteRtDispatch_Qualcomm.so, libLiteRtCompilerPlugin_Qualcomm.so	litert_npu_runtime_libraries_jit.zip, a release asset of google-ai-edge/LiteRT
libQnnHtp.so, libQnnSystem.so, libQnnHtpV81Stub.so, libQnnHtpV81Skel.so, libQnnHtpPrepare.so, libQnnIr.so, libQnnSaver.so	Qualcomm QAIRT — the same zip ships fetch_qualcomm_library.sh, which downloads the SDK and copies them for you

Pick the runtime matching the device's Hexagon version: SM8550 → v73, SM8650 → v75, SM8750 → v79, SM8850 → v81.

val env = Environment.create(
    context,
    mapOf(
        Environment.Option.DispatchLibraryDir to context.applicationInfo.nativeLibraryDir,
        // Required for on-device compilation. Without it the model silently runs on CPU.
        Environment.Option.CompilerPluginLibraryDir to context.applicationInfo.nativeLibraryDir,
    ),
)
val options = CompiledModel.Options(Accelerator.NPU).apply {
    qualcommOptions = CompiledModel.QualcommOptions(
        htpPerformanceMode = CompiledModel.QualcommOptions.HtpPerformanceMode.BURST
    )
}
val model = CompiledModel.create(context.assets, "faceparsing.tflite", options, env)

Build settings: useLegacyPackaging = true under packaging { jniLibs { … } }, so the DSP can open the skel from a real path, and Kotlin 2.3+ for LiteRT 2.2.0's metadata.

Every NPU failure here is silent. There is no error when the NPU is unavailable — you get a plausible CPU number instead. Confirm from logcat which delegate took the graph: Replacing 1 out of 1 node(s) with delegate (DispatchDelegate) is the NPU, while ... (TfLiteXNNPackDelegate) is the CPU. A missing library is reported only as a W-level dlopen failed line under a generic No compiler plugin found summary.

On the conditions. Thermal headroom is reported as measured, where 1.0 is the throttling threshold. All rows were taken at a comparable headroom and compare directly; figures taken at a different headroom will differ. Each accelerator ran in its own process, because LiteRT's Environment is shared within one and the first model load fixes the options for every later one.

License

MIT (BiSeNet / zllrunning/face-parsing.PyTorch). CelebAMask-HQ label taxonomy.