lightweight-OpenPose — LiteRT (TFLite) GPU, FP16

On-device LiteRT (.tflite) conversion of lightweight-OpenPose for human pose estimation. The model is a MobileNet-based heatmap network; it outputs keypoint heatmaps only and the keypoint decode (argmax) is done in app code.

The model runs fully on the LiteRT CompiledModel GPU accelerator (ML Drift): every op is GPU-native, no CPU fallback. Converted with litert-torch with no patches.

Why heatmaps-only: MoveNet's official .tflite bakes the keypoint decode into the graph (GATHER_ND), which the GPU delegate can't run — so it only partially offloads to the GPU. Keeping the graph pure-conv and decoding in app code keeps it 100% on the GPU.

Files

File	Precision	Size
pose_256_fp16.tflite	fp16 weights	~8.3 MB
pose_256.tflite	fp32	~16.4 MB

I/O

• Input: [1, 256, 256, 3] float32, NHWC, RGB, normalized (px - 128) / 256.
• Output: [1, 32, 32, 19] float32, NHWC, keypoint heatmaps (18 body keypoints + background). Argmax each of the 18 keypoint channels over the 32 x 32 grid to get the normalized keypoint locations; connect them into a skeleton.

Keypoint order (18): nose, neck, r-shoulder, r-elbow, r-wrist, l-shoulder, l-elbow, l-wrist, r-hip, r-knee, r-ankle, l-hip, l-knee, l-ankle, r-eye, l-eye, r-ear, l-ear.

Ops

CONV_2D x41, DEPTHWISE_CONV_2D x14, TRANSPOSE x14, EXP x6, SUB x6,
GREATER_EQUAL x6, SELECT x6, ADD x6, PAD x3, CONCATENATION x1

(The ELU activations lower to EXP/SUB/GREATER_EQUAL/SELECT, all GPU-supported.) No GATHER_ND, no Flex/Custom.

On-device (Pixel 8a, verified)

The fp16 model compiles to 158 / 158 nodes on the LiteRT GPU delegate (LITERT_CL) — full GPU residency, no CPU fallback.

Minimal usage

Android (Kotlin, CompiledModel GPU)

val model = CompiledModel.create(context.assets, "pose_256_fp16.tflite",
    CompiledModel.Options(Accelerator.GPU), null)
val inputs = model.createInputBuffers()
val outputs = model.createOutputBuffers()
inputs[0].writeFloat(nhwc)             // [1,256,256,3] RGB, (px - 128) / 256
model.run(inputs, outputs)
val heatmaps = outputs[0].readFloat()  // [1,32,32,19] -> argmax per keypoint channel

Python (desktop verification)

import numpy as np
from PIL import Image
from ai_edge_litert.interpreter import Interpreter

img = Image.open("person.jpg").convert("RGB").resize((256, 256))
x = ((np.asarray(img, np.float32) - 128.0) / 256.0)[None]        # [1,256,256,3] NHWC

it = Interpreter(model_path="pose_256_fp16.tflite"); it.allocate_tensors()
it.set_tensor(it.get_input_details()[0]["index"], x); it.invoke()
hm = it.get_tensor(it.get_output_details()[0]["index"])[0]       # [32,32,19]

NAMES = ["nose","neck","r_sho","r_elb","r_wri","l_sho","l_elb","l_wri",
         "r_hip","r_knee","r_ank","l_hip","l_knee","l_ank","r_eye","l_eye","r_ear","l_ear"]
for k, name in enumerate(NAMES):                                  # channel 18 = background
    gy, gx = divmod(hm[:, :, k].argmax(), 32)
    print(f"{name}: ({gx/32:.2f}, {gy/32:.2f}) conf {hm[gy, gx, k]:.2f}")

A complete Android sample (camera + gallery, skeleton overlay) is available in google-ai-edge/litert-samples.

Training data & PII

This is a weights-exact format conversion of the public Lightweight OpenPose model; no new training was performed. It was trained for 2D human-pose estimation on the COCO 2017 keypoints dataset (web photos of people with keypoint annotations). These images contain people; the model outputs anonymous keypoint coordinates only and performs no identification. No PII was deliberately collected and this conversion adds none. Apply your own content/PII handling as appropriate. See the original lightweight-human-pose-estimation repo for dataset details.

Performance

Measured on a Pixel 8a (Tensor G3, Android 16) with the standard TFLite benchmark_model tool — 10 warm-up runs then 50 timed runs, reported as the tool's mean.

Runtime	Backend	Graph on GPU	Latency
TFLite benchmark_model (TfLiteGpuDelegateV2) — pose_256.tflite	GPU (OpenCL)	103 / 103	15.0 ms
TFLite benchmark_model (TfLiteGpuDelegateV2) — pose_256_fp16.tflite	GPU (OpenCL)	158 / 158	21.1 ms
TFLite benchmark_model — pose_256.tflite	CPU (XNNPACK, 4 threads)	—	157.3 ms
TFLite benchmark_model — pose_256_fp16.tflite	CPU (XNNPACK, 4 threads)	—	XNNPACK declined the graph

Any on-device figure recorded when this model shipped came from a different runtime. It was taken through LiteRT's own CompiledModel accelerator (logcat reports it as LITERT_CL), which is the path the Kotlin sample app and the LiteRT API use, and it appears elsewhere on this card. The rows above are the classic TFLite OpenCL delegate, measured with a tool anyone can download and re-run. The two are not comparable, so read the rows above as a reproducible floor rather than as this model's speed on LiteRT.

XNNPACK declines these fp16 graphs — it reports failed to delegate DEPTHWISE_CONV_2D and then fails to allocate tensors — so there is no usable CPU number. Disabling XNNPACK falls back to reference kernels, which measured about 20× slower than the GPU on models of this size and would not represent CPU inference anyone would ship.

License & attribution

• License: Apache-2.0. Weights/model from Daniil-Osokin/lightweight-human-pose-estimation.pytorch. Based on "Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose" (Osokin, 2018). Format conversion only; all credit to the original authors.