README.md

metadata

license: apache-2.0
library_name: litert
pipeline_tag: keypoint-detection
tags:
  - litert
  - tflite
  - on-device
  - android
  - pose-estimation
  - openpose
  - gpu

lightweight-OpenPose — LiteRT (TFLite) GPU, FP16

On-device LiteRT (.tflite) conversion of lightweight-OpenPose for human pose estimation. The model is a MobileNet-based heatmap network; it outputs keypoint heatmaps only and the keypoint decode (argmax) is done in app code.

The model runs fully on the LiteRT CompiledModel GPU accelerator (ML Drift): every op is GPU-native, no CPU fallback. Converted with litert-torch with no patches.

Why heatmaps-only: MoveNet's official .tflite bakes the keypoint decode into the graph (GATHER_ND), which the GPU delegate can't run — so it only partially offloads to the GPU. Keeping the graph pure-conv and decoding in app code keeps it 100% on the GPU.

Files

File	Precision	Size
pose_256_fp16.tflite	fp16 weights	~8.3 MB
pose_256.tflite	fp32	~16.4 MB

I/O

• Input: [1, 256, 256, 3] float32, NHWC, RGB, normalized (px - 128) / 256.
• Output: [1, 32, 32, 19] float32, NHWC, keypoint heatmaps (18 body keypoints + background). Argmax each of the 18 keypoint channels over the 32 x 32 grid to get the normalized keypoint locations; connect them into a skeleton.

Keypoint order (18): nose, neck, r-shoulder, r-elbow, r-wrist, l-shoulder, l-elbow, l-wrist, r-hip, r-knee, r-ankle, l-hip, l-knee, l-ankle, r-eye, l-eye, r-ear, l-ear.

Ops

CONV_2D x41, DEPTHWISE_CONV_2D x14, TRANSPOSE x14, EXP x6, SUB x6,
GREATER_EQUAL x6, SELECT x6, ADD x6, PAD x3, CONCATENATION x1

(The ELU activations lower to EXP/SUB/GREATER_EQUAL/SELECT, all GPU-supported.) No GATHER_ND, no Flex/Custom.

On-device (Pixel 8a, verified)

The fp16 model compiles to 158 / 158 nodes on the LiteRT GPU delegate (LITERT_CL) — full GPU residency, no CPU fallback.

Minimal usage

Android (Kotlin, CompiledModel GPU)

val model = CompiledModel.create(context.assets, "pose_256_fp16.tflite",
    CompiledModel.Options(Accelerator.GPU), null)
val inputs = model.createInputBuffers()
val outputs = model.createOutputBuffers()
inputs[0].writeFloat(nhwc)             // [1,256,256,3] RGB, (px - 128) / 256
model.run(inputs, outputs)
val heatmaps = outputs[0].readFloat()  // [1,32,32,19] -> argmax per keypoint channel

Python (desktop verification)

import numpy as np
from PIL import Image
from ai_edge_litert.interpreter import Interpreter

img = Image.open("person.jpg").convert("RGB").resize((256, 256))
x = ((np.asarray(img, np.float32) - 128.0) / 256.0)[None]        # [1,256,256,3] NHWC

it = Interpreter(model_path="pose_256_fp16.tflite"); it.allocate_tensors()
it.set_tensor(it.get_input_details()[0]["index"], x); it.invoke()
hm = it.get_tensor(it.get_output_details()[0]["index"])[0]       # [32,32,19]

NAMES = ["nose","neck","r_sho","r_elb","r_wri","l_sho","l_elb","l_wri",
         "r_hip","r_knee","r_ank","l_hip","l_knee","l_ank","r_eye","l_eye","r_ear","l_ear"]
for k, name in enumerate(NAMES):                                  # channel 18 = background
    gy, gx = divmod(hm[:, :, k].argmax(), 32)
    print(f"{name}: ({gx/32:.2f}, {gy/32:.2f}) conf {hm[gy, gx, k]:.2f}")

A complete Android sample (camera + gallery, skeleton overlay) is available in google-ai-edge/litert-samples.

Training data & PII

This is a weights-exact format conversion of the public Lightweight OpenPose model; no new training was performed. It was trained for 2D human-pose estimation on the COCO 2017 keypoints dataset (web photos of people with keypoint annotations). These images contain people; the model outputs anonymous keypoint coordinates only and performs no identification. No PII was deliberately collected and this conversion adds none. Apply your own content/PII handling as appropriate. See the original lightweight-human-pose-estimation repo for dataset details.

License & attribution

• License: Apache-2.0. Weights/model from Daniil-Osokin/lightweight-human-pose-estimation.pytorch. Based on "Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose" (Osokin, 2018). Format conversion only; all credit to the original authors.