README.md

---
license: apache-2.0
library_name: litert
pipeline_tag: keypoint-detection
tags:
  - litert
  - tflite
  - on-device
  - android
  - pose-estimation
  - openpose
  - gpu
---

# lightweight-OpenPose — LiteRT (TFLite) GPU, FP16

On-device [LiteRT](https://ai.google.dev/edge/litert) (`.tflite`) conversion of
**[lightweight-OpenPose](https://github.com/Daniil-Osokin/lightweight-human-pose-estimation.pytorch)**
for human pose estimation. The model is a MobileNet-based heatmap network; it outputs
**keypoint heatmaps only** and the keypoint decode (argmax) is done in app code.

![lightweight-OpenPose — 18-keypoint skeleton (on-device LiteRT GPU)](samples/sample.png)

The model runs **fully on the LiteRT `CompiledModel` GPU accelerator** (ML Drift): every op is
GPU-native, no CPU fallback. Converted with
[`litert-torch`](https://github.com/google-ai-edge/ai-edge-torch) **with no patches**.

> Why heatmaps-only: MoveNet's official `.tflite` bakes the keypoint decode into the graph
> (`GATHER_ND`), which the GPU delegate can't run — so it only partially offloads to the GPU.
> Keeping the graph pure-conv and decoding in app code keeps it 100% on the GPU.

## Files

| File | Precision | Size |
|------|-----------|------|
| `pose_256_fp16.tflite` | fp16 weights | ~8.3 MB |
| `pose_256.tflite` | fp32 | ~16.4 MB |

## I/O

- **Input**: `[1, 256, 256, 3]` float32, **NHWC**, RGB, normalized `(px - 128) / 256`.
- **Output**: `[1, 32, 32, 19]` float32, **NHWC**, keypoint heatmaps (18 body keypoints +
  background). Argmax each of the 18 keypoint channels over the `32 x 32` grid to get the
  normalized keypoint locations; connect them into a skeleton.

Keypoint order (18): nose, neck, r-shoulder, r-elbow, r-wrist, l-shoulder, l-elbow, l-wrist,
r-hip, r-knee, r-ankle, l-hip, l-knee, l-ankle, r-eye, l-eye, r-ear, l-ear.

## Ops

```
CONV_2D x41, DEPTHWISE_CONV_2D x14, TRANSPOSE x14, EXP x6, SUB x6,
GREATER_EQUAL x6, SELECT x6, ADD x6, PAD x3, CONCATENATION x1
```

(The `ELU` activations lower to `EXP/SUB/GREATER_EQUAL/SELECT`, all GPU-supported.) No
`GATHER_ND`, no Flex/Custom.

## On-device (Pixel 8a, verified)

The fp16 model compiles to **158 / 158 nodes on the LiteRT GPU delegate (LITERT_CL)** — full
GPU residency, no CPU fallback.

## Minimal usage

**Android (Kotlin, CompiledModel GPU)**

```kotlin
val model = CompiledModel.create(context.assets, "pose_256_fp16.tflite",
    CompiledModel.Options(Accelerator.GPU), null)
val inputs = model.createInputBuffers()
val outputs = model.createOutputBuffers()
inputs[0].writeFloat(nhwc)             // [1,256,256,3] RGB, (px - 128) / 256
model.run(inputs, outputs)
val heatmaps = outputs[0].readFloat()  // [1,32,32,19] -> argmax per keypoint channel
```

**Python (desktop verification)**

```python
import numpy as np
from PIL import Image
from ai_edge_litert.interpreter import Interpreter

img = Image.open("person.jpg").convert("RGB").resize((256, 256))
x = ((np.asarray(img, np.float32) - 128.0) / 256.0)[None]        # [1,256,256,3] NHWC

it = Interpreter(model_path="pose_256_fp16.tflite"); it.allocate_tensors()
it.set_tensor(it.get_input_details()[0]["index"], x); it.invoke()
hm = it.get_tensor(it.get_output_details()[0]["index"])[0]       # [32,32,19]

NAMES = ["nose","neck","r_sho","r_elb","r_wri","l_sho","l_elb","l_wri",
         "r_hip","r_knee","r_ank","l_hip","l_knee","l_ank","r_eye","l_eye","r_ear","l_ear"]
for k, name in enumerate(NAMES):                                  # channel 18 = background
    gy, gx = divmod(hm[:, :, k].argmax(), 32)
    print(f"{name}: ({gx/32:.2f}, {gy/32:.2f}) conf {hm[gy, gx, k]:.2f}")
```

A complete Android sample (camera + gallery, skeleton overlay) is available in
[google-ai-edge/litert-samples](https://github.com/google-ai-edge/litert-samples).

## Training data & PII

This is a weights-exact format conversion of the public **Lightweight OpenPose** model; no
new training was performed. It was trained for 2D human-pose estimation on the **COCO 2017
keypoints** dataset (web photos of people with keypoint annotations). These images contain
people; the model outputs anonymous keypoint coordinates only and performs no
identification. No PII was deliberately collected and this conversion adds none. Apply your
own content/PII handling as appropriate. See the original
[lightweight-human-pose-estimation](https://github.com/Daniil-Osokin/lightweight-human-pose-estimation.pytorch)
repo for dataset details.

## License & attribution

- License: **Apache-2.0**. Weights/model from
  [`Daniil-Osokin/lightweight-human-pose-estimation.pytorch`](https://github.com/Daniil-Osokin/lightweight-human-pose-estimation.pytorch).
  Based on *"Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose"* (Osokin,
  2018). Format conversion only; all credit to the original authors.