litert-community/L2CS-Gaze360-LiteRT

L2CS-Net — LiteRT (on-device gaze estimation, fully-GPU)

L2CS-Net (Ahmednull) gaze estimation, converted to LiteRT and running fully on the CompiledModel GPU (ML Drift) on Android. Predicts where a centered face is looking (yaw/pitch). ResNet50 backbone trained on Gaze360.

On-device (Pixel 8a, Tensor G3 — verified)

nodes on GPU	139 / 139 LITERT_CL (full residency)
inference	~3 ms (448×448)
size	47.9 MB (fp16)
accuracy	device-vs-PyTorch corr 0.9999, gaze angle within ~0.1°

face[1,3,448,448] (ImageNet-normalized) →[GPU: ResNet50]→ yaw[1,90], pitch[1,90] (softmax over angle bins)

The 90 bins span [-180,180]° (4° each); gaze angle = softmax expectation Σ p_i·i · 4 − 180 (softmax baked in).

Minimal usage

Android (Kotlin, CompiledModel GPU)

val model = CompiledModel.create(context.assets, "gaze_fp16.tflite",
    CompiledModel.Options(Accelerator.GPU), null)
val inputs = model.createInputBuffers()
val outputs = model.createOutputBuffers()
inputs[0].writeFloat(chw)              // [1,3,448,448] ImageNet-normalized RGB, NCHW
model.run(inputs, outputs)
val yawProbs = outputs[0].readFloat()   // [1,90] softmax over 4-deg bins
val pitchProbs = outputs[1].readFloat() // [1,90]; deg = sum(p_i * i) * 4 - 180

Python (desktop verification)

MEAN = np.array([0.485, 0.456, 0.406], np.float32)
STD  = np.array([0.229, 0.224, 0.225], np.float32)
import numpy as np
from PIL import Image
from ai_edge_litert.interpreter import Interpreter

img = Image.open("face.jpg").convert("RGB").resize((448, 448))   # centered face crop
x = ((np.asarray(img, np.float32) / 255 - MEAN) / STD).transpose(2, 0, 1)[None]

it = Interpreter(model_path="gaze_fp16.tflite"); it.allocate_tensors()
it.set_tensor(it.get_input_details()[0]["index"], x); it.invoke()
od = it.get_output_details()                       # output 0 = yaw, 1 = pitch (both [1,90])
deg = lambda p: float((p * np.arange(90)).sum() * 4 - 180)
yaw, pitch = (deg(it.get_tensor(o["index"])[0]) for o in od)
print(f"yaw {yaw:+.1f} deg, pitch {pitch:+.1f} deg")

How it converts (litert-torch)

Pure CNN (ResNet50 + 2 FC heads). Two numerically-exact ResNet fixes:

1. stem MaxPool2d(3,s2,p1) → zero-pad + valid max-pool — PyTorch's max-pool pads with -inf → a PADV2 the Mali delegate won't delegate (compile fail); since the pool follows a ReLU, a 0-pad is exactly equivalent → PAD, full GPU residency.
2. global AdaptiveAvgPool2d(1) → mean(3).mean(2).

Result: banned ops NONE, all tensors ≤4D, tflite-vs-torch corr 1.0, device-vs-torch corr 0.9999.

Preprocessing & decode

Center-crop to a (centered) face, resize 448×448, /255, ImageNet mean/std, NCHW. Decode: softmax expectation over the 90 bins → yaw/pitch degrees → gaze direction.

License

MIT. Upstream: Ahmednull/L2CS-Net.