litert-community/PP-OCRv5-LiteRT

PP-OCRv5 — LiteRT on-device (fully GPU)

On-device LiteRT conversion of PP-OCRv5 (PaddleOCR 2025, Apache-2.0) text detection + recognition, running fully on the CompiledModel GPU delegate (LITERT_CL). Detects text regions in an image and reads each line. The recognizer uses a CTC head (no autoregressive decoder), so both stages ride the GPU with no CPU/ONNX fallback — unlike VLM-based OCR (Florence-2 / GOT-OCR) whose AR decoder must run on CPU. Device-verified on a Pixel 8a.

Files

File	Size	Delegate	In → Out
ppocr_det_fp16.tflite	10 MB	GPU	image [1,3,640,640] → prob map [1,1,640,640]
ppocr_rec_fp16.tflite	17 MB	GPU	line [1,3,48,320] → CTC logits [1,T,18385]
ppocr_rec_fp32.tflite	33 MB	CPU (browser)	line [1,3,48,320] → CTC logits [1,T,18385]
ppocrv5_dict.txt	—	CPU	18383-char dictionary (CTC layout: blank + dict + space)

Pixel 8a: detector 777/777 + recognizer 827/827 on LITERT_CL, ~9 ms each; a 3-line image read 3/3 correct ("Hello OCR 2026" / "PP-OCRv5 on GPU" / "LiteRT CompiledModel").

ppocr_rec_fp32.tflite is the recognizer before the fp16 cast, added for the browser. In LiteRT.js 2.5.3, wasm XNNPACK declines the fp16 recognizer graph (reference-kernel fallback, ~430 ms/line) but fully delegates the fp32 one: ~21 ms/line on an M4 Max. The WebGPU delegate flips some recognizer argmaxes on this architecture regardless of weight precision, so in the browser run the detector on WebGPU and the recognizer on wasm with the fp32 file.

Pipeline

image →[GPU detector]→ prob map → [CPU: threshold + connected components + unclip] → boxes
   → crop+resize →[GPU recognizer]→ CTC logits → [CPU: CTC greedy decode] → text

Minimal usage

Android (Kotlin, CompiledModel GPU)

val det = CompiledModel.create(context.assets, "ppocr_det_fp16.tflite",
    CompiledModel.Options(Accelerator.GPU), null)
val rec = CompiledModel.create(context.assets, "ppocr_rec_fp16.tflite",
    CompiledModel.Options(Accelerator.GPU), null)
val dIn = det.createInputBuffers(); val dOut = det.createOutputBuffers()
dIn[0].writeFloat(image)            // [1,3,640,640] NCHW, /255 then ImageNet mean/std
det.run(dIn, dOut)
val prob = dOut[0].readFloat()      // [1,1,640,640] text probability -> boxes (CPU)
val rIn = rec.createInputBuffers(); val rOut = rec.createOutputBuffers()
rIn[0].writeFloat(lineCrop)         // [1,3,48,320] NCHW, (x/255 - 0.5)/0.5
rec.run(rIn, rOut)
val logits = rOut[0].readFloat()    // [1,T,18385] -> CTC greedy decode (Python below)

Python (desktop verification)

import cv2, numpy as np
from ai_edge_litert.interpreter import Interpreter

MEAN, STD = np.array([0.485, 0.456, 0.406]), np.array([0.229, 0.224, 0.225])
im640 = cv2.resize(cv2.cvtColor(cv2.imread("doc.jpg"), cv2.COLOR_BGR2RGB), (640, 640))
x = ((im640 / 255 - MEAN) / STD).transpose(2, 0, 1)[None].astype(np.float32)

det = Interpreter(model_path="ppocr_det_fp16.tflite"); det.allocate_tensors()
det.set_tensor(det.get_input_details()[0]["index"], x); det.invoke()
prob = det.get_tensor(det.get_output_details()[0]["index"])[0, 0]     # [640,640]

rec = Interpreter(model_path="ppocr_rec_fp16.tflite"); rec.allocate_tensors()
chars = [""] + open("ppocrv5_dict.txt", encoding="utf-8").read().splitlines() + [" "]

n, labels, stats, _ = cv2.connectedComponentsWithStats((prob > 0.3).astype(np.uint8))
for i in range(1, n):                                                 # each text region
    x0, y0, w, h, _ = stats[i]
    if w < 6 or h < 6 or prob[labels == i].mean() < 0.5: continue
    pad = int(np.clip(0.35 * min(w, h), 2, 24))                       # approx DB unclip
    crop = im640[max(y0 - pad, 0):y0 + h + pad, max(x0 - pad, 0):x0 + w + pad]
    rw = min(max(round(48 * crop.shape[1] / crop.shape[0]), 1), 320)  # keep-aspect h=48
    line = np.zeros((48, 320, 3), np.float32)                         # pad to width 320
    line[:, :rw] = cv2.resize(crop, (rw, 48))
    lx = ((line / 255 - 0.5) / 0.5).transpose(2, 0, 1)[None].astype(np.float32)
    rec.set_tensor(rec.get_input_details()[0]["index"], lx); rec.invoke()
    ids = rec.get_tensor(rec.get_output_details()[0]["index"])[0].argmax(-1)   # [T]
    text = "".join(chars[c] for t, c in enumerate(ids)                # CTC: collapse repeats,
                   if c != 0 and (t == 0 or c != ids[t - 1]))         # drop blank (id 0)
    print((x0, y0), text)

Re-authoring (litert-torch, parity corr 1.0)

• Detector DB-head ConvTranspose2d → ZeroStuffConvT2d (2D nearest-upsample × stride zero-stuff mask
  • flipped conv2d; TRANSPOSE_CONV is Mali-rejected). Numerically exact.
• Recognizer SVTR attention fused-QKV 5D reshape → split q/k/v into 4D (numerically identical).

Preprocessing: detector = ImageNet mean/std, /255, NCHW, 640×640. recognizer = resize to h=48 keep-aspect, pad to width 320, (img/255−0.5)/0.5.

Sample app

A complete Android sample app + the conversion scripts are in the official LiteRT samples repository under compiled_model_api/ocr (google-ai-edge/litert-samples). Push these files to the app's filesDir with that sample's install_to_device.sh.

Weights are converted from PaddleOCR via the PaddleOCR2Pytorch port (Apache-2.0). License follows upstream PaddleOCR (Apache-2.0).

Performance

Measured on a Pixel 8a (Tensor G3, Android 16) with the standard TFLite benchmark_model tool — 10 warm-up runs then 50 timed runs, reported as the tool's mean.

Runtime	Backend	Graph on GPU	Latency
TFLite benchmark_model (TfLiteGpuDelegateV2) — ppocr_rec_fp16.tflite	GPU (OpenCL)	579 / 827	91.7 ms
TFLite benchmark_model (TfLiteGpuDelegateV2) — ppocr_det_fp16.tflite	GPU (OpenCL)	777 / 777	45.8 ms
TFLite benchmark_model — ppocr_rec_fp16.tflite	CPU (XNNPACK, 4 threads)	—	XNNPACK declined the graph
TFLite benchmark_model — ppocr_det_fp16.tflite	CPU (XNNPACK, 4 threads)	—	XNNPACK declined the graph

Any on-device figure recorded when this model shipped came from a different runtime. It was taken through LiteRT's own CompiledModel accelerator (logcat reports it as LITERT_CL), which is the path the Kotlin sample app and the LiteRT API use, and it appears elsewhere on this card. The rows above are the classic TFLite OpenCL delegate, measured with a tool anyone can download and re-run. The two are not comparable, so read the rows above as a reproducible floor rather than as this model's speed on LiteRT.

XNNPACK declines these fp16 graphs — it reports failed to delegate DEPTHWISE_CONV_2D and then fails to allocate tensors — so there is no usable CPU number. Disabling XNNPACK falls back to reference kernels, which measured about 20× slower than the GPU on models of this size and would not represent CPU inference anyone would ship.

Note that the GPU does not take the whole graph here (579 / 827 in ppocr_rec_fp16.tflite); the remainder runs on the CPU and the split costs a per-partition round trip.