Hexagon NPU: measure the published file via on-device JIT, add reproduction steps

README.md

@@ -94,20 +94,69 @@ Measured on a **Pixel 8a** (Tensor G3, Android 16) with the standard TFLite [`be
 
 **The two GPU rows are different runtimes, not a contradiction.** The `LITERT_CL` figure is the one recorded when this model shipped, taken through LiteRT's own `CompiledModel` accelerator — the path the Kotlin sample app and the LiteRT API use. The `TfLiteGpuDelegateV2` figure is the classic TFLite OpenCL delegate, measured with a tool anyone can download and re-run. They agree on how much of the graph the GPU takes; they disagree on speed, and the classic delegate is the slower of the two here. Read the `TfLiteGpuDelegateV2` row as a reproducible floor, not as this model's speed on LiteRT.
 
-### Snapdragon NPU (Hexagon, AOT)
+### Snapdragon NPU (Hexagon)
 
-Measured on a physical **Samsung Galaxy S26** (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81) over adb, using LiteRT `CompiledModel` from a purpose-built harness — 5 warm-up runs then 50 timed runs, one accelerator per process.
+This file runs on the Qualcomm Hexagon NPU **as published** — no conversion and no
+pre-compiled artifact. LiteRT compiles it on the device and caches the result.
 
-| Compute unit | Inference (median / min) | First load | Start thermal headroom |
+Measured on a physical **Samsung Galaxy S26** (Snapdragon 8 Elite Gen 5 / SM8850,
+Hexagon v81) with LiteRT `CompiledModel` 2.2.0 — 5 warm-up runs then 50 timed runs, one
+accelerator per process, every row taken at device thermal status `NONE`.
+
+| Compute unit | Inference (median / min) | Load | Start headroom |
 |---|---|---|---|
-| NPU (Hexagon, AOT context) | 13.86 ms / 13.57 ms | 121 ms | 0.66 |
-| GPU (Adreno) | 13.56 ms / 10.57 ms | 1387 ms | 0.66 |
+| NPU (Hexagon) — first launch | 13.81 ms / 13.65 ms | 17003 ms | 0.58 |
+| NPU (Hexagon) — cached | 13.72 ms / 13.55 ms | **121 ms** | 0.61 |
+| GPU (Adreno) | 13.82 ms / 13.31 ms | 1401 ms | 0.61 |
+
+Inference is close to a tie (13.72 ms against 13.82 ms, 1.01x). The first launch pays once for on-device compilation; every launch after that
+loads in 121 ms against 1401 ms for the GPU (11.6x), because the GPU rebuilds its
+shaders each time. The file is fp16 and needs no int8 quantization to reach the NPU.
+
+#### Running it on the NPU
+
+Put these in `jniLibs/arm64-v8a/`. **None of them are distributed from this repository** —
+the first two come from Google, the rest from Qualcomm's own SDK:
+
+| Library | Source |
+|---|---|
+| `libLiteRtDispatch_Qualcomm.so`, `libLiteRtCompilerPlugin_Qualcomm.so` | `litert_npu_runtime_libraries_jit.zip`, a release asset of [google-ai-edge/LiteRT](https://github.com/google-ai-edge/LiteRT/releases) |
+| `libQnnHtp.so`, `libQnnSystem.so`, `libQnnHtpV81Stub.so`, `libQnnHtpV81Skel.so`, `libQnnHtpPrepare.so`, `libQnnIr.so`, `libQnnSaver.so` | Qualcomm QAIRT — the same zip ships `fetch_qualcomm_library.sh`, which downloads the SDK and copies them for you |
+
+Pick the runtime matching the device's Hexagon version: SM8550 → v73, SM8650 → v75,
+SM8750 → v79, SM8850 → v81.
+
+```kotlin
+val env = Environment.create(
+    context,
+    mapOf(
+        Environment.Option.DispatchLibraryDir to context.applicationInfo.nativeLibraryDir,
+        // Required for on-device compilation. Without it the model silently runs on CPU.
+        Environment.Option.CompilerPluginLibraryDir to context.applicationInfo.nativeLibraryDir,
+    ),
+)
+val options = CompiledModel.Options(Accelerator.NPU).apply {
+    qualcommOptions = CompiledModel.QualcommOptions(
+        htpPerformanceMode = CompiledModel.QualcommOptions.HtpPerformanceMode.BURST
+    )
+}
+val model = CompiledModel.create(context.assets, "twinlite.tflite", options, env)
+```
 
-**Inference is a tie here** — the GPU is ahead by 2%, which is inside the run-to-run spread. Where the NPU wins outright is startup: an AOT-compiled context loads in 121 ms against 1387 ms for the GPU, which builds its shaders on every launch. That is **11x**, and in a sweep of 20 models from this zoo the NPU loaded faster than the GPU on every single one. Pick the NPU when cold start matters, and treat the two as equivalent for sustained throughput on this model.
+Build settings: `useLegacyPackaging = true` under `packaging { jniLibs { … } }`, so the
+DSP can open the skel from a real path, and Kotlin **2.3+** for LiteRT 2.2.0's metadata.
 
-The file is fp16 and needs no int8 quantization to reach the NPU — the AOT compile places all 270 operations there with no fallback.
+> **Every NPU failure here is silent.** There is no error when the NPU is unavailable —
+> you get a plausible CPU number instead. Confirm from logcat which delegate took the
+> graph: `Replacing 1 out of 1 node(s) with delegate (DispatchDelegate)` is the NPU,
+> while `... (TfLiteXNNPackDelegate)` is the CPU. A missing library is reported only as
+> a `W`-level `dlopen failed` line under a generic `No compiler plugin found` summary.
 
-**On the measurement conditions.** Thermal headroom is reported as measured, where 1.0 is the throttling threshold; the value is disclosed rather than held to a threshold. Both rows were taken at the same headroom and compare directly. Figures taken at a different headroom will differ — in our tests the same measurement moved 13.5% across a headroom change of 0.05, while device thermal status still read `NONE`. Each accelerator ran in its own process, because LiteRT's `Environment` is shared within a process and the first model load fixes the dispatch options for every later one.
+**On the conditions.** Thermal headroom is reported as measured, where 1.0 is the
+throttling threshold. All rows were taken at a comparable headroom and compare directly;
+figures taken at a different headroom will differ. Each accelerator ran in its own
+process, because LiteRT's `Environment` is shared within one and the first model load
+fixes the options for every later one.
 
 ## License