| TITLE (one line): |
| npu-linux-kit: practical AMD XDNA NPU apps for Linux — a working NPU webcam-effects daemon (220 FPS pipeline), plus an honest roadmap |
|
|
| CATEGORY: Showcase (also fits Projects & Demos) |
|
|
| ----- BODY ----- |
|
|
| **TL;DR.** A kit of practical Linux applications for the AMD XDNA NPU — built on my open-source |
| gen-1 (XDNA1 / Phoenix) bring-up. The first module is a **working real-time NPU webcam-effects |
| daemon** that outputs a v4l2 virtual camera (so Zoom/OBS/browsers use it). The other modules |
| (background-blur, embeddings, audio, a ggml backend) are on a roadmap and **clearly marked as |
| candidate/planned — no working claims until built.** Repo: https://github.com/Scottcjn/npu-linux-kit |
| (foundation: https://github.com/Scottcjn/open-xdna) |
|
|
| **Why.** AMD's NPU is underused on Linux — the official LLM stack targets XDNA2, and the NPU's real |
| strengths (low-power, conv/vision, batchable compute, offload) aren't packaged into usable Linux |
| apps. This kit does that, one measured module at a time. |
|
|
| **Working today — NPU webcam effects.** Live camera → NPU conv/colorspace effect (edge-stylize: |
| rgba2gray → 3×3 filter2d → threshold → blend) → **v4l2loopback virtual camera**. Measured on a |
| Ryzen 7 8845HS (XDNA1), Ubuntu 25.10 / kernel 6.17: |
|
|
| - Full per-frame pipeline incl. host DMA round-trip: **220 FPS @1080p, 451 FPS @720p** (7–32× headroom over 30fps). |
| - In the live daemon, the NPU effect adds **~2 ms/frame at ~6.6 W** — end-to-end is **camera-capture-bound** (~15 FPS on a USB webcam), *never* NPU-bound. |
|
|
| So the NPU runs the effect essentially for free, leaving the CPU/iGPU idle — exactly the low-power |
| offload role it's built for. There's huge headroom for heavier effects. |
|
|
| **Roadmap (honestly marked):** |
| - **Background blur** — the marquee "Studio Effects" feature; needs a segmentation model on the NPU. The headroom says feasible; not built. |
| - **Embeddings** — batch sentence/RAG embeddings; matmul-heavy + batchable → amortizes the NPU's dispatch overhead. Candidate. |
| - **Audio** — wake-word / denoise; the NPU's classic edge job, but **untested here** — listed as a candidate, not a claim. |
| - **ggml backend** — an XDNA backend for ggml/llama.cpp; the foundational enabler. Biggest lift. |
|
|
| **Honesty / scope.** Verified on XDNA1 (Phoenix). The camera module is real and measured; the rest |
| make no working claims. Research/community project, not a production stack. The kit builds on |
| open-xdna, where I also filed the bring-up issues/PRs upstream (amd/xdna-driver #1447/#1448/#1449, |
| plus an mlir-aie discussion on the AIE2 vector ISA). |
|
|
| Feedback very welcome — especially from anyone on Phoenix/Hawk Point hardware who wants NPU camera |
| effects on Linux. 👉 https://github.com/Scottcjn/npu-linux-kit |
|
|
