File size: 2,767 Bytes
200d040 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | TITLE (one line):
npu-linux-kit: practical AMD XDNA NPU apps for Linux — a working NPU webcam-effects daemon (220 FPS pipeline), plus an honest roadmap
CATEGORY: Showcase (also fits Projects & Demos)
----- BODY -----
**TL;DR.** A kit of practical Linux applications for the AMD XDNA NPU — built on my open-source
gen-1 (XDNA1 / Phoenix) bring-up. The first module is a **working real-time NPU webcam-effects
daemon** that outputs a v4l2 virtual camera (so Zoom/OBS/browsers use it). The other modules
(background-blur, embeddings, audio, a ggml backend) are on a roadmap and **clearly marked as
candidate/planned — no working claims until built.** Repo: https://github.com/Scottcjn/npu-linux-kit
(foundation: https://github.com/Scottcjn/open-xdna)
**Why.** AMD's NPU is underused on Linux — the official LLM stack targets XDNA2, and the NPU's real
strengths (low-power, conv/vision, batchable compute, offload) aren't packaged into usable Linux
apps. This kit does that, one measured module at a time.
**Working today — NPU webcam effects.** Live camera → NPU conv/colorspace effect (edge-stylize:
rgba2gray → 3×3 filter2d → threshold → blend) → **v4l2loopback virtual camera**. Measured on a
Ryzen 7 8845HS (XDNA1), Ubuntu 25.10 / kernel 6.17:
- Full per-frame pipeline incl. host DMA round-trip: **220 FPS @1080p, 451 FPS @720p** (7–32× headroom over 30fps).
- In the live daemon, the NPU effect adds **~2 ms/frame at ~6.6 W** — end-to-end is **camera-capture-bound** (~15 FPS on a USB webcam), *never* NPU-bound.
So the NPU runs the effect essentially for free, leaving the CPU/iGPU idle — exactly the low-power
offload role it's built for. There's huge headroom for heavier effects.
**Roadmap (honestly marked):**
- **Background blur** — the marquee "Studio Effects" feature; needs a segmentation model on the NPU. The headroom says feasible; not built.
- **Embeddings** — batch sentence/RAG embeddings; matmul-heavy + batchable → amortizes the NPU's dispatch overhead. Candidate.
- **Audio** — wake-word / denoise; the NPU's classic edge job, but **untested here** — listed as a candidate, not a claim.
- **ggml backend** — an XDNA backend for ggml/llama.cpp; the foundational enabler. Biggest lift.
**Honesty / scope.** Verified on XDNA1 (Phoenix). The camera module is real and measured; the rest
make no working claims. Research/community project, not a production stack. The kit builds on
open-xdna, where I also filed the bring-up issues/PRs upstream (amd/xdna-driver #1447/#1448/#1449,
plus an mlir-aie discussion on the AIE2 vector ISA).
Feedback very welcome — especially from anyone on Phoenix/Hawk Point hardware who wants NPU camera
effects on Linux. 👉 https://github.com/Scottcjn/npu-linux-kit
|