darkmaniac7/TokForge-SDXL-QNN-NPU

TokForge

• Website: https://tokforge.ai
• Discord: https://discord.gg/Acv3CBtfVm
• Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
• iOS TestFlight: https://testflight.apple.com/join/jnufjzRr

Runs on-device in the TokForge app.

TokForge — RealVisXL V4.0 Lightning (Qualcomm Hexagon NPU)

RealVisXL V4.0 Lightning image generation for the Qualcomm Hexagon NPU (HTP), packaged for on-device image generation in the TokForge Android app (dev.tokforge). This is the higher-quality 1024×1024 "Faithful" tier alongside the SD1.5 NPU bundle.

vc410: this repo replaces the previous SDXL-Turbo bins (few-step distillation gave fused heads / doubled bodies / extra limbs) with RealVisXL V4.0 Lightning, which is adversarially distilled for deterministic few-step Euler and renders clean, photoreal people. The W8A16 NPU quant is unchanged — the checkpoint was the cure.

The model is quantized to W8A16 (8-bit weights, 16-bit activations) and compiled to QNN HTP context binaries that run on the phone's Hexagon DSP. The pipeline uses fp16 text encoders and a TAESDXL tiny-VAE decoder, runs 6-step EulerDiscrete, and is guidance-free (one UNet pass per step).

Based on

SG161222/RealVisXL_V4.0_Lightning — CreativeML OpenRAIL++-M license (commercial use permitted, no revenue cap).

Format

QNN HTP context binaries (W8A16), native 1024px (128×128 latent). These are compiled for the Hexagon DSP and are not a portable format like GGUF. The repo ships native sets for V73 / V75 / V79 / V81; the app reads the device Hexagon arch (dsp_arch) and downloads the matching set. Forward-compatibility (a lower-arch binary also runs on a higher-arch DSP) still applies as a fallback. Device-verified clean on V75 (Lenovo SM8650) and V81 (RedMagic SM8850).

Pipeline

Stage	File
CLIP-L text encoder (fp16)	<arch>/text_encoder_1_fp16.bin
OpenCLIP-bigG text encoder (fp16)	<arch>/text_encoder_2_fp16.bin
Combined-emb MLP (host CPU)	<arch>/sdxl_emb_mlp.bin
UNet (W8A16, DSP)	<arch>/unet.bin
TAESDXL tiny-VAE (CPU MNN)	<arch>/taesdxl_decoder.mnn
Dual CLIP BPE tokenizers	<arch>/tokenizer/, <arch>/tokenizer_2/

Scheduler: EulerDiscrete (deterministic), trailing spacing, epsilon prediction, 6 steps, guidance_scale 0. Runs via the license-clean libsdxl_qnn_driver in the TokForge app.

See manifest.json for the per-arch file list, sizes, and md5 checksums.

Reference-image (IP-Adapter / identity-preserving) — ip/ subfolder

The ip/ subfolder ships the IP-Adapter context bins for identity-preserving (reference-image) rendering. When a reference photo is attached, the driver swaps the base unet.bin for the IP-baked UNet (unet_ip.bin) and renders with the reference's identity; no-reference renders are unchanged.

File	What
ip/v75/unet_ip.bin	IP-Adapter UNet context bin, Hexagon V75 (device-verified)
ip/v81/unet_ip.bin	IP-Adapter UNet context bin, Hexagon V81 (compiled from the same DLC; on-device validation pending)
ip/model_w8a16.dlc	arch-agnostic W8A16 QAIRT 2.40 DLC the per-arch unet_ip.bin are compiled from
ip/ip_kv_proj_weights.npz	70× to_k_ip/to_v_ip Linear weights for the CPU IP side (k_0..k_69, v_0..v_69, inner[70], heads[70])

The bins are identity-agnostic and safe to publish: the 70 cross-attention k_ip/v_ip are render-constant inputs computed on the CPU per render (reference → CLIP-ViT-H penultimate → plus-face Resampler → 70× to_k_ip/to_v_ip), so no likeness is baked into the published files. The CLIP-ViT-H encoder + plus-face adapter weights are the standard public h94/IP-Adapter SDXL plus-face files and are not re-hosted here. The app downloads unet_ip.bin as an optional bundle file (paired in each set's files map); its absence degrades to a plain (no-identity) render and never fails the install. IO contract, checksums, and per-arch details are in manifest.json under the top-level ip block.