Upload 17 files

README.md

@@ -1,3 +1,107 @@
 ---
-license: bsd-3-clause
+license: bsd-3-clause-clear
+base_model: Qualcomm-AI-Research/Neodragon
+tags:
+  - text-to-video
+  - litert
+  - tflite
+  - on-device
+  - android
+library_name: litert
 ---
+
+# Neodragon — LiteRT (.tflite) conversions for on-device text-to-video
+
+LiteRT conversions of [Qualcomm-AI-Research/Neodragon](https://huggingface.co/Qualcomm-AI-Research/Neodragon),
+prepared for [Box](https://github.com/jegly/Box) — an open-source, fully offline
+on-device AI app for Android.
+
+These are **converted artifacts only**. The models, research and original weights
+are Qualcomm AI Research's work; see the
+[upstream repo](https://huggingface.co/Qualcomm-AI-Research/Neodragon) and
+[paper](https://qualcomm-ai-research.github.io/neodragon).
+
+Generates a 320×512 clip entirely on-device — no network, no server, no account.
+
+## Licence — please read before use
+
+The weights are released by Qualcomm under **two** licences, and both carry over
+to these conversions:
+
+1. **[BSD 3-Clause Clear](https://spdx.org/licenses/BSD-3-Clause-Clear.html)** —
+   permits redistribution and derivative works (a conversion is a derivative)
+   with notices retained. Note the "Clear" variant grants **no patent licence**.
+2. **[Qualcomm Responsible AI License](https://www.qualcomm.com/site/responsible-ai-license)** —
+   adds use restrictions. It is not a grant licence; it sits on top of the BSD
+   grant and **flows down to you**. Prohibited uses include military use,
+   criminal/predictive-policing applications, social scoring, biometric
+   categorisation, emotion recognition in workplaces or education, and
+   distributing generated content without disclosing that it is machine
+   generated. Read the full text before using these files.
+
+Box surfaces these restrictions in a consent screen before download and marks
+generated output as AI-generated.
+
+## What's here
+
+Fixed-shape LiteRT graphs. Filenames are deliberately **flat** — Box's downloader
+flattens remote paths into a single directory.
+
+### Video pipeline (picture → video)
+| file | size | shape |
+|---|---|---|
+| `dt5_text_encoder.tflite` | 495 MB | `(1,128)` ids + mask → `(1,128,4096)` |
+| `context_adapter_dynamic_int8.tflite` | 124 MB | `(1,128,4096)` → `(1,128,1536)` |
+| `clip_l_text_encoder.tflite` | 472 MB | `(1,77)` → pooled 768 |
+| `clip_g_text_encoder_int8.tflite` | 670 MB | `(1,77)` → pooled 1280 |
+| `dit_stage0_10x16_int4.tflite` | 816 MB | pyramid stage 0 |
+| `dit_stage1_20x32_int4.tflite` | 817 MB | pyramid stage 1 |
+| `dit_stage2_40x64_int4.tflite` | 822 MB | pyramid stage 2 |
+| `causal_video_vae_encoder_320x512.tflite` | 419 MB | `(1,3,1,320,512)` → `(1,16,1,40,64)` |
+| `causal_video_vae_decoder.tflite` | 38 MB | `(1,16,2,40,64)` → `(1,3,9,320,512)` |
+
+The three DiT graphs are the same 1.5B weights at three input resolutions; the
+pipeline calls one per pyramid stage. Each accepts **exactly two** conditioning
+latents, which is what limits this set to 9-frame clips.
+
+### First-frame pipeline (text → picture, SSD1B at 640×1024)
+| file | size | shape |
+|---|---|---|
+| `ssd_1b_clip_l.tflite` | 443 MB | `(1,77)` → `(1,77,768)` `hidden_states[-2]` |
+| `ssd_1b_clip_g.tflite` | 2.65 GB | `(1,77)` → `(1,1280)` + `(1,77,1280)` |
+| `ssd_1b_unet_640x1024.tflite` | 5.07 GB | `(1,4,80,128)` + t + ehs + text_embeds + time_ids |
+| `ssd_1b_vae_decoder_640x1024.tflite` | 189 MB | `(1,4,80,128)` → `(1,3,640,1024)` |
+
+### Tokenizer assets
+| file | note |
+|---|---|
+| `clip_vocab.json`, `clip_merges.txt` | CLIP BPE, shared by all four CLIP encoders |
+| `t5_spiece.model` | DT5 unigram SentencePiece |
+
+`clip_merges.txt` retains its `#version: 0.2` header line — **skip it when
+loading**, or every merge rank shifts by one and tokenization silently corrupts.
+
+## Notes for anyone reusing these
+
+- **CPU only.** The LiteRT GPU delegate supports at most rank-4 tensors, and
+  Neodragon's latents are 5D `(b,c,t,h,w)`. On a Tensor G5, `--gpu` places 256 of
+  3402 ops on the GPU and then segfaults (`RESHAPE: Tensor dimensions must be
+  less than 5`). This is structural, not a configuration problem.
+- **Two different CLIP pairs.** `clip_*` are Neodragon's own encoders (for the
+  DiT's pooled vector); `ssd_1b_clip_*` are SSD1B's (for SDXL conditioning).
+  Their weights differ. The tokenizer vocabularies are identical, so one
+  tokenizer serves both.
+- **SDXL uses `hidden_states[-2]`**, not `last_hidden_state`, from both CLIP
+  encoders. Using the last layer does not crash — it quietly degrades output.
+- **The SSD1B UNet is fp32** because int8 quantisation fails in
+  `ai_edge_quantizer` 0.8.0 on one layer
+  (`CrossAttnDownBlock2D_2/ResnetBlock2D_0/conv2`: *"input operand has more
+  dimensions than allowed by the axis remapping"*).
+- Measured on a Pixel 10 Pro Fold (Tensor G5), CPU: DiT stages ≈ 1.1 / 3.9 /
+  17.3 s per call.
+
+## Credit
+
+All credit for the model and research to **Qualcomm AI Research** — Animesh
+Karnewar and the Neodragon authors. These conversions exist only to make that
+work runnable on more devices.

causal_video_vae_decoder.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24717e1cb2d1b1ee249ac2397a6911169cfb35db7cc5ed41d5333ca8063c3cb2
+size 39720268

causal_video_vae_encoder_320x512.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78ea025166a77df00db1ecfb1ead93d0f5e4cb7f40d1aa87f9a78cb710c50dc4
+size 438938584

clip_g_text_encoder_int8.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:920b4c071f5e5dade89a6326b2fc613d38764458bd3870074f325cf779b19fff
+size 702166032

clip_l_text_encoder.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69cf4cd3f148a025cdad38258b7779e1b9139bab189ce1c72402ccca144eada2
+size 494994068

context_adapter_dynamic_int8.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f104b49625df78a8c6718aa61912e2b3d5161baf2714cce0b11a27f95eaf241
+size 130314976

dit_stage0_10x16_int4.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db7bb98dd08b3674c4635997f06027d76b9b074a262ffad990f9040cc9eaa60c
+size 855641104

dit_stage1_20x32_int4.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0cd4dc5ea02ff3ba0bc91634d5c74221b47e9a78e3565f60df18a68ef08ac45b
+size 856659680

dit_stage2_40x64_int4.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8a5937d904ac374f5dad6f98a45db8110392b7bbac1aff2f5771985137eb8d2
+size 861885920

dt5_text_encoder.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d46dd2e3f3dbe1892659551358f1bb4c7dac9930f4bfb79d6e56b2d6c5329b2b
+size 518898176

ssd_1b_clip_g.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42323338f0d10b4996819ea1def61ac10ee0eb098c8f0ce264aa9e7866af5cb0
+size 2779194992

ssd_1b_clip_l.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18a2403570beaae0068afd014cd93436ae690767a092617a63fc8b9e889c027d
+size 464132792

ssd_1b_unet_640x1024.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc8c6fe749a565c1b9c52feb494aef775c580b9094fa361401f1da2c8589d822
+size 5314769280

ssd_1b_vae_decoder_640x1024.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c45862baaff2e68223b7d3e37f330da5702afd0f49f2d89ffb8b0544772f03
+size 198303220

t5_spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d60acb128cf7b7f2536e8f38a5b18a05535c9e14c7a355904270e15b0945ea86
+size 791656