---
license: cc-by-4.0
base_model: kyutai/mimi
tags:
- litert
- on-device
- neural-codec
- audio
library_name: litert
---

# Mimi (Kyutai 2024) — LiteRT on-device (hybrid GPU/CPU)

On-device [LiteRT](https://ai.google.dev/edge/litert) conversion of [**kyutai/mimi**](https://huggingface.co/kyutai/mimi),
the Kyutai/Moshi streaming neural audio codec (24 kHz, 12.5 Hz frame rate). The heavy SEANet
convolutional halves run on the **CompiledModel GPU** (ML Drift / `LITERT_CL`); the two 8-layer
Transformers and the split RVQ run on **CPU**. Device-verified on a Pixel 8a (Tensor G3): full
round-trip at **RTF ≈ 0.35** (faster than real-time), reconstruction at the codec's quality floor.

## Files

| File | Size | Placement | Input → Output |
|------|------|-----------|----------------|
| `mimi_enc_conv_fp16.tflite` | 24 MB | GPU | audio `[1,1,L]` → feat `[1,512,Se]` |
| `mimi_enc_tx_fp16.tflite` | 50 MB | CPU | feat `[1,Se,512]` → emb `[1,512,Tc]` |
| `mimi_dec_tx_fp16.tflite` | 48 MB | CPU | emb `[1,512,Tc]` → conv_in `[1,512,seq]` |
| `mimi_deconly_fp16.tflite` | 28 MB | GPU | conv_in `[1,512,seq]` → audio `[1,1,L]` |
| `mimi_rvq.bin` | 69 MB | CPU | codes ↔ emb (32 codebooks, float32 LE) |

Graphs are fixed-length (built per duration). The example set is for a 2 s clip (Se=50, Tc=25, seq=50).

## Pipeline

```
audio →[GPU enc_conv]→ feat →[CPU enc_tx]→ emb →[CPU RVQ.encode]→ codes
      →[CPU RVQ.decode]→ emb →[CPU dec_tx]→ conv_in →[GPU deconly]→ audio
```

## Why hybrid

Every op in all four graphs is GPU-clean (re-authored), and the convs are **fp16-exact on Mali**
(decoder-only fed the exact transformer output = 48 dB SNR). But the decoder transformer's residual
stream reaches **|x|=27**, where the Mali GPU delegate's internal fp16 compute loses precision —
full-GPU decode drops to ~12 dB on real speech. The transformer behaves **identically standalone and
fused** on device, so this is fp16 *precision*, not a fusion collapse. The transformers are tiny
(8 layers × 512, seq ~50), so CPU is trivial and exact; the heavy SEANet convs stay on GPU. The split
RVQ (1 semantic + 31 acoustic, Euclidean argmin + int64 indices) runs on CPU.

## Re-authoring (litert-torch, parity ~1.0)

tanh-GELU · baked RoPE cos/sin + rotate_half · baked causal additive bias · `MimiLayerScale`→Linear ·
grouped `ZeroStuffConvT1d` (depthwise upsample, no `TRANSPOSE_CONV`) · baked constant conv pad ·
`nn.ELU`→`relu(x)−relu(1−exp(min(x,0)))` · replicate-pad→SLICE+CONCAT.

## Usage

Sample app + conversion scripts: [**LiteRT-Models / mimi**](https://github.com/john-rocky/LiteRT-Models/tree/main/mimi).
Push the files to the app's `filesDir` with `mimi/scripts/install_to_device.sh`.

This conversion uses model-op rewrites + a GPU/CPU placement split, so it is hosted in a personal
namespace (not a patch-free "clean" sample). License follows upstream Mimi (CC-BY-4.0).