README.md

---
license: apache-2.0
tags:
- mlx
- mlx-creator
- apple-silicon
- text-to-image
- text-to-audio
- text-to-video
---

# MLX Creator — Models

Local generative-media studio for **Apple Silicon** — images, music, and video, fully
on-device via [MLX](https://github.com/ml-explore/mlx). No PyTorch at runtime, no cloud.

- **App / source:** https://github.com/rynky2436/MLX-Creator
- **These repos** (`MLXCreator/*`) host MLX-format weights the app downloads on first run.

---

## How to add or convert a model so it works in the app

The app loads models from its local `models/` folder. Each model lives in **its own
subfolder** with a small `mlxstudio.json` manifest that tells the app what it is and
which engine loads it. Drop a compatible folder in, and it appears in the right tab.

### 1. Is it already MLX format?

The app's **inference is torch-free** — it loads pre-converted MLX weights. The easy path:

- **Yes (drop-in):** weights already converted to MLX (e.g. from `mlx-community`, or any
  `MLXCreator/*` repo here). Just add the manifest (step 3) — no conversion needed.
- **No (convert first):** the model only ships PyTorch/diffusers `safetensors`. Convert it
  to MLX **once, offline**, then drop in the result. Conversion may use torch on another
  machine/run; the app itself never needs it. Use the upstream MLX tool for that family:
  - Flux: [`mlx-examples/flux`](https://github.com/ml-explore/mlx-examples/tree/main/flux)
  - SD3 / SD3.5: [`DiffusionKit`](https://github.com/argmaxinc/DiffusionKit)
  - Qwen-Image: [`mflux`](https://github.com/filipstrand/mflux)
  - Language/planner models: `mlx_lm.convert`

### 2. Does the app already have an engine for that architecture?

A manifest only **routes** a model to an engine that already exists. So a model works if it
is **(a) MLX format** and **(b) one of the supported families** below:

| Modality | Engine (`engine`) | Architectures it loads | Drop-in source |
|---|---|---|---|
| `image` | `flux`  | FLUX.1 schnell / dev | `mlx-community` Flux, `MLXCreator/MLXCreator-Flux-Schnell` |
| `image` | `sd35`  | SD3 / SD3.5 (needs `SD3-encoders` companion) | `MLXCreator/MLXCreator-SD3.5-Large` |
| `image` | `qwen`  | Qwen-Image (quantized MLX) | `MLXCreator/MLXCreator-QwenImage-4bit / -8bit` |
| `audio` | `ace_step` | ACE-Step 1.5 (needs an `acestep-*-lm` planner companion) | `MLXCreator/MLXCreator-ACEStep-1.5` |
| `video` | `wan`   | Wan 2.2 TI2V (needs `umt5-xxl-tokenizer` companion) | `MLXCreator/MLXCreator-Wan2.2-TI2V-5B` |

A **brand-new architecture** (not in this list) needs a small new engine in the app — see
[CONTRIBUTING in the repo](https://github.com/rynky2436/MLX-Creator). A new *variant* of a
supported family just needs MLX weights + a manifest.

### 3. Write the manifest

Put `mlxstudio.json` in the model's folder:

```json
{
  "modality": "image",
  "engine": "flux",
  "arch": "schnell",
  "role": "model",
  "display": "FLUX.1 schnell"
}
```

| Field | Required | Values |
|---|---|---|
| `modality` | yes | `image` · `audio` · `video` |
| `engine`   | yes | `flux` · `sd35` · `qwen` · `ace_step` · `wan` |
| `arch`     | flux only | `schnell` · `dev` |
| `role`     | yes | `model` (selectable) · `companion` (shared encoder/tokenizer/planner, hidden) |
| `display`  | optional | label shown in the UI (defaults to the folder name) |

The app also **auto-detects** common layouts and writes this file for you, but adding it
explicitly is the reliable way to register an unusual folder.

### 4. Install it — two ways

- **In-app (recommended):** open the **Models** tab, search Hugging Face, and install. The
  browser filters for MLX-compatible repos and writes the manifest automatically.
- **Manual:** copy the folder into the app's `models/` directory (see the path in the
  **Settings** tab). It shows up on next launch / refresh.

### 5. Publishing your own MLX model so the in-app browser finds it

When you upload converted weights to Hugging Face, tag them so MLX Creator's browser
surfaces them:

- `library_name: mlx` (or include `mlx` in `tags`)
- a `pipeline_tag` that matches the modality (`text-to-image`, `text-to-audio`, `text-to-video`)
- keep the upstream **license** and attribution in the model card

---

## Companions

Some engines share a heavy encoder/tokenizer/planner across models. These are marked
`"role": "companion"` and are **not** shown as selectable models — they're loaded behind the
scenes. Examples here: `SD3-encoders` (CLIP/T5 for `sd35`), `umt5-xxl-tokenizer` (for `wan`),
`acestep-5Hz-lm-0.6B` / `-4B` (the ACE-Step "thinking" planners).

---

*Each model repo here keeps its original source + license in its card. Apache-2.0 for the
Flux / ACE-Step / Wan / Qwen / umt5 weights; the SD3 / SD3.5 weights are under the Stability
AI Community License.*