point examples at renamed repos

805b0fd verified 18 days ago

4.82 kB

license: apache-2.0
tags:
  - mlx
  - mlx-creator
  - apple-silicon
  - text-to-image
  - text-to-audio
  - text-to-video

MLX Creator — Models

Local generative-media studio for Apple Silicon — images, music, and video, fully on-device via MLX. No PyTorch at runtime, no cloud.

App / source: https://github.com/rynky2436/MLX-Creator
These repos (MLXCreator/*) host MLX-format weights the app downloads on first run.

How to add or convert a model so it works in the app

The app loads models from its local models/ folder. Each model lives in its own subfolder with a small mlxstudio.json manifest that tells the app what it is and which engine loads it. Drop a compatible folder in, and it appears in the right tab.

1. Is it already MLX format?

The app's inference is torch-free — it loads pre-converted MLX weights. The easy path:

Yes (drop-in): weights already converted to MLX (e.g. from mlx-community, or any MLXCreator/* repo here). Just add the manifest (step 3) — no conversion needed.
No (convert first): the model only ships PyTorch/diffusers safetensors. Convert it to MLX once, offline, then drop in the result. Conversion may use torch on another machine/run; the app itself never needs it. Use the upstream MLX tool for that family:
- Flux: mlx-examples/flux
- SD3 / SD3.5: DiffusionKit
- Qwen-Image: mflux
- Language/planner models: mlx_lm.convert

2. Does the app already have an engine for that architecture?

A manifest only routes a model to an engine that already exists. So a model works if it is (a) MLX format and (b) one of the supported families below:

Modality	Engine (`engine`)	Architectures it loads	Drop-in source
`image`	`flux`	FLUX.1 schnell / dev	`mlx-community` Flux, `MLXCreator/MLXCreator-Flux-Schnell`
`image`	`sd35`	SD3 / SD3.5 (needs `SD3-encoders` companion)	`MLXCreator/MLXCreator-SD3.5-Large`
`image`	`qwen`	Qwen-Image (quantized MLX)	`MLXCreator/MLXCreator-QwenImage-4bit / -8bit`
`audio`	`ace_step`	ACE-Step 1.5 (needs an `acestep-*-lm` planner companion)	`MLXCreator/MLXCreator-ACEStep-1.5`
`video`	`wan`	Wan 2.2 TI2V (needs `umt5-xxl-tokenizer` companion)	`MLXCreator/MLXCreator-Wan2.2-TI2V-5B`

A brand-new architecture (not in this list) needs a small new engine in the app — see CONTRIBUTING in the repo. A new variant of a supported family just needs MLX weights + a manifest.

3. Write the manifest

Put mlxstudio.json in the model's folder:

{
  "modality": "image",
  "engine": "flux",
  "arch": "schnell",
  "role": "model",
  "display": "FLUX.1 schnell"
}

Field	Required	Values
`modality`	yes	`image` · `audio` · `video`
`engine`	yes	`flux` · `sd35` · `qwen` · `ace_step` · `wan`
`arch`	flux only	`schnell` · `dev`
`role`	yes	`model` (selectable) · `companion` (shared encoder/tokenizer/planner, hidden)
`display`	optional	label shown in the UI (defaults to the folder name)

The app also auto-detects common layouts and writes this file for you, but adding it explicitly is the reliable way to register an unusual folder.

4. Install it — two ways

In-app (recommended): open the Models tab, search Hugging Face, and install. The browser filters for MLX-compatible repos and writes the manifest automatically.
Manual: copy the folder into the app's models/ directory (see the path in the Settings tab). It shows up on next launch / refresh.

5. Publishing your own MLX model so the in-app browser finds it

When you upload converted weights to Hugging Face, tag them so MLX Creator's browser surfaces them:

library_name: mlx (or include mlx in tags)
a pipeline_tag that matches the modality (text-to-image, text-to-audio, text-to-video)
keep the upstream license and attribution in the model card

Companions

Some engines share a heavy encoder/tokenizer/planner across models. These are marked "role": "companion" and are not shown as selectable models — they're loaded behind the scenes. Examples here: SD3-encoders (CLIP/T5 for sd35), umt5-xxl-tokenizer (for wan), acestep-5Hz-lm-0.6B / -4B (the ACE-Step "thinking" planners).

Each model repo here keeps its original source + license in its card. Apache-2.0 for the Flux / ACE-Step / Wan / Qwen / umt5 weights; the SD3 / SD3.5 weights are under the Stability AI Community License.