Add Hardware envelope section: memory_mode=relay opens 8-16 GB Apple Silicon Macs (PR #6)
Browse files
README.md
CHANGED
|
@@ -42,6 +42,19 @@ For video tasks (t2v, video_edit, x2t_video), see [`mlx-community/Lance-3B-Video
|
|
| 42 |
|
| 43 |
The 48-channel Wan2.2 VAE is bundled here for convenience but also published standalone at [`mlx-community/Wan2.2-VAE-Lance-bf16`](https://huggingface.co/mlx-community/Wan2.2-VAE-Lance-bf16) — both image_edit and the video pipelines need it.
|
| 44 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
## Quickstart
|
| 46 |
|
| 47 |
Install from the source repo (will be on PyPI in a follow-up release):
|
|
|
|
| 42 |
|
| 43 |
The 48-channel Wan2.2 VAE is bundled here for convenience but also published standalone at [`mlx-community/Wan2.2-VAE-Lance-bf16`](https://huggingface.co/mlx-community/Wan2.2-VAE-Lance-bf16) — both image_edit and the video pipelines need it.
|
| 44 |
|
| 45 |
+
### Hardware envelope (`memory_mode`, 2026-06-02)
|
| 46 |
+
|
| 47 |
+
The `lance-mlx` source repo's [`memory_mode`](https://github.com/xocialize/lance-mlx) knob (auto / parallel / relay) brings bf16 image generation within reach of 8–16 GB Apple Silicon Macs:
|
| 48 |
+
|
| 49 |
+
| RAM | Mode | t2i / image_edit | Notes |
|
| 50 |
+
|---|---|---|---|
|
| 51 |
+
| **8–16 GB** | `relay` (auto-resolved) | ✅ 768² bf16, no swap | Single-shot per pipeline load — re-prefill reloads the UND tower. ~5 min/image at 768² on a base-RAM M-series Mac. |
|
| 52 |
+
| **24 GB+** | `parallel` (auto-resolved) | ✅ 768² bf16, reusable | Pipeline stays resident across calls. |
|
| 53 |
+
|
| 54 |
+
`relay` produces **byte-identical output** to `parallel` (MD5-verified on real Lance-3B-bf16) — it sheds the UND tower after prefill and frees the GEN tower before VAE decode, so peak memory ≈ heaviest single phase rather than the sum of all three. Default `auto` resolves by `mx.device_info()`'s recommended working-set size with the split at ~18 GiB. The same envelope and modes apply to [`mlx-community/Lance-3B-Video-bf16`](https://huggingface.co/mlx-community/Lance-3B-Video-bf16) for video tasks.
|
| 55 |
+
|
| 56 |
+
For VQA-only use cases on small Macs, the compressed [`mlx-community/Lance-3B-AWQ-INT4`](https://huggingface.co/mlx-community/Lance-3B-AWQ-INT4) variant is also available (3.3 GB LLM, 6-9× faster long-form decode; VQA only, not for image generation).
|
| 57 |
+
|
| 58 |
## Quickstart
|
| 59 |
|
| 60 |
Install from the source repo (will be on PyPI in a follow-up release):
|