Instructions to use SceneWorks/lens-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use SceneWorks/lens-mlx with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir lens-mlx SceneWorks/lens-mlx
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,62 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
pipeline_tag: text-to-image
|
| 6 |
+
library_name: mlx-gen
|
| 7 |
+
base_model: microsoft/Lens
|
| 8 |
+
tags:
|
| 9 |
+
- mlx
|
| 10 |
+
- apple-silicon
|
| 11 |
+
- lens
|
| 12 |
+
- text-to-image
|
| 13 |
+
- quantized
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# Lens (base) — MLX pre-quantized tiers (SceneWorks)
|
| 17 |
+
|
| 18 |
+
Native-MLX, **pre-quantized** re-host of the **base** Lens model (`microsoft/Lens`, MIT)
|
| 19 |
+
for on-device Apple-Silicon inference via [`mlx-gen`](https://github.com/SceneWorks/mlx-gen)'s
|
| 20 |
+
`mlx-gen-lens` provider (SceneWorks). The heavy components are packed offline so a tier loads
|
| 21 |
+
directly with **no dense transient and no in-app quantization** (epic 8506, sc-8767).
|
| 22 |
+
|
| 23 |
+
Microsoft removed `microsoft/Lens` from the Hub; the base DiT here was recovered from the public
|
| 24 |
+
ungated re-package [`Comfy-Org/Lens`](https://huggingface.co/Comfy-Org/Lens)
|
| 25 |
+
(`diffusion_models/lens_bf16.safetensors`), whose keys are byte-identical to the diffusers
|
| 26 |
+
`LensTransformer2DModel` state dict. Base Lens and Lens-Turbo differ **only** in the DiT weights;
|
| 27 |
+
this re-host reuses the **shared** gpt-oss-20b text encoder + Flux.2 VAE + tokenizer + scheduler
|
| 28 |
+
from [`SceneWorks/lens-turbo-mlx`](https://huggingface.co/SceneWorks/lens-turbo-mlx).
|
| 29 |
+
|
| 30 |
+
Base Lens is **undistilled** — use a higher step count (~20–26) with CFG ~5.0 (the `mlx-gen-lens`
|
| 31 |
+
`lens` id defaults to 20 steps / CFG 5.0), unlike the distilled Turbo (4 steps / guidance 1.0).
|
| 32 |
+
|
| 33 |
+
## Tiers
|
| 34 |
+
|
| 35 |
+
Each subdirectory is a full, self-contained turnkey snapshot (the diffusers multi-component tree —
|
| 36 |
+
`transformer/`, `text_encoder/`, `vae/`, `tokenizer/`, `scheduler/`, `model_index.json`):
|
| 37 |
+
|
| 38 |
+
| Tier | Dir | What is packed |
|
| 39 |
+
|------|-----|----------------|
|
| 40 |
+
| **Q4** (default) | `q4/` | DiT + gpt-oss encoder MoE experts → MLX group-64 affine 4-bit |
|
| 41 |
+
| **Q8** | `q8/` | DiT + gpt-oss encoder MoE experts → MLX group-64 affine 8-bit |
|
| 42 |
+
| **bf16** | `bf16/` | dense mirror of the source (no quantization) |
|
| 43 |
+
|
| 44 |
+
Two components are quantized (matching the load-time `.quantize` scope):
|
| 45 |
+
|
| 46 |
+
- **DiT** — `img_in`/`txt_in`/`proj_out` + every block's fused-QKV attention projections
|
| 47 |
+
(`img_qkv`/`txt_qkv`/`to_out.0`/`to_add_out`) and SwiGLU MLPs. The timestep embedder, AdaLN
|
| 48 |
+
modulations, and all norms stay full precision.
|
| 49 |
+
- **gpt-oss-20b encoder MoE experts** — the source ships these as MXFP4; the packed tiers store them
|
| 50 |
+
as MLX group-64 affine Q4/Q8 (stacked `experts.{gate_up,down}_proj.{weight,scales,biases}`). The
|
| 51 |
+
router / attention / embeddings / norms stay dense.
|
| 52 |
+
|
| 53 |
+
The **VAE** (the shared Flux.2 decoder) always runs f32 and is shipped dense in every tier.
|
| 54 |
+
|
| 55 |
+
The pack is **byte-identical** to what the load-time quantizer produces (bf16 cast, group 64), verified
|
| 56 |
+
in-repo (`mlx-gen-lens` `convert`/`quant` byte-identity tests) and by an on-device render gate.
|
| 57 |
+
|
| 58 |
+
## License
|
| 59 |
+
|
| 60 |
+
MIT, inherited from `microsoft/Lens`. The shared text encoder is `openai/gpt-oss-20b` (Apache-2.0)
|
| 61 |
+
and the VAE is `black-forest-labs/FLUX.2-dev` (Apache-2.0). This is a format re-host; all model
|
| 62 |
+
weights and credit belong to the original authors (Microsoft Research; OpenAI; Black Forest Labs).
|