SceneWorks commited on
Commit
8e453f3
·
verified ·
1 Parent(s): a3b6635

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +62 -0
README.md ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - en
5
+ pipeline_tag: text-to-image
6
+ library_name: mlx-gen
7
+ base_model: microsoft/Lens
8
+ tags:
9
+ - mlx
10
+ - apple-silicon
11
+ - lens
12
+ - text-to-image
13
+ - quantized
14
+ ---
15
+
16
+ # Lens (base) — MLX pre-quantized tiers (SceneWorks)
17
+
18
+ Native-MLX, **pre-quantized** re-host of the **base** Lens model (`microsoft/Lens`, MIT)
19
+ for on-device Apple-Silicon inference via [`mlx-gen`](https://github.com/SceneWorks/mlx-gen)'s
20
+ `mlx-gen-lens` provider (SceneWorks). The heavy components are packed offline so a tier loads
21
+ directly with **no dense transient and no in-app quantization** (epic 8506, sc-8767).
22
+
23
+ Microsoft removed `microsoft/Lens` from the Hub; the base DiT here was recovered from the public
24
+ ungated re-package [`Comfy-Org/Lens`](https://huggingface.co/Comfy-Org/Lens)
25
+ (`diffusion_models/lens_bf16.safetensors`), whose keys are byte-identical to the diffusers
26
+ `LensTransformer2DModel` state dict. Base Lens and Lens-Turbo differ **only** in the DiT weights;
27
+ this re-host reuses the **shared** gpt-oss-20b text encoder + Flux.2 VAE + tokenizer + scheduler
28
+ from [`SceneWorks/lens-turbo-mlx`](https://huggingface.co/SceneWorks/lens-turbo-mlx).
29
+
30
+ Base Lens is **undistilled** — use a higher step count (~20–26) with CFG ~5.0 (the `mlx-gen-lens`
31
+ `lens` id defaults to 20 steps / CFG 5.0), unlike the distilled Turbo (4 steps / guidance 1.0).
32
+
33
+ ## Tiers
34
+
35
+ Each subdirectory is a full, self-contained turnkey snapshot (the diffusers multi-component tree —
36
+ `transformer/`, `text_encoder/`, `vae/`, `tokenizer/`, `scheduler/`, `model_index.json`):
37
+
38
+ | Tier | Dir | What is packed |
39
+ |------|-----|----------------|
40
+ | **Q4** (default) | `q4/` | DiT + gpt-oss encoder MoE experts → MLX group-64 affine 4-bit |
41
+ | **Q8** | `q8/` | DiT + gpt-oss encoder MoE experts → MLX group-64 affine 8-bit |
42
+ | **bf16** | `bf16/` | dense mirror of the source (no quantization) |
43
+
44
+ Two components are quantized (matching the load-time `.quantize` scope):
45
+
46
+ - **DiT** — `img_in`/`txt_in`/`proj_out` + every block's fused-QKV attention projections
47
+ (`img_qkv`/`txt_qkv`/`to_out.0`/`to_add_out`) and SwiGLU MLPs. The timestep embedder, AdaLN
48
+ modulations, and all norms stay full precision.
49
+ - **gpt-oss-20b encoder MoE experts** — the source ships these as MXFP4; the packed tiers store them
50
+ as MLX group-64 affine Q4/Q8 (stacked `experts.{gate_up,down}_proj.{weight,scales,biases}`). The
51
+ router / attention / embeddings / norms stay dense.
52
+
53
+ The **VAE** (the shared Flux.2 decoder) always runs f32 and is shipped dense in every tier.
54
+
55
+ The pack is **byte-identical** to what the load-time quantizer produces (bf16 cast, group 64), verified
56
+ in-repo (`mlx-gen-lens` `convert`/`quant` byte-identity tests) and by an on-device render gate.
57
+
58
+ ## License
59
+
60
+ MIT, inherited from `microsoft/Lens`. The shared text encoder is `openai/gpt-oss-20b` (Apache-2.0)
61
+ and the VAE is `black-forest-labs/FLUX.2-dev` (Apache-2.0). This is a format re-host; all model
62
+ weights and credit belong to the original authors (Microsoft Research; OpenAI; Black Forest Labs).