File size: 2,925 Bytes
059ac97
 
 
 
 
 
 
 
 
 
 
 
 
1ac2be1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
---
license: other
license_name: modilify-non-commercial-model-license-1.0
license_link: LICENSE
library_name: mlx
pipeline_tag: text-generation
tags:
  - diffusion
  - mixture-of-experts
  - custom-code
  - modilify-mk2
---

# Modilify Mk2 MLX

Modilify Mk2 MLX is a self-contained Apple Silicon inference release of Modilify Mk2 step 4400. The text-only DiffusionGemma 26B-A4B backbone revises a rolling 256-token canvas and commits a prefix at each denoising step when its confidence and entropy meet the commit policy. Working GDN2 memory follows canvas revisions. Persistent GDN2 memory updates when tokens are committed.

| Property | Value |
| --- | --- |
| Architecture | schema25 `compact_gdn2_v2` |
| Weights | 16 Safetensors shards, about 48.23 GiB |
| Precision | 1,297 BF16 tensors and 26 FP32 tensors |
| Runtime | MLX Metal on Apple Silicon |

## Run locally

Use Python 3.12 on Apple Silicon. Allow unified memory for the weights, prompt cache, rolling state and generation workspace.

```bash
cd ~/Modilify-Mk2-mlx
python -m pip install -r requirements.txt
python inference.py --prompt "Why is the sky blue?" --stream
```

The CLI loads the model and tokenizer from the script directory by default. Move the entire directory together to use it elsewhere. `--model /path/to/Modilify-Mk2-mlx` selects a local copy of this release. The loader verifies the manifest and all 16 shard checksums before generation. After dependency installation, loading and inference work offline.

`--no-thinking` disables the thinking prompt. `--seed` defaults to 42. `--max-new-tokens` and `--max-denoising-steps` both default to 256. Reaching either limit can leave a response unfinished. Streaming emits text as tokens are committed.

The runtime computes router probabilities and expert aggregation in FP32, then returns the aggregate in BF16 before the next backbone operation. Seeds reproduce results within this runtime version. The precision correction can change generated tokens relative to the earlier MLX release.

## Python

```python
from modilify_mk2_mlx import load

model = load()
result = model.generate(
    [{"role": "user", "content": "Explain how rainbows form."}],
    enable_thinking=False,
    max_new_tokens=256,
    max_denoising_steps=256,
    seed=42,
)
print(result.text)
print(result.stop_reason, result.denoise_steps)
```

Pass system, user, and assistant messages in conversation order. `result.token_ids` contains the committed response tokens. To stream, pass `on_text=callback`. The callback receives text increments that concatenate to `result.text`. A seed makes requests reproducible in the same MLX environment. The MLX and PyTorch releases use separate random number streams.

## License

This model is available for non-commercial research and experimentation under the [Modilify Non-Commercial Model License 1.0](LICENSE). See [NOTICE.md](NOTICE.md) for attribution and upstream terms.