Update README.md

README.md

@@ -10,3 +10,92 @@ tags:
 - comfyui
 - diffusion-single-file
 ---
+For more information (including how to compress models yourself), check out https://huggingface.co/DFloat11 and https://github.com/LeanModels/DFloat11
+
+Feel free to request for other models for compression as well, although compressing models that do not use the Flux architecture might be slightly tricky for me.
+
+### How to Use
+
+#### ComfyUI
+Install my own fork of the DF11 ComfyUI custom node: https://github.com/mingyi456/ComfyUI-DFloat11-Extended. After installing the DF11 custom node, use the provided workflow [json](flux1-schnell-DF11-workflow.json), or simply replace the "Load Diffusion Model" node of an existing Flux workflow with the "DFloat11 Model Loader" node. If you run into any issues, feel free to leave a comment. The workflow is also embedded in the below [png](flux1-schnell-DF11-workflow.png) image.
+
+![](flux1-schnell-DF11-workflow.png)
+
+Alternatively, the official DF11 custom node can be used instead, but it will not work out of the box with this model, due to ComfyUI differentiating between Dev and Schnell versions of Flux. The code within the dfloat11_model_loader.py file needs to be patched, by explicitly adding an entry for `"FluxSchnell"` in the `MODEL_TO_PATTERN_DICT` dictionary. The patched code should be as follows:
+
+```python
+class DFloat11ModelLoader:
+    """
+    A custom node to load a DFloat11 diffusion model from the `diffusion_models` directory.
+    DFloat11 models are >30% smaller than their float16 counterparts, yet produce bit-for-bit identical outputs.
+    """
+    MODEL_TO_PATTERN_DICT = {
+        "Flux": {
+            "double_blocks\.\d+": (
+                "img_mod.lin",
+                "img_attn.qkv",
+                "img_attn.proj",
+                "img_mlp.0",
+                "img_mlp.2",
+                "txt_mod.lin",
+                "txt_attn.qkv",
+                "txt_attn.proj",
+                "txt_mlp.0",
+                "txt_mlp.2",
+            ),
+            "single_blocks\.\d+": (
+                "linear1",
+                "linear2",
+                "modulation.lin",
+            ),
+        },
+        "FluxSchnell": {
+            "double_blocks\.\d+": (
+                "img_mod.lin",
+                "img_attn.qkv",
+                "img_attn.proj",
+                "img_mlp.0",
+                "img_mlp.2",
+                "txt_mod.lin",
+                "txt_attn.qkv",
+                "txt_attn.proj",
+                "txt_mlp.0",
+                "txt_mlp.2",
+            ),
+            "single_blocks\.\d+": (
+                "linear1",
+                "linear2",
+                "modulation.lin",
+            ),
+        },
+    }
+```
+
+#### `diffusers`
+Refer to this [model](https://huggingface.co/DFloat11/FLUX.1-schnell-DF11) instead.
+
+### Compression Details
+
+This is the `pattern_dict` for compressing Flux-based models in ComfyUI:
+
+```python
+pattern_dict_comfyui = {
+    "double_blocks\.\d+": (
+        "img_mod.lin",
+        "img_attn.qkv",
+        "img_attn.proj",
+        "img_mlp.0",
+        "img_mlp.2",
+        "txt_mod.lin",
+        "txt_attn.qkv",
+        "txt_attn.proj",
+        "txt_mlp.0",
+        "txt_mlp.2",
+    ),
+    "single_blocks\.\d+": (
+        "linear1",
+        "linear2",
+        "modulation.lin",
+    ),
+}
+```