Ill-Ness commited on
Commit
cc5f2f0
·
verified ·
1 Parent(s): 4c6ca2a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +146 -1
README.md CHANGED
@@ -1,3 +1,148 @@
1
  ---
2
- license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: other
3
+ license_name: rift-non-commercial-license-v1.0
4
+ license_link: https://huggingface.co/Rift-ai/Rift.1-decoder/blob/main/LICENSE.md
5
+ language:
6
+ - en
7
+ pipeline_tag: image-to-image
8
+ library_name: diffusers
9
+ tags:
10
+ - text-to-image
11
+ - image-editing
12
+ - flux
13
+ - diffusion-single-file
14
+ - vae
15
+ - decoder
16
+ - rift
17
+ - rift-ai
18
  ---
19
+
20
+ ![Comparison Panel](./comparison_panel.jpeg)
21
+
22
+ `Rift.1-decoder` is a FLUX-compatible VAE decoder made for the Rift model line. It is designed as a **drop-in decoder component** for compatible FLUX.2-style Diffusers pipelines that use `AutoencoderKLFlux2`. The encoder path remains compatible with FLUX-style latent encoding, while the decoder has been trained as the Rift image reconstruction component.
23
+
24
+ The exported Diffusers runtime class remains `AutoencoderKLFlux2` for loader compatibility. The model metadata identifies the architecture as `Rift1Decoder` with model type `rift1_decoder`.
25
+
26
+ # **Key Features**
27
+
28
+ 1. **FLUX-compatible decoder interface** using `AutoencoderKLFlux2`.
29
+ 2. **Rift1Decoder metadata** in `config.json` for clear model identity.
30
+ 3. **32 latent channels** for compatibility with FLUX.2-style latent spaces.
31
+ 4. **512px reconstruction training** with edge and frequency losses for sharper detail retention.
32
+ 5. **Single-file artifacts included** for decoder-focused workflows:
33
+ - `diffusion_pytorch_model.safetensors`
34
+ - `full_encoder_small_decoder.safetensors`
35
+ - `small_decoder.safetensors`
36
+ 6. Released under the **Rift Non-Commercial License v1.0**.
37
+
38
+ Compatible target pipeline family:
39
+
40
+ - FLUX.2-style Diffusers pipelines using `AutoencoderKLFlux2`
41
+ - [FLUX.2-klein-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B)
42
+ - [FLUX.2-klein-9B](https://huggingface.co/black-forest-labs/FLUX.2-klein-9B)
43
+ - [FLUX.2-klein-9b-kv](https://huggingface.co/black-forest-labs/FLUX.2-klein-9b-kv)
44
+ - [FLUX.2-dev](https://huggingface.co/black-forest-labs/FLUX.2-dev)
45
+
46
+ # **Comparison**
47
+
48
+ | Reference Decoder | Rift1Decoder |
49
+ |:---:|:---:|
50
+ | ![Reference Decoder](./compare_full_decoder.png) | ![Rift1Decoder](./compare_small_decoder.png) |
51
+
52
+ # **Detail View**
53
+
54
+ ![Detail Zoom](./detail_zoom.jpeg)
55
+
56
+ # **Usage**
57
+
58
+ ```shell
59
+ pip install git+https://github.com/huggingface/diffusers.git transformers accelerate torch
60
+ ```
61
+
62
+ ```python
63
+ import torch
64
+ from diffusers import AutoencoderKLFlux2
65
+
66
+ vae = AutoencoderKLFlux2.from_pretrained(
67
+ "Rift-ai/Rift.1-decoder",
68
+ torch_dtype=torch.bfloat16,
69
+ )
70
+ ```
71
+
72
+ If using a compatible FLUX.2 pipeline, pass this VAE when loading the pipeline:
73
+
74
+ ```python
75
+ import torch
76
+ from diffusers import Flux2KleinPipeline, AutoencoderKLFlux2
77
+
78
+ device = "cuda"
79
+ dtype = torch.bfloat16
80
+
81
+ vae = AutoencoderKLFlux2.from_pretrained("Rift-ai/Rift.1-decoder", torch_dtype=dtype)
82
+ pipe = Flux2KleinPipeline.from_pretrained(
83
+ "black-forest-labs/FLUX.2-klein-4B",
84
+ vae=vae,
85
+ torch_dtype=dtype,
86
+ )
87
+ pipe.enable_model_cpu_offload()
88
+
89
+ prompt = "A black cat holding a sign that says 'hello world' in typewriter font"
90
+ image = pipe(
91
+ prompt=prompt,
92
+ height=1024,
93
+ width=1024,
94
+ guidance_scale=1.0,
95
+ num_inference_steps=4,
96
+ generator=torch.Generator(device=device).manual_seed(0),
97
+ ).images[0]
98
+ image.save("rift-decoder-output.png")
99
+ ```
100
+
101
+ ---
102
+
103
+ # **Artifact Files**
104
+
105
+ | File | Purpose |
106
+ |:---|:---|
107
+ | `config.json` | Diffusers config with Rift metadata |
108
+ | `diffusion_pytorch_model.safetensors` | Standard Diffusers weights |
109
+ | `full_encoder_small_decoder.safetensors` | Full autoencoder-format weights |
110
+ | `small_decoder.safetensors` | Decoder-only and post-quant-conv weights |
111
+ | `comparison_panel.jpeg` | Full reference/Rift comparison |
112
+ | `compare_full_decoder.png` | Reference decoder reconstruction sample |
113
+ | `compare_small_decoder.png` | Rift decoder reconstruction sample |
114
+ | `detail_zoom.jpeg` | Zoomed detail comparison |
115
+ | `editing.jpg` | Additional visual sample |
116
+
117
+ ---
118
+
119
+ # **Limitations**
120
+
121
+ - This repository contains a VAE decoder component, not a complete text-to-image model.
122
+ - Visual quality depends on the surrounding diffusion model, scheduler, prompt, latent distribution, and inference settings.
123
+ - The decoder may introduce color shifts, texture smoothing, edge artifacts, or small structural artifacts.
124
+ - Text rendered in generated images may be inaccurate or distorted.
125
+ - Prompt following is handled primarily by the surrounding generation pipeline, not the VAE decoder alone.
126
+ - This model should be evaluated visually and quantitatively before production use.
127
+
128
+ # **Out-of-Scope Use**
129
+
130
+ This model and its derivatives may not be used outside the scope of the Rift Non-Commercial License v1.0, including for unlawful, fraudulent, defamatory, abusive, exploitative, privacy-invasive, or otherwise harmful purposes.
131
+
132
+ ---
133
+
134
+ # **Responsible AI Development**
135
+
136
+ Rift.1-decoder should be evaluated as part of a complete image generation or image reconstruction system. A decoder can affect visual fidelity and artifacts, but safety behavior also depends on the text encoder, diffusion transformer, prompt filters, data pipeline, deployment environment, and downstream product policy.
137
+
138
+ Users are responsible for applying appropriate safeguards, content review, watermarking or provenance notices where required, and compliance with applicable law.
139
+
140
+ ---
141
+
142
+ # **License**
143
+
144
+ This model is licensed under the [Rift Non-Commercial License v1.0](./LICENSE.md).
145
+
146
+ # **Trademarks & IP**
147
+
148
+ This project may contain trademarks or references to third-party projects, products, or services. Use of Rift, Rift-ai, or associated marks in modified versions of this project must not imply sponsorship, endorsement, approval, or official status unless explicitly authorized. Third-party trademarks, intellectual property, and logos remain subject to their respective owners' policies.