Update README.md
Browse files
README.md
CHANGED
|
@@ -24,20 +24,19 @@ datasets:
|
|
| 24 |
|
| 25 |
**TinyFlux-Lailah** is an expanded TinyFlux architecture with increased depth and width. Originally ported from [TinyFlux](https://huggingface.co/AbstractPhil/tiny-flux) with strategic layer expansion and attention head doubling, now training end-to-end on teacher latents.
|
| 26 |
|
| 27 |
-
> **Current checkpoint:** `step_286250` | **Status:** Active training
|
| 28 |
|
| 29 |
## Quick Start (Colab)
|
| 30 |
|
| 31 |
The easiest way to test Lailah:
|
| 32 |
|
| 33 |
1. Open [Google Colab](https://colab.research.google.com/)
|
| 34 |
-
2. Copy the contents of [`
|
| 35 |
3. Run the cells
|
| 36 |
|
| 37 |
```python
|
| 38 |
# Or fetch directly:
|
| 39 |
-
!wget https://huggingface.co/AbstractPhil/tiny-flux-deep/raw/main/
|
| 40 |
-
%run
|
| 41 |
```
|
| 42 |
|
| 43 |
## Fair Weights
|
|
@@ -93,9 +92,9 @@ Training on [AbstractPhil/flux-schnell-teacher-latents](https://huggingface.co/d
|
|
| 93 |
|
| 94 |
### Checkpoints
|
| 95 |
|
| 96 |
-
Checkpoints are saved every
|
| 97 |
- `checkpoints/step_XXXXX.safetensors` - Training weights
|
| 98 |
-
- `checkpoints/step_XXXXX_ema.safetensors` - EMA weights (
|
| 99 |
|
| 100 |
## Usage
|
| 101 |
|
|
@@ -116,10 +115,10 @@ from safetensors.torch import load_file
|
|
| 116 |
config = TinyFluxDeepConfig()
|
| 117 |
model = TinyFluxDeep(config).to("cuda", torch.bfloat16)
|
| 118 |
|
| 119 |
-
# Load EMA weights (
|
| 120 |
weights = load_file(hf_hub_download(
|
| 121 |
"AbstractPhil/tiny-flux-deep",
|
| 122 |
-
"checkpoints/step_286250_ema.safetensors" #
|
| 123 |
))
|
| 124 |
model.load_state_dict(weights, strict=False)
|
| 125 |
model.eval()
|
|
@@ -174,7 +173,7 @@ AbstractPhil/tiny-flux-deep/
|
|
| 174 |
βββ train_tinyflux_deep.py # Training script
|
| 175 |
βββ checkpoints/
|
| 176 |
β βββ step_286250.safetensors # Training weights
|
| 177 |
-
β βββ step_286250_ema.safetensors # EMA weights (
|
| 178 |
βββ samples/ # Generated samples during training
|
| 179 |
βββ README.md
|
| 180 |
```
|
|
@@ -240,4 +239,4 @@ MIT License
|
|
| 240 |
|
| 241 |
---
|
| 242 |
|
| 243 |
-
**Status**: Active training. Checkpoints updated regularly. Use
|
|
|
|
| 24 |
|
| 25 |
**TinyFlux-Lailah** is an expanded TinyFlux architecture with increased depth and width. Originally ported from [TinyFlux](https://huggingface.co/AbstractPhil/tiny-flux) with strategic layer expansion and attention head doubling, now training end-to-end on teacher latents.
|
| 26 |
|
|
|
|
| 27 |
|
| 28 |
## Quick Start (Colab)
|
| 29 |
|
| 30 |
The easiest way to test Lailah:
|
| 31 |
|
| 32 |
1. Open [Google Colab](https://colab.research.google.com/)
|
| 33 |
+
2. Copy the contents of [`inference_v3.py`](./inference_v3.py)
|
| 34 |
3. Run the cells
|
| 35 |
|
| 36 |
```python
|
| 37 |
# Or fetch directly:
|
| 38 |
+
!wget https://huggingface.co/AbstractPhil/tiny-flux-deep/raw/main/inference_v3.py
|
| 39 |
+
%run inference_v3.py
|
| 40 |
```
|
| 41 |
|
| 42 |
## Fair Weights
|
|
|
|
| 92 |
|
| 93 |
### Checkpoints
|
| 94 |
|
| 95 |
+
Checkpoints are saved every epoch or so with both main and EMA weights:
|
| 96 |
- `checkpoints/step_XXXXX.safetensors` - Training weights
|
| 97 |
+
- `checkpoints/step_XXXXX_ema.safetensors` - EMA weights (currently very broken and retraining, use standard step to inference)
|
| 98 |
|
| 99 |
## Usage
|
| 100 |
|
|
|
|
| 115 |
config = TinyFluxDeepConfig()
|
| 116 |
model = TinyFluxDeep(config).to("cuda", torch.bfloat16)
|
| 117 |
|
| 118 |
+
# Load EMA weights (broken) or main weights
|
| 119 |
weights = load_file(hf_hub_download(
|
| 120 |
"AbstractPhil/tiny-flux-deep",
|
| 121 |
+
"checkpoints/step_286250_ema.safetensors" # EMA will be better later, for now it's broken.
|
| 122 |
))
|
| 123 |
model.load_state_dict(weights, strict=False)
|
| 124 |
model.eval()
|
|
|
|
| 173 |
βββ train_tinyflux_deep.py # Training script
|
| 174 |
βββ checkpoints/
|
| 175 |
β βββ step_286250.safetensors # Training weights
|
| 176 |
+
β βββ step_286250_ema.safetensors # EMA weights (currently broken)
|
| 177 |
βββ samples/ # Generated samples during training
|
| 178 |
βββ README.md
|
| 179 |
```
|
|
|
|
| 239 |
|
| 240 |
---
|
| 241 |
|
| 242 |
+
**Status**: Active training. Checkpoints updated regularly. Use standard weights for best results.
|