nicolas-dufour commited on
Commit
3678ae1
Β·
verified Β·
1 Parent(s): ed75745

Add root model card

Browse files
Files changed (1) hide show
  1. README.md +97 -0
README.md ADDED
@@ -0,0 +1,97 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: miro-t2i
4
+ tags:
5
+ - text-to-image
6
+ - diffusion
7
+ - flow-matching
8
+ - miro
9
+ - reward-conditioning
10
+ - ablations
11
+ pipeline_tag: text-to-image
12
+ ---
13
+
14
+ # MIRO β€” ablations and single-reward specialists
15
+
16
+ This repository hosts the **15 ablation / baseline checkpoints** that accompany the main MIRO release at [`nicolas-dufour/miro`](https://huggingface.co/nicolas-dufour/miro).
17
+
18
+ > Dufour, Degeorge, Ghosh, Kalogeiton, Picard. _MIRO: MultI-Reward cOnditioned pretraining improves T2I quality and efficiency_. **ICML 2026**.
19
+ >
20
+ > πŸ“„ [Paper](https://arxiv.org/abs/2510.25897) Β· 🌐 [Project page](https://nicolas-dufour.github.io/miro/) Β· πŸ’» [Code](https://github.com/nicolas-dufour/miro) Β· 🐍 `pip install miro-t2i`
21
+
22
+ <table style="width:100%;border-collapse:separate;border-spacing:4px">
23
+ <tr><td><img src="https://huggingface.co/nicolas-dufour/miro/resolve/main/teaser.jpg" alt="MIRO samples"></td></tr>
24
+ </table>
25
+
26
+ ## Layout
27
+
28
+ Every variant lives in its own subfolder and is loaded via the `variant=` argument:
29
+
30
+ ```python
31
+ from miro import MiroPipeline
32
+ import torch
33
+
34
+ pipe = MiroPipeline.from_pretrained(
35
+ "nicolas-dufour/miro-ablations",
36
+ variant="miro-no-clip", # ← the subfolder name
37
+ ).to("cuda", torch.float16)
38
+ ```
39
+
40
+ Each `MiroPipeline` instance exposes `pipe.coherence_keys`, which lists the reward axes the loaded checkpoint was trained on. `reward_targets={...}` will raise `ValueError` if you pass a key that's not in this list.
41
+
42
+ ## Variants
43
+
44
+ ### Reward ablations (8) β€” full MIRO recipe minus one signal
45
+
46
+ Same architecture and training data as the main MIRO, with one reward signal turned off so you can isolate its contribution.
47
+
48
+ | Subfolder | What's ablated | `coherence_keys` size |
49
+ |---|---|:-:|
50
+ | `miro-no-synthetic-captions` | Trained on original captions only (no synthetic-caption augmentation) | 7 |
51
+ | `miro-no-aesthetic` | LAION aesthetic-quality reward | 6 |
52
+ | `miro-no-clip` | CLIP text-image alignment | 6 |
53
+ | `miro-no-hpsv2` | HPSv2 human preference | 6 |
54
+ | `miro-no-image-reward` | ImageReward | 6 |
55
+ | `miro-no-pickscore` | PickScore human preference | 6 |
56
+ | `miro-no-sciscore` | SciScore | 6 |
57
+ | `miro-no-vqa` | VQAScore | 6 |
58
+
59
+ ### Single-reward specialists (7) β€” paper baselines
60
+
61
+ Each is trained on **only one** reward signal β€” the controls the paper compares MIRO against. `pipe.coherence_keys` is a 1-tuple for these.
62
+
63
+ | Subfolder | The one reward it knows about |
64
+ |---|---|
65
+ | `miro-only-aesthetic` | `aesthetic_score` |
66
+ | `miro-only-clip` | `clip_score` |
67
+ | `miro-only-hpsv2` | `hpsv2_score` |
68
+ | `miro-only-image-reward` | `image_reward_score` |
69
+ | `miro-only-pickscore` | `pick_a_score_score` |
70
+ | `miro-only-sciscore` | `sciscore_score` |
71
+ | `miro-only-vqa` | `vqa_score` |
72
+
73
+ ## What's in each subfolder
74
+
75
+ ```
76
+ miro-<variant>/
77
+ β”œβ”€β”€ model.safetensors # fp32 EMA weights (~1.4 GB) β€” ready for finetuning
78
+ β”œβ”€β”€ config.json # network kwargs + sampler defaults
79
+ β”œβ”€β”€ uncond_embedding.npy # precomputed FLAN-T5-XL unconditional embedding
80
+ β”œβ”€β”€ teaser.jpg # shared masonry gallery
81
+ └── README.md # per-variant model card
82
+ ```
83
+
84
+ ## Citation
85
+
86
+ ```bibtex
87
+ @inproceedings{dufour2026miro,
88
+ title = {{MIRO}: {M}ult{I}-{R}eward c{O}nditioned pretraining improves {T2I} quality and efficiency},
89
+ author = {Dufour, Nicolas and Degeorge, Lucas and Ghosh, Arijit and Kalogeiton, Vicky and Picard, David},
90
+ booktitle = {International Conference on Machine Learning (ICML)},
91
+ year = {2026}
92
+ }
93
+ ```
94
+
95
+ ## License
96
+
97
+ MIT β€” see [LICENSE](https://github.com/nicolas-dufour/miro/blob/main/LICENSE).