Initial public release

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,61 @@
+---
+license: cc-by-nc-4.0
+pipeline_tag: text-to-image
+tags:
+- depth-estimation
+- text-to-3d
+- diffusion
+- flux
+library_name: flux_rgbd
+---
+
+# Modality Forcing for Scalable Spatial Generation
+
+Joint **text → RGB + depth** generation with a single diffusion
+transformer, built on FLUX.2. Modality Forcing assigns separate noise levels
+per modality during post-training, so one model supports joint generation
+(text → RGB-D), image-to-depth, and depth-to-image at inference.
+
+- 📄 Paper: [arXiv:2606.13676](https://arxiv.org/abs/2606.13676)
+- 💻 Code: [github.com/Duisterhof/modality-forcing](https://github.com/Duisterhof/modality-forcing)
+- 🚀 Demo: [Hugging Face Space](https://huggingface.co/spaces/bartduis/modality_forcing)
+- 🌐 Project page: [modality-forcing.github.io](https://modality-forcing.github.io/)
+
+## Files
+
+| File | Description |
+|------|-------------|
+| `model.safetensors` | FluxRGBD DiT (12B total — 9B-class FLUX.2 backbone + depth streams, bf16) |
+| `config.json` | Model variant config (`flux_rgbd_9b_v2`) |
+| `ae_encoder.safetensors` / `ae_decoder.safetensors` | FLUX.2 autoencoder |
+
+The Qwen3-8B text encoder is pulled separately from
+[`Qwen/Qwen3-8B`](https://huggingface.co/Qwen/Qwen3-8B).
+
+## Usage
+
+```bash
+git clone https://github.com/Duisterhof/modality-forcing.git
+cd modality-forcing
+bash install.sh
+python scripts/joint.py --prompt "a cozy sunlit kitchen with wooden cabinets"
+```
+
+The scripts download these weights automatically (`bartduis/modality_forcing`
+is the default `--model`).
+
+## License
+
+The model weights are released under **CC BY-NC 4.0** (non-commercial). The
+inference code is Apache-2.0; see the GitHub repository.
+
+## Citation
+
+```bibtex
+@article{duisterhof2026mofo,
+  title   = {Modality Forcing for Scalable Spatial Generation},
+  author  = {Duisterhof, Bardienus Pieter and Ramanan, Deva and Ichnowski, Jeffrey and Johnson, Justin and Park, Keunhong},
+  journal = {arXiv preprint arXiv:2606.13676},
+  year    = {2026}
+}
+```

ae_decoder.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:393477cff9c15055e512525c7b90cda02b06e0378c93dfc75b261d10825842b1
+size 198495428

ae_encoder.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fde73aad493e1343b138660c87e6a376dc5b2d6bf9b061206b580088b78ff6e3
+size 137714880

config.json

@@ -0,0 +1,5 @@
+{
+  "variant": "flux_rgbd_9b_v2",
+  "model_type": "flux_rgbd",
+  "torch_dtype": "bfloat16"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3da9bbd5e45ba92e22bfb7c338f6bbcce7ba11e9119c409381177396e17b1f0f
+size 23972604108