Improve model card: Add `library_name`, abstract, links, and usage example

README.md

@@ -1,7 +1,9 @@
 ---
-license: cc-by-4.0
 base_model:
 - ashawkey/mvdream-sd2.1-diffusers
+datasets:
+- yosepyossi/OOD-Eval
+license: cc-by-4.0
 pipeline_tag: text-to-3d
 tags:
 - multiview
@@ -9,6 +11,83 @@ tags:
 - RAG
 - retrieval
 - diffusion
-datasets:
-- yosepyossi/OOD-Eval
----
+library_name: diffusers
+---
+
+# MV-RAG: Retrieval Augmented Multiview Diffusion
+
+| [Project Page](https://yosefdayani.github.io/MV-RAG/) | [Paper](https://huggingface.co/papers/2508.16577) | [GitHub](https://github.com/yosefdayani/MV-RAG) | [Weights](https://huggingface.co/yosepyossi/mvrag) | [Benchmark (OOD-Eval)](https://huggingface.co/datasets/yosepyossi/OOD-Eval) |
+
+![teaser](https://yosefdayani.github.io/MV-RAG/static/images/teaser.jpg)
+
+## Abstract
+Text-to-3D generation approaches have advanced significantly by leveraging pretrained 2D diffusion priors, producing high-quality and 3D-consistent outputs. However, they often fail to produce out-of-domain (OOD) or rare concepts, yielding inconsistent or inaccurate results. To this end, we propose MV-RAG, a novel text-to-3D pipeline that first retrieves relevant 2D images from a large in-the-wild 2D database and then conditions a multiview diffusion model on these images to synthesize consistent and accurate multiview outputs. Training such a retrieval-conditioned model is achieved via a novel hybrid strategy bridging structured multiview data and diverse 2D image collections. This involves training on multiview data using augmented conditioning views that simulate retrieval variance for view-specific reconstruction, alongside training on sets of retrieved real-world 2D images using a distinctive held-out view prediction objective: the model predicts the held-out view from the other views to infer 3D consistency from 2D data. To facilitate a rigorous OOD evaluation, we introduce a new collection of challenging OOD prompts. Experiments against state-of-the-art text-to-3D, image-to-3D, and personalization baselines show that our approach significantly improves 3D consistency, photorealism, and text adherence for OOD/rare concepts, while maintaining competitive performance on standard benchmarks.
+
+## Overview
+MV-RAG is a text-to-3D generation method that retrieves 2D reference images to guide a multiview diffusion model. By conditioning on both text and multiple real-world 2D images, MV-RAG improves realism and consistency for rare/out-of-distribution or newly emerging objects.
+
+## Installation
+
+We recommend creating a fresh conda environment to run MV-RAG:
+
+```bash
+# Clone the repository
+git clone https://github.com/yosefdayani/MV-RAG.git
+cd MV-RAG
+
+# Create new environment
+conda create -n mvrag python=3.9 -y
+conda activate mvrag
+
+# Install PyTorch (adjust CUDA version as needed)
+# Example: CUDA 12.4, PyTorch 2.5.1
+conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.4 -c pytorch -c nvidia
+
+# Install other dependencies
+pip install -r requirements.txt
+```
+
+## Weights
+
+MV-RAG weights are available on [Hugging Face](https://huggingface.co/yosepyossi/mvrag).
+```bash
+# Make sure git-lfs is installed (https://git-lfs.com)
+git lfs install
+
+git clone https://huggingface.co/yosepyossi/mvrag
+```
+
+Then the model weights should appear as MV-RAG/mvrag/...
+
+## Usage Example
+You could prompt the model on your retrieved local images by:
+```bash
+python main.py \
+--prompt "Cadillac 341 automobile car" \
+--retriever simple \
+--folder_path "assets/Cadillac 341 automobile car" \
+--seed 0 \
+--k 4 \
+--azimuth_start 45  # or 0 for front view
+```
+
+To see all command options run
+```bash
+python main.py --help
+```
+
+## Acknowledgement
+This repository is based on [MVDream](https://github.com/bytedance/MVDream) and adapted from [MVDream Diffusers](https://github.com/ashawkey/mvdream_diffusers). We would like to thank the authors of these works for publicly releasing their code.
+
+## Citation
+``` bibtex
+@misc{dayani2025mvragretrievalaugmentedmultiview,
+      title={MV-RAG: Retrieval Augmented Multiview Diffusion}, 
+      author={Yosef Dayani and Omer Benishu and Sagie Benaim},
+      year={2025},
+      eprint={2508.16577},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV},
+      url={https://arxiv.org/abs/2508.16577}, 
+}
+```