Update README.md

README.md

@@ -13,70 +13,43 @@ pipeline_tag: text-to-image
 <table>
   <tr>
     <td><img src="assets/teaser_info.png" alt="teaser example 0" width="1200"/></td>
+  </tr>
+  <tr>
     <td><img src="assets/teaser_slide.png" alt="teaser example 1" width="1200"/></td>
   </tr>
 </table>
 
 ## Abstract
 <p>
-    Generating visually appealing images is fundamental to modern text-to-image generation models. 
-    A potential solution to better aesthetics is direct preference optimization (DPO), 
-    which has been applied to diffusion models to improve general image quality including prompt alignment and aesthetics. 
-    Popular DPO methods propagate preference labels from clean image pairs to all the intermediate steps along the two generation trajectories. 
-    However, preference labels provided in existing datasets are blended with layout and aesthetic opinions, which would disagree with aesthetic preference. 
-    Even if aesthetic labels were provided (at substantial cost), it would be hard for the two-trajectory methods to capture nuanced visual differences at different steps.
+  Recently, state-of-the-art text-to-image generation models, such as Flux and Ideogram 2.0, have made 
+  significant progress in sentence-level visual text rendering. In this paper, we focus on the more 
+  challenging scenarios of article-level visual text rendering and address a novel task of generating 
+  high-quality business content, including infographics and slides, based on user provided article-level 
+  descriptive prompts and ultra-dense layouts.  The fundamental challenges are twofold: significantly 
+  longer context lengths and the scarcity of high-quality business content data.
 </p>
 <p>
-    To improve aesthetics economically, this paper uses existing generic preference data and introduces step-by-step preference optimization 
-    (SPO) that discards the propagation strategy and allows fine-grained image details to be assessed. Specifically, 
-    at each denoising step, we 1) sample a pool of candidates by denoising from a shared noise latent, 
-    2) use a step-aware preference model to find a suitable win-lose pair to supervise the diffusion model, and 
-    3) randomly select one from the pool to initialize the next denoising step. 
-    This strategy ensures that diffusion models focus on the subtle, fine-grained visual differences 
-    instead of layout aspect. We find that aesthetic can be significantly enhanced by accumulating these 
-    improved minor differences.
+  In contrast to most previous works that focus on a limited number of sub-regions and sentence-level 
+  prompts, ensuring precise adherence to ultra-dense layouts with tens or even hundreds of sub-regions in
+  business content is far more challenging. We make two key technical contributions: (i) the construction 
+  of scalable, high-quality business content dataset, i.e., Infographics-650K, equipped with 
+  ultra-dense layouts and prompts by implementing a layer-wise retrieval-augmented infographic generation 
+  scheme; and (ii) a layout-guided cross attention scheme, which injects tens of region-wise prompts into 
+  a set of cropped region latent space according to the ultra-dense layouts, and refine each sub-regions 
+  flexibly during inference using a layout conditional CFG. 
 </p>
 <p>
-    When fine-tuning Stable Diffusion v1.5 and SDXL, SPO yields significant 
-    improvements in aesthetics compared with existing DPO methods while not sacrificing image-text alignment 
-    compared with vanilla models. Moreover, SPO converges much faster than DPO methods due to the step-by-step 
-    alignment of fine-grained visual details.
+  We demonstrate the strong results of our system compared to previous SOTA systems such as Flux and SD3 
+  on our BizEval prompt set. Additionally, we conduct thorough ablation experiments to verify the 
+  effectiveness of each component. We hope our constructed Infographics-650K and BizEval can encourage 
+  the broader community to advance the progress of business content generation.
 </p>
 
 ## Model Description
 
-This model is fine-tuned from [stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0). It has been trained on 4,000 prompts for 10 epochs.
-
-This is a merged checkpoint that combines the LoRA checkpoint with the base model [stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0). If you want to access the LoRA checkpoint, please visit [SPO-SDXL_4k-p_10ep_LoRA](https://huggingface.co/SPO-Diffusion-Models/SPO-SDXL_4k-p_10ep_LoRA). We also provide a LoRA checkpoint compatible with [stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui), which can be accessed [here](https://civitai.com/models/510261?modelVersionId=567119).
-
-
-## A quick example
-```python
-from diffusers import StableDiffusionXLPipeline, UNet2DConditionModel
-import torch
-
-# load pipeline
-inference_dtype = torch.float16
-pipe = StableDiffusionXLPipeline.from_pretrained(
-    "SPO-Diffusion-Models/SPO-SDXL_4k-p_10ep", 
-    torch_dtype=inference_dtype,
-)
-vae = AutoencoderKL.from_pretrained(
-    'madebyollin/sdxl-vae-fp16-fix',
-    torch_dtype=inference_dtype,
-)
-pipe.vae = vae
-pipe.to('cuda')
-
-generator=torch.Generator(device='cuda').manual_seed(42)
-image = pipe(
-    prompt='a child and a penguin sitting in front of the moon',
-    guidance_scale=5.0,
-    generator=generator,
-    output_type='pil',
-).images[0]
-image.save('moon.png')
-```
+The ByT5 model is finetuned from [Glyph-ByT5-v2](https://arxiv.org/abs/2406.10208), which supports accerate visual text rendering in ten different languages.
+The [SPO](https://huggingface.co/SPO-Diffusion-Models) model is a substitute for the original sdxl-base-1.0 for aesthetic improvement.
+You can follow our [github]() to organize and run the model.
 
 ## Citation
 If you find our work or codebase useful, please consider giving us a star and citing our work.