unilm/textdiffuser/eval/README.md

# Evaluation

We provide the code for sampling from Stable Diffusion, ControlNet, DeepFloyd at ```MARIOEval_generate.py```. Since these methods rely on *diffusers* of the original version, it is recommended to create a **NEW** environment and install packages with command ```pip install requirements.txt```. It is recommended to install pytorch with version >= 2.0 to avoid the OOM error.

Once the generation is complete, evaluation of FID and CLIPScore can be performed using the ```MARIOEval_evaluate.py``` file. For OCR metrics, please install [MaskTextSpotterV3](https://github.com/MhLiao/MaskTextSpotterV3) to obtain the OCR result of each image and refer to ```ocr_eval.py``` for evaluation. It should be noted that the output image of DeepFloyd contains a watermark "IF" at the right-bottom corner, which needs to be masked before performing OCR.

```python
if method is 'deepfloyd':
    image[-64:, -64:] = 0 # remove watermark, the input image is resized to 512x512
```