|
|
---
|
|
|
license: cc-by-nc-4.0
|
|
|
datasets:
|
|
|
- salamnocap/ml-figs
|
|
|
language:
|
|
|
- en
|
|
|
pipeline_tag: text-to-image
|
|
|
---
|
|
|
|
|
|
# [ML-FIGS-LDM](https://github.com/salamnocap/ml-figs-ldm) |
|
|
## EDUCATIONAL FIGURE GENERATION USING TEXT PERCEPTUAL LOSS |
|
|
|
|
|
ML-FIGS-LDM is a Latent Diffusion Model (LDM) for generating educational figures. The AutoencoderKL is trained using a Text Perceptual Loss to reconstruct more readable text within the figures. |
|
|
|
|
|
### Autoencoder Performance Comparison: |
|
|
**Note**: |
|
|
- `AutoencoderKL_SD`: Stable Diffusion v1-4 autoencoder trained on LAION. |
|
|
- `AutoencoderKL_TPL`: Autoencoders trained with Text Perceptual Loss (TPL). |
|
|
- [Model A](https://huggingface.co/salamnocap/ml-figs-ldm/blob/main/AutoencoderKL_MlFigs.ckpt) is trained on **ML-Figs**. |
|
|
- [Model B](https://huggingface.co/salamnocap/ml-figs-ldm/blob/main/AutoencoderKL_MlFigsSciCap.ckpt) is trained on **ML-Figs + SciCap**. |
|
|
|
|
|
| Dataset | Method | PSNR ↑ | SSIM ↑ | FID ↓ | LPIPS ↓ | MSE ↓ | TPL ↓ | |
|
|
|--------------------------|---------------------------|--------|--------|---------|---------|--------|--------| |
|
|
| **ML-Figs Test** | `AutoencoderKL_SD` | 33.01 | 0.970 | 20.51 | 0.022 | 0.003 | 0.043 | |
|
|
| | `AutoencoderKL_TPL` A | 30.71 | 0.954 | 16.13 | 0.056 | 0.002 | 0.017 | |
|
|
| **ML-Figs + SciCap Test**| `AutoencoderKL_SD` | 32.60 | 0.970 | 12.69 | 0.023 | 0.004 | 0.061 | |
|
|
| | `AutoencoderKL_TPL` A | 29.94 | 0.954 | 9.235 | 0.057 | 0.003 | 0.028 | |
|
|
| | `AutoencoderKL_TPL` B | 31.47 | 0.979 | 6.256 | 0.016 | 0.001 | 0.010 | |
|
|
|
|
|
### Latent Diffusion Model: |
|
|
[MlFigs_LDM_12.ckpt](https://huggingface.co/salamnocap/ml-figs-ldm/blob/main/MlFigs_LDM_12.ckpt) |
