File size: 2,317 Bytes
b57606c 4ef394d b57606c 538b2d3 b57606c 538b2d3 b57606c 4ef394d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
---
license: apache-2.0
base_model: runwayml/stable-diffusion-v1-5
tags:
- stable-diffusion
- diffusers
- lora
- scientific-machine-learning
- physics-simulation
- visual-reasoning
---
# VR-LoRA: N-Body Physics Simulator (SD-v1.5)
This repository contains a Visual Reasoning LoRA (VR-LoRA) fine-tuned on `runwayml/stable-diffusion-v1-5` to predict the temporal evolution of a 3-body gravitational system.
This model is the primary output of the paper: **"Visual Reasoning Transfer: Leveraging Pretrained Visual Models for Physical and Temporal Prediction"** (link to paper coming soon).
## Model Description
The model does not generate images. Instead, it acts as a dynamics engine in latent space. When given a latent representation of a physical state (encoded as a "spatial field image"), it predicts the latent representation of the next physical state.
This LoRA was trained for 15,000 steps on a synthetic dataset of 10,000 N-body trajectories.
* **Research Project Repository:** [https://github.com/sandner-art/SC-Visual-Reasoning](https://github.com/sandner-art/SC-Visual-Reasoning)
* **Training Dataset:** [huggingface.co/datasets/sandner/n-body-trajectories-for-vrlora](https://huggingface.co/datasets/sandner/n-body-trajectories-for-vrlora) (Replace with your link)
## How to Use
This LoRA is designed to be used with the evaluation script found in the main research repository. It is not intended for standard text-to-image generation.
```python
# See the evaluate_vr_lora.py script in the main GitHub repo for a full example.
from diffusers import StableDiffusionPipeline
base_model = "runwayml/stable-diffusion-v1-5"
lora_model = "sandner/vr-lora-physics-sd15" # Replace with your repo name
pipeline = StableDiffusionPipeline.from_pretrained(base_model)
pipeline.load_lora_weights(lora_model)
pipeline.to("cuda")
# Now the `pipeline.unet` component is ready for physics simulation.
unet = pipeline.unet
# ...
```
## Training Procedure
The model was trained using the `train_vr_lora.py` script from the project repository. Key hyperparameters:
* **Learning Rate:** 1e-4
* **Batch Size:** 32
* **Max Steps:** 15,000
* **Optimizer:** AdamW
* **Scheduler:** Cosine
## Citing this Work
If you use this model in your research, please cite our paper (BibTeX entry coming soon).
|