# SciVisual-Agent: A Physics and Mathematics Virtual Animation Lab
SciVisual-Agent is a Hugging Face Space that turns math and physics prompts into animated Manim videos. The idea is simple: describe a concept in natural language, then get back a visual explanation that moves.
We built it for the Build Small hackathon as a compact virtual animation lab for scientific ideas. It combines a fine-tuned code generation model, ManimCE rendering, LaTeX support, and a self-correction loop that can repair generated code when rendering fails.
## Motivation
Physics and mathematics are deeply visual, but making good animations still takes time. A teacher might know exactly how a damped pendulum should move, or how the identity `(a+b)^2 = a^2 + 2ab + b^2` should unfold geometrically, but turning that intuition into polished animation code is a separate skill.
SciVisual-Agent is meant to reduce that gap. Instead of asking users to write Manim code directly, the app asks for the concept, the visual constraints, and the teaching goal. From there, it generates ManimCE Python code, renders the scene, and shows both the final video and the source code.
## What It Does
SciVisual-Agent currently works best when the user gives a detailed prompt with enough visual and physical constraints. For example, prompts that specify the objects, equations, layout, timing, labels, and animation behavior tend to produce much better Manim code.
The long-term goal is to make the app work from much shorter prompts, such as:
- "Create a damped simple pendulum simulation with a fading trace and the governing differential equation."
- "Show a spring-mass system with a synchronized position graph."
- "Create a geometric proof of `(a+b)^2 = a^2 + 2ab + b^2` with quadrant-aligned blocks."
Those short prompts are still challenging because animation generation needs many hidden decisions: scene layout, scale, labels, equations, motion timing, updater logic, and visual style. A human animator naturally fills in those gaps. An AI system needs a planning step to do the same.
For each prompt, the app follows a full prompt-to-video workflow:
1. Build a structured prompt for scientific animation code generation.
2. Generate ManimCE Python code from a Hugging Face-hosted adapter model.
3. Extract clean executable Python from the model response.
4. Render the scene with ManimCE in an isolated subprocess.
5. Capture tracebacks or render logs if Manim fails.
6. Feed the error back into the model for self-correction.
7. Return the rendered MP4 and the final clean source code.
## Example Outputs
### Damped Simple Pendulum
This example shows a damped pendulum with numerical integration, a dynamic rod updater, a moving bob, a fading trace, and a LaTeX equation overlay.
### Geometric Identity
This example turns `(a+b)^2 = a^2 + 2ab + b^2` into a geometric decomposition of one larger square.
### Spring-Mass System
This example synchronizes a horizontal spring-mass oscillator with a live position graph. We explicitly use a linear animation rate so the motion does not visually ease in or out.
## How The System Flows
The app is built around one feedback loop: generate code, render it, inspect the result, and repair it when needed.
```mermaid
flowchart TD
A["User describes a math or physics animation"] --> B["SciVisual prompt builder"]
B --> C["Fine-tuned adapter model"]
C --> D["Generated ManimCE Python code"]
D --> E["Code extractor and safety checks"]
E --> F["ManimCE render subprocess"]
F --> G{"Render successful?"}
G -->|Yes| H["Return MP4 video and clean source code"]
G -->|No| I["Capture traceback and render logs"]
I --> J["Self-correction prompt"]
J --> C
```
The important design choice is that rendering is not treated as the end of the process. It becomes feedback. If Manim fails, the app captures the exact error and asks the model to produce a corrected version.
In the next version, we want to add a planning phase before code generation:
```mermaid
flowchart TD
A["Short user prompt"] --> B["Planning agent"]
B --> C{"Enough detail?"}
C -->|No| D["Ask clarifying questions"]
D --> B
C -->|Yes| E["Detailed animation specification"]
E --> F["Manim code generation"]
F --> G["Render and self-correct"]
```
This planning phase would turn a simple idea like "show a damped pendulum" into a more complete animation brief: what objects to draw, what equation to show, how long to animate, where labels should go, and what physical assumptions to use. That detailed brief is what the code generation model can handle much more reliably.
## Self-Correction Loop
Generated Manim code can fail for small but important reasons: missing imports, invalid LaTeX, incorrect updater logic, wrong scene names, or layout assumptions that do not work at render time.
SciVisual-Agent treats those failures as useful signals. When Manim returns an error, the app sends the traceback back to the model and asks for a complete corrected scene. The default retry count is 3, and the app supports up to 5 retries.
This makes the system feel less like a one-shot code generator and more like a small repair loop. The first answer does not have to be perfect for the final result to be useful.
## Hugging Face Space Details
The Space is designed for ZeroGPU:
- Model loading is deferred until the render callback.
- The GPU callback uses `@spaces.GPU`.
- The default model path points to a Hugging Face adapter repository.
- 4-bit loading is enabled by default.
- ManimCE, LaTeX, `dvisvgm`, Ghostscript, ffmpeg, Cairo, and Pango are installed through Space dependency files.
### Model Selection
By default the app loads the fine-tuned SciVisual LoRA adapter from HF model card [vankhieu/Seed_Coder_8B_Instruct_unsloth_bnb_4bit_lora_r8_sft_grpo_rw_mean_text_visual](https://huggingface.co/vankhieu/Seed_Coder_8B_Instruct_unsloth_bnb_4bit_lora_r8_sft_grpo_rw_mean_text_visual)
and applies it to its base model [unsloth/Seed-Coder-8B-Instruct-unsloth-bnb-4bit](https://huggingface.co/unsloth/Seed-Coder-8B-Instruct-unsloth-bnb-4bit).
The app can also test a pure base model through environment variables:
```bash
SCIVISUAL_MODEL_KIND=base
SCIVISUAL_MODEL_PATH=unsloth/Qwen3-30B-A3B-Instruct-2507-bnb-4bit
SCIVISUAL_SELECTED_MODEL=unsloth/Qwen3-30B-A3B-Instruct-2507-bnb-4bit
SCIVISUAL_BACKEND=transformers
SCIVISUAL_LOAD_IN_4BIT=1
```
That makes it easy to compare the fine-tuned adapter with larger or newer base models without changing the application code.
## Design
We wanted the interface to feel like a scientific control surface instead of a standard web form. The UI uses:
- A dark `#0b0f19` dashboard background.
- Cyber cyan accents with hot pink alert highlights.
- Glassmorphism panels.
- Restyled native Gradio inputs and outputs.
- A side-by-side layout with controls on the left and the rendered viewport on the right.
The visual style is intentionally different from a default Gradio app. We wanted users to feel like they were operating a small virtual lab.
## Current Strengths
- End-to-end prompt-to-video workflow.
- Fine-tuned adapter support from Hugging Face.
- Consistent prompt templates for more stable code generation.
- Render-aware self-correction.
- ManimCE and LaTeX support.
- Example prompts for physics and mathematics.
- Clean source-code display for learning and debugging.
- Environment-variable switch for testing base models.
## Limitations
This is still a hackathon prototype, so there are clear limits:
- Some generated scenes may still fail after all retries, especially when the prompt is very broad.
- The model can create animations that look plausible but are not physically precise unless the prompt is explicit.
- Longer animations increase render time and can hit Space timeout constraints.
## Improvements We Want Next
The biggest next step is evaluation. We want to compare the fine-tuned adapter against strong base models such as larger Qwen or Gemma variants, using the same prompt suite and measuring render success rate, visual quality, and number of correction attempts.
We also want to:
- Save successful prompt/code/video triples for future fine-tuning.
- Add a planning phase that expands short prompts into detailed animation briefs.
- Ask clarifying questions when the user prompt is too vague for reliable code generation.
- Build a larger gallery of examples for education-focused demos.
- Improve targeted repair for LaTeX and Manim updater errors.
## What We Learned
Code generation for animation is different from ordinary code generation. A Manim scene can be syntactically valid but visually wrong. It can render successfully but still teach the wrong intuition. Good prompts need to specify geometry, timing, updater behavior, labels, and animation flow.
We also learned that prompt format matters. A consistent generation template improved output stability and reduced formatting problems. Small extraction details matter too: a stray markdown fence can become a Python syntax error.
Most importantly, self-correction is especially useful for visual code. Manim gives concrete errors, and those errors can help the model produce a better scene on the next attempt.
## Closing
SciVisual-Agent is a small step toward interactive scientific visualization. It does not replace expert-made educational animations, but it makes the first draft much easier to create.
For us, the exciting part is the loop: prompt, generate, render, inspect, repair, and learn. That loop can become a practical assistant for teachers, students, and builders who want math and physics ideas to move on screen.
## Acknowledgements
- Special thanks to the author of [Manim Trainer](https://github.com/SuienS/manim-trainer) for providing the fine-tuning code. The forked version of it is available at [KhieuNguyen/manim-trainer](https://github.com/khieunguyen/manim-trainer) which fixes some bugs.
- Special thanks to [Hugging Face](https://huggingface.co/) and [Gradio](https://gradio.app/) for providing the platform and resources to build and deploy the app, and for organizing the hackathon.
- Special thanks to [Modal](https://modal.com) for providing the platform and GPU resources to fine-tune the model.
- Thanks to [Codex](https://chatgpt.com/codex) for helping write and refine the project code and this blog post.
- Thanks for all sponsors and judges for making this event possible.