manim-mcp / MIGRATION_TO_BLAXEL.md
bhaveshgoel07's picture
Deploy code fixes (clean history)
fff13d1
|
raw
history blame
17.3 kB

Migration Guide: Local to Blaxel Cloud Rendering

This document explains how the Manim rendering has been migrated from local execution to Blaxel cloud sandboxes with pre-installed dependencies.

Table of Contents

Overview

Before: Manim code execution and video rendering happened on your local machine.

After: Rendering happens in Blaxel cloud sandboxes with Manim and FFmpeg pre-installed.

Why: Better security, scalability, and no local dependency management.

Why Migrate?

Benefits of Blaxel Cloud Rendering

  1. Security πŸ”’

    • Isolated execution environment
    • No risk of malicious code affecting your system
    • AI-generated code runs in sandboxed containers
  2. No Local Dependencies πŸ“¦

    • Don't need to install Manim locally
    • Don't need FFmpeg on your machine
    • Don't need LaTeX packages
    • Works on any OS without configuration
  3. Scalability πŸ“ˆ

    • Parallel rendering of multiple animations
    • No resource constraints from local machine
    • Auto-scaling based on workload
  4. Consistency βœ…

    • Same environment every time
    • No "works on my machine" issues
    • Reproducible builds
  5. Resource Management πŸ’ͺ

    • Heavy rendering doesn't slow down your computer
    • Can allocate more memory (4GB-8GB) as needed
    • Automatic cleanup of temporary files

What Changed

New Files Added

manim-agent/
β”œβ”€β”€ Dockerfile.sandbox          # Custom Docker image definition
β”œβ”€β”€ entrypoint.sh               # Sandbox initialization script
β”œβ”€β”€ Makefile.sandbox            # Build automation
β”œβ”€β”€ deploy_sandbox.sh           # Automated deployment script
β”œβ”€β”€ BLAXEL_SANDBOX_SETUP.md     # Detailed setup guide
β”œβ”€β”€ BLAXEL_QUICKSTART.md        # Quick reference
└── MIGRATION_TO_BLAXEL.md      # This file

Modified Files

  1. mcp_servers/renderer.py

    • Added MANIM_SANDBOX_IMAGE environment variable
    • Uses custom image instead of blaxel/py-app:latest
    • No more runtime installation of Manim/FFmpeg
  2. .gitignore

    • Added Docker build artifacts
    • Added backup files from scripts

Environment Variables

New required variable:

MANIM_SANDBOX_IMAGE=blaxel/your-workspace/manim-sandbox:latest

Existing variables:

BLAXEL_API_KEY=your_api_key_here
BL_WORKSPACE=your_workspace_id  # Optional

The Problem

Old Approach: Runtime Installation

# Old code in renderer.py
sandbox = await SandboxInstance.create({
    "name": f"manim-render-{scene_name}",
    "image": "blaxel/py-app:latest",  # Generic Python image
    "memory": 4096,
})

# Then install dependencies at runtime (slow and unreliable)
await sandbox.process.exec({
    "command": "pip install manim",
    "wait_for_completion": True,
})

Issues with Runtime Installation

  1. Slow ⏱️

    • Installing Manim takes 2-3 minutes every time
    • FFmpeg installation requires apt-get
    • LaTeX packages are huge downloads
  2. Unreliable ❌

    • Network failures during pip install
    • Version conflicts
    • Missing system dependencies
    • Race conditions with process management
  3. No FFmpeg 🚫

    • Generic images don't have FFmpeg
    • Installing FFmpeg requires root access
    • System dependencies complex to manage
  4. Wasteful πŸ’Έ

    • Pay for installation time every render
    • Same packages downloaded repeatedly
    • No caching between renders

The Solution

Custom Docker Image with Pre-installed Dependencies

# Dockerfile.sandbox
FROM python:3.12-slim

# Install FFmpeg, LaTeX, and system dependencies
RUN apt-get update && apt-get install -y \
    ffmpeg \
    texlive \
    libcairo2-dev \
    # ... other dependencies

# Pre-install Manim
RUN pip install manim>=0.18.1

# Copy Blaxel sandbox API
COPY --from=ghcr.io/blaxel-ai/sandbox:latest /sandbox-api /usr/local/bin/sandbox-api

ENTRYPOINT ["/entrypoint.sh"]

Advantages

  1. Fast ⚑

    • Dependencies already installed
    • Sandbox ready in seconds
    • Start rendering immediately
  2. Reliable βœ…

    • Pre-tested environment
    • Consistent versions
    • No installation failures
  3. Complete 🎯

    • FFmpeg included
    • LaTeX support
    • All system dependencies
  4. Cost-effective πŸ’°

    • Pay only for rendering time
    • No repeated installations
    • Efficient resource usage

Migration Steps

Step 1: Prerequisites

Ensure you have:

  • Docker installed locally
  • Blaxel CLI: npm install -g @blaxel/cli
  • Blaxel API key from blaxel.ai

Step 2: Set Environment Variables

# Add to your .env or shell profile
export BLAXEL_API_KEY="your_api_key_here"
export BL_WORKSPACE="your_workspace_id"  # Optional

Step 3: Deploy Custom Sandbox (Automated)

# Run the automated deployment script
./deploy_sandbox.sh

This script will:

  1. βœ… Check all prerequisites
  2. βœ… Build Docker image locally
  3. βœ… Test the image
  4. βœ… Deploy to Blaxel
  5. βœ… Retrieve your image ID
  6. βœ… Update your .env file

Step 4: Deploy Custom Sandbox (Manual)

If you prefer manual steps:

# 1. Build the image
docker build -f Dockerfile.sandbox -t manim-sandbox .

# 2. Test locally
docker run -d --name test -p 8080:8080 manim-sandbox
curl -X POST http://localhost:8080/process \
  -H "Content-Type: application/json" \
  -d '{"command": "manim --version", "waitForCompletion": true}'
docker stop test && docker rm test

# 3. Login to Blaxel
bl login

# 4. Deploy
bl deploy

# 5. Get your image ID
bl get sandboxes -ojson | jq -r '.[0].spec.runtime.image'

Step 5: Configure Your Application

Add to your .env file:

MANIM_SANDBOX_IMAGE=blaxel/your-workspace/manim-sandbox:latest

The renderer code automatically reads this variable:

# This is already in renderer.py
MANIM_SANDBOX_IMAGE = os.getenv(
    "MANIM_SANDBOX_IMAGE",
    "blaxel/py-app:latest",  # Fallback
)

Step 6: Test End-to-End

# Run your animation pipeline
python main_new.py

# Or launch Gradio UI
python app.py

Step 7: Verify Success

Check that:

  • Sandbox creates quickly (< 10 seconds)
  • No "Installing Manim..." messages in logs
  • Rendering completes successfully
  • Video output is correct
  • FFmpeg processing works

Architecture Comparison

Before: Local + Runtime Installation

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Your Machine                                           β”‚
β”‚                                                         β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”‚
β”‚  β”‚   Python    │───▢│  MCP Server  │───▢│  Manim   β”‚  β”‚
β”‚  β”‚   Script    β”‚    β”‚  (renderer)  β”‚    β”‚  Local   β”‚  β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β”‚
β”‚                            β”‚                            β”‚
β”‚                            β–Ό                            β”‚
β”‚                     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                    β”‚
β”‚                     β”‚   Blaxel     β”‚                    β”‚
β”‚                     β”‚  Sandbox API β”‚                    β”‚
β”‚                     β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                    β”‚
β”‚                            β”‚                            β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                             β”‚
                             β–Ό
              β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
              β”‚  Blaxel Cloud                β”‚
              β”‚                              β”‚
              β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”‚
              β”‚  β”‚  Generic Python Image  β”‚  β”‚
              β”‚  β”‚  (No Manim/FFmpeg)     β”‚  β”‚
              β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β”‚
              β”‚           β”‚                  β”‚
              β”‚           β–Ό                  β”‚
              β”‚  ⏱️ Install Manim (3 min)    β”‚
              β”‚  ⏱️ Install FFmpeg (fail)    β”‚
              β”‚  ❌ Render (error)           β”‚
              β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

After: Cloud with Pre-installed Dependencies

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Your Machine                                           β”‚
β”‚                                                         β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                   β”‚
β”‚  β”‚   Python    │───▢│  MCP Server  β”‚                   β”‚
β”‚  β”‚   Script    β”‚    β”‚  (renderer)  β”‚                   β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                   β”‚
β”‚                            β”‚                            β”‚
β”‚                            β–Ό                            β”‚
β”‚                     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                    β”‚
β”‚                     β”‚   Blaxel     β”‚                    β”‚
β”‚                     β”‚  Sandbox API β”‚                    β”‚
β”‚                     β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                    β”‚
β”‚                            β”‚                            β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                             β”‚
                             β–Ό
              β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
              β”‚  Blaxel Cloud                β”‚
              β”‚                              β”‚
              β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”‚
              β”‚  β”‚ Custom Manim Image     β”‚  β”‚
              β”‚  β”‚ βœ… Manim pre-installed  β”‚  β”‚
              β”‚  β”‚ βœ… FFmpeg ready         β”‚  β”‚
              β”‚  β”‚ βœ… LaTeX included       β”‚  β”‚
              β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β”‚
              β”‚           β”‚                  β”‚
              β”‚           β–Ό                  β”‚
              β”‚  ⚑ Start (< 10 sec)        β”‚
              β”‚  🎬 Render (works!)         β”‚
              β”‚  βœ… Output video            β”‚
              β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Configuration Changes

Environment Variables

Variable Before After Required
BLAXEL_API_KEY Optional Required Yes
BL_WORKSPACE N/A Optional No
MANIM_SANDBOX_IMAGE N/A Required Yes

Code Changes

The renderer code now uses the custom image:

# Before
sandbox = await SandboxInstance.create({
    "name": f"manim-render-{scene_name}",
    "image": "blaxel/py-app:latest",
    "memory": 4096,
})

# After
MANIM_SANDBOX_IMAGE = os.getenv("MANIM_SANDBOX_IMAGE")

sandbox = await SandboxInstance.create({
    "name": f"manim-render-{scene_name}",
    "image": MANIM_SANDBOX_IMAGE,  # Your custom image
    "memory": 4096,
})

Render Flow

# Before: Install then render
1. Create generic sandbox
2. ⏱️ Install Manim (3 minutes)
3. ⏱️ Try to install FFmpeg (fails)
4. Upload code
5. ❌ Render (error - no FFmpeg)

# After: Just render
1. Create custom sandbox (Manim + FFmpeg ready)
2. Upload code
3. βœ… Render (works immediately)
4. Download result

Testing

Test Local Build

# Build and test locally
make -f Makefile.sandbox build
make -f Makefile.sandbox run
make -f Makefile.sandbox test

Expected output:

βœ“ Manim is installed and working
βœ“ FFmpeg is installed and working

Test Blaxel Deployment

# Deploy to Blaxel
bl deploy

# Check deployment
bl get sandboxes

# Test in cloud
bl connect sandbox manim-sandbox
# Inside sandbox:
manim --version
ffmpeg -version

Test Full Pipeline

# Generate an animation
python main_new.py

# Check logs for:
# - "Creating Blaxel sandbox"
# - No "Installing Manim" messages
# - "Successfully rendered animation"

Rollback Plan

If you need to rollback to local rendering:

Option 1: Keep Using Cloud but Fallback Image

Remove from .env:

# MANIM_SANDBOX_IMAGE=...  # Comment out

The code will fallback to blaxel/py-app:latest (but slower).

Option 2: Complete Rollback to Local

In mcp_servers/renderer.py, find the render_manim_animation function around line 374:

# Change from:
return await _render_manim_with_sandbox(...)

# To:
return await _render_manim_locally(...)

This completely disables Blaxel and uses local Manim.

Option 3: Environment Flag

You could add a flag to toggle between local and cloud:

USE_CLOUD_RENDERING = os.getenv("USE_CLOUD_RENDERING", "true").lower() == "true"

if USE_CLOUD_RENDERING and BLAXEL_API_KEY:
    return await _render_manim_with_sandbox(...)
else:
    return await _render_manim_locally(...)

FAQ

Q: Do I need Manim installed locally anymore?

A: No! That's the beauty of this approach. Your local machine only needs Python and the Blaxel SDK. All rendering happens in the cloud.

Q: How much does this cost?

A: You pay for sandbox usage time. With pre-installed dependencies, rendering is much faster, so costs are actually lower than the runtime-installation approach.

Q: Can I still render locally?

A: Yes. The local rendering code is still in _render_manim_locally(). You can switch back anytime.

Q: What if Blaxel is down?

A: Implement the rollback to local rendering as described above.

Q: How do I update the sandbox image?

A: Rebuild and redeploy:

# Make changes to Dockerfile.sandbox
# Then:
./deploy_sandbox.sh

Q: Can I use a different base image?

A: Yes. Edit Dockerfile.sandbox to use any base image. Just ensure the Blaxel sandbox API is included.

Q: How do I add more LaTeX packages?

A: Update Dockerfile.sandbox:

RUN apt-get install -y \
    texlive-full \  # Complete LaTeX distribution
    && rm -rf /var/lib/apt/lists/*

Then rebuild and redeploy.

Q: What about Python package versions?

A: They're pinned in the Dockerfile. To update:

RUN pip install manim==0.18.2  # Specific version

Q: Can I test without deploying?

A: Yes! Use the local Docker testing:

make -f Makefile.sandbox build
make -f Makefile.sandbox run
make -f Makefile.sandbox test

Q: How do I debug render failures?

A:

  1. Check sandbox logs: bl logs
  2. Connect to sandbox: bl connect sandbox <name>
  3. Check process logs in the renderer code
  4. Test Manim command manually in sandbox

Q: Can I run multiple renders in parallel?

A: Yes! Each render creates a unique sandbox, so they run in parallel automatically.

Resources

Support

If you encounter issues:

  1. Check this migration guide
  2. Review BLAXEL_SANDBOX_SETUP.md troubleshooting section
  3. Test locally first: make -f Makefile.sandbox test
  4. Verify environment variables: echo $MANIM_SANDBOX_IMAGE
  5. Check Blaxel status: bl get sandboxes

Next Steps

After successful migration:

  1. βœ… Remove local Manim installation (optional)
  2. βœ… Update your documentation
  3. βœ… Train team on new workflow
  4. βœ… Set up CI/CD with Blaxel
  5. βœ… Monitor usage and costs
  6. βœ… Optimize sandbox memory/timeout settings

Migration Complete! πŸŽ‰

You're now rendering animations in the cloud with Blaxel sandboxes. Enjoy faster, more reliable, and more secure animation generation!