|
|
--- |
|
|
title: SAM3D |
|
|
emoji: 😻 |
|
|
colorFrom: pink |
|
|
colorTo: blue |
|
|
sdk: gradio |
|
|
sdk_version: 6.0.1 |
|
|
app_file: app.py |
|
|
pinned: false |
|
|
license: mit |
|
|
short_description: Creating and editing 3d models using text prompts |
|
|
tags: |
|
|
- text-to-3d |
|
|
- mcp |
|
|
- gemini |
|
|
- sam-3d |
|
|
- building-mcp-track-multimodal |
|
|
--- |
|
|
|
|
|
# 🎨 Text to 3D – MCP Server |
|
|
|
|
|
Generate and edit 3D models from text prompts using **Gemini** + **Meta SAM-3D**. |
|
|
|
|
|
🎬 **Demo:** [[YouTube Link](https://drive.google.com/file/d/1tghBt4iaNEMtUT9Er0UdkfQrsy-3FxD0/view?usp=sharing)] |
|
|
📱 **Post:** [[X/LinkedIn Link](https://www.linkedin.com/feed/update/urn:li:activity:7401045673531420672/)] |
|
|
|
|
|
## 💡 Use Cases |
|
|
|
|
|
This tool enables rapid 3D asset creation through natural language: |
|
|
- **Game Developers** – Quickly prototype 3D assets by describing them ("a medieval sword", "a sci-fi helmet") |
|
|
- **Designers & Artists** – Generate concept models and iterate with text edits ("make it larger", "change color to gold") |
|
|
- **E-commerce** – Create product mockups from descriptions for visualization |
|
|
- **Education** – Generate 3D models for teaching (anatomy, architecture, objects) |
|
|
- **AI Agents** – Claude/Cursor can autonomously create 3D assets as part of larger workflows via MCP |
|
|
|
|
|
## 🚀 Future Scope |
|
|
|
|
|
- Multi-object scene generation |
|
|
- Style transfer for 3D models |
|
|
- Animation support |
|
|
- Direct 3D file editing without regeneration |
|
|
- Integration with game engines (Unity/Unreal) |
|
|
|
|
|
## 🏗️ Architecture |
|
|
|
|
|
``` |
|
|
┌─────────────────┐ MCP Protocol ┌─────────────────────┐ |
|
|
│ Claude Desktop │ ◄────────────────────► │ Gradio MCP Server │ |
|
|
│ Cursor / Cline │ │ (HF Space) │ |
|
|
└─────────────────┘ └──────────┬──────────┘ |
|
|
│ |
|
|
┌─────────────────────────────┼─────────────────────────────┐ |
|
|
▼ ▼ ▼ |
|
|
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ |
|
|
│ Gemini │ │ Gemini │ │ SAM-3D │ |
|
|
│ Generate │──────────────│ Edit │──────────────│ (Modal) │ |
|
|
│ Image │ │ / Remove │ │ 3D Recon │ |
|
|
└─────────────┘ └─────────────┘ └─────────────┘ |
|
|
``` |
|
|
|
|
|
**Flow:** `Text → Image → Remove BG → Grayscale Mask → SAM-3D → 3D Model (GLB/PLY)` |
|
|
|
|
|
## 🔌 MCP Tools |
|
|
|
|
|
| Tool | Description | |
|
|
|------|-------------| |
|
|
| `generate_3d` | Text prompt → 3D model | |
|
|
| `edit_3d` | Edit existing model with text | |
|
|
|
|
|
### Connect from Claude Desktop |
|
|
|
|
|
```json |
|
|
{ |
|
|
"mcpServers": { |
|
|
"text-to-3d": { |
|
|
"command": "npx", |
|
|
"args": ["mcp-remote", "https://MCP-1st-Birthday-text-to-3d-SAM3D.hf.space/gradio_api/mcp/sse"] |
|
|
} |
|
|
} |
|
|
} |
|
|
``` |
|
|
|
|
|
## 🛠️ Tech Stack |
|
|
|
|
|
| Component | Purpose | |
|
|
|-----------|---------| |
|
|
| Gemini 2.5 Flash | Image generation | |
|
|
| Gemini 3 Pro | Background removal & editing | |
|
|
| SAM-3D (Meta) | 3D reconstruction | |
|
|
| Modal | Serverless GPU | |
|
|
| Gradio | UI + MCP server | |
|
|
|
|
|
## 📦 Outputs |
|
|
|
|
|
- **GLB** – 3D mesh |
|
|
- **PLY** – Gaussian splat |
|
|
|
|
|
--- |
|
|
|
|
|
**Built for [MCP 1st Birthday Hackathon](https://huggingface.co/MCP-1st-Birthday)** 🎂 |
