Spaces:
Sleeping
A newer version of the Gradio SDK is available: 6.13.0
UMCP Test Plan for Claude Desktop
Connection: localhost:9877 | PIN: 4336 Server: KarianaUMCP v1.0.0 Test Duration: 15-20 minutes
Pre-Test Checklist
- Unreal Engine is running
- KarianaUMCP plugin is loaded
- Socket server is active on port 9877
- Claude Desktop is configured to connect to localhost:9877
- Current level has some space for spawning actors
Test Suite 1: Basic Connectivity (2 min)
Test 1.1: Ping Server
Prompt for Claude:
Test the UMCP server connection with a ping command. Show me the server information.
Expected Response:
- Server responds with "pong"
- Server name: KarianaUMCP
- Version: 1.0.0
Pass Criteria: ✓ Connection successful, server info returned
Test 1.2: List Available Tools
Prompt for Claude:
List all available UMCP tools and categorize them by functionality.
Expected Response:
- 29 tools listed
- Categories: Actors, Assets, Blueprints, Editor, Skills, Python, Screenshot, System
Pass Criteria: ✓ All 29 tools discovered and categorized
Test Suite 2: Actor Lifecycle (5 min)
Test 2.1: Spawn Actor
Prompt for Claude:
Spawn a cube actor at location (0, 0, 100) and name it "TestCube1".
Expected Response:
- Actor spawned successfully
- Name: TestCube1
- Location confirmed at (0, 0, 100)
Pass Criteria: ✓ Actor appears in World Outliner
Test 2.2: List Actors
Prompt for Claude:
List all actors in the current level.
Expected Response:
- Actor list includes "TestCube1"
- Other actors: DirectionalLight, SkyLight, PlayerStart, etc.
Pass Criteria: ✓ TestCube1 is in the list
Test 2.3: Move Actor
Prompt for Claude:
Move TestCube1 to location (500, 500, 100).
Expected Response:
- Location updated successfully
- New position confirmed
Pass Criteria: ✓ Actor moved in viewport (verify visually)
Test 2.4: Rotate Actor
Prompt for Claude:
Rotate TestCube1 45 degrees around the Z axis (yaw).
Expected Response:
- Rotation updated successfully
- New rotation: [0, 45, 0]
Pass Criteria: ✓ Actor rotated visually
Test 2.5: Scale Actor
Prompt for Claude:
Scale TestCube1 to 2x its current size in all dimensions.
Expected Response:
- Scale updated successfully
- New scale: [2, 2, 2]
Pass Criteria: ✓ Actor is larger (visual confirmation)
Test 2.6: Delete Actor
Prompt for Claude:
Delete TestCube1.
Expected Response:
- Actor deleted successfully
Pass Criteria: ✓ TestCube1 removed from World Outliner
Test Suite 3: Scene Discovery (3 min)
Test 3.1: Current Level Info
Prompt for Claude:
What level am I currently working in? How many actors does it have?
Expected Response:
- Level name (e.g., "Untitled_1")
- Actor count (e.g., 9 actors)
- Level path
Pass Criteria: ✓ Accurate level information returned
Test 3.2: Asset Discovery
Prompt for Claude:
List all assets in the /Game folder. How many are there?
Expected Response:
- Asset list with names, paths, and classes
- Count: ~61 assets
- Includes: Materials, Meshes, Blueprints, Animations
Pass Criteria: ✓ Asset list matches Content Browser
Test 3.3: Capture Screenshot
Prompt for Claude:
Capture a screenshot of the current viewport at 1920x1080 resolution.
Expected Response:
- Screenshot captured
- Base64 encoded image data returned (or saved to file)
Pass Criteria: ✓ Screenshot file created or data returned
Test Suite 4: Python Execution (3 min)
Test 4.1: Simple Python
Prompt for Claude:
Execute this Python code in Unreal Engine:
print("Hello from Claude Desktop!")
Expected Response:
- Code executed successfully
- Output: "Hello from Claude Desktop!"
Pass Criteria: ✓ Python output returned
Test 4.2: Query Unreal API
Prompt for Claude:
Use Python to get the Unreal Engine version.
Expected Response:
- Engine version string (e.g., "5.6.0")
Pass Criteria: ✓ Version matches running UE version
Test 4.3: Create Actor via Python
Prompt for Claude:
Use Python to spawn a sphere at (1000, 0, 100) and name it "PythonSphere".
Expected Response:
- Actor spawned via Python
- Name: PythonSphere
- Location: (1000, 0, 100)
Pass Criteria: ✓ PythonSphere appears in World Outliner
Test Suite 5: Skill System (3 min)
Test 5.1: List Skills
Prompt for Claude:
What skills are available in the UMCP system?
Expected Response:
- 6 skills listed:
- screenshot
- blueprint-helper
- spawn-actor
- level-organizer
- organize_level
- create_lighting_setup
Pass Criteria: ✓ All 6 skills discovered
Test 5.2: Execute Screenshot Skill
Prompt for Claude:
Use the screenshot skill to capture a high-quality render of the viewport.
Expected Response:
- Skill executed successfully
- High-quality screenshot generated
Pass Criteria: ✓ Screenshot created with higher quality than basic capture
Test 5.3: Execute Level Organizer Skill
Prompt for Claude:
Use the level-organizer skill to organize the World Outliner.
Expected Response:
- Actors organized into folders
- Folders created by type (Lights, Geometry, etc.)
Pass Criteria: ✓ World Outliner shows folder hierarchy
Test Suite 6: Editor Control (2 min)
Test 6.1: Play in Editor
Prompt for Claude:
Start Play in Editor mode.
Expected Response:
- PIE started successfully
- Editor enters play mode
Pass Criteria: ✓ Game starts playing in viewport
Test 6.2: Stop Play in Editor
Prompt for Claude:
Stop Play in Editor.
Expected Response:
- PIE stopped successfully
- Editor returns to edit mode
Pass Criteria: ✓ Game stops, editor returns to normal
Test Suite 7: Complex Workflows (5 min)
Test 7.1: Create Scene with Multiple Actors
Prompt for Claude:
Create a simple scene with:
- A floor plane at (0, 0, 0) scaled to (10, 10, 1)
- Three cubes at (0, 0, 100), (200, 0, 100), (400, 0, 100)
- Name them Floor, Cube1, Cube2, Cube3
Expected Response:
- All 4 actors spawned
- Positioned correctly
- Named correctly
Pass Criteria: ✓ All actors visible in viewport at correct positions
Test 7.2: Batch Transformation
Prompt for Claude:
Move all three cubes up by 50 units (Z+50).
Expected Response:
- All cubes moved up
- New Z positions: 150, 150, 150
Pass Criteria: ✓ All cubes elevated visually
Test 7.3: Scene Cleanup
Prompt for Claude:
Delete all actors I just created (Floor, Cube1, Cube2, Cube3).
Expected Response:
- All actors deleted
Pass Criteria: ✓ Actors removed from World Outliner
Test 7.4: Asset Query and Material Creation
Prompt for Claude:
Find all materials in the project, then create a new material called "M_TestMaterial".
Expected Response:
- Material list returned
- New material created
- Path: /Game/Materials/M_TestMaterial (or similar)
Pass Criteria: ✓ New material appears in Content Browser
Test Suite 8: Error Handling (2 min)
Test 8.1: Invalid Actor Name
Prompt for Claude:
Try to delete an actor named "NonExistentActor".
Expected Response:
- Error message: Actor not found
- Graceful error handling
Pass Criteria: ✓ Error message returned, no crash
Test 8.2: Invalid Command
Prompt for Claude:
Try to execute a non-existent command called "do_magic_things".
Expected Response:
- Error message: Unknown command
- List of available commands
Pass Criteria: ✓ Helpful error message with alternatives
Test 8.3: Malformed Parameters
Prompt for Claude:
Try to spawn an actor without specifying the actor_class parameter.
Expected Response:
- Error message: Missing required parameter
- Parameter documentation
Pass Criteria: ✓ Clear error about missing parameter
Test Suite 9: Performance & Stress (Optional, 3 min)
Test 9.1: Batch Spawn
Prompt for Claude:
Spawn 100 cubes in a 10x10 grid pattern on the ground plane.
Expected Response:
- 100 actors spawned
- Grid pattern visible
Pass Criteria: ✓ All actors spawned, no timeout
Test 9.2: Large Asset List
Prompt for Claude:
List all assets in the entire Content folder, including all subfolders.
Expected Response:
- Complete asset list
- Response time <5 seconds
Pass Criteria: ✓ All assets returned without timeout
Test Results Summary
Scorecard
| Test Suite | Tests Passed | Tests Failed | Pass Rate |
|---|---|---|---|
| 1. Basic Connectivity | __ / 2 | __ | __% |
| 2. Actor Lifecycle | __ / 6 | __ | __% |
| 3. Scene Discovery | __ / 3 | __ | __% |
| 4. Python Execution | __ / 3 | __ | __% |
| 5. Skill System | __ / 3 | __ | __% |
| 6. Editor Control | __ / 2 | __ | __% |
| 7. Complex Workflows | __ / 4 | __ | __% |
| 8. Error Handling | __ / 3 | __ | __% |
| 9. Performance (Optional) | __ / 2 | __ | __% |
| TOTAL | __ / 28 | __ | __% |
Target Pass Rate: ≥ 90% (25/28 tests)
Issues Encountered
Issue 1:
- Test: _________
- Description: _________
- Severity: Critical / Major / Minor
- Workaround: _________
Issue 2:
- Test: _________
- Description: _________
- Severity: Critical / Major / Minor
- Workaround: _________
Issue 3:
- Test: _________
- Description: _________
- Severity: Critical / Major / Minor
- Workaround: _________
Observations
What Worked Well:
What Needs Improvement:
Unexpected Behaviors:
Performance Metrics
| Metric | Value |
|---|---|
| Average Response Time | _____ ms |
| Longest Response Time | _____ ms (Test: _____) |
| Failed Requests | _____ |
| Timeout Events | _____ |
| Server Uptime | _____ seconds |
Next Steps
Based on test results:
If pass rate ≥ 90%:
- System is production-ready
- Begin real-world usage with Claude Desktop
- Monitor for edge cases
If pass rate 70-89%:
- Investigate failed tests
- Check parameter documentation
- Review error logs in UE
If pass rate < 70%:
- Critical issues detected
- Review server configuration
- Check plugin installation
- Verify Unreal Engine version compatibility
Recommended Follow-Up Tests
After completing this test plan, consider:
Multi-Instance Testing
- Test with multiple Unreal instances
- Verify instance discovery
Long-Running Operations
- Test large-scale scene creation
- Monitor memory usage
Blueprint Automation
- Test blueprint creation
- Test auto-wiring
Asset Import Pipeline
- Test asset import
- Test batch import
Real-World Workflows
- Level design tasks
- Asset organization
- Scene setup automation
Test Plan Version: 1.0.0 Last Updated: 2025-11-30 Prepared By: MCP Tool Analyzer Agent Estimated Duration: 15-20 minutes (core tests), 30+ minutes (with optional tests)