| # Unity Agent |
|
|
| A Python package that generates complete, playable **Unity 2022.3 LTS** |
| projects on disk. The agent exposes a registry of code-generation tools |
| that emit C# scripts (MonoBehaviours), `.unity` scene files, project |
| configuration (manifest, ProjectSettings, .meta) and a full open-world |
| city example game. The output is a real Unity project you can open in |
| Unity Hub and press Play. |
|
|
| ## What it does |
|
|
| * Writes complete, compilable **C#** MonoBehaviours (player, vehicle, |
| camera, AI, weapons, inventory, quests, save system, ...). |
| * Scaffolds a Unity project structure (`Assets/`, `Packages/manifest.json`, |
| `ProjectSettings/`, `.meta` files) exactly the way Unity expects. |
| * Provides a **system prompt** that teaches an LLM to act like a senior |
| Unity engineer and to call the right tool in the right order. |
| * Ships a **knowledge base** of focused markdown references |
| (physics, navigation, input, UI, audio, rendering, open-world design). |
| * Runs **static QA** against generated projects to catch unbalanced |
| braces, missing meta files, missing manifest, etc. |
| * Comes with a fully working **open-world city example** generated |
| end-to-end from the tool layer. |
|
|
| ## Installation |
|
|
| ```bash |
| git clone <this-repo> unity-agent |
| cd unity-agent |
| pip install -e . # optional; the package also works with PYTHONPATH=. |
| ``` |
|
|
| The package has **no third-party Python dependencies** -- it only uses the |
| standard library. Python 3.9+ is required. |
|
|
| ## Quick start |
|
|
| ### Generate the bundled example |
|
|
| ```bash |
| # Generate the full open-world city example into examples/open_world_city/ |
| python -m unity_agent generate --game open_world_city --preset open_world_city --output-dir examples |
| |
| # Run static QA on it |
| python -m unity_agent qa examples/open_world_city |
| ``` |
|
|
| Open `examples/open_world_city/` in Unity Hub (2022.3 LTS or newer) and |
| press Play. |
|
|
| ### Use the package from Python |
|
|
| ```python |
| from unity_agent.config import Settings |
| from unity_agent.transport.unity_transport import UnityTransport |
| from unity_agent.tools.unity_tools import ( |
| SetupUnityProjectTool, |
| CreatePlayerControllerTool, |
| CreateProceduralCityTool, |
| WriteSceneFileTool, |
| ) |
| |
| settings = Settings(output_dir="my_games") |
| transport = UnityTransport(settings) |
| |
| SetupUnityProjectTool(settings, transport).run(project_name="MyGame") |
| CreatePlayerControllerTool(settings, transport).run(project_name="MyGame") |
| CreateProceduralCityTool(settings, transport).run(project_name="MyGame") |
| WriteSceneFileTool(settings, transport).run(project_name="MyGame", scene_name="MainScene") |
| |
| print(transport.summary()) |
| ``` |
|
|
| ### CLI |
|
|
| ```bash |
| python -m unity_agent --help |
| |
| # Commands |
| python -m unity_agent list-tools |
| python -m unity_agent setup --project-name MyGame --output-dir my_games |
| python -m unity_agent generate --game MyGame --preset open_world_city |
| python -m unity_agent generate --game MyFPS --preset fps_arena |
| python -m unity_agent generate --game Minimal --preset minimal |
| python -m unity_agent qa path/to/MyGame |
| ``` |
|
|
| ## Project layout |
|
|
| ``` |
| unity-agent/ |
| unity_agent/ |
| __init__.py # Package entry point |
| __main__.py # CLI: `python -m unity_agent ...` |
| config.py # Settings (Unity path, LLM keys, output dirs) |
| tests.py # Self-test suite (run with `python -m unity_agent.tests`) |
| transport/ |
| __init__.py |
| unity_transport.py # Writes .cs / .unity / .meta / config files |
| tools/ |
| __init__.py |
| base.py # ToolBase + ToolRegistry + @register_tool |
| unity_tools.py # 24 tools (see below) |
| orchestrator/ |
| __init__.py |
| prompts.py # SYSTEM_PROMPT + build_user_prompt + tool_catalog |
| knowledge/ |
| README.md |
| __init__.py # load_entry / list_entries / load_all |
| entries/ |
| unity_fundamentals.md |
| unity_physics.md |
| unity_rendering.md |
| unity_navigation.md |
| unity_input_system.md |
| unity_ui.md |
| unity_audio.md |
| open_world_design.md |
| qa/ |
| __init__.py # ProjectQA + Issue + QAReport + CLI |
| examples/ |
| generate_example.py # Regenerates examples/open_world_city/ |
| open_world_city/ # A complete generated Unity project |
| Assets/Scripts/*.cs |
| Assets/Scenes/MainScene.unity |
| Packages/manifest.json |
| ProjectSettings/* |
| README.md |
| pyproject.toml |
| README.md |
| ``` |
|
|
| ## Tools (24 total) |
|
|
| | Tool | Output | |
| |-------------------------------|--------------------------------------------------------------| |
| | `setup_unity_project` | Project scaffold (manifest, ProjectSettings, asmdef) | |
| | `create_player_controller` | PlayerController.cs (WASD + jump + physics) | |
| | `create_vehicle_controller` | VehicleController.cs (arcade car physics) | |
| | `create_procedural_city` | CityGenerator.cs + BuildingGenerator.cs (50+ buildings) | |
| | `create_third_person_camera` | ThirdPersonCamera.cs (smooth follow + collision) | |
| | `create_fps_controller` | FPSController.cs (mouse-look + WASD + jump) | |
| | `create_day_night_cycle` | DayNightCycle.cs (sun rotation + sky/ambient lerp) | |
| | `create_ai_npc` | NPCController.cs (NavMesh wander/chase FSM) | |
| | `create_pickup_system` | PickupSystem.cs + Pickup.cs (score + collectibles) | |
| | `create_health_system` | HealthSystem.cs + HealthBar.cs | |
| | `create_weapon_system` | WeaponSystem.cs (raycast hitscan + ammo + reload) | |
| | `create_audio_manager` | AudioManager.cs (SFX + music singleton) | |
| | `create_ui_manager` | UIManager.cs (TMP HUD: score / health / messages) | |
| | `create_terrain_generator` | TerrainGenerator.cs (Perlin heightmap + splat) | |
| | `create_water_shader` | WaterMaterial.cs (animated standard-material water) | |
| | `create_particle_effects` | ParticleSpawner.cs (bursts + explosion helper) | |
| | `create_save_system` | SaveSystem.cs (JSON save/load to persistentDataPath) | |
| | `create_inventory_system` | InventorySystem.cs (stacks + capacity + events) | |
| | `create_quest_system` | QuestSystem.cs (state machine + progress + completion) | |
| | `create_building_generator` | BuildingGenerator.cs (walls + emissive windows + collider) | |
| | `create_road_network` | RoadNetwork.cs (grid layout + intersections) | |
| | `write_csharp_script` | Any C# script to Assets/Scripts/ | |
| | `write_scene_file` | .unity scene file to Assets/Scenes/ | |
| | `generate_complete_game` | Full game in one call (presets: open_world_city, fps_arena, minimal) | |
| |
| ## The system prompt |
| |
| `unity_agent.orchestrator.prompts.SYSTEM_PROMPT` is a long, prescriptive |
| prompt that teaches the LLM to: |
| |
| * Think like a senior Unity engineer (lifecycle, physics step, idiomatic |
| C#). |
| * Know the tool catalog and the recommended workflow (scaffold -> |
| components -> bootstrap script -> scene). |
| * Always emit code that compiles on the first try in Unity 2022.3 LTS. |
| * Prefer procedural content (runtime `Start()` generation) over |
| inspector-wired prefabs so the example "just plays" on open. |
| * Use the `UnityAgent` namespace and the `UnityAgent.Scripts` asmdef. |
| |
| Use it like this from your own LLM driver: |
| |
| ```python |
| from unity_agent.orchestrator import SYSTEM_PROMPT, build_user_prompt |
| |
| prompt = build_user_prompt("Build me an open-world city game I can drive around in") |
| # Send SYSTEM_PROMPT + prompt to your LLM, then dispatch tool calls. |
| ``` |
| |
| ## Knowledge base |
| |
| Eight focused markdown references in `unity_agent/knowledge/entries/`: |
| |
| 1. `unity_fundamentals.md` -- MonoBehaviour lifecycle, GameObjects, |
| components, coroutines, inspector best practices. |
| 2. `unity_physics.md` -- Rigidbody, colliders, raycasting, ForceModes, |
| physics vs frame step. |
| 3. `unity_rendering.md` -- Materials, Standard shader, lighting, post- |
| processing, LOD, fog. |
| 4. `unity_navigation.md` -- NavMesh, NavMeshAgent, AI state machines, |
| NavMeshObstacle, off-mesh links. |
| 5. `unity_input_system.md` -- Legacy Input Manager, mouse-look recipes, |
| new Input System, touch. |
| 6. `unity_ui.md` -- Canvas, anchoring, HUD pattern, TMP, performance. |
| 7. `unity_audio.md` -- AudioSource, 3D audio, AudioMixer, AudioManager |
| singleton. |
| 8. `open_world_design.md` -- Procedural generation, LOD, streaming, |
| day/night cycle, save systems. |
| |
| Load them programmatically: |
| |
| ```python |
| from unity_agent.knowledge import list_entries, load_entry, load_all |
|
|
| for slug in list_entries(): |
| print(slug, len(load_entry(slug))) |
| ``` |
| |
| ## QA |
| |
| `unity_agent.qa.ProjectQA` runs static checks against a generated project: |
| |
| * Required directories exist (`Assets/`, `Assets/Scripts/`, `Packages/`, |
| `ProjectSettings/`). |
| * Required files exist (`Packages/manifest.json`, |
| `ProjectSettings/ProjectVersion.txt`, |
| `ProjectSettings/ProjectSettings.asset`). |
| * Every `.cs` file has balanced braces and at least one type declaration. |
| * Every `.cs` and `.unity` file has a sibling `.meta`. |
| * Scene files start with the `%YAML` header. |
| * `manifest.json` parses as JSON and has a `dependencies` object. |
| |
| ```python |
| from unity_agent.qa import ProjectQA |
| report = ProjectQA("examples/open_world_city").run() |
| print(report.summary()) |
| print("PASS" if report.passed else "FAIL") |
| ``` |
| |
| ## Tests |
| |
| ```bash |
| python -m unity_agent.tests |
| ``` |
| |
| Verifies the tool registry has 24 entries, the project scaffolder writes |
| the expected files, generated C# has balanced braces, the full game |
| generator produces 12+ scripts, and QA passes on a freshly generated |
| project. |
| |
| ## Requirements |
| |
| * Python 3.9+ (no third-party dependencies). |
| * Unity Hub + Unity 2022.3 LTS (or newer) to actually open and play the |
| generated projects. The Python layer works without Unity installed. |
| |
| ## License |
| |
| MIT. |
| |
