# Unity Agent A Python package that generates complete, playable **Unity 2022.3 LTS** projects on disk. The agent exposes a registry of code-generation tools that emit C# scripts (MonoBehaviours), `.unity` scene files, project configuration (manifest, ProjectSettings, .meta) and a full open-world city example game. The output is a real Unity project you can open in Unity Hub and press Play. ## What it does * Writes complete, compilable **C#** MonoBehaviours (player, vehicle, camera, AI, weapons, inventory, quests, save system, ...). * Scaffolds a Unity project structure (`Assets/`, `Packages/manifest.json`, `ProjectSettings/`, `.meta` files) exactly the way Unity expects. * Provides a **system prompt** that teaches an LLM to act like a senior Unity engineer and to call the right tool in the right order. * Ships a **knowledge base** of focused markdown references (physics, navigation, input, UI, audio, rendering, open-world design). * Runs **static QA** against generated projects to catch unbalanced braces, missing meta files, missing manifest, etc. * Comes with a fully working **open-world city example** generated end-to-end from the tool layer. ## Installation ```bash git clone unity-agent cd unity-agent pip install -e . # optional; the package also works with PYTHONPATH=. ``` The package has **no third-party Python dependencies** -- it only uses the standard library. Python 3.9+ is required. ## Quick start ### Generate the bundled example ```bash # Generate the full open-world city example into examples/open_world_city/ python -m unity_agent generate --game open_world_city --preset open_world_city --output-dir examples # Run static QA on it python -m unity_agent qa examples/open_world_city ``` Open `examples/open_world_city/` in Unity Hub (2022.3 LTS or newer) and press Play. ### Use the package from Python ```python from unity_agent.config import Settings from unity_agent.transport.unity_transport import UnityTransport from unity_agent.tools.unity_tools import ( SetupUnityProjectTool, CreatePlayerControllerTool, CreateProceduralCityTool, WriteSceneFileTool, ) settings = Settings(output_dir="my_games") transport = UnityTransport(settings) SetupUnityProjectTool(settings, transport).run(project_name="MyGame") CreatePlayerControllerTool(settings, transport).run(project_name="MyGame") CreateProceduralCityTool(settings, transport).run(project_name="MyGame") WriteSceneFileTool(settings, transport).run(project_name="MyGame", scene_name="MainScene") print(transport.summary()) ``` ### CLI ```bash python -m unity_agent --help # Commands python -m unity_agent list-tools python -m unity_agent setup --project-name MyGame --output-dir my_games python -m unity_agent generate --game MyGame --preset open_world_city python -m unity_agent generate --game MyFPS --preset fps_arena python -m unity_agent generate --game Minimal --preset minimal python -m unity_agent qa path/to/MyGame ``` ## Project layout ``` unity-agent/ unity_agent/ __init__.py # Package entry point __main__.py # CLI: `python -m unity_agent ...` config.py # Settings (Unity path, LLM keys, output dirs) tests.py # Self-test suite (run with `python -m unity_agent.tests`) transport/ __init__.py unity_transport.py # Writes .cs / .unity / .meta / config files tools/ __init__.py base.py # ToolBase + ToolRegistry + @register_tool unity_tools.py # 24 tools (see below) orchestrator/ __init__.py prompts.py # SYSTEM_PROMPT + build_user_prompt + tool_catalog knowledge/ README.md __init__.py # load_entry / list_entries / load_all entries/ unity_fundamentals.md unity_physics.md unity_rendering.md unity_navigation.md unity_input_system.md unity_ui.md unity_audio.md open_world_design.md qa/ __init__.py # ProjectQA + Issue + QAReport + CLI examples/ generate_example.py # Regenerates examples/open_world_city/ open_world_city/ # A complete generated Unity project Assets/Scripts/*.cs Assets/Scenes/MainScene.unity Packages/manifest.json ProjectSettings/* README.md pyproject.toml README.md ``` ## Tools (24 total) | Tool | Output | |-------------------------------|--------------------------------------------------------------| | `setup_unity_project` | Project scaffold (manifest, ProjectSettings, asmdef) | | `create_player_controller` | PlayerController.cs (WASD + jump + physics) | | `create_vehicle_controller` | VehicleController.cs (arcade car physics) | | `create_procedural_city` | CityGenerator.cs + BuildingGenerator.cs (50+ buildings) | | `create_third_person_camera` | ThirdPersonCamera.cs (smooth follow + collision) | | `create_fps_controller` | FPSController.cs (mouse-look + WASD + jump) | | `create_day_night_cycle` | DayNightCycle.cs (sun rotation + sky/ambient lerp) | | `create_ai_npc` | NPCController.cs (NavMesh wander/chase FSM) | | `create_pickup_system` | PickupSystem.cs + Pickup.cs (score + collectibles) | | `create_health_system` | HealthSystem.cs + HealthBar.cs | | `create_weapon_system` | WeaponSystem.cs (raycast hitscan + ammo + reload) | | `create_audio_manager` | AudioManager.cs (SFX + music singleton) | | `create_ui_manager` | UIManager.cs (TMP HUD: score / health / messages) | | `create_terrain_generator` | TerrainGenerator.cs (Perlin heightmap + splat) | | `create_water_shader` | WaterMaterial.cs (animated standard-material water) | | `create_particle_effects` | ParticleSpawner.cs (bursts + explosion helper) | | `create_save_system` | SaveSystem.cs (JSON save/load to persistentDataPath) | | `create_inventory_system` | InventorySystem.cs (stacks + capacity + events) | | `create_quest_system` | QuestSystem.cs (state machine + progress + completion) | | `create_building_generator` | BuildingGenerator.cs (walls + emissive windows + collider) | | `create_road_network` | RoadNetwork.cs (grid layout + intersections) | | `write_csharp_script` | Any C# script to Assets/Scripts/ | | `write_scene_file` | .unity scene file to Assets/Scenes/ | | `generate_complete_game` | Full game in one call (presets: open_world_city, fps_arena, minimal) | ## The system prompt `unity_agent.orchestrator.prompts.SYSTEM_PROMPT` is a long, prescriptive prompt that teaches the LLM to: * Think like a senior Unity engineer (lifecycle, physics step, idiomatic C#). * Know the tool catalog and the recommended workflow (scaffold -> components -> bootstrap script -> scene). * Always emit code that compiles on the first try in Unity 2022.3 LTS. * Prefer procedural content (runtime `Start()` generation) over inspector-wired prefabs so the example "just plays" on open. * Use the `UnityAgent` namespace and the `UnityAgent.Scripts` asmdef. Use it like this from your own LLM driver: ```python from unity_agent.orchestrator import SYSTEM_PROMPT, build_user_prompt prompt = build_user_prompt("Build me an open-world city game I can drive around in") # Send SYSTEM_PROMPT + prompt to your LLM, then dispatch tool calls. ``` ## Knowledge base Eight focused markdown references in `unity_agent/knowledge/entries/`: 1. `unity_fundamentals.md` -- MonoBehaviour lifecycle, GameObjects, components, coroutines, inspector best practices. 2. `unity_physics.md` -- Rigidbody, colliders, raycasting, ForceModes, physics vs frame step. 3. `unity_rendering.md` -- Materials, Standard shader, lighting, post- processing, LOD, fog. 4. `unity_navigation.md` -- NavMesh, NavMeshAgent, AI state machines, NavMeshObstacle, off-mesh links. 5. `unity_input_system.md` -- Legacy Input Manager, mouse-look recipes, new Input System, touch. 6. `unity_ui.md` -- Canvas, anchoring, HUD pattern, TMP, performance. 7. `unity_audio.md` -- AudioSource, 3D audio, AudioMixer, AudioManager singleton. 8. `open_world_design.md` -- Procedural generation, LOD, streaming, day/night cycle, save systems. Load them programmatically: ```python from unity_agent.knowledge import list_entries, load_entry, load_all for slug in list_entries(): print(slug, len(load_entry(slug))) ``` ## QA `unity_agent.qa.ProjectQA` runs static checks against a generated project: * Required directories exist (`Assets/`, `Assets/Scripts/`, `Packages/`, `ProjectSettings/`). * Required files exist (`Packages/manifest.json`, `ProjectSettings/ProjectVersion.txt`, `ProjectSettings/ProjectSettings.asset`). * Every `.cs` file has balanced braces and at least one type declaration. * Every `.cs` and `.unity` file has a sibling `.meta`. * Scene files start with the `%YAML` header. * `manifest.json` parses as JSON and has a `dependencies` object. ```python from unity_agent.qa import ProjectQA report = ProjectQA("examples/open_world_city").run() print(report.summary()) print("PASS" if report.passed else "FAIL") ``` ## Tests ```bash python -m unity_agent.tests ``` Verifies the tool registry has 24 entries, the project scaffolder writes the expected files, generated C# has balanced braces, the full game generator produces 12+ scripts, and QA passes on a freshly generated project. ## Requirements * Python 3.9+ (no third-party dependencies). * Unity Hub + Unity 2022.3 LTS (or newer) to actually open and play the generated projects. The Python layer works without Unity installed. ## License MIT.