test/test_environment_20260406_180407.log

2026-04-06 18:04:07 | INFO     | __main__ | Logging initialized. Log file: F:\RL\OpenEnv\Tool_forge_project\openenv-SSV\toolforge_env\test\test_environment_20260406_180407.log
2026-04-06 18:04:07 | INFO     | __main__ | Starting tests. Output log: F:\RL\OpenEnv\Tool_forge_project\openenv-SSV\toolforge_env\test\test_environment_20260406_180407.log
2026-04-06 18:04:07 | INFO     | __main__ | ================================================================================
2026-04-06 18:04:07 | INFO     | __main__ | TEST: Environment Reset and Step
2026-04-06 18:04:07 | INFO     | __main__ | ================================================================================
2026-04-06 18:04:07 | INFO     | __main__ | Task selector created successfully: <server.inputs.simulated.task_selector.TaskSelector object at 0x00000232A4D6ABD0>
2026-04-06 18:04:07 | INFO     | __main__ | ✓ Environment created successfully: <server.toolforge_env_environment.ToolforgeEnvironment object at 0x00000232C5715340>
2026-04-06 18:04:07 | INFO     | server.tools | Building fresh atomic tools list.
2026-04-06 18:04:07 | INFO     | __main__ | ✓ Reset successful
2026-04-06 18:04:07 | INFO     | __main__ |   Episode ID: test-episode-001
2026-04-06 18:04:07 | INFO     | __main__ |   Initial Task: easy-deploy-notify
2026-04-06 18:04:07 | INFO     | __main__ |   Task Prompt: Deploy the 'frontend-web' service version 'v2.1.0', check its health, and notify the '#deployments' channel that it is done.
2026-04-06 18:04:07 | INFO     | __main__ |   Task Difficulty: easy
2026-04-06 18:04:07 | INFO     | __main__ |   Required Slots: ['deployment_execution', 'deployment_verification', 'deployment_notification']
2026-04-06 18:04:07 | INFO     | __main__ |   Baseline Call Count: 3
2026-04-06 18:04:07 | INFO     | __main__ |   Available Tools: [{'name': 'deploy', 'description': 'Deploys a specified service or application version to a target environment.', 'is_macro': False, 'token_cost': 10, 'required_params': ['service_name', 'version'], 'params_schema': {'service_name': {'type': 'string', 'description': 'Name of the service to deploy'}, 'version': {'type': 'string', 'description': 'Version tag or commit hash'}}, 'composed_of': []}, {'name': 'healthcheck', 'description': 'Checks the current health status of a running service.', 'is_macro': False, 'token_cost': 2, 'required_params': ['service_name'], 'params_schema': {'service_name': {'type': 'string', 'description': 'Name of the service to check'}}, 'composed_of': []}, {'name': 'notify', 'description': 'Sends a notification to a specific channel (e.g., Slack, Email) regarding system status.', 'is_macro': False, 'token_cost': 1, 'required_params': ['channel', 'message'], 'params_schema': {'channel': {'type': 'string', 'description': 'Target communication channel'}, 'message': {'type': 'string', 'description': 'The message body to send'}}, 'composed_of': []}, {'name': 'rollback', 'description': 'Reverts a deployed service to the previous stable version if something goes wrong.', 'is_macro': False, 'token_cost': 15, 'required_params': ['service_name'], 'params_schema': {'service_name': {'type': 'string', 'description': 'Name of the service to rollback'}}, 'composed_of': []}, {'name': 'scale', 'description': 'Adjusts the number of running replicas for a given service.', 'is_macro': False, 'token_cost': 5, 'required_params': ['service_name', 'replicas'], 'params_schema': {'service_name': {'type': 'string', 'description': 'Name of the service to scale'}, 'replicas': {'type': 'integer', 'description': 'Desired number of running instances'}}, 'composed_of': []}, {'name': 'restart', 'description': 'Performs a rolling restart on a specified service.', 'is_macro': False, 'token_cost': 8, 'required_params': ['service_name'], 'params_schema': {'service_name': {'type': 'string', 'description': 'Name of the service to restart'}}, 'composed_of': []}]
2026-04-06 18:04:07 | INFO     | __main__ |   Done: False
2026-04-06 18:04:07 | INFO     | __main__ |   Reward: 0.0
2026-04-06 18:04:07 | INFO     | __main__ | 
================================================================================
2026-04-06 18:04:07 | INFO     | __main__ | THREE-STEP MACRO FLOW
2026-04-06 18:04:07 | INFO     | __main__ | ================================================================================
2026-04-06 18:04:07 | INFO     | __main__ | 
--- Step 1: Propose valid macro ---
2026-04-06 18:04:07 | INFO     | server.evaluation_pipeline | [step=1] Evaluation pipeline start: task_id=easy-deploy-notify, plan_calls=2
2026-04-06 18:04:07 | INFO     | server.evaluation_pipeline | [step=1] Stage-3 plan accuracy: score=-0.665, slot_completion=0.667, unnecessary_penalty=-0.000
2026-04-06 18:04:07 | INFO     | server.evaluation_pipeline | [step=1] Stage-4 token cost: used=12, baseline=60, efficiency_ratio=0.800, efficiency_score=0.800, macro_bonus=0.000
2026-04-06 18:04:07 | INFO     | toolforge_env.server.plan_evaluator | Final Reward: -0.226
2026-04-06 18:04:07 | INFO     | server.evaluation_pipeline | [step=1] Stage-5 final reward: -0.226
2026-04-06 18:04:07 | INFO     | server.toolforge_env_environment | Task advanced from 'easy-deploy-notify' to 'easy-deploy-restart'. Remaining tasks=0
2026-04-06 18:04:07 | INFO     | server.toolforge_env_environment | Macro approved: name='deploy_and_verify_macro', steps=['deploy', 'healthcheck'], token_cost=10
2026-04-06 18:04:07 | INFO     | __main__ | Step 1 reward=-0.226 done=False
2026-04-06 18:04:07 | INFO     | __main__ | Step 1 macro_decision=approved
2026-04-06 18:04:07 | INFO     | __main__ | 
--- Step 2: Use created macro ---
2026-04-06 18:04:07 | INFO     | server.evaluation_pipeline | [step=2] Evaluation pipeline start: task_id=easy-deploy-restart, plan_calls=1
2026-04-06 18:04:07 | INFO     | server.evaluation_pipeline | [step=2] Stage-3 plan accuracy: score=-0.957, slot_completion=0.200, unnecessary_penalty=-0.000
2026-04-06 18:04:07 | INFO     | server.evaluation_pipeline | [step=2] Stage-4 token cost: used=10, baseline=100, efficiency_ratio=0.900, efficiency_score=1.000, macro_bonus=0.100
2026-04-06 18:04:07 | INFO     | toolforge_env.server.plan_evaluator | Final Reward: -0.370
2026-04-06 18:04:07 | INFO     | server.evaluation_pipeline | [step=2] Stage-5 final reward: -0.370
2026-04-06 18:04:07 | INFO     | server.toolforge_env_environment | Task advanced from 'easy-deploy-restart' to 'easy-deploy-scale'. Remaining tasks=0
2026-04-06 18:04:07 | INFO     | __main__ | Step 2 reward=-0.370 done=False
2026-04-06 18:04:07 | INFO     | __main__ | Step 2 plan_accepted=True
2026-04-06 18:04:07 | INFO     | __main__ | 
--- Step 3: Propose invalid macro with wrong_tool ---
2026-04-06 18:04:07 | INFO     | server.evaluation_pipeline | [step=3] Evaluation pipeline start: task_id=easy-deploy-scale, plan_calls=1
2026-04-06 18:04:07 | INFO     | server.evaluation_pipeline | [step=3] Stage-3 plan accuracy: score=-0.941, slot_completion=0.250, unnecessary_penalty=-0.000
2026-04-06 18:04:07 | INFO     | server.evaluation_pipeline | [step=3] Stage-4 token cost: used=10, baseline=80, efficiency_ratio=0.875, efficiency_score=0.875, macro_bonus=0.000
2026-04-06 18:04:07 | INFO     | toolforge_env.server.plan_evaluator | Final Reward: -0.397
2026-04-06 18:04:07 | INFO     | server.evaluation_pipeline | [step=3] Stage-5 final reward: -0.397
2026-04-06 18:04:07 | INFO     | server.toolforge_env_environment | Episode complete. Final task 'easy-deploy-scale' finished; no tasks remain.
2026-04-06 18:04:07 | INFO     | server.toolforge_env_environment | Macro rejected: name='invalid_macro', reason='macro_contains_unknown_tools:wrong_tool'
2026-04-06 18:04:07 | INFO     | __main__ | Step 3 reward=-0.397 done=True
2026-04-06 18:04:07 | INFO     | __main__ | Step 3 macro_decision=rejected
2026-04-06 18:04:07 | INFO     | __main__ | Step 3 macro_reason=macro_contains_unknown_tools:wrong_tool
2026-04-06 18:04:07 | INFO     | __main__ | 
================================================================================
2026-04-06 18:04:07 | INFO     | __main__ | EPISODE SUMMARY
2026-04-06 18:04:07 | INFO     | __main__ | ================================================================================
2026-04-06 18:04:07 | INFO     | __main__ | Total Steps Executed: 3
2026-04-06 18:04:07 | INFO     | __main__ | Final State Step Count: 3
2026-04-06 18:04:07 | INFO     | __main__ | Tasks Completed: 3
2026-04-06 18:04:07 | INFO     | __main__ | Accepted Macros: 1
2026-04-06 18:04:07 | INFO     | __main__ | Rejected Macros: 1
2026-04-06 18:04:07 | INFO     | __main__ | Total Tokens Used: 32
2026-04-06 18:04:07 | INFO     | __main__ | Episode Done: True
2026-04-06 18:04:07 | INFO     | __main__ | Final Observation Done: True
2026-04-06 18:04:07 | INFO     | __main__ | 
Completed Tasks:
2026-04-06 18:04:07 | INFO     | __main__ |   - easy-deploy-notify (easy)
2026-04-06 18:04:07 | INFO     | __main__ |   - easy-deploy-restart (easy)
2026-04-06 18:04:07 | INFO     | __main__ |   - easy-deploy-scale (easy)
2026-04-06 18:04:07 | INFO     | __main__ | 
✓ All assertions passed!
2026-04-06 18:04:07 | INFO     | __main__ | ================================================================================
2026-04-06 18:04:07 | INFO     | __main__ | 

🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 
2026-04-06 18:04:07 | INFO     | __main__ | ALL TESTS PASSED!
2026-04-06 18:04:07 | INFO     | __main__ | 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉