You are an expert systems engineer, RL researcher, and backend architect.
Your task is to build a COMPLETE, PRODUCTION-GRADE OpenEnv environment for:
"Noise-aware, hardware-constrained quantum circuit optimization using reinforcement learning"
This is NOT a toy project. It must fully comply with OpenEnv specification and be deployable.
HARD REQUIREMENTS (DO NOT VIOLATE)
Must follow OpenEnv spec EXACTLY:
- step(action) -> observation, reward, done, info
- reset()
- state()
Use strict Pydantic models
Include openenv.yaml
Include 3+ tasks (easy, medium, hard)
Each task must have deterministic graders (0.0-1.0)
Include meaningful reward shaping (NOT sparse)
Include baseline inference.py (OpenAI client)
Include Dockerfile (must run)
Must be HF Spaces deployable
Must pass "openenv validate"
PROBLEM DEFINITION
We are building an RL environment where an agent learns to:
- Construct and optimize quantum circuits
- Maximize fidelity to a target state/unitary
- Minimize circuit depth and gate count
- Minimize noise impact
- Respect hardware connectivity constraints
Use Qiskit for simulation (statevector + noisy Aer simulator).
REQUIRED FILE STRUCTURE
envs/my_env/ |---- init.py |---- models.py |---- client.py |---- README.md |---- openenv.yaml |---- server/ |---- init.py |---- my_environment.py |---- graders/ | |---- fidelity.py | |---- efficiency.py | |---- noise.py | |---- constraints.py | |---- aggregate.py |---- tasks/ | |---- easy.py | |---- medium.py | |---- hard.py |---- app.py |---- Dockerfile
Also include:
- inference.py (root)
- requirements.txt
MODEL DEFINITIONS (STRICT)
Define:
Action:
- action_type: ADD, REMOVE, SWAP, PARAM, STOP
- gate
- qubits
- parameter (optional)
Observation:
- circuit (structured representation)
- fidelity (float)
- depth (int)
- gate_count (int)
- noise_estimate (float)
- valid_actions
State:
- full circuit object
- step count
- internal simulator state
ENVIRONMENT LOGIC
Implement:
reset()
- initialize empty circuit
- load task config (target, noise model, connectivity)
- return initial observation
step(action)
validate action
apply gate using Qiskit
simulate:
- ideal (statevector)
- noisy (Aer)
compute:
- fidelity
- depth
- noise estimate
compute reward via modular graders
return observation, reward, done, info
state()
- return full internal state
ACTION SPACE
- ADD gate (H, X, CNOT, RX, RZ)
- REMOVE gate
- SWAP qubits
- PARAM tuning (continuous angle)
- STOP
REWARD SYSTEM (VERY IMPORTANT)
Implement modular grading:
fidelity.py
- compute overlap with target state
efficiency.py
- penalize depth and gate count
noise.py
- penalize noisy circuits
constraints.py
- penalize invalid hardware usage
aggregate.py
- combine scores into final score (0-1)
Final reward:
- shaped reward = current_score - previous_score
TASKS (MANDATORY)
EASY
- Bell state (2 qubits)
- no noise
MEDIUM
- GHZ state (3 qubits)
- noise + connectivity constraints
HARD
- arbitrary unitary approximation
- strict depth + noisy simulation
Each task must:
- define target
- define constraints
- define grading thresholds
OPENENV INTEGRATION
Include openenv.yaml:
- name
- description
- tasks
- action/observation schema
Ensure environment passes openenv validate.
API SERVER
Use FastAPI:
- POST /reset
- POST /step
- GET /state
DOCKER
Dockerfile must:
- install qiskit, fastapi, uvicorn, openenv
- expose port
- run server
BASELINE INFERENCE
Create inference.py:
uses OpenAI client
reads:
- API_BASE_URL
- MODEL_NAME
- OPENAI_API_KEY
runs all 3 tasks
prints logs in EXACT format:
[START] [STEP] [END]
This is critical.
LOGGING FORMAT (STRICT)
Each step must log:
- step number
- action
- reward
- cumulative score
README REQUIREMENTS
Include:
- problem description
- real-world relevance (quantum compiler optimization)
- action space
- observation space
- reward design
- task descriptions
- setup instructions
- how to run inference
- expected baseline results
IMPORTANT DESIGN REQUIREMENTS
- Reward must NOT be sparse
- Must include partial progress signals
- Must penalize bad actions
- Must prevent infinite loops
- Must be deterministic (same input -> same score)
OUTPUT FORMAT
Generate ALL files with FULL CODE.
Do NOT:
- skip files
- leave TODOs
- give partial implementations
Everything must be runnable.
BONUS (IF POSSIBLE)
- Add circuit visualization utility
- Add simple logging middleware
- Add reproducibility seed
FINAL GOAL
Produce a COMPLETE, CLEAN, PROFESSIONAL repository that:
- passes OpenEnv validation
- runs locally
- deploys on HF Spaces
- produces meaningful RL signals
- demonstrates real-world quantum optimization
Think like a Meta/Hugging Face engineer reviewing submissions.
Quality > speed.
Now generate the full implementation.