You are an expert systems engineer, RL researcher, and backend architect.

Your task is to build a COMPLETE, PRODUCTION-GRADE OpenEnv environment for:

"Noise-aware, hardware-constrained quantum circuit optimization using reinforcement learning"

This is NOT a toy project. It must fully comply with OpenEnv specification and be deployable.

---

#  HARD REQUIREMENTS (DO NOT VIOLATE)

* Must follow OpenEnv spec EXACTLY:

  * step(action) -> observation, reward, done, info
  * reset()
  * state()
* Use strict Pydantic models
* Include openenv.yaml
* Include 3+ tasks (easy, medium, hard)
* Each task must have deterministic graders (0.0-1.0)
* Include meaningful reward shaping (NOT sparse)
* Include baseline inference.py (OpenAI client)
* Include Dockerfile (must run)
* Must be HF Spaces deployable
* Must pass "openenv validate"

---

#  PROBLEM DEFINITION

We are building an RL environment where an agent learns to:

* Construct and optimize quantum circuits
* Maximize fidelity to a target state/unitary
* Minimize circuit depth and gate count
* Minimize noise impact
* Respect hardware connectivity constraints

Use Qiskit for simulation (statevector + noisy Aer simulator).

---

#  REQUIRED FILE STRUCTURE

envs/my_env/
|---- **init**.py
|---- models.py
|---- client.py
|---- README.md
|---- openenv.yaml
|---- server/
|---- **init**.py
|---- my_environment.py
|---- graders/
|   |---- fidelity.py
|   |---- efficiency.py
|   |---- noise.py
|   |---- constraints.py
|   |---- aggregate.py
|---- tasks/
|   |---- easy.py
|   |---- medium.py
|   |---- hard.py
|---- app.py
|---- Dockerfile

Also include:

* inference.py (root)
* requirements.txt

---

#  MODEL DEFINITIONS (STRICT)

Define:

Action:

* action_type: ADD, REMOVE, SWAP, PARAM, STOP
* gate
* qubits
* parameter (optional)

Observation:

* circuit (structured representation)
* fidelity (float)
* depth (int)
* gate_count (int)
* noise_estimate (float)
* valid_actions

State:

* full circuit object
* step count
* internal simulator state

---

#  ENVIRONMENT LOGIC

Implement:

## reset()

* initialize empty circuit
* load task config (target, noise model, connectivity)
* return initial observation

## step(action)

* validate action
* apply gate using Qiskit
* simulate:

  * ideal (statevector)
  * noisy (Aer)
* compute:

  * fidelity
  * depth
  * noise estimate
* compute reward via modular graders
* return observation, reward, done, info

## state()

* return full internal state

---

#  ACTION SPACE

* ADD gate (H, X, CNOT, RX, RZ)
* REMOVE gate
* SWAP qubits
* PARAM tuning (continuous angle)
* STOP

---

#  REWARD SYSTEM (VERY IMPORTANT)

Implement modular grading:

## fidelity.py

* compute overlap with target state

## efficiency.py

* penalize depth and gate count

## noise.py

* penalize noisy circuits

## constraints.py

* penalize invalid hardware usage

## aggregate.py

* combine scores into final score (0-1)

Final reward:

* shaped reward = current_score - previous_score

---

#  TASKS (MANDATORY)

## EASY

* Bell state (2 qubits)
* no noise

## MEDIUM

* GHZ state (3 qubits)
* noise + connectivity constraints

## HARD

* arbitrary unitary approximation
* strict depth + noisy simulation

Each task must:

* define target
* define constraints
* define grading thresholds

---

#  OPENENV INTEGRATION

Include openenv.yaml:

* name
* description
* tasks
* action/observation schema

Ensure environment passes openenv validate.

---

#  API SERVER

Use FastAPI:

* POST /reset
* POST /step
* GET /state

---

#  DOCKER

Dockerfile must:

* install qiskit, fastapi, uvicorn, openenv
* expose port
* run server

---

#  BASELINE INFERENCE

Create inference.py:

* uses OpenAI client
* reads:

  * API_BASE_URL
  * MODEL_NAME
  * OPENAI_API_KEY
* runs all 3 tasks
* prints logs in EXACT format:

[START]
[STEP]
[END]

This is critical.

---

#  LOGGING FORMAT (STRICT)

Each step must log:

* step number
* action
* reward
* cumulative score

---

#  README REQUIREMENTS

Include:

* problem description
* real-world relevance (quantum compiler optimization)
* action space
* observation space
* reward design
* task descriptions
* setup instructions
* how to run inference
* expected baseline results

---

#  IMPORTANT DESIGN REQUIREMENTS

* Reward must NOT be sparse
* Must include partial progress signals
* Must penalize bad actions
* Must prevent infinite loops
* Must be deterministic (same input -> same score)

---

#  OUTPUT FORMAT

Generate ALL files with FULL CODE.

Do NOT:

* skip files
* leave TODOs
* give partial implementations

Everything must be runnable.

---

#  BONUS (IF POSSIBLE)

* Add circuit visualization utility
* Add simple logging middleware
* Add reproducibility seed

---

#  FINAL GOAL

Produce a COMPLETE, CLEAN, PROFESSIONAL repository that:

* passes OpenEnv validation
* runs locally
* deploys on HF Spaces
* produces meaningful RL signals
* demonstrates real-world quantum optimization

Think like a Meta/Hugging Face engineer reviewing submissions.

Quality > speed.

Now generate the full implementation.