feat: add low-fi PPO smoke workflow

AGENTS.md

@@ -149,3 +149,243 @@ A strong change in this repo usually does at least one of these:
 - makes the demo evidence easier to trust
 
 If a change does not help one of those, question whether it belongs in this hackathon repo.
+
+## **OpenEnv Hackathon Participant Guide**
+
+Welcome to the [OpenEnv Hackathon](https://cerebralvalley.ai/e/open-env-hackathon), hacker! 👋 We’re thrilled to have you on board.
+
+This guide is your all-in-one resource for the event, including schedule, rules, technical resources, problem statements, judging information, and more. Please read this carefully; most answers can be found here.
+
+## **1. Join the [PyTorch Discord Server](https://discord.gg/VBcf6VtfY6)**
+
+- You’ll be given a Hackathon Participant role by an admin, which will give you access to the hackathon-specific channels.
+
+- Here, you’ll be able to interact with hackers and sponsors, introduce yourselves, and form teams (for a maximum team size of **3**).
+
+- If you don't receive your role within **24 hours of joining,** please ping @CV.
+
+- Please submit your Discord username below so we can grant you the role
+
+[linkEmbed]
+
+## **2. Location**
+
+**|** Shack15 (1 Ferry Building, Suite 201, San Francisco CA. 94111)
+
+- **Venue Access:** Shack15 is on the 2nd floor of the Ferry Building. Go up the Ferry Building elevator to the second floor, and turn left. Here you will see the main entrance to Shack15. 
+
+- **Parking:** Parking near the Ferry Building is extremely limited. Consider parking farther out and taking Uber, Lyft, or Public Transportation. 
+
+[youtube]
+
+## **3. WiFi Information**
+
+- **Username:** SHACK15_Members
+
+- **Password:** M3mb3r$4L!f3
+
+## **4. Hackathon Schedule**
+
+**Saturday, March 7 (Outline)**
+
+- **9:00 AM:** Doors Open •󠁏 Breakfast Served •󠁏 Team Formation
+
+- **10:00 AM – 11:30AM**: Kick-off presentations with Meta, Hugging Face, UC Berkeley, CoreWeave, OpenPipe, Unsloth AI, Fleet AI, Mercor, Scaler AI Labs, Snorkel AI, Patronus AI, Halluminate and Scale AI
+
+- **11:30 AM:** Hacking Begins
+
+- **1:00 PM:** Lunch Served
+
+- **6:00 PM:** Dinner Served
+
+- **10:00 PM:** Doors Close •󠁏 Re-entry not permitted
+
+**Sunday, March 8 (Outline)**
+
+- **9:00AM:** Doors Open •󠁏 Breakfast Served
+
+- **1:00PM:** Hacking stops •󠁏 Submissions Due
+
+- **1:15PM:** First Round Judging Begins
+
+- **2:00PM:** Lunch Served
+
+- **3:00PM:** Final Round Judging Begins
+
+- **4:00PM:** Winners Announced and Closing
+
+- **5:00PM:** Doors Close
+
+All presentation slides can be found here
+
+[linkEmbed]
+
+## **5. Hackathon and Submission Rules**
+
+To keep things fair and aligned with our goals, all teams must follow these rules:
+
+- **Open Source:** Please ensure your repository is public.
+
+- **New Work Only:** All projects must be started from scratch during the hackathon with no previous work.
+
+- **Team Size:** Teams may have up to **3** members.
+
+- **Banned Projects:** Projects will be disqualified if they: violate legal, ethical, or platform policies, use code, data, or assets you do not have the rights to.
+
+- Your project **must** use OpenEnv (stable release 0.2.1) deployed on HF spaces
+
+- You must show a minimal training script for your environment using Unsloth or HF TRL in Colab.
+
+- You must upload a **one minute** demo video to YouTube talking about your submission.
+
+## **6. Hackathon Problem Statements**
+
+Your project must address at least **one of the five** required problem statements.
+
+- Some problem statements include **optional partner-sponsored sub-problem statements**, which are additional focus areas related to the main theme.
+
+- Your project may align with **multiple partner sub-problem statements**, but you can only be **judged for a maximum of two**. Please **select up to two** when submitting.
+
+- Projects that match these partner sub-problem statements are eligible for **extra partner prizes**, judged separately from the main track winners.
+
+- Each partner sub-problem statement carries a prize of **$10,000 USD**.
+
+**Statement 1: Multi-Agent Interactions**
+
+Environments for this theme involve cooperation, competition, negotiation, and coalition formation. Learning from these environments will enable agents to model the beliefs and incentives of others in partially observable settings. This drives theory-of-mind reasoning and emergent strategic behavior.
+
+- **Expected Outcome:** an environment that can be used to train multi-agent task handling in a LLM
+
+- **Example Environments:** Market simulations, compute-allocation negotiations, collaborative puzzle worlds, mixed cooperative/competitive strategy games.
+
+- **Partner Sub-Themes:**
+  - **Fleet AI:** Scalable Oversight: Environments that train oversight agents to monitor, analyze, and explain the behavior of other AI agents operating in complex, multi-agent settings.
+  - **Halluminate:** Multi-Actor Environments: Build a realistic environment where an agent interacts with and manages multiple actors (agents) to discover and achieve the task
+
+**Statement 2: (Super) Long-Horizon Planning & Instruction Following**
+
+You will build environments that require deep, multi-step reasoning with sparse or delayed rewards. After using these environments, the goal is to enable agents to decompose goals, track state over extended trajectories, and recover from early mistakes. The aim is to push beyond shallow next-token reasoning toward structured planning and durable internal representations. 
+
+- **Expected Outcome:** an environment that can capture and improve LLM behaviour on challenging long horizon tasks that need long running sessions beyond context memory limits. 
+
+- **Example Environments:** Research-planning simulators, large-scale codebase refactoring tasks, strategic resource management worlds, long-horizon logistics optimization, extremely complicated long-horizon instruction following (e.g., 300 instructions scattered around).
+
+- **Partner Sub-Themes:**
+  - **Mercor:** Make an environment with capped/uncapped rewards where frontier model rewards scale with token output.
+
+  - **Scale AI:** Environments for long horizon workflows for non-code use cases within a business setting: focusing on either Sales, Project management, or HR & IT.
+
+**Statement 3: World Modeling**
+
+- **Statement 3.1: Professional Tasks:** Here you will develop environments that require real interaction with tools, APIs, or dynamic systems where the model is expected to do real hard work instead of exploiting short-cuts to arrive at the desired outcome. Learning from these environments will enable agents to maintain consistent internal state, update beliefs based on outcomes, and orchestrate multi-step workflows. The goal is to strengthen causal reasoning and persistent world models.
+  - **Expected Outcome:** an environment capturing nuances of a defined partially observable world and improve LLM interaction with it
+
+  - **Example Environments:** Dynamic browser/API ecosystems, enterprise applications, scientific workflow loops (papers → code → experiments), economic simulations with feedback, tool-discovery benchmarks.
+
+  - **Partner Sub-Theme:**
+    - **Scaler AI Labs:** Multi-App RL Environment for Enterprise Workflows: Create RL environments to demonstrate complex workflows, business rule nuances etc in a large enterprise
+
+- **Statement 3.2: Personalized Tasks:** Here we will develop an environment that offers real personalized task handling, imagine replying to personal messages or handling dinner conflicts due to work conflicts, replying to tough emails. Think any personal assistant tasks.
+  - **Expected Outcome:** An environment that gives the model a realistic simulation of handling personal tasks, conflicts and managing them as delegations
+
+  - **Example Environments:** Executive Assistant Meeting Planner, Dinner and drive planning, email and message replying, etc
+
+  - **Partner Sub-Theme:**
+    - **Patronus AI:** Consumer Workflows with Schema Drift: Multi-step consumer workflow environments where the underlying data schemas, API contracts, and t&cs/policies/rules change.
+
+**Statement 4: Self-Improvement**
+
+The focus here is to create environments where agents can learn to generate new challenges, escalate difficulty, and improve through self-play or adaptive curricula. Rather than optimizing fixed tasks, the goal is for agents to learn to drive their own capability growth. The objective is recursive skill amplification.
+
+- **Expected Outcome:** an environment for improving self-play of a LLM over a defined set of tasks
+
+- **Example Environments:** Self-play negotiation arenas, auto-generated math/proof tasks, evolving coding competitions, adaptive RL curricula.
+
+- **Partner Sub-Theme:**
+  - **Snorkel AI:** Simulated Experts-in-the-Loop: Environment that simulates interactions with real subject-matter experts, with changing requirements / preferences.
+
+**Statement 5: Wild Card - Impress Us!**
+
+We do not want to limit your focus if your idea doesn’t fit the boxes above, we want and WILL reward out of box tasks, please be creative but remember to add submissions that meaningfully add value to LLM training on a certain task.
+
+More details about each theme can be found here:
+
+[linkEmbed]
+
+## **7. CV Hackathon Winners**
+
+[linkEmbed]
+
+## **8. OpenEnv Provided Resources**
+
+**Please read through the entire slideshow here. This includes:**
+
+- OpenEnv Fundamentals, Architecture
+- Local Dev, Docker, and HF Spaces Deployment
+- OpenEnv in Practice
+- Training (TRL & Unsloth)
+- How-to-Access-Infrastructure (including GPU Request Form)
+
+[linkEmbed]
+
+## **9. Partner Provided Resources**
+
+- **Unsloth AI Resources**
+  - <https://unsloth.ai/docs/get-started/unsloth-notebooks#grpo-reasoning-rl-notebooks>
+- **Mercor Resources**
+  - Dataset: <https://huggingface.co/datasets/mercor/apex-agents>
+  - Archipelago repo to run the eval: <https://github.com/Mercor-Intelligence/archipelago>
+  - APEX-Agents paper: <https://arxiv.org/abs/2601.14242>
+- **Hugging Face Resources**
+  - **$30** in Compute and Inference Credits
+  - To claim your credits, set up a HF account here: <https://huggingface.co/join>
+  - Then, follow this link: <https://huggingface.co/openenv-community>
+  - You will be granted **$30** of compute and inference credits!
+- **Northflank Resources**
+  - Each team gets an H100
+  - Northflank instructions
+
+    [linkEmbed]
+
+  - Join the NorthFlank discord channel for any questions
+  - Please fill out this form:
+
+    [linkEmbed]
+
+- **Cursor Resources**
+  - **$50** in Cursor Credits, **apply below**
+
+    [linkEmbed]
+
+## **10. Judging & Submissions**
+
+Judges will be taking place on **Sunday, March 8**. These judges are evaluating your **technical demos** in the following categories. _Show us what you have built_ to solve our problem statements. Please **do not** show us a presentation. We'll be checking to ensure your project was built **entirely during the event**; no previous work is allowed. 
+
+**|** **Teams should submit [here](https://cerebralvalley.ai/e/openenv-hackathon-sf/hackathon/submit) when they have completed hacking.** In the submission form, you will have to upload a **one minute** demo video on YouTube talking about your submission. You must also show a minimal training script for your environment using Unsloth or HF TRL in Colab.
+
+**Please ensure your project uses** use OpenEnv (stable release 0.2.1) deployed on HF spaces.
+
+[linkEmbed]
+
+**Judging Criteria**
+
+- **Environment Innovation (40%) -** Is the environment novel, creative, or challenging? Does it meaningfully test the agent’s behavior?
+- **Storytelling (30%) -** Does the team clearly explain the problem, environment, and agent behavior? Is the demo engaging and easy to follow?
+- **Training Script Showing Improvement in Rewards (20%) -** Does the demo provide observable evidence of training progress (reward curves, metrics, or before/after behavior)? 
+- **Reward and Training Pipeline Setup (10%) -** Is the reward logic coherent, and does the pipeline produce meaningful improvement in the agent’s inference (how it acts in the environment)?
+
+**Judging Process**
+
+**|** Judging proceeds in two rounds:
+
+- Hackers will be assigned groups of judges; \~3 minutes to pitch followed by 1-2 minutes of Q/A
+
+- The top **six** teams in ranking will get to demo on stage to a panel of judges; \~3 minutes to pitch followed by 2-3 minutes for Q/A.
+
+## **11. Prizes**
+
+- **1st Place:** $15,000 USD Cash
+
+- **2nd Place:** $9,000 USD Cash
+
+- **3rd Place:** $6,000 USD Cash

docs/P1_PPO_SMOKE_NOTE.md

@@ -0,0 +1,61 @@
+# P1 PPO Smoke Note
+
+This note records the first tiny low-fidelity PPO smoke pass on the repaired 4-knob `P1` environment.
+
+## Purpose
+
+This run is diagnostic-only.
+
+It exists to answer:
+
+- can a small PPO policy interact with the low-fidelity environment without code-path failures
+- does the reward surface produce a readable early failure mode
+- what is the first obvious behavior problem before any broader training push
+
+It does **not** validate the high-fidelity `submit` contract.
+
+## Command
+
+```bash
+uv sync --extra training
+uv run --extra training python training/ppo_smoke.py --eval-episodes 1
+```
+
+## Artifact
+
+- ignored runtime artifact: `training/artifacts/ppo_smoke/ppo_smoke_20260308T062412Z.json`
+
+## Configuration
+
+- training mode: low-fidelity only
+- action space: 24 `run` actions + `restore_best`
+- `submit`: intentionally excluded from the smoke loop
+- total timesteps: `64`
+- evaluation episodes: `1`
+- device: `cpu`
+
+## Result
+
+- the smoke path executed successfully and wrote a trajectory artifact
+- the trained policy did **not** reach feasibility in the evaluation episode
+- summary metrics:
+  - `mean_eval_reward = -1.1`
+  - `constraint_satisfaction_rate = 0.0`
+
+## First failure mode
+
+The policy collapsed to a repeated low-fidelity action:
+
+- `aspect_ratio increase medium`
+
+Observed behavior:
+
+- the same action repeated for the full 6-step budget
+- feasibility stayed near `0.050653`
+- final reward was negative because the agent burned the budget without finding a repair path
+
+This is useful smoke evidence because it shows:
+
+- the PPO training path is wired correctly enough to produce trajectories
+- the current low-fidelity surface still permits an obvious local-behavior failure
+- the next step should remain paired high-fidelity fixture checks plus at least one submit-side manual trace, not a broader training push

pyproject.toml

@@ -18,6 +18,10 @@ notebooks = [
   "ipykernel>=6.29.0",
   "jupyterlab>=4.3.0",
 ]
+training = [
+  "gymnasium>=1.0.0",
+  "stable-baselines3>=2.5.0",
+]
 dev = [
   "pre-commit>=4.0.0",
   "pytest>=8.3.0",

training/README.md

@@ -12,4 +12,10 @@ Training policy:
 
 - [ ] Northflank notebook artifacts saved
 - [ ] Colab notebook saved
+- [x] tiny low-fi PPO smoke artifact saved
 - [ ] trained-policy evidence saved
+
+## Runnable paths
+
+- install the training dependencies: `uv sync --extra training`
+- tiny low-fi PPO smoke run: `uv run --extra training python training/ppo_smoke.py`

training/ppo_smoke.py

@@ -0,0 +1,339 @@
+from __future__ import annotations
+
+import argparse
+import json
+from dataclasses import asdict, dataclass
+from datetime import UTC, datetime
+from pathlib import Path
+from typing import Final
+
+import gymnasium as gym
+import numpy as np
+from gymnasium import spaces
+from stable_baselines3 import PPO
+
+from fusion_lab.models import StellaratorAction, StellaratorObservation
+from server.contract import RESET_SEEDS
+from server.environment import BUDGET, StellaratorEnvironment
+
+DEFAULT_OUTPUT_DIR: Final[Path] = Path("training/artifacts/ppo_smoke")
+DEFAULT_TOTAL_TIMESTEPS: Final[int] = 128
+DEFAULT_EVAL_EPISODES: Final[int] = 3
+
+RUN_ACTION_SPECS: Final[tuple[tuple[str, str, str], ...]] = (
+    ("aspect_ratio", "increase", "small"),
+    ("aspect_ratio", "increase", "medium"),
+    ("aspect_ratio", "increase", "large"),
+    ("aspect_ratio", "decrease", "small"),
+    ("aspect_ratio", "decrease", "medium"),
+    ("aspect_ratio", "decrease", "large"),
+    ("elongation", "increase", "small"),
+    ("elongation", "increase", "medium"),
+    ("elongation", "increase", "large"),
+    ("elongation", "decrease", "small"),
+    ("elongation", "decrease", "medium"),
+    ("elongation", "decrease", "large"),
+    ("rotational_transform", "increase", "small"),
+    ("rotational_transform", "increase", "medium"),
+    ("rotational_transform", "increase", "large"),
+    ("rotational_transform", "decrease", "small"),
+    ("rotational_transform", "decrease", "medium"),
+    ("rotational_transform", "decrease", "large"),
+    ("triangularity_scale", "increase", "small"),
+    ("triangularity_scale", "increase", "medium"),
+    ("triangularity_scale", "increase", "large"),
+    ("triangularity_scale", "decrease", "small"),
+    ("triangularity_scale", "decrease", "medium"),
+    ("triangularity_scale", "decrease", "large"),
+)
+LOW_FI_ACTION_COUNT: Final[int] = len(RUN_ACTION_SPECS) + 1
+LOW_FI_RESTORE_ACTION_INDEX: Final[int] = len(RUN_ACTION_SPECS)
+
+
+@dataclass(frozen=True)
+class TraceStep:
+    step: int
+    action_index: int
+    action_label: str
+    reward: float
+    score: float
+    feasibility: float
+    constraints_satisfied: bool
+    evaluation_failed: bool
+    budget_remaining: int
+    max_elongation: float
+    average_triangularity: float
+    edge_iota_over_nfp: float
+
+
+@dataclass(frozen=True)
+class EpisodeTrace:
+    episode: int
+    seed: int
+    total_reward: float
+    final_score: float
+    final_feasibility: float
+    constraints_satisfied: bool
+    evaluation_failed: bool
+    steps: list[TraceStep]
+
+
+class LowFiSmokeEnv(gym.Env[np.ndarray, int]):
+    metadata = {"render_modes": []}
+
+    def __init__(self) -> None:
+        super().__init__()
+        self._env = StellaratorEnvironment()
+        self._seed = 0
+        self._episode_index = 0
+        self.observation_space = spaces.Box(
+            low=-np.inf,
+            high=np.inf,
+            shape=(12,),
+            dtype=np.float32,
+        )
+        self.action_space = spaces.Discrete(LOW_FI_ACTION_COUNT)
+
+    def reset(
+        self,
+        *,
+        seed: int | None = None,
+        options: dict[str, object] | None = None,
+    ) -> tuple[np.ndarray, dict[str, object]]:
+        super().reset(seed=seed)
+        self._seed = self._next_seed(seed)
+        obs = self._env.reset(seed=self._seed)
+        return self._encode_observation(obs), self._info(obs)
+
+    def _next_seed(self, seed: int | None) -> int:
+        if seed is not None:
+            self._episode_index = 0
+            return seed % len(RESET_SEEDS)
+        next_seed = self._episode_index % len(RESET_SEEDS)
+        self._episode_index += 1
+        return next_seed
+
+    def step(
+        self,
+        action: int,
+    ) -> tuple[np.ndarray, float, bool, bool, dict[str, object]]:
+        obs = self._env.step(self._decode_action(action))
+        return (
+            self._encode_observation(obs),
+            float(obs.reward or 0.0),
+            bool(obs.done),
+            False,
+            self._info(obs),
+        )
+
+    def _decode_action(self, action: int) -> StellaratorAction:
+        if action == LOW_FI_RESTORE_ACTION_INDEX:
+            return StellaratorAction(intent="restore_best")
+        parameter, direction, magnitude = RUN_ACTION_SPECS[action]
+        return StellaratorAction(
+            intent="run",
+            parameter=parameter,
+            direction=direction,
+            magnitude=magnitude,
+        )
+
+    def action_label(self, action: int) -> str:
+        if action == LOW_FI_RESTORE_ACTION_INDEX:
+            return "restore_best"
+        parameter, direction, magnitude = RUN_ACTION_SPECS[action]
+        return f"{parameter} {direction} {magnitude}"
+
+    def _encode_observation(self, obs: StellaratorObservation) -> np.ndarray:
+        budget_fraction = obs.budget_remaining / BUDGET
+        step_fraction = obs.step_number / BUDGET
+        return np.array(
+            [
+                obs.max_elongation,
+                obs.aspect_ratio,
+                obs.average_triangularity,
+                obs.edge_iota_over_nfp,
+                obs.p1_score,
+                obs.p1_feasibility,
+                obs.vacuum_well,
+                budget_fraction,
+                step_fraction,
+                obs.best_low_fidelity_score,
+                obs.best_low_fidelity_feasibility,
+                float(obs.constraints_satisfied) - float(obs.evaluation_failed),
+            ],
+            dtype=np.float32,
+        )
+
+    def _info(self, obs: StellaratorObservation) -> dict[str, object]:
+        return {
+            "diagnostics_text": obs.diagnostics_text,
+            "budget_remaining": obs.budget_remaining,
+            "constraints_satisfied": obs.constraints_satisfied,
+            "evaluation_failed": obs.evaluation_failed,
+            "p1_score": obs.p1_score,
+            "p1_feasibility": obs.p1_feasibility,
+        }
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description=(
+            "Run a tiny low-fidelity PPO smoke pass against the repaired Fusion Design Lab "
+            "environment and save a small trajectory artifact."
+        )
+    )
+    parser.add_argument(
+        "--total-timesteps",
+        type=int,
+        default=DEFAULT_TOTAL_TIMESTEPS,
+        help=f"Total PPO timesteps for the smoke run (default: {DEFAULT_TOTAL_TIMESTEPS}).",
+    )
+    parser.add_argument(
+        "--eval-episodes",
+        type=int,
+        default=DEFAULT_EVAL_EPISODES,
+        help=f"Number of deterministic evaluation episodes to record (default: {DEFAULT_EVAL_EPISODES}).",
+    )
+    parser.add_argument(
+        "--seed",
+        type=int,
+        default=0,
+        help="Base seed for training and evaluation.",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=Path,
+        default=DEFAULT_OUTPUT_DIR,
+        help="Directory where the JSON artifact should be written.",
+    )
+    return parser.parse_args()
+
+
+def build_model(env: LowFiSmokeEnv, seed: int) -> PPO:
+    return PPO(
+        policy="MlpPolicy",
+        env=env,
+        seed=seed,
+        verbose=0,
+        device="cpu",
+        n_steps=32,
+        batch_size=32,
+        n_epochs=4,
+        gamma=0.98,
+        learning_rate=3e-4,
+        ent_coef=0.01,
+    )
+
+
+def evaluate_policy(model: PPO, *, eval_episodes: int, base_seed: int) -> list[EpisodeTrace]:
+    traces: list[EpisodeTrace] = []
+    for episode in range(eval_episodes):
+        env = LowFiSmokeEnv()
+        seed = base_seed + episode
+        obs, _ = env.reset(seed=seed)
+        done = False
+        total_reward = 0.0
+        steps: list[TraceStep] = []
+        step_index = 0
+        final_info: dict[str, object] = {}
+
+        while not done:
+            action, _ = model.predict(obs, deterministic=True)
+            action_index = int(action)
+            obs, reward, terminated, truncated, info = env.step(action_index)
+            done = terminated or truncated
+            total_reward += reward
+            step_index += 1
+            final_info = info
+            steps.append(
+                TraceStep(
+                    step=step_index,
+                    action_index=action_index,
+                    action_label=env.action_label(action_index),
+                    reward=reward,
+                    score=float(info["p1_score"]),
+                    feasibility=float(info["p1_feasibility"]),
+                    constraints_satisfied=bool(info["constraints_satisfied"]),
+                    evaluation_failed=bool(info["evaluation_failed"]),
+                    budget_remaining=int(info["budget_remaining"]),
+                    max_elongation=float(obs[0]),
+                    average_triangularity=float(obs[2]),
+                    edge_iota_over_nfp=float(obs[3]),
+                )
+            )
+
+        traces.append(
+            EpisodeTrace(
+                episode=episode,
+                seed=seed,
+                total_reward=round(total_reward, 4),
+                final_score=float(final_info["p1_score"]),
+                final_feasibility=float(final_info["p1_feasibility"]),
+                constraints_satisfied=bool(final_info["constraints_satisfied"]),
+                evaluation_failed=bool(final_info["evaluation_failed"]),
+                steps=steps,
+            )
+        )
+    return traces
+
+
+def artifact_payload(
+    *,
+    total_timesteps: int,
+    eval_episodes: int,
+    seed: int,
+    traces: list[EpisodeTrace],
+) -> dict[str, object]:
+    mean_reward = sum(trace.total_reward for trace in traces) / max(len(traces), 1)
+    success_rate = sum(1 for trace in traces if trace.constraints_satisfied) / max(len(traces), 1)
+    return {
+        "created_at_utc": datetime.now(UTC).isoformat(),
+        "mode": "low_fidelity_ppo_smoke",
+        "total_timesteps": total_timesteps,
+        "eval_episodes": eval_episodes,
+        "seed": seed,
+        "train_reset_seed_indices": list(range(len(RESET_SEEDS))),
+        "action_space_size": LOW_FI_ACTION_COUNT,
+        "notes": (
+            "Diagnostic-only PPO smoke run. Submit is intentionally excluded here so the "
+            "smoke loop stays low-fidelity and fast. Training resets cycle through the "
+            "frozen low-fidelity reset seeds to surface positive repair signal sooner."
+        ),
+        "summary": {
+            "mean_eval_reward": round(mean_reward, 4),
+            "constraint_satisfaction_rate": round(success_rate, 4),
+        },
+        "episodes": [asdict(trace) for trace in traces],
+    }
+
+
+def write_artifact(output_dir: Path, payload: dict[str, object]) -> Path:
+    output_dir.mkdir(parents=True, exist_ok=True)
+    timestamp = datetime.now(UTC).strftime("%Y%m%dT%H%M%SZ")
+    output_path = output_dir / f"ppo_smoke_{timestamp}.json"
+    output_path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
+    return output_path
+
+
+def main() -> None:
+    args = parse_args()
+    env = LowFiSmokeEnv()
+    model = build_model(env, seed=args.seed)
+    model.learn(total_timesteps=args.total_timesteps, progress_bar=False)
+    traces = evaluate_policy(
+        model,
+        eval_episodes=args.eval_episodes,
+        base_seed=args.seed,
+    )
+    payload = artifact_payload(
+        total_timesteps=args.total_timesteps,
+        eval_episodes=args.eval_episodes,
+        seed=args.seed,
+        traces=traces,
+    )
+    output_path = write_artifact(args.output_dir, payload)
+    print(output_path)
+
+
+if __name__ == "__main__":
+    main()

uv.lock

@@ -698,6 +698,15 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/98/78/01c019cdb5d6498122777c1a43056ebb3ebfeef2076d9d026bfe15583b2b/click-8.3.1-py3-none-any.whl", hash = "sha256:981153a64e25f12d547d3426c367a4857371575ee7ad18df2a6183ab0545b2a6", size = 108274, upload-time = "2025-11-15T20:45:41.139Z" },
 ]
 
+[[package]]
+name = "cloudpickle"
+version = "3.1.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/27/fb/576f067976d320f5f0114a8d9fa1215425441bb35627b1993e5afd8111e5/cloudpickle-3.1.2.tar.gz", hash = "sha256:7fda9eb655c9c230dab534f1983763de5835249750e85fbcef43aaa30a9a2414", size = 22330, upload-time = "2025-11-03T09:25:26.604Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/88/39/799be3f2f0f38cc727ee3b4f1445fe6d5e4133064ec2e4115069418a5bb6/cloudpickle-3.1.2-py3-none-any.whl", hash = "sha256:9acb47f6afd73f60dc1df93bb801b472f05ff42fa6c84167d25cb206be1fbf4a", size = 22228, upload-time = "2025-11-03T09:25:25.534Z" },
+]
+
 [[package]]
 name = "cma"
 version = "4.4.4"
@@ -909,6 +918,30 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/bc/58/6b3d24e6b9bc474a2dcdee65dfd1f008867015408a271562e4b690561a4d/cryptography-46.0.5-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:8456928655f856c6e1533ff59d5be76578a7157224dbd9ce6872f25055ab9ab7", size = 3407605, upload-time = "2026-02-10T19:18:29.233Z" },
 ]
 
+[[package]]
+name = "cuda-bindings"
+version = "12.9.4"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "cuda-pathfinder", marker = "platform_machine != 'ARM64' or sys_platform != 'win32'" },
+]
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/45/e7/b47792cc2d01c7e1d37c32402182524774dadd2d26339bd224e0e913832e/cuda_bindings-12.9.4-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c912a3d9e6b6651853eed8eed96d6800d69c08e94052c292fec3f282c5a817c9", size = 12210593, upload-time = "2025-10-21T14:51:36.574Z" },
+    { url = "https://files.pythonhosted.org/packages/a9/c1/dabe88f52c3e3760d861401bb994df08f672ec893b8f7592dc91626adcf3/cuda_bindings-12.9.4-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fda147a344e8eaeca0c6ff113d2851ffca8f7dfc0a6c932374ee5c47caa649c8", size = 12151019, upload-time = "2025-10-21T14:51:43.167Z" },
+    { url = "https://files.pythonhosted.org/packages/63/56/e465c31dc9111be3441a9ba7df1941fe98f4aa6e71e8788a3fb4534ce24d/cuda_bindings-12.9.4-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:32bdc5a76906be4c61eb98f546a6786c5773a881f3b166486449b5d141e4a39f", size = 11906628, upload-time = "2025-10-21T14:51:49.905Z" },
+    { url = "https://files.pythonhosted.org/packages/a3/84/1e6be415e37478070aeeee5884c2022713c1ecc735e6d82d744de0252eee/cuda_bindings-12.9.4-cp313-cp313t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:56e0043c457a99ac473ddc926fe0dc4046694d99caef633e92601ab52cbe17eb", size = 11925991, upload-time = "2025-10-21T14:51:56.535Z" },
+    { url = "https://files.pythonhosted.org/packages/d1/af/6dfd8f2ed90b1d4719bc053ff8940e494640fe4212dc3dd72f383e4992da/cuda_bindings-12.9.4-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8b72ee72a9cc1b531db31eebaaee5c69a8ec3500e32c6933f2d3b15297b53686", size = 11922703, upload-time = "2025-10-21T14:52:03.585Z" },
+    { url = "https://files.pythonhosted.org/packages/6c/19/90ac264acc00f6df8a49378eedec9fd2db3061bf9263bf9f39fd3d8377c3/cuda_bindings-12.9.4-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d80bffc357df9988dca279734bc9674c3934a654cab10cadeed27ce17d8635ee", size = 11924658, upload-time = "2025-10-21T14:52:10.411Z" },
+]
+
+[[package]]
+name = "cuda-pathfinder"
+version = "1.4.1"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/07/02/59a5bc738a09def0b49aea0e460bdf97f65206d0d041246147cf6207e69c/cuda_pathfinder-1.4.1-py3-none-any.whl", hash = "sha256:40793006082de88e0950753655e55558a446bed9a7d9d0bcb48b2506d50ed82a", size = 43903, upload-time = "2026-03-06T21:05:24.372Z" },
+]
+
 [[package]]
 name = "cycler"
 version = "0.12.1"
@@ -1191,6 +1224,15 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/21/f2/4454eefc15cc326b46530d230c58cc0bb91a1e9797f2842b2a1720cbb233/f90nml-1.5.0-py2.py3-none-any.whl", hash = "sha256:bdf616dbe7e83619feb86d54358fb8d97038133bfd8f9ba9a01eeca5dc4691a7", size = 51994, upload-time = "2025-10-07T15:25:09.064Z" },
 ]
 
+[[package]]
+name = "farama-notifications"
+version = "0.0.4"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/2e/2c/8384832b7a6b1fd6ba95bbdcae26e7137bb3eedc955c42fd5cdcc086cfbf/Farama-Notifications-0.0.4.tar.gz", hash = "sha256:13fceff2d14314cf80703c8266462ebf3733c7d165336eee998fc58e545efd18", size = 2131, upload-time = "2023-02-27T18:28:41.047Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/05/2c/ffc08c54c05cdce6fbed2aeebc46348dbe180c6d2c541c7af7ba0aa5f5f8/Farama_Notifications-0.0.4-py3-none-any.whl", hash = "sha256:14de931035a41961f7c056361dc7f980762a143d05791ef5794a751a2caf05ae", size = 2511, upload-time = "2023-02-27T18:28:39.447Z" },
+]
+
 [[package]]
 name = "fastapi"
 version = "0.135.1"
@@ -1489,11 +1531,16 @@ notebooks = [
     { name = "ipykernel" },
     { name = "jupyterlab" },
 ]
+training = [
+    { name = "gymnasium" },
+    { name = "stable-baselines3" },
+]
 
 [package.metadata]
 requires-dist = [
     { name = "constellaration" },
     { name = "fastapi", specifier = ">=0.115.0" },
+    { name = "gymnasium", marker = "extra == 'training'", specifier = ">=1.0.0" },
     { name = "ipykernel", marker = "extra == 'notebooks'", specifier = ">=6.29.0" },
     { name = "jupyterlab", marker = "extra == 'notebooks'", specifier = ">=4.3.0" },
     { name = "numpy", specifier = ">=2.0.0" },
@@ -1502,9 +1549,10 @@ requires-dist = [
     { name = "pydantic", specifier = ">=2.10.0" },
     { name = "pytest", marker = "extra == 'dev'", specifier = ">=8.3.0" },
     { name = "ruff", marker = "extra == 'dev'", specifier = ">=0.11.0" },
+    { name = "stable-baselines3", marker = "extra == 'training'", specifier = ">=2.5.0" },
     { name = "uvicorn", specifier = ">=0.34.0" },
 ]
-provides-extras = ["notebooks", "dev"]
+provides-extras = ["notebooks", "training", "dev"]
 
 [[package]]
 name = "graphemeu"
@@ -1515,6 +1563,21 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/69/18/36503ea63e1ecd0a95590d7b6b8b7d227a1e4541a154e1612a231def1bdc/graphemeu-0.7.2-py3-none-any.whl", hash = "sha256:1444520f6899fd30114fc2a39f297d86d10fa0f23bf7579f772f8bc7efaa2542", size = 22670, upload-time = "2025-01-15T09:48:57.241Z" },
 ]
 
+[[package]]
+name = "gymnasium"
+version = "1.2.3"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "cloudpickle" },
+    { name = "farama-notifications" },
+    { name = "numpy" },
+    { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/76/59/653a9417d98ed3e29ef9734ba52c3495f6c6823b8d5c0c75369f25111708/gymnasium-1.2.3.tar.gz", hash = "sha256:2b2cb5b5fbbbdf3afb9f38ca952cc48aa6aa3e26561400d940747fda3ad42509", size = 829230, upload-time = "2025-12-18T16:51:10.234Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/56/d3/ea5f088e3638dbab12e5c20d6559d5b3bdaeaa1f2af74e526e6815836285/gymnasium-1.2.3-py3-none-any.whl", hash = "sha256:e6314bba8f549c7fdcc8677f7cd786b64908af6e79b57ddaa5ce1825bffb5373", size = 952113, upload-time = "2025-12-18T16:51:08.445Z" },
+]
+
 [[package]]
 name = "h11"
 version = "0.16.0"
@@ -3130,6 +3193,108 @@ dependencies = [
 ]
 sdist = { url = "https://files.pythonhosted.org/packages/1a/95/5b99a5798b366ab242fe0b2190f3814b9321eb98c6e1e9c6b599b2b4ce84/nvgpu-0.10.0.tar.gz", hash = "sha256:c415f757e0c375357f8904a6ea0cee084ab0ce97ed11e4840f2c8839196b3918", size = 8445, upload-time = "2023-03-30T03:17:01.622Z" }
 
+[[package]]
+name = "nvidia-cublas-cu12"
+version = "12.8.4.1"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/dc/61/e24b560ab2e2eaeb3c839129175fb330dfcfc29e5203196e5541a4c44682/nvidia_cublas_cu12-12.8.4.1-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:8ac4e771d5a348c551b2a426eda6193c19aa630236b418086020df5ba9667142", size = 594346921, upload-time = "2025-03-07T01:44:31.254Z" },
+]
+
+[[package]]
+name = "nvidia-cuda-cupti-cu12"
+version = "12.8.90"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/f8/02/2adcaa145158bf1a8295d83591d22e4103dbfd821bcaf6f3f53151ca4ffa/nvidia_cuda_cupti_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:ea0cb07ebda26bb9b29ba82cda34849e73c166c18162d3913575b0c9db9a6182", size = 10248621, upload-time = "2025-03-07T01:40:21.213Z" },
+]
+
+[[package]]
+name = "nvidia-cuda-nvrtc-cu12"
+version = "12.8.93"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/05/6b/32f747947df2da6994e999492ab306a903659555dddc0fbdeb9d71f75e52/nvidia_cuda_nvrtc_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl", hash = "sha256:a7756528852ef889772a84c6cd89d41dfa74667e24cca16bb31f8f061e3e9994", size = 88040029, upload-time = "2025-03-07T01:42:13.562Z" },
+]
+
+[[package]]
+name = "nvidia-cuda-runtime-cu12"
+version = "12.8.90"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/0d/9b/a997b638fcd068ad6e4d53b8551a7d30fe8b404d6f1804abf1df69838932/nvidia_cuda_runtime_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:adade8dcbd0edf427b7204d480d6066d33902cab2a4707dcfc48a2d0fd44ab90", size = 954765, upload-time = "2025-03-07T01:40:01.615Z" },
+]
+
+[[package]]
+name = "nvidia-cudnn-cu12"
+version = "9.10.2.21"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "nvidia-cublas-cu12", marker = "platform_machine != 'ARM64' or sys_platform != 'win32'" },
+]
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/ba/51/e123d997aa098c61d029f76663dedbfb9bc8dcf8c60cbd6adbe42f76d049/nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:949452be657fa16687d0930933f032835951ef0892b37d2d53824d1a84dc97a8", size = 706758467, upload-time = "2025-06-06T21:54:08.597Z" },
+]
+
+[[package]]
+name = "nvidia-cufft-cu12"
+version = "11.3.3.83"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "nvidia-nvjitlink-cu12", marker = "platform_machine != 'ARM64' or sys_platform != 'win32'" },
+]
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/1f/13/ee4e00f30e676b66ae65b4f08cb5bcbb8392c03f54f2d5413ea99a5d1c80/nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:4d2dd21ec0b88cf61b62e6b43564355e5222e4a3fb394cac0db101f2dd0d4f74", size = 193118695, upload-time = "2025-03-07T01:45:27.821Z" },
+]
+
+[[package]]
+name = "nvidia-cufile-cu12"
+version = "1.13.1.3"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/bb/fe/1bcba1dfbfb8d01be8d93f07bfc502c93fa23afa6fd5ab3fc7c1df71038a/nvidia_cufile_cu12-1.13.1.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:1d069003be650e131b21c932ec3d8969c1715379251f8d23a1860554b1cb24fc", size = 1197834, upload-time = "2025-03-07T01:45:50.723Z" },
+]
+
+[[package]]
+name = "nvidia-curand-cu12"
+version = "10.3.9.90"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/fb/aa/6584b56dc84ebe9cf93226a5cde4d99080c8e90ab40f0c27bda7a0f29aa1/nvidia_curand_cu12-10.3.9.90-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:b32331d4f4df5d6eefa0554c565b626c7216f87a06a4f56fab27c3b68a830ec9", size = 63619976, upload-time = "2025-03-07T01:46:23.323Z" },
+]
+
+[[package]]
+name = "nvidia-cusolver-cu12"
+version = "11.7.3.90"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "nvidia-cublas-cu12", marker = "platform_machine != 'ARM64' or sys_platform != 'win32'" },
+    { name = "nvidia-cusparse-cu12", marker = "platform_machine != 'ARM64' or sys_platform != 'win32'" },
+    { name = "nvidia-nvjitlink-cu12", marker = "platform_machine != 'ARM64' or sys_platform != 'win32'" },
+]
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/85/48/9a13d2975803e8cf2777d5ed57b87a0b6ca2cc795f9a4f59796a910bfb80/nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:4376c11ad263152bd50ea295c05370360776f8c3427b30991df774f9fb26c450", size = 267506905, upload-time = "2025-03-07T01:47:16.273Z" },
+]
+
+[[package]]
+name = "nvidia-cusparse-cu12"
+version = "12.5.8.93"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "nvidia-nvjitlink-cu12", marker = "platform_machine != 'ARM64' or sys_platform != 'win32'" },
+]
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/c2/f5/e1854cb2f2bcd4280c44736c93550cc300ff4b8c95ebe370d0aa7d2b473d/nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:1ec05d76bbbd8b61b06a80e1eaf8cf4959c3d4ce8e711b65ebd0443bb0ebb13b", size = 288216466, upload-time = "2025-03-07T01:48:13.779Z" },
+]
+
+[[package]]
+name = "nvidia-cusparselt-cu12"
+version = "0.7.1"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/56/79/12978b96bd44274fe38b5dde5cfb660b1d114f70a65ef962bcbbed99b549/nvidia_cusparselt_cu12-0.7.1-py3-none-manylinux2014_x86_64.whl", hash = "sha256:f1bb701d6b930d5a7cea44c19ceb973311500847f81b634d802b7b539dc55623", size = 287193691, upload-time = "2025-02-26T00:15:44.104Z" },
+]
+
 [[package]]
 name = "nvidia-ml-py"
 version = "13.590.48"
@@ -3139,6 +3304,38 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/fd/72/fb2af0d259a651affdce65fd6a495f0e07a685a0136baf585c5065204ee7/nvidia_ml_py-13.590.48-py3-none-any.whl", hash = "sha256:fd43d30ee9cd0b7940f5f9f9220b68d42722975e3992b6c21d14144c48760e43", size = 50680, upload-time = "2026-01-22T01:14:55.281Z" },
 ]
 
+[[package]]
+name = "nvidia-nccl-cu12"
+version = "2.27.5"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/6e/89/f7a07dc961b60645dbbf42e80f2bc85ade7feb9a491b11a1e973aa00071f/nvidia_nccl_cu12-2.27.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:ad730cf15cb5d25fe849c6e6ca9eb5b76db16a80f13f425ac68d8e2e55624457", size = 322348229, upload-time = "2025-06-26T04:11:28.385Z" },
+]
+
+[[package]]
+name = "nvidia-nvjitlink-cu12"
+version = "12.8.93"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/f6/74/86a07f1d0f42998ca31312f998bd3b9a7eff7f52378f4f270c8679c77fb9/nvidia_nvjitlink_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl", hash = "sha256:81ff63371a7ebd6e6451970684f916be2eab07321b73c9d244dc2b4da7f73b88", size = 39254836, upload-time = "2025-03-07T01:49:55.661Z" },
+]
+
+[[package]]
+name = "nvidia-nvshmem-cu12"
+version = "3.4.5"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/b5/09/6ea3ea725f82e1e76684f0708bbedd871fc96da89945adeba65c3835a64c/nvidia_nvshmem_cu12-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:042f2500f24c021db8a06c5eec2539027d57460e1c1a762055a6554f72c369bd", size = 139103095, upload-time = "2025-09-06T00:32:31.266Z" },
+]
+
+[[package]]
+name = "nvidia-nvtx-cu12"
+version = "12.8.90"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/a2/eb/86626c1bbc2edb86323022371c39aa48df6fd8b0a1647bc274577f72e90b/nvidia_nvtx_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:5b17e2001cc0d751a5bc2c6ec6d26ad95913324a4adb86788c944f8ce9ba441f", size = 89954, upload-time = "2025-03-07T01:42:44.131Z" },
+]
+
 [[package]]
 name = "openai"
 version = "2.26.0"
@@ -5031,6 +5228,23 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/61/28/8cb142d3fe80c4a2d8af54ca0b003f47ce0ba920974e7990fa6e016402d1/sse_starlette-3.3.2-py3-none-any.whl", hash = "sha256:5c3ea3dad425c601236726af2f27689b74494643f57017cafcb6f8c9acfbb862", size = 14270, upload-time = "2026-02-28T11:24:32.984Z" },
 ]
 
+[[package]]
+name = "stable-baselines3"
+version = "2.7.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "cloudpickle" },
+    { name = "gymnasium" },
+    { name = "matplotlib" },
+    { name = "numpy" },
+    { name = "pandas" },
+    { name = "torch" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/c9/42/f284c28272422262a99cdf35ecd2e283fded2f75327e6d5e82a9f6d6fe62/stable_baselines3-2.7.1.tar.gz", hash = "sha256:cd90d12d9ee0d9584053f12215c1682b313be4e3a8d8007739319799c3d2c071", size = 220719, upload-time = "2025-12-05T11:22:03.691Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/df/cc/a3038d3833f329dcd03b2dce8b778e4b41044caff88b48429473b8629623/stable_baselines3-2.7.1-py3-none-any.whl", hash = "sha256:b017e76dfe5ca0ce6eabb29e79c42e8c7e125d5862bfcd43ce04ec19732348d0", size = 188039, upload-time = "2025-12-05T11:22:00.819Z" },
+]
+
 [[package]]
 name = "stack-data"
 version = "0.6.3"
@@ -5198,6 +5412,66 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/c7/18/c86eb8e0202e32dd3df50d43d7ff9854f8e0603945ff398974c1d91ac1ef/tomli_w-1.2.0-py3-none-any.whl", hash = "sha256:188306098d013b691fcadc011abd66727d3c414c571bb01b1a174ba8c983cf90", size = 6675, upload-time = "2025-01-15T12:07:22.074Z" },
 ]
 
+[[package]]
+name = "torch"
+version = "2.10.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "cuda-bindings", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "filelock" },
+    { name = "fsspec" },
+    { name = "jinja2" },
+    { name = "networkx" },
+    { name = "nvidia-cublas-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cuda-cupti-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cuda-nvrtc-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cuda-runtime-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cudnn-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cufft-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cufile-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-curand-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cusolver-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cusparse-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cusparselt-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-nccl-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-nvjitlink-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-nvshmem-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-nvtx-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "setuptools", marker = "python_full_version >= '3.12'" },
+    { name = "sympy" },
+    { name = "triton", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "typing-extensions" },
+]
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/0f/8b/4b61d6e13f7108f36910df9ab4b58fd389cc2520d54d81b88660804aad99/torch-2.10.0-2-cp311-none-macosx_11_0_arm64.whl", hash = "sha256:418997cb02d0a0f1497cf6a09f63166f9f5df9f3e16c8a716ab76a72127c714f", size = 79423467, upload-time = "2026-02-10T21:44:48.711Z" },
+    { url = "https://files.pythonhosted.org/packages/d3/54/a2ba279afcca44bbd320d4e73675b282fcee3d81400ea1b53934efca6462/torch-2.10.0-2-cp312-none-macosx_11_0_arm64.whl", hash = "sha256:13ec4add8c3faaed8d13e0574f5cd4a323c11655546f91fbe6afa77b57423574", size = 79498202, upload-time = "2026-02-10T21:44:52.603Z" },
+    { url = "https://files.pythonhosted.org/packages/ec/23/2c9fe0c9c27f7f6cb865abcea8a4568f29f00acaeadfc6a37f6801f84cb4/torch-2.10.0-2-cp313-none-macosx_11_0_arm64.whl", hash = "sha256:e521c9f030a3774ed770a9c011751fb47c4d12029a3d6522116e48431f2ff89e", size = 79498254, upload-time = "2026-02-10T21:44:44.095Z" },
+    { url = "https://files.pythonhosted.org/packages/78/89/f5554b13ebd71e05c0b002f95148033e730d3f7067f67423026cc9c69410/torch-2.10.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:3282d9febd1e4e476630a099692b44fdc214ee9bf8ee5377732d9d9dfe5712e4", size = 145992610, upload-time = "2026-01-21T16:25:26.327Z" },
+    { url = "https://files.pythonhosted.org/packages/ae/30/a3a2120621bf9c17779b169fc17e3dc29b230c29d0f8222f499f5e159aa8/torch-2.10.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:a2f9edd8dbc99f62bc4dfb78af7bf89499bca3d753423ac1b4e06592e467b763", size = 915607863, upload-time = "2026-01-21T16:25:06.696Z" },
+    { url = "https://files.pythonhosted.org/packages/6f/3d/c87b33c5f260a2a8ad68da7147e105f05868c281c63d65ed85aa4da98c66/torch-2.10.0-cp311-cp311-win_amd64.whl", hash = "sha256:29b7009dba4b7a1c960260fc8ac85022c784250af43af9fb0ebafc9883782ebd", size = 113723116, upload-time = "2026-01-21T16:25:21.916Z" },
+    { url = "https://files.pythonhosted.org/packages/61/d8/15b9d9d3a6b0c01b883787bd056acbe5cc321090d4b216d3ea89a8fcfdf3/torch-2.10.0-cp311-none-macosx_11_0_arm64.whl", hash = "sha256:b7bd80f3477b830dd166c707c5b0b82a898e7b16f59a7d9d42778dd058272e8b", size = 79423461, upload-time = "2026-01-21T16:24:50.266Z" },
+    { url = "https://files.pythonhosted.org/packages/cc/af/758e242e9102e9988969b5e621d41f36b8f258bb4a099109b7a4b4b50ea4/torch-2.10.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:5fd4117d89ffd47e3dcc71e71a22efac24828ad781c7e46aaaf56bf7f2796acf", size = 145996088, upload-time = "2026-01-21T16:24:44.171Z" },
+    { url = "https://files.pythonhosted.org/packages/23/8e/3c74db5e53bff7ed9e34c8123e6a8bfef718b2450c35eefab85bb4a7e270/torch-2.10.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:787124e7db3b379d4f1ed54dd12ae7c741c16a4d29b49c0226a89bea50923ffb", size = 915711952, upload-time = "2026-01-21T16:23:53.503Z" },
+    { url = "https://files.pythonhosted.org/packages/6e/01/624c4324ca01f66ae4c7cd1b74eb16fb52596dce66dbe51eff95ef9e7a4c/torch-2.10.0-cp312-cp312-win_amd64.whl", hash = "sha256:2c66c61f44c5f903046cc696d088e21062644cbe541c7f1c4eaae88b2ad23547", size = 113757972, upload-time = "2026-01-21T16:24:39.516Z" },
+    { url = "https://files.pythonhosted.org/packages/c9/5c/dee910b87c4d5c0fcb41b50839ae04df87c1cfc663cf1b5fca7ea565eeaa/torch-2.10.0-cp312-none-macosx_11_0_arm64.whl", hash = "sha256:6d3707a61863d1c4d6ebba7be4ca320f42b869ee657e9b2c21c736bf17000294", size = 79498198, upload-time = "2026-01-21T16:24:34.704Z" },
+    { url = "https://files.pythonhosted.org/packages/c9/6f/f2e91e34e3fcba2e3fc8d8f74e7d6c22e74e480bbd1db7bc8900fdf3e95c/torch-2.10.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:5c4d217b14741e40776dd7074d9006fd28b8a97ef5654db959d8635b2fe5f29b", size = 146004247, upload-time = "2026-01-21T16:24:29.335Z" },
+    { url = "https://files.pythonhosted.org/packages/98/fb/5160261aeb5e1ee12ee95fe599d0541f7c976c3701d607d8fc29e623229f/torch-2.10.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:6b71486353fce0f9714ca0c9ef1c850a2ae766b409808acd58e9678a3edb7738", size = 915716445, upload-time = "2026-01-21T16:22:45.353Z" },
+    { url = "https://files.pythonhosted.org/packages/6a/16/502fb1b41e6d868e8deb5b0e3ae926bbb36dab8ceb0d1b769b266ad7b0c3/torch-2.10.0-cp313-cp313-win_amd64.whl", hash = "sha256:c2ee399c644dc92ef7bc0d4f7e74b5360c37cdbe7c5ba11318dda49ffac2bc57", size = 113757050, upload-time = "2026-01-21T16:24:19.204Z" },
+    { url = "https://files.pythonhosted.org/packages/1a/0b/39929b148f4824bc3ad6f9f72a29d4ad865bcf7ebfc2fa67584773e083d2/torch-2.10.0-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:3202429f58309b9fa96a614885eace4b7995729f44beb54d3e4a47773649d382", size = 79851305, upload-time = "2026-01-21T16:24:09.209Z" },
+    { url = "https://files.pythonhosted.org/packages/d8/14/21fbce63bc452381ba5f74a2c0a959fdf5ad5803ccc0c654e752e0dbe91a/torch-2.10.0-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:aae1b29cd68e50a9397f5ee897b9c24742e9e306f88a807a27d617f07adb3bd8", size = 146005472, upload-time = "2026-01-21T16:22:29.022Z" },
+    { url = "https://files.pythonhosted.org/packages/54/fd/b207d1c525cb570ef47f3e9f836b154685011fce11a2f444ba8a4084d042/torch-2.10.0-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:6021db85958db2f07ec94e1bc77212721ba4920c12a18dc552d2ae36a3eb163f", size = 915612644, upload-time = "2026-01-21T16:21:47.019Z" },
+    { url = "https://files.pythonhosted.org/packages/36/53/0197f868c75f1050b199fe58f9bf3bf3aecac9b4e85cc9c964383d745403/torch-2.10.0-cp313-cp313t-win_amd64.whl", hash = "sha256:ff43db38af76fda183156153983c9a096fc4c78d0cd1e07b14a2314c7f01c2c8", size = 113997015, upload-time = "2026-01-21T16:23:00.767Z" },
+    { url = "https://files.pythonhosted.org/packages/0e/13/e76b4d9c160e89fff48bf16b449ea324bda84745d2ab30294c37c2434c0d/torch-2.10.0-cp313-none-macosx_11_0_arm64.whl", hash = "sha256:cdf2a523d699b70d613243211ecaac14fe9c5df8a0b0a9c02add60fb2a413e0f", size = 79498248, upload-time = "2026-01-21T16:23:09.315Z" },
+    { url = "https://files.pythonhosted.org/packages/4f/93/716b5ac0155f1be70ed81bacc21269c3ece8dba0c249b9994094110bfc51/torch-2.10.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:bf0d9ff448b0218e0433aeb198805192346c4fd659c852370d5cc245f602a06a", size = 79464992, upload-time = "2026-01-21T16:23:05.162Z" },
+    { url = "https://files.pythonhosted.org/packages/69/2b/51e663ff190c9d16d4a8271203b71bc73a16aa7619b9f271a69b9d4a936b/torch-2.10.0-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:233aed0659a2503b831d8a67e9da66a62c996204c0bba4f4c442ccc0c68a3f60", size = 146018567, upload-time = "2026-01-21T16:22:23.393Z" },
+    { url = "https://files.pythonhosted.org/packages/5e/cd/4b95ef7f293b927c283db0b136c42be91c8ec6845c44de0238c8c23bdc80/torch-2.10.0-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:682497e16bdfa6efeec8cde66531bc8d1fbbbb4d8788ec6173c089ed3cc2bfe5", size = 915721646, upload-time = "2026-01-21T16:21:16.983Z" },
+    { url = "https://files.pythonhosted.org/packages/56/97/078a007208f8056d88ae43198833469e61a0a355abc0b070edd2c085eb9a/torch-2.10.0-cp314-cp314-win_amd64.whl", hash = "sha256:6528f13d2a8593a1a412ea07a99812495bec07e9224c28b2a25c0a30c7da025c", size = 113752373, upload-time = "2026-01-21T16:22:13.471Z" },
+    { url = "https://files.pythonhosted.org/packages/d8/94/71994e7d0d5238393df9732fdab607e37e2b56d26a746cb59fdb415f8966/torch-2.10.0-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:f5ab4ba32383061be0fb74bda772d470140a12c1c3b58a0cfbf3dae94d164c28", size = 79850324, upload-time = "2026-01-21T16:22:09.494Z" },
+    { url = "https://files.pythonhosted.org/packages/e2/65/1a05346b418ea8ccd10360eef4b3e0ce688fba544e76edec26913a8d0ee0/torch-2.10.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:716b01a176c2a5659c98f6b01bf868244abdd896526f1c692712ab36dbaf9b63", size = 146006482, upload-time = "2026-01-21T16:22:18.42Z" },
+    { url = "https://files.pythonhosted.org/packages/1d/b9/5f6f9d9e859fc3235f60578fa64f52c9c6e9b4327f0fe0defb6de5c0de31/torch-2.10.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:d8f5912ba938233f86361e891789595ff35ca4b4e2ac8fe3670895e5976731d6", size = 915613050, upload-time = "2026-01-21T16:20:49.035Z" },
+    { url = "https://files.pythonhosted.org/packages/66/4d/35352043ee0eaffdeff154fad67cd4a31dbed7ff8e3be1cc4549717d6d51/torch-2.10.0-cp314-cp314t-win_amd64.whl", hash = "sha256:71283a373f0ee2c89e0f0d5f446039bdabe8dbc3c9ccf35f0f784908b0acd185", size = 113995816, upload-time = "2026-01-21T16:22:05.312Z" },
+]
+
 [[package]]
 name = "tornado"
 version = "6.5.4"
@@ -5238,6 +5512,19 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/00/c0/8f5d070730d7836adc9c9b6408dec68c6ced86b304a9b26a14df072a6e8c/traitlets-5.14.3-py3-none-any.whl", hash = "sha256:b74e89e397b1ed28cc831db7aea759ba6640cb3de13090ca145426688ff1ac4f", size = 85359, upload-time = "2024-04-19T11:11:46.763Z" },
 ]
 
+[[package]]
+name = "triton"
+version = "3.6.0"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/e0/12/b05ba554d2c623bffa59922b94b0775673de251f468a9609bc9e45de95e9/triton-3.6.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e8e323d608e3a9bfcc2d9efcc90ceefb764a82b99dea12a86d643c72539ad5d3", size = 188214640, upload-time = "2026-01-20T16:00:35.869Z" },
+    { url = "https://files.pythonhosted.org/packages/ab/a8/cdf8b3e4c98132f965f88c2313a4b493266832ad47fb52f23d14d4f86bb5/triton-3.6.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:74caf5e34b66d9f3a429af689c1c7128daba1d8208df60e81106b115c00d6fca", size = 188266850, upload-time = "2026-01-20T16:00:43.041Z" },
+    { url = "https://files.pythonhosted.org/packages/f9/0b/37d991d8c130ce81a8728ae3c25b6e60935838e9be1b58791f5997b24a54/triton-3.6.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:10c7f76c6e72d2ef08df639e3d0d30729112f47a56b0c81672edc05ee5116ac9", size = 188289450, upload-time = "2026-01-20T16:00:49.136Z" },
+    { url = "https://files.pythonhosted.org/packages/35/f8/9c66bfc55361ec6d0e4040a0337fb5924ceb23de4648b8a81ae9d33b2b38/triton-3.6.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d002e07d7180fd65e622134fbd980c9a3d4211fb85224b56a0a0efbd422ab72f", size = 188400296, upload-time = "2026-01-20T16:00:56.042Z" },
+    { url = "https://files.pythonhosted.org/packages/df/3d/9e7eee57b37c80cec63322c0231bb6da3cfe535a91d7a4d64896fcb89357/triton-3.6.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a17a5d5985f0ac494ed8a8e54568f092f7057ef60e1b0fa09d3fd1512064e803", size = 188273063, upload-time = "2026-01-20T16:01:07.278Z" },
+    { url = "https://files.pythonhosted.org/packages/f6/56/6113c23ff46c00aae423333eb58b3e60bdfe9179d542781955a5e1514cb3/triton-3.6.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:46bd1c1af4b6704e554cad2eeb3b0a6513a980d470ccfa63189737340c7746a7", size = 188397994, upload-time = "2026-01-20T16:01:14.236Z" },
+]
+
 [[package]]
 name = "typer"
 version = "0.24.1"