Fix Space inference loading and switch evidence plots to SVG

.gitignore

@@ -223,3 +223,10 @@ __marimo__/
 
 # local codebase maps
 CODEBASE_STRUCTURE_MAIN_V5.md
+
+# recovered local-only artifacts
+artifacts/events.jsonl
+artifacts/reward_curve.csv
+artifacts/run_summary.json
+artifacts/test_plot.png
+artifacts/*.png

BLOG.md

@@ -0,0 +1,57 @@
+# ADAPT: training a model to debug algorithm solutions instead of guessing once
+
+ADAPT is an OpenEnv environment for reinforcement learning on algorithmic problem solving. The core idea is simple: instead of rewarding a model only for a perfect first answer, we make it solve a programming task, inspect verifier feedback, and repair its code over up to three attempts.
+
+That makes the environment closer to real software work. The agent sees a problem statement, input format, constraints, and visible examples. It writes Python code, gets hidden-test feedback, and is rewarded for improving hidden correctness and eventually meeting an efficiency target.
+
+## What we built
+
+- an OpenEnv-compatible environment with `reset`, `step`, and `state`
+- a verifier that executes submitted Python safely and scores hidden correctness, visible correctness, and efficiency
+- a reward-aware curriculum that shifts toward the most educational problem families
+- a GRPO training pipeline built with Unsloth + Hugging Face TRL
+
+## Why this environment is interesting
+
+Most code-generation tasks only ask whether a model can get the answer immediately. ADAPT focuses on a harder and more realistic behavior: can the model recover from failure, use feedback, and converge toward a correct solution?
+
+The environment includes `20` DSA problem families across easy, medium, and hard tiers, hidden evaluation tests, efficiency-aware rewards, and a curriculum that adapts based on what seems to teach the agent the most.
+
+## Real training run
+
+We trained `Qwen/Qwen2.5-3B-Instruct` with the overnight preset:
+
+- Run ID: `15940d1d-7d8c-4253-8810-2ea934bedee4`
+- Optimizer steps: `950`
+- Training episodes logged: `7,600`
+- Wall-clock time: `8.96 hours`
+- Uploaded model revision: `6c957e7c6bdb25ff086775fb8692570aee4501c9`
+
+From the recovered training logs:
+
+- average reward improved from `0.4441` in the first `500` episodes to `0.5951` in the last `500`
+- average hidden pass rate improved from `0.4625` to `0.6488`
+- completion rate improved from `43.6%` to `59.6%`
+
+Late in training, the model was solving many easy and medium tasks reliably and was noticeably stronger on hard tasks as well:
+
+| Difficulty | Avg reward | Avg hidden pass rate | Completion rate |
+| --- | ---: | ---: | ---: |
+| Easy | 0.8390 | 0.8477 | 84.38% |
+| Medium | 0.7892 | 0.8425 | 78.85% |
+| Hard | 0.5182 | 0.5759 | 51.92% |
+
+## Evidence
+
+![Reward curve](artifacts/reward_curve.svg)
+
+![Pass rate by difficulty](artifacts/pass_rate_by_difficulty.svg)
+
+![Family productivity](artifacts/family_productivity.svg)
+
+## Links
+
+- Environment Space: [Dishaaa25/meta-rl-dsa-solver](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver)
+- Live app: [Dishaaa25-meta-rl-dsa-solver.hf.space](https://Dishaaa25-meta-rl-dsa-solver.hf.space)
+- Training notebook: [Qwen2.5_(3B)-GRPO.ipynb](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/blob/main/Qwen2.5_%283B%29-GRPO.ipynb)
+- Trained model: [Dishaaa25/adapt-dsa-tutor-model](https://huggingface.co/Dishaaa25/adapt-dsa-tutor-model)

README.md

@@ -15,6 +15,17 @@ tags:
 
 LLMs are getting better at one-shot code generation, but they still struggle with the thing real engineers do all day: read feedback, debug, and repair. ADAPT closes that gap by turning algorithm practice into a self-repair RL environment where the model must improve over multiple attempts instead of guessing once.
 
+## Submission links
+
+- Hugging Face Space: [Dishaaa25/meta-rl-dsa-solver](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver)
+- Live environment: [Dishaaa25-meta-rl-dsa-solver.hf.space](https://Dishaaa25-meta-rl-dsa-solver.hf.space)
+- Training repo URL: [Space repository root](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/tree/main)
+- Training script: [training/train_grpo.py](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/blob/main/training/train_grpo.py)
+- Training evidence: [TRAINING_EVIDENCE.md](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/blob/main/TRAINING_EVIDENCE.md)
+- Mini blog / writeup: [BLOG.md](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/blob/main/BLOG.md)
+- Trained adapter repo: [Dishaaa25/adapt-dsa-tutor-model](https://huggingface.co/Dishaaa25/adapt-dsa-tutor-model)
+- Source repo mirror: [s-shah4/meta-rl-dsa-solver](https://github.com/s-shah4/meta-rl-dsa-solver)
+
 ## Why ADAPT exists
 
 Most code-generation benchmarks test whether a model can land the answer immediately. They do not test whether the model can recover from partial failure, use examples productively, or adapt as the task distribution changes.
@@ -187,17 +198,70 @@ That makes ADAPT more than a static benchmark. The environment actively searches
 
 ## Results
 
-[INSERT: reward curve plot]
+We ran a real overnight GRPO training job on `Qwen/Qwen2.5-3B-Instruct` and logged `7,600` training episodes over `950` optimizer steps.
+
+Training run:
+
+- Run ID: `15940d1d-7d8c-4253-8810-2ea934bedee4`
+- Started: `2026-04-25 22:21 UTC`
+- Finished: `2026-04-26 07:19 UTC`
+- Wall-clock time: `8.96 hours`
+- Train preset: `overnight`
+- Curriculum mode: `reward_aware`
+- Trained model revision: `6c957e7c6bdb25ff086775fb8692570aee4501c9`
+
+Proof of learning from the actual run logs:
+
+- Average reward improved from `0.4441` in the first `500` episodes to `0.5951` in the last `500` episodes.
+- Average hidden pass rate improved from `0.4625` to `0.6488`.
+- Completion rate improved from `43.6%` to `59.6%`.
+
+Late-training performance over the final `500` episodes:
+
+| Difficulty | Episodes | Avg reward | Avg hidden pass rate | Completion rate |
+| --- | ---: | ---: | ---: | ---: |
+| Easy | 32 | 0.8390 | 0.8477 | 84.38% |
+| Medium | 104 | 0.7892 | 0.8425 | 78.85% |
+| Hard | 364 | 0.5182 | 0.5759 | 51.92% |
+
+### Reward curve
+
+![ADAPT reward curve](artifacts/reward_curve.svg)
+
+### Pass rate by difficulty
+
+![Pass rate by difficulty](artifacts/pass_rate_by_difficulty.svg)
+
+### Reward-aware family productivity
+
+![Family productivity](artifacts/family_productivity.svg)
+
+Top productive families near the end of training:
+
+- `smallest_most_frequent`
+- `merge_intervals`
+- `matrix_diagonal_sum`
+- `fizzbuzz_variant`
+- `group_anagrams_count`
+- `max_subarray_sum`
+- `two_sum_count`
+- `balanced_brackets`
+
+### A concrete improvement story
+
+This run did not include a separate baseline-eval sweep, so we do not claim a strict benchmark-style baseline-vs-trained score table. Instead, we show the model improving *inside the environment itself* over the same overnight run:
+
+- early training is dominated by low-reward, low-pass-rate episodes and safety failures
+- by the end of the run, the agent is regularly solving hard tasks like `reverse_words` with `reward=1.0`, `hidden pass rate=1.0`, and `efficiency score=1.0`
 
-[INSERT: baseline vs trained table]
+The raw summary used for the table above is stored in `artifacts/results_summary.json`.
 
-Recommended artifacts to include here:
+For the hackathon submission form, the "Training Run Notebook URL" field can point to the public Hugging Face Space repository because this project trained directly on Hugging Face rather than from a separate Colab notebook. The repo includes:
 
-- reward curve from `training/reward_curve.csv`
-- `reward_curve.png`
-- `pass_rate_by_difficulty.png`
-- `family_productivity.png`
-- one before/after repair example from baseline vs trained evaluation
+- the full training entrypoint in `training/train_grpo.py`
+- the recovered training evidence in `TRAINING_EVIDENCE.md`
+- embedded reward / pass-rate plots committed as image files
+- the lightweight structured summary in `artifacts/results_summary.json`
 
 ## How to run
 
@@ -302,7 +366,10 @@ openenv push --repo-id <your-hf-username>/adapt-dsa-tutor
 
 ## Links
 
-- HuggingFace Space URL: [HuggingFace Space URL]
-- Colab Training Notebook: [Colab Training Notebook]
-- HF Blog Post: [HF Blog Post]
-- YouTube Demo: [YouTube Demo]
+- Hugging Face Space URL: [https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver)
+- Live app URL: [https://Dishaaa25-meta-rl-dsa-solver.hf.space](https://Dishaaa25-meta-rl-dsa-solver.hf.space)
+- Training repo URL: [https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/tree/main](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/tree/main)
+- Training script URL: [https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/blob/main/training/train_grpo.py](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/blob/main/training/train_grpo.py)
+- Training evidence URL: [https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/blob/main/TRAINING_EVIDENCE.md](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/blob/main/TRAINING_EVIDENCE.md)
+- Blog post URL: [https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/blob/main/BLOG.md](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/blob/main/BLOG.md)
+- Trained model URL: [https://huggingface.co/Dishaaa25/adapt-dsa-tutor-model](https://huggingface.co/Dishaaa25/adapt-dsa-tutor-model)

TRAINING_EVIDENCE.md

@@ -0,0 +1,69 @@
+# Training Evidence
+
+This project was trained directly on Hugging Face infrastructure rather than from a separate Colab notebook. For the hackathon submission form, the public Hugging Face Space repository can be used as the "Training Run Notebook URL" because it contains the runnable training script and the evidence artifacts below.
+
+## Public training assets
+
+- Space repository root: [Dishaaa25/meta-rl-dsa-solver](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/tree/main)
+- Training entrypoint: [training/train_grpo.py](https://huggingface.co/spaces/Dishaaa25/meta-rl-dsa-solver/blob/main/training/train_grpo.py)
+- Trained adapter repo: [Dishaaa25/adapt-dsa-tutor-model](https://huggingface.co/Dishaaa25/adapt-dsa-tutor-model)
+- Model revision from this run: `6c957e7c6bdb25ff086775fb8692570aee4501c9`
+
+## Confirmed run
+
+- Run ID: `15940d1d-7d8c-4253-8810-2ea934bedee4`
+- Status: `succeeded`
+- Base model: `Qwen/Qwen2.5-3B-Instruct`
+- Optimizer steps: `950`
+- Logged training episodes: `7,600`
+- Duration: `8.964 hours`
+- Generator mode: `reward_aware`
+- Dataset size: `1024`
+- Checkpoint upload enabled: `true`
+
+## Evidence that training actually happened
+
+- Reward curve image committed to the repo: `artifacts/reward_curve.svg`
+- Pass-rate-by-difficulty image committed to the repo: `artifacts/pass_rate_by_difficulty.svg`
+- Family-productivity image committed to the repo: `artifacts/family_productivity.svg`
+- Structured summary committed to the repo: `artifacts/results_summary.json`
+
+## Key outcomes from the recovered run logs
+
+- Average reward improved from `0.4441` over the first `500` episodes to `0.5951` over the last `500`.
+- Average hidden pass rate improved from `0.4625` to `0.6488`.
+- Completion rate improved from `43.6%` to `59.6%`.
+
+Final-500-episode breakdown:
+
+| Difficulty | Episodes | Avg reward | Avg hidden pass rate | Completion rate |
+| --- | ---: | ---: | ---: | ---: |
+| Easy | 32 | 0.8390 | 0.8477 | 84.38% |
+| Medium | 104 | 0.7892 | 0.8425 | 78.85% |
+| Hard | 364 | 0.5182 | 0.5759 | 51.92% |
+
+## Plots
+
+![Reward curve](artifacts/reward_curve.svg)
+
+![Pass rate by difficulty](artifacts/pass_rate_by_difficulty.svg)
+
+![Family productivity](artifacts/family_productivity.svg)
+
+## Training configuration snapshot
+
+The full lightweight snapshot is committed in `artifacts/training_status_snapshot.json`. Key settings:
+
+- learning rate: `5e-6`
+- batch size: `1`
+- gradient accumulation steps: `8`
+- num generations: `4`
+- max prompt length: `1024`
+- max completion length: `384`
+- LoRA rank: `16`
+- LoRA alpha: `32`
+- 4-bit loading: `true`
+
+## Why the repo URL is the right form entry
+
+The hackathon form explicitly allows a public Hugging Face repository URL in place of a Colab notebook. Since this team trained on Hugging Face directly, the repository URL is the most accurate submission target: it contains the training code, the evidence plots, and the run summary in one public place.

artifacts/results_summary.json

@@ -0,0 +1,46 @@
+{
+  "run_id": "15940d1d-7d8c-4253-8810-2ea934bedee4",
+  "episodes": 7600,
+  "train_steps_reported": 950,
+  "wall_clock_hours": 8.964,
+  "first_500": {
+    "avg_reward": 0.4441,
+    "avg_pass_rate": 0.4625,
+    "completion_rate": 0.436
+  },
+  "last_500": {
+    "avg_reward": 0.5951,
+    "avg_pass_rate": 0.6488,
+    "completion_rate": 0.596
+  },
+  "difficulty_last_500": {
+    "easy": {
+      "episodes": 32,
+      "avg_reward": 0.839,
+      "avg_pass_rate": 0.8477,
+      "completion_rate": 0.8438
+    },
+    "medium": {
+      "episodes": 104,
+      "avg_reward": 0.7892,
+      "avg_pass_rate": 0.8425,
+      "completion_rate": 0.7885
+    },
+    "hard": {
+      "episodes": 364,
+      "avg_reward": 0.5182,
+      "avg_pass_rate": 0.5759,
+      "completion_rate": 0.5192
+    }
+  },
+  "top_productive_families_final": [
+    "smallest_most_frequent",
+    "merge_intervals",
+    "matrix_diagonal_sum",
+    "fizzbuzz_variant",
+    "group_anagrams_count",
+    "max_subarray_sum",
+    "two_sum_count",
+    "balanced_brackets"
+  ]
+}

artifacts/training_status_snapshot.json

@@ -0,0 +1,59 @@
+{
+  "status": "succeeded",
+  "run_id": "15940d1d-7d8c-4253-8810-2ea934bedee4",
+  "started_at": "2026-04-25T22:21:10.584881+00:00",
+  "finished_at": "2026-04-26T07:19:01.232852+00:00",
+  "model_repo_id": "Dishaaa25/adapt-dsa-tutor-model",
+  "uploaded_revision": "6c957e7c6bdb25ff086775fb8692570aee4501c9",
+  "phase": "completed",
+  "completed_steps": 950,
+  "total_steps": 950,
+  "train_episode_index": 7600,
+  "current_difficulty": "hard",
+  "last_problem_id": "reverse_words_8c8524b3",
+  "last_problem_family": "reverse_words",
+  "last_pass_rate": 1.0,
+  "last_visible_pass_rate": 1.0,
+  "last_reward": 1.0,
+  "last_execution_status": "completed",
+  "timing_summary": {
+    "wall_clock_seconds": 32254.3,
+    "wall_clock_minutes": 537.57,
+    "wall_clock_hours": 8.96,
+    "completed_steps": 950,
+    "train_episode_count": 7600,
+    "configured_dataset_size": 1024,
+    "configured_batch_size": 1,
+    "configured_gradient_accumulation_steps": 8,
+    "configured_num_generations": 4,
+    "avg_seconds_per_step": 33.95,
+    "steps_per_hour": 106.03,
+    "avg_seconds_per_episode": 4.24,
+    "episodes_per_hour": 848.26
+  },
+  "config": {
+    "model_name": "Qwen/Qwen2.5-3B-Instruct",
+    "dataset_size": 1024,
+    "max_steps": 950,
+    "batch_size": 1,
+    "gradient_accumulation_steps": 8,
+    "num_generations": 4,
+    "max_seq_length": 2048,
+    "max_prompt_length": 1024,
+    "max_completion_length": 384,
+    "learning_rate": 5e-6,
+    "lora_rank": 16,
+    "lora_alpha": 32,
+    "load_in_4bit": true,
+    "gradient_checkpointing": true,
+    "bf16": false,
+    "baseline_eval": false,
+    "evaluation_episodes": 20,
+    "disable_wandb": true,
+    "generator_mode": "reward_aware",
+    "trace_logging_enabled": true,
+    "save_steps": 50,
+    "save_total_limit": 3,
+    "upload_checkpoints_to_hub": true
+  }
+}

requirements.txt

@@ -7,3 +7,4 @@ huggingface_hub>=0.34.0,<0.36
 datasets>=2.18.0
 transformers>=4.51.3,<5.7
 peft>=0.18.0,<0.20
+unsloth==2026.3.5

server/requirements.txt

@@ -6,3 +6,4 @@ httpx>=0.28.0
 huggingface_hub>=0.34.0,<0.36
 transformers>=4.51.3,<5.7
 peft>=0.18.0,<0.20
+unsloth==2026.3.5

server/runtime.py

@@ -174,6 +174,32 @@ class SpaceModelRegistry:
             ) from exc
         return torch, AutoPeftModelForCausalLM, (AutoModelForCausalLM, AutoTokenizer)
 
+    def _load_with_unsloth(
+        self,
+        *,
+        model_name: str,
+        dtype: Any,
+        load_in_4bit: bool,
+        max_seq_length: int = 2048,
+    ) -> tuple[Any, Any] | None:
+        try:
+            from unsloth import FastLanguageModel
+        except ImportError:
+            return None
+
+        model, tokenizer = FastLanguageModel.from_pretrained(
+            model_name=model_name,
+            max_seq_length=max_seq_length,
+            dtype=dtype,
+            load_in_4bit=load_in_4bit,
+        )
+        if tokenizer.pad_token is None and tokenizer.eos_token is not None:
+            tokenizer.pad_token = tokenizer.eos_token
+        if hasattr(FastLanguageModel, "for_inference"):
+            FastLanguageModel.for_inference(model)
+        model.eval()
+        return model, tokenizer
+
     def _base_model_name(self) -> str:
         return os.getenv("BASE_MODEL_NAME") or os.getenv("MODEL_NAME") or "Qwen/Qwen2.5-3B-Instruct"
 
@@ -264,15 +290,23 @@ class SpaceModelRegistry:
                 dtype = torch.float16
             else:
                 dtype = torch.float32
-            tokenizer = AutoTokenizer.from_pretrained(base_model_name)
-            if tokenizer.pad_token is None and tokenizer.eos_token is not None:
-                tokenizer.pad_token = tokenizer.eos_token
-            model = AutoModelForCausalLM.from_pretrained(
-                base_model_name,
-                device_map="auto",
-                torch_dtype=dtype,
+            unsloth_stack = self._load_with_unsloth(
+                model_name=base_model_name,
+                dtype=dtype,
+                load_in_4bit=torch.cuda.is_available(),
             )
-            model.eval()
+            if unsloth_stack is not None:
+                model, tokenizer = unsloth_stack
+            else:
+                tokenizer = AutoTokenizer.from_pretrained(base_model_name)
+                if tokenizer.pad_token is None and tokenizer.eos_token is not None:
+                    tokenizer.pad_token = tokenizer.eos_token
+                model = AutoModelForCausalLM.from_pretrained(
+                    base_model_name,
+                    device_map="auto",
+                    torch_dtype=dtype,
+                )
+                model.eval()
             self._base_model = model
             self._base_tokenizer = tokenizer
             self._set_state(
@@ -307,17 +341,27 @@ class SpaceModelRegistry:
                 dtype = torch.float16
             else:
                 dtype = torch.float32
-            tokenizer = AutoTokenizer.from_pretrained(str(artifact_dir))
-            if tokenizer.pad_token is None and tokenizer.eos_token is not None:
-                tokenizer.pad_token = tokenizer.eos_token
-
             if (artifact_dir / "adapter_config.json").exists():
-                model = AutoPeftModelForCausalLM.from_pretrained(
-                    str(artifact_dir),
-                    device_map="auto",
-                    torch_dtype=dtype,
+                unsloth_stack = self._load_with_unsloth(
+                    model_name=str(artifact_dir),
+                    dtype=dtype,
+                    load_in_4bit=torch.cuda.is_available(),
                 )
+                if unsloth_stack is not None:
+                    model, tokenizer = unsloth_stack
+                else:
+                    tokenizer = AutoTokenizer.from_pretrained(str(artifact_dir))
+                    if tokenizer.pad_token is None and tokenizer.eos_token is not None:
+                        tokenizer.pad_token = tokenizer.eos_token
+                    model = AutoPeftModelForCausalLM.from_pretrained(
+                        str(artifact_dir),
+                        device_map="auto",
+                        torch_dtype=dtype,
+                    )
             else:
+                tokenizer = AutoTokenizer.from_pretrained(str(artifact_dir))
+                if tokenizer.pad_token is None and tokenizer.eos_token is not None:
+                    tokenizer.pad_token = tokenizer.eos_token
                 model = AutoModelForCausalLM.from_pretrained(
                     str(artifact_dir),
                     device_map="auto",