pushpam14 commited on
Commit
7f76aff
·
verified ·
1 Parent(s): 979f467

Clarify submitted HF Jobs run vs Colab demo path

Browse files
Files changed (1) hide show
  1. training/grpo_colab.ipynb +7 -5
training/grpo_colab.ipynb CHANGED
@@ -8,11 +8,13 @@
8
  "\n",
9
  "**Meta PyTorch OpenEnv Hackathon \u00d7 Scaler School of Technology \u2014 Grand Finale**\n",
10
  "\n",
11
- "This notebook trains a small open-weight model (Qwen2.5-1.5B by default) on the API Contract Validator environment using GRPO from TRL. The reward signal comes directly from the deployed environment, not from a static dataset \u2014 the model learns by interacting with the env on every training step.\n",
12
  "\n",
13
- "**Hardware**: T4 GPU (15 GB VRAM) is enough. Colab free tier or HF Jobs `--flavor t4-small` both work.\n",
14
  "\n",
15
- "**Estimated runtime**: ~45 min for 200 steps on Qwen2.5-1.5B at LoRA r=16."
 
 
16
  ]
17
  },
18
  {
@@ -38,9 +40,9 @@
38
  "| **baseline_7b_scores.json** (apples-to-apples baseline) | [`../baseline_7b_scores.json`](../baseline_7b_scores.json) |\n",
39
  "| **baseline_72b_v2_scores.json** (size-baseline) | [`../baseline_72b_v2_scores.json`](../baseline_72b_v2_scores.json) |\n",
40
  "\n",
41
- "**Why no embedded cell outputs?** Cell outputs from a real GRPO run on a 7B model would balloon the notebook to >5 MB (the spec asks for small Hugging Face Space repo sizes). Instead we ship: the public WandB report for live metrics, the reward-curve PNG and full log in `results/`, and the trained adapter on HuggingFace Hub. Anyone can verify the numbers without rerunning.\n",
42
  "\n",
43
- "**To reproduce the run yourself**: open this notebook in [Colab](https://colab.research.google.com/) \u2192 set `HF_TOKEN` and `WANDB_API_KEY` in the Secrets pane \u2192 Runtime \u2192 Run all. ~45 min on a T4. The HF Jobs path (~$2 on L4, ~2 hours) is documented in [`README.md`](README.md).\n"
44
  ]
45
  },
46
  {
 
8
  "\n",
9
  "**Meta PyTorch OpenEnv Hackathon \u00d7 Scaler School of Technology \u2014 Grand Finale**\n",
10
  "\n",
11
+ "This notebook is the **reviewer-friendly Colab version** of the training pipeline. The final submitted model was trained on Hugging Face Jobs using `training/run_in_hf_jobs.py`, which calls `training/train.py`. That submitted run used **Qwen2.5-7B-Instruct (4-bit) + LoRA r=16 for 300 GRPO steps on an L4 GPU** and took about **1 h 56 min**.\n",
12
  "\n",
13
+ "The Colab cells below are kept runnable for judges who want to reproduce the pipeline on lighter hardware. By default they use a smaller Qwen2.5-1.5B model because free Colab/T4 is less reliable for the full 7B run.\n",
14
  "\n",
15
+ "**Submitted run**: Qwen2.5-7B + LoRA, 300 GRPO steps, HF Jobs L4, ~1 h 56 min.\n",
16
+ "\n",
17
+ "**Colab demo path**: Qwen2.5-1.5B by default, ~45 min for 200 steps on a T4.\n"
18
  ]
19
  },
20
  {
 
40
  "| **baseline_7b_scores.json** (apples-to-apples baseline) | [`../baseline_7b_scores.json`](../baseline_7b_scores.json) |\n",
41
  "| **baseline_72b_v2_scores.json** (size-baseline) | [`../baseline_72b_v2_scores.json`](../baseline_72b_v2_scores.json) |\n",
42
  "\n",
43
+ "**Why not rerun the full 7B job inside this notebook?** The submitted 7B run was done on HF Jobs L4 for reliability and produced the public WandB report, adapter, logs, and committed result artifacts. This notebook includes a saved proof-output cell that displays those submitted artifacts without changing the metrics.\n",
44
  "\n",
45
+ "**To reproduce the lightweight Colab path yourself**: open this notebook in [Colab](https://colab.research.google.com/) \u2192 set `HF_TOKEN` and `WANDB_API_KEY` in the Secrets pane \u2192 Runtime \u2192 Run all. This runs the smaller default model. The exact HF Jobs path used for the submitted 7B run is documented in [`training/README.md`](README.md) and implemented in `run_in_hf_jobs.py`.\n"
46
  ]
47
  },
48
  {