--- license: apache-2.0 base_model: convaiinnovations/laya tags: - reinforcement-learning - highway-env - autonomous-driving - safety-shield - laya --- # CS654 Final Project: model weights These are the final weights for a CS 654 (Reinforcement Learning) course project. The project compares DQN, Laya, and a Laya-shielded DQN on [highway-env](https://highway-env.farama.org/) (`highway-fast-v0`). | Folder | Contents | |---|---| | `laya-highway/` | [Laya](https://huggingface.co/convaiinnovations/laya), fine-tuned to judge driving actions as safe or unsafe and to choose actions | ## Download ```bash hf download adamlizerbram/cs654-final-project --include "laya-highway/*" --local-dir models/final # -> models/final/laya-highway ``` ```python import laya agent = laya.load("adamlizerbram/cs654-final-project", subfolder="laya-highway") ``` ## laya-highway One checkpoint answers two questions about a text description of the highway around the ego car: - **Safety** (`noul`): will a proposed action lead to a collision or a gap under 5 m within 3 s? This yes/no answer drives the safety shield. - **Action** (`choice`): which legal action is safest while still making good progress? This is the Laya-alone driving policy. **Data.** Labels come from simulation, not humans. For each visited state and each legal action, a copy of the simulator applies the action and then keeps lane and speed for 2 more steps. - An action is **unsafe** if the copy crashes or the same-lane gap drops below 5 m. - The **action label** is the safe action with the most forward progress. - Train episodes use seeds 0–299 and validation episodes use seeds 10000–10049. Evaluation seeds (100000+) were never used for training or tuning. **Training.** `laya-train` from laya 0.4.30 on an A100: - 4 epochs with the default `rlcd` objective, `--shuffle-options`, and an effective batch of 64 - 16,805 train items and 400 calibration items - Temperature scaling fitted T = 5.0 for both question types **Validation results** (before → after fine-tuning): | Question | Items | Accuracy | ECE | |---|---|---|---| | Safety | 2,754 | 0.503 → 0.947 | 0.303 → 0.025 | | Action | 697 | 0.144 → 0.891 | 0.586 → 0.048 | The fine-tuned P(unsafe) is nearly binary: 99% of predictions fall below 0.1 or at 0.9 and above, and both clusters are well calibrated. 6.2% of truly unsafe actions are scored below 0.1. **Limitations.** - The labels imitate a 3-second lookahead rule, not an optimal driver. - `highway-fast-v0` uses coarse 5 Hz physics. - The model reads a fixed text format, so other state descriptions are out of distribution.