CS654 Final Project: model weights

These are the final weights for a CS 654 (Reinforcement Learning) course project. The project compares DQN, Laya, and a Laya-shielded DQN on highway-env (highway-fast-v0).

Folder Contents
laya-highway/ Laya, fine-tuned to judge driving actions as safe or unsafe and to choose actions

Download

hf download adamlizerbram/cs654-final-project --include "laya-highway/*" --local-dir models/final  # -> models/final/laya-highway
import laya
agent = laya.load("adamlizerbram/cs654-final-project", subfolder="laya-highway")

laya-highway

One checkpoint answers two questions about a text description of the highway around the ego car:

  • Safety (noul): will a proposed action lead to a collision or a gap under 5 m within 3 s? This yes/no answer drives the safety shield.
  • Action (choice): which legal action is safest while still making good progress? This is the Laya-alone driving policy.

Data. Labels come from simulation, not humans. For each visited state and each legal action, a copy of the simulator applies the action and then keeps lane and speed for 2 more steps.

  • An action is unsafe if the copy crashes or the same-lane gap drops below 5 m.
  • The action label is the safe action with the most forward progress.
  • Train episodes use seeds 0–299 and validation episodes use seeds 10000–10049. Evaluation seeds (100000+) were never used for training or tuning.

Training. laya-train from laya 0.4.30 on an A100:

  • 4 epochs with the default rlcd objective, --shuffle-options, and an effective batch of 64
  • 16,805 train items and 400 calibration items
  • Temperature scaling fitted T = 5.0 for both question types

Validation results (before β†’ after fine-tuning):

Question Items Accuracy ECE
Safety 2,754 0.503 β†’ 0.947 0.303 β†’ 0.025
Action 697 0.144 β†’ 0.891 0.586 β†’ 0.048

The fine-tuned P(unsafe) is nearly binary: 99% of predictions fall below 0.1 or at 0.9 and above, and both clusters are well calibrated. 6.2% of truly unsafe actions are scored below 0.1.

Limitations.

  • The labels imitate a 3-second lookahead rule, not an optimal driver.
  • highway-fast-v0 uses coarse 5 Hz physics.
  • The model reads a fixed text format, so other state descriptions are out of distribution.
Downloads last month
-
Video Preview
loading

Model tree for adamlizerbram/cs654-final-project

Finetuned
(253)
this model