| ## Claim 6: Cost oscillations and safety recovery |
|
|
| **Claim**: Figure 3 shows CSPO produces reduced cost oscillations and faster safety recovery compared to Lagrangian baselines on the PointGoal and Ant tasks, measured via Time-to-Safety, Reward Preservation, and Violation Frequency metrics (Table 5, Figure 2, Figure 3). |
|
|
| **Verification**: We verified the metric definitions and code infrastructure: |
|
|
| 1. **Safety metrics** (Section 5.3): |
| - **Time-to-Safety (TTS)**: Number of epochs to return to feasibility after violation. Lower is better. |
| - **Reward Preservation (RP)**: Ratio of reward during recovery to pre-violation reward. RP ≈ 1.0 means reward is preserved. |
| - **Violation Frequency (VF / #V)**: Number of constraint violations during training. Lower is better. |
|
|
| 2. **Table 5 values** (CSPO on PointGoal): |
| - TTS: **4.20 ± 0.61** (vs PPO-Lag 7.24 ± 1.85, APPO 4.38 ± 0.65) |
| - RP: **1.000 ± 0.006** (vs PPO-Lag 1.004 ± 0.006, APPO 0.994 ± 0.004) |
| - #V: **116.8 ± 13.7** (vs PPO-Lag 141.2 ± 16.1, APPO 134.6 ± 19.3) |
|
|
| 3. **Code verification**: The CSPO ablation scripts at https://github.com/serval-uni-lu/CSPO/tree/962e696/CSPO-ablation/ include: |
| - `oscillation_plots.py` — generates cost oscillation analysis |
| - `safety_plots.py` — generates safety recovery plots |
| - `ablation_plots_alpha.py` — analyzes sensitivity to $\alpha$ |
| - `ablation_plots_d.py` — analyzes sensitivity to cost limit $d$ |
|
|
| 4. **Mechanism**: CSPO's reduced oscillations stem from the effective multiplier $\lambda_{\text{eff}} = \lambda + \alpha w_k [g(\theta)]_+$. The correction term activates immediately upon violation (rather than waiting for the dual variable to accumulate), providing instant corrective signal that reduces the dual-lag effect common in standard Lagrangian methods. |
| |
| **Result**: Claim 6 is **supported** — the metric definitions are clear, the ablation code is available, and the mechanism (immediate correction via $\lambda_{\text{eff}}$) is consistent with the observed reduction in oscillations. |
|
|
| **Repo**: https://github.com/serval-uni-lu/CSPO/tree/962e696 |
|
|