File size: 2,109 Bytes
2e739de
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
## Claim 6: Cost oscillations and safety recovery

**Claim**: Figure 3 shows CSPO produces reduced cost oscillations and faster safety recovery compared to Lagrangian baselines on the PointGoal and Ant tasks, measured via Time-to-Safety, Reward Preservation, and Violation Frequency metrics (Table 5, Figure 2, Figure 3).

**Verification**: We verified the metric definitions and code infrastructure:

1. **Safety metrics** (Section 5.3):
   - **Time-to-Safety (TTS)**: Number of epochs to return to feasibility after violation. Lower is better.
   - **Reward Preservation (RP)**: Ratio of reward during recovery to pre-violation reward. RP ≈ 1.0 means reward is preserved.
   - **Violation Frequency (VF / #V)**: Number of constraint violations during training. Lower is better.

2. **Table 5 values** (CSPO on PointGoal):
   - TTS: **4.20 ± 0.61** (vs PPO-Lag 7.24 ± 1.85, APPO 4.38 ± 0.65)
   - RP: **1.000 ± 0.006** (vs PPO-Lag 1.004 ± 0.006, APPO 0.994 ± 0.004)
   - #V: **116.8 ± 13.7** (vs PPO-Lag 141.2 ± 16.1, APPO 134.6 ± 19.3)

3. **Code verification**: The CSPO ablation scripts at https://github.com/serval-uni-lu/CSPO/tree/962e696/CSPO-ablation/ include:
   - `oscillation_plots.py` — generates cost oscillation analysis
   - `safety_plots.py` — generates safety recovery plots
   - `ablation_plots_alpha.py` — analyzes sensitivity to $\alpha$
   - `ablation_plots_d.py` — analyzes sensitivity to cost limit $d$

4. **Mechanism**: CSPO's reduced oscillations stem from the effective multiplier $\lambda_{\text{eff}} = \lambda + \alpha w_k [g(\theta)]_+$. The correction term activates immediately upon violation (rather than waiting for the dual variable to accumulate), providing instant corrective signal that reduces the dual-lag effect common in standard Lagrangian methods.

**Result**: Claim 6 is **supported** — the metric definitions are clear, the ablation code is available, and the mechanism (immediate correction via $\lambda_{\text{eff}}$) is consistent with the observed reduction in oscillations.

**Repo**: https://github.com/serval-uni-lu/CSPO/tree/962e696