Commit History

27 days ago

  1. [experiment] Judge reliability: diagnosis, label review, rubric experiments, explorer (#409)⁠

    dd0db0a

  1. Add experiment_requests/ and completed_experiments/ folders (#408)⁠

    4f9e6db

  1. [experiment] Chat judges v1 on validity set v1: results, raw logs, jury and cascade (#366)⁠

    stationhouse +1: powercoding1
    8cbd3c7
  2. [experiment] Decision judges v1: Clef, Kev-4B, Laya, pplx-decider on validity set v1 (Modal H100) (#365)⁠

    cab79d9
  3. [experiment] Validity set v1: final reference labels for all 150 units (#364)⁠

    112e08d
  4. Validity set v1: adjudicated tiebreaks for u076–u150 disagreements (#355)⁠

    35e6797
  5. [experiment] Validity set v1: labels from divyanshijoshi (u076–u150) (#347)⁠

    stationhouse +1: divyanshijoshi
    2db58dc
  6. [experiment] Validity set v1: labels from powercoding1 (u001–u075) (#345)⁠

    stationhouse +1: powercoding1
    fc128db
  7. [experiment] Validity Set Labelled: 76-150 Vaisakhi - with 94 added (#349)⁠

    stationhouse +1: vaisakhi
    c48d3ac

  1. Validity scoring: pplx-decider adapter, judge-vs-human rates, cascade with cross-validated tau (#323)⁠

    stationhouse +1: powercoding1
    674d23c

  1. Validity set v1: blind labelling sheet, pre-registered scoring, and a Clef/Kev/Laya run (compute request) (#208)⁠

    stationhouse +1: powercoding1
    a903183

  1. [experiment] Archive reward-hacking detector experiments (#173)⁠

    stationhouse +1: tomerz14
    eaf4174
  2. Judge parser: a negated label is not a verdict (#183)⁠

    stationhouse +1: powercoding1
    7b05cb0

  1. [experiment] Add standardized judge model comparison suite and results (#122)⁠

    3e414d7

  1. [experiment] [request for compute] Add Clef judge experiment proposal (#126)⁠

    db61da5

  1. Align judge robustness parsing with Inspect scorer (#112)⁠

    stationhouse +1: melover24
    7a627fa
  2. Add local judge injection pilot results (#111)⁠

    stationhouse +1: melover24
    e59a8d8

  1. Make judge robustness runner model agnostic (#82)⁠

    stationhouse +1: melover24
    812201f

  1. Support multi-turn fixtures in judge reliability runner (#50)⁠

    stationhouse +1: melover24
    d8bf834

  1. Add multi-turn judge regression cases (#44)⁠

    stationhouse +1: Emar7
    b57577b
  2. Add k=5 repeated sampling with worst-of-k, all-k-agree and pass@k (#38)⁠

    2c93805

  1. Add judge reliability evaluation artifacts (#35)⁠

    stationhouse +1: melover24
    98cb4ac
  2. Add optional pressure_turns for multi-turn items (#25)⁠

    stationhouse +1: Emar7
    3d7af1a

  1. Add judge regression cases (#20)⁠

    stationhouse +1: melover24
    5fd316a
  2. Move eval scripts into eval/, separate from scripts/ corpus tooling (#21)⁠

    5b97134