arturzolkowski commited on
Commit
51ee80f
·
1 Parent(s): fdbdd8c

Compare per-iteration seeds as a set in the parallel-generation test

Browse files

Samples within an iteration are generated concurrently, so the order in which the
fake generator records their seeds is not deterministic. The test compared the
recorded list directly and failed intermittently (about 1 run in 12 locally).

Compare each iteration's seeds as a sorted group, which still asserts that the same
seeds are reused across iterations.

Files changed (1) hide show
  1. tests/test_agentic_upsampling.py +3 -1
tests/test_agentic_upsampling.py CHANGED
@@ -434,7 +434,9 @@ def test_runner_can_disable_early_stop_and_select_best_sample(tmp_path: Path) ->
434
 
435
  result = runner.run_item(_item("8", "exactly 12 balloons with exact color counts"))
436
 
437
- assert generator.seeds == [1000, 1001, 1002, 1000, 1001, 1002]
 
 
438
  assert rewriter.previous_scores == [9.0]
439
  assert result["best_iteration"] == 1
440
  assert result["best"]["selected_sample_index"] == 1
 
434
 
435
  result = runner.run_item(_item("8", "exactly 12 balloons with exact color counts"))
436
 
437
+ # Samples are generated concurrently, so compare each iteration's seeds as a set.
438
+ assert sorted(generator.seeds[:3]) == [1000, 1001, 1002]
439
+ assert sorted(generator.seeds[3:]) == [1000, 1001, 1002]
440
  assert rewriter.previous_scores == [9.0]
441
  assert result["best_iteration"] == 1
442
  assert result["best"]["selected_sample_index"] == 1