training/eval.log

[transformers] `torch_dtype` is deprecated! Use `dtype` instead!

Loading weights:   0%|          | 0/291 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/291 [00:00<02:19,  2.08it/s]
Loading weights:   3%|▎         | 10/291 [00:00<00:13, 20.73it/s]
Loading weights:   5%|▌         | 15/291 [00:00<00:12, 22.80it/s]
Loading weights:   8%|▊         | 22/291 [00:00<00:08, 32.32it/s]
Loading weights:   9%|▉         | 27/291 [00:01<00:08, 29.55it/s]
Loading weights:  11%|█         | 32/291 [00:01<00:08, 31.24it/s]
Loading weights:  13%|█▎        | 37/291 [00:01<00:07, 34.91it/s]
Loading weights:  14%|█▍        | 42/291 [00:01<00:08, 30.41it/s]
Loading weights:  17%|█▋        | 49/291 [00:01<00:06, 36.79it/s]
Loading weights:  19%|█▊        | 54/291 [00:01<00:07, 32.80it/s]
Loading weights:  20%|█▉        | 58/291 [00:02<00:08, 27.64it/s]
Loading weights:  22%|██▏       | 64/291 [00:02<00:08, 26.41it/s]
Loading weights:  23%|██▎       | 68/291 [00:02<00:08, 25.82it/s]
Loading weights:  24%|██▍       | 71/291 [00:02<00:10, 21.91it/s]
Loading weights:  26%|██▌       | 76/291 [00:02<00:08, 25.71it/s]
Loading weights:  27%|██▋       | 79/291 [00:03<00:09, 22.11it/s]
Loading weights:  28%|██▊       | 82/291 [00:03<00:09, 22.28it/s]
Loading weights:  30%|██▉       | 86/291 [00:03<00:09, 20.67it/s]
Loading weights:  31%|███▏      | 91/291 [00:03<00:08, 22.51it/s]
Loading weights:  32%|███▏      | 94/291 [00:03<00:09, 21.13it/s]
Loading weights:  34%|███▍      | 99/291 [00:03<00:07, 25.20it/s]
Loading weights:  36%|███▌      | 104/291 [00:04<00:07, 24.02it/s]
Loading weights:  38%|███▊      | 112/291 [00:04<00:07, 23.92it/s]
Loading weights:  40%|███▉      | 115/291 [00:04<00:07, 23.72it/s]
Loading weights:  41%|████      | 120/291 [00:04<00:06, 25.66it/s]
Loading weights:  44%|████▎     | 127/291 [00:05<00:06, 23.58it/s]
Loading weights:  45%|████▌     | 131/291 [00:05<00:06, 24.10it/s]
Loading weights:  47%|████▋     | 136/291 [00:05<00:05, 26.12it/s]
Loading weights:  48%|████▊     | 140/291 [00:05<00:06, 21.70it/s]
Loading weights:  49%|████▉     | 143/291 [00:05<00:06, 21.55it/s]
Loading weights:  51%|█████     | 149/291 [00:06<00:06, 20.42it/s]
Loading weights:  54%|█████▍    | 157/291 [00:06<00:05, 24.73it/s]
Loading weights:  56%|█████▌    | 163/291 [00:06<00:04, 27.02it/s]
Loading weights:  57%|█████▋    | 167/291 [00:06<00:05, 21.74it/s]
Loading weights:  59%|█████▉    | 173/291 [00:06<00:04, 26.04it/s]
Loading weights:  60%|██████    | 176/291 [00:07<00:05, 21.49it/s]
Loading weights:  62%|██████▏   | 181/291 [00:07<00:04, 24.61it/s]
Loading weights:  63%|██████▎   | 184/291 [00:07<00:06, 16.75it/s]
Loading weights:  64%|██████▍   | 187/291 [00:07<00:05, 18.34it/s]
Loading weights:  65%|██████▌   | 190/291 [00:08<00:06, 16.44it/s]
Loading weights:  67%|██████▋   | 194/291 [00:08<00:05, 18.06it/s]
Loading weights:  68%|██████▊   | 198/291 [00:08<00:05, 18.53it/s]
Loading weights:  69%|██████▉   | 201/291 [00:08<00:04, 20.39it/s]
Loading weights:  70%|███████   | 204/291 [00:09<00:06, 13.43it/s]
Loading weights:  73%|███████▎  | 212/291 [00:09<00:05, 15.35it/s]
Loading weights:  75%|███████▍  | 217/291 [00:09<00:04, 18.05it/s]
Loading weights:  76%|███████▌  | 221/291 [00:09<00:03, 20.34it/s]
Loading weights:  78%|███████▊  | 226/291 [00:09<00:02, 22.41it/s]
Loading weights:  79%|███████▉  | 230/291 [00:10<00:03, 16.86it/s]
Loading weights:  80%|████████  | 234/291 [00:10<00:03, 16.92it/s]
Loading weights:  82%|████████▏ | 239/291 [00:10<00:03, 16.69it/s]
Loading weights:  84%|████████▍ | 244/291 [00:11<00:02, 19.59it/s]
Loading weights:  85%|████████▌ | 248/291 [00:11<00:02, 17.47it/s]
Loading weights:  86%|████████▌ | 250/291 [00:11<00:02, 15.33it/s]
Loading weights:  88%|████████▊ | 256/291 [00:11<00:01, 22.00it/s]
Loading weights:  89%|████████▉ | 259/291 [00:11<00:01, 16.85it/s]
Loading weights:  91%|█████████ | 265/291 [00:12<00:01, 16.79it/s]
Loading weights:  92%|█████████▏| 268/291 [00:12<00:02, 10.68it/s]
Loading weights:  94%|█████████▍| 274/291 [00:13<00:01,  9.66it/s]
Loading weights:  95%|█████████▍| 276/291 [00:14<00:02,  6.03it/s]
Loading weights:  98%|█████████▊| 284/291 [00:14<00:00, 10.52it/s]
Loading weights: 100%|██████████| 291/291 [00:14<00:00, 19.59it/s]
[transformers] The following generation flags are not valid and may be ignored: ['temperature']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
[transformers] The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
  permissions_7ae14a1411a7: success (terminal=exit verified=True steps=6)
  text_replace_b71816a16fa8: success (terminal=exit verified=True steps=6)
  permissions_917866eaadda: success (terminal=exit verified=True steps=5)
  text_replace_c0b4ca9bbb91: success (terminal=exit verified=True steps=7)
  permissions_c46cbef0e1a1: incomplete (terminal=None verified=True steps=30)
  text_replace_c155c881228a: success (terminal=exit verified=True steps=14)
  permissions_1845d061f412: success (terminal=exit verified=True steps=6)
  text_replace_c1a29ebfe948: premature_exit (terminal=exit verified=False steps=13)
  permissions_367ba331169e: success (terminal=exit verified=True steps=8)
  text_replace_891a0c52f02e: success (terminal=exit verified=True steps=3)
  permissions_332774800f96: success (terminal=exit verified=True steps=4)
  text_replace_2e730553893e: success (terminal=exit verified=True steps=8)
  permissions_eff856415243: success (terminal=exit verified=True steps=11)
  text_replace_a0621d984fc0: success (terminal=exit verified=True steps=7)
  permissions_7030f2fc275a: success (terminal=exit verified=True steps=9)
  text_replace_2f46bf6b47ea: success (terminal=exit verified=True steps=10)

=== behavioural metrics ===
{
  "n": 16,
  "operate_rate": 1.0,
  "terminate_rate": 0.938,
  "verified_rate": 0.938,
  "clean_rate": 0.875
}