training/eval.log

[transformers] `torch_dtype` is deprecated! Use `dtype` instead!

Loading weights:   0%|          | 0/381 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/381 [00:00<04:42,  1.34it/s]
Loading weights:   1%|          | 2/381 [00:01<03:15,  1.94it/s]
Loading weights:   5%|▍         | 19/381 [00:01<00:14, 25.44it/s]
Loading weights:   7%|▋         | 27/381 [00:01<00:10, 34.24it/s]
Loading weights:   9%|▉         | 35/381 [00:01<00:09, 38.32it/s]
Loading weights:  11%|█         | 42/381 [00:01<00:08, 38.87it/s]
Loading weights:  13%|█▎        | 51/381 [00:01<00:07, 46.75it/s]
Loading weights:  15%|█▌        | 58/381 [00:02<00:09, 34.15it/s]
Loading weights:  17%|█▋        | 64/381 [00:02<00:08, 35.72it/s]
Loading weights:  18%|█▊        | 69/381 [00:02<00:11, 26.48it/s]
Loading weights:  20%|█▉        | 76/381 [00:02<00:10, 30.09it/s]
Loading weights:  21%|██        | 80/381 [00:02<00:10, 27.45it/s]
Loading weights:  22%|██▏       | 84/381 [00:03<00:10, 29.37it/s]
Loading weights:  24%|██▍       | 91/381 [00:03<00:08, 34.74it/s]
Loading weights:  26%|██▌       | 99/381 [00:03<00:06, 43.59it/s]
Loading weights:  28%|██▊       | 105/381 [00:03<00:06, 40.45it/s]
Loading weights:  29%|██▉       | 110/381 [00:03<00:06, 40.59it/s]
Loading weights:  30%|███       | 115/381 [00:03<00:07, 37.20it/s]
Loading weights:  33%|███▎      | 126/381 [00:03<00:04, 52.67it/s]
Loading weights:  35%|███▍      | 132/381 [00:04<00:06, 40.34it/s]
Loading weights:  36%|███▌      | 138/381 [00:04<00:05, 43.38it/s]
Loading weights:  38%|███▊      | 144/381 [00:04<00:06, 36.75it/s]
Loading weights:  41%|████      | 157/381 [00:04<00:04, 52.73it/s]
Loading weights:  43%|████▎     | 164/381 [00:04<00:04, 52.14it/s]
Loading weights:  45%|████▍     | 170/381 [00:05<00:05, 35.80it/s]
Loading weights:  46%|████▌     | 176/381 [00:05<00:05, 35.47it/s]
Loading weights:  48%|████▊     | 181/381 [00:05<00:06, 29.91it/s]
Loading weights:  49%|████▉     | 186/381 [00:05<00:05, 32.98it/s]
Loading weights:  50%|████▉     | 190/381 [00:05<00:05, 32.83it/s]
Loading weights:  51%|█████     | 194/381 [00:06<00:09, 18.79it/s]
Loading weights:  53%|█████▎    | 203/381 [00:06<00:06, 27.80it/s]
Loading weights:  55%|█████▍    | 208/381 [00:06<00:05, 30.43it/s]
Loading weights:  56%|█████▌    | 213/381 [00:06<00:07, 23.16it/s]
Loading weights:  58%|█████▊    | 221/381 [00:07<00:06, 24.59it/s]
Loading weights:  59%|█████▉    | 225/381 [00:07<00:05, 26.60it/s]
Loading weights:  60%|██████    | 230/381 [00:07<00:05, 26.22it/s]
Loading weights:  61%|██████▏   | 234/381 [00:07<00:05, 26.48it/s]
Loading weights:  63%|██████▎   | 239/381 [00:07<00:05, 27.08it/s]
Loading weights:  64%|██████▍   | 244/381 [00:07<00:04, 28.11it/s]
Loading weights:  65%|██████▌   | 248/381 [00:07<00:04, 30.11it/s]
Loading weights:  66%|██████▋   | 253/381 [00:08<00:03, 33.96it/s]
Loading weights:  67%|██████▋   | 257/381 [00:08<00:05, 22.51it/s]
Loading weights:  69%|██████▉   | 263/381 [00:08<00:04, 25.73it/s]
Loading weights:  70%|███████   | 267/381 [00:08<00:05, 20.65it/s]
Loading weights:  72%|███████▏  | 274/381 [00:09<00:04, 25.81it/s]
Loading weights:  73%|███████▎  | 280/381 [00:09<00:04, 23.87it/s]
Loading weights:  76%|███████▌  | 289/381 [00:09<00:03, 27.84it/s]
Loading weights:  78%|███████▊  | 298/381 [00:09<00:02, 34.85it/s]
Loading weights:  80%|████████  | 305/381 [00:09<00:01, 40.69it/s]
Loading weights:  81%|████████▏ | 310/381 [00:10<00:02, 27.38it/s]
Loading weights:  82%|████████▏ | 314/381 [00:10<00:02, 28.73it/s]
Loading weights:  84%|████████▍ | 320/381 [00:10<00:01, 31.80it/s]
Loading weights:  86%|████████▌ | 328/381 [00:10<00:01, 36.86it/s]
Loading weights:  87%|████████▋ | 333/381 [00:10<00:01, 29.36it/s]
Loading weights:  88%|████████▊ | 337/381 [00:11<00:01, 29.58it/s]
Loading weights:  90%|████████▉ | 341/381 [00:11<00:01, 28.79it/s]
Loading weights:  91%|█████████ | 347/381 [00:11<00:01, 25.21it/s]
Loading weights:  92%|█████████▏| 352/381 [00:11<00:01, 27.10it/s]
Loading weights:  93%|█████████▎| 355/381 [00:11<00:01, 25.39it/s]
Loading weights:  94%|█████████▍| 358/381 [00:11<00:00, 25.41it/s]
Loading weights:  95%|█████████▌| 362/381 [00:12<00:00, 26.38it/s]
Loading weights:  96%|█████████▌| 365/381 [00:12<00:00, 17.21it/s]
Loading weights:  97%|█████████▋| 370/381 [00:13<00:01, 10.69it/s]
Loading weights:  98%|█████████▊| 373/381 [00:13<00:00, 10.55it/s]
Loading weights:  98%|█████████▊| 375/381 [00:14<00:01,  5.73it/s]
Loading weights: 100%|██████████| 381/381 [00:14<00:00, 26.11it/s]
[transformers] The following generation flags are not valid and may be ignored: ['temperature']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
[transformers] The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
  permissions_7ae14a1411a7: success (terminal=exit verified=True steps=5)
  text_replace_b71816a16fa8: success (terminal=exit verified=True steps=7)
  permissions_917866eaadda: success (terminal=exit verified=True steps=5)
  text_replace_c0b4ca9bbb91: incomplete (terminal=None verified=False steps=30)
  permissions_c46cbef0e1a1: incomplete (terminal=None verified=True steps=9)
  text_replace_c155c881228a: success (terminal=exit verified=True steps=7)
  permissions_1845d061f412: incomplete (terminal=None verified=True steps=30)
  text_replace_c1a29ebfe948: incomplete (terminal=None verified=False steps=30)
  permissions_367ba331169e: success (terminal=exit verified=True steps=13)
  text_replace_891a0c52f02e: success (terminal=exit verified=True steps=3)
  permissions_332774800f96: success (terminal=exit verified=True steps=4)
  text_replace_2e730553893e: premature_exit (terminal=exit verified=False steps=8)
  permissions_eff856415243: incomplete (terminal=None verified=True steps=29)
  text_replace_a0621d984fc0: incomplete (terminal=None verified=True steps=22)
  permissions_7030f2fc275a: incomplete (terminal=None verified=True steps=16)
  text_replace_2f46bf6b47ea: incomplete (terminal=None verified=False steps=7)

=== behavioural metrics ===
{
  "n": 16,
  "operate_rate": 1.0,
  "terminate_rate": 0.5,
  "verified_rate": 0.75,
  "clean_rate": 0.438
}