"avg_ts": 32.210311
PROF total compute 3571.2 ms, barrier-wait 353.2 ms
PROF              ADD n= 26316  compute    34.30 ms  barrier    18.01 ms  (1.3 us/op compute, 0.7 us/op barrier)
PROF              RMS_NORM n=  8901  compute     0.35 ms  barrier     3.25 ms  (0.0 us/op compute, 0.4 us/op barrier)
PROF              GET_ROWS n=   387  compute     0.61 ms  barrier     0.19 ms  (1.6 us/op compute, 0.5 us/op barrier)
PROF              SET_ROWS n=  8772  compute    11.75 ms  barrier     5.78 ms  (1.3 us/op compute, 0.7 us/op barrier)
PROF              ROPE n=  8772  compute    12.65 ms  barrier    22.85 ms  (1.4 us/op compute, 2.6 us/op barrier)
PROF              FLASH_ATTN_EXT n=  4386  compute    82.19 ms  barrier    13.96 ms  (18.7 us/op compute, 3.2 us/op barrier)
PROF              GLU n=  4386  compute    40.82 ms  barrier     3.01 ms  (9.3 us/op compute, 0.7 us/op barrier)
PROF MUL_MAT      q4_K n= 26316  compute  2470.86 ms  barrier   236.91 ms  (93.9 us/op compute, 9.0 us/op barrier)
PROF MUL_MAT      q6_K n=  4515  compute   917.64 ms  barrier    49.19 ms  (203.2 us/op compute, 10.9 us/op barrier)
PROF_DONE