Yet another updated weights (did some more post training on sampling per group, then culled the dataset to focus more on my own corpus to help bolster it), seems to help?
Received additional training on my 7900XTX (AMP bfloat16, SDPA):
* a number of unshuffled samples from 12 seconds to 24 seconds, to verify that my ROCm setup works.
* another number of shuffled samples from 3 seconds to 32 seconds, to re-"teach" the model to work for any duration rather than the last duration it was trained against.
* another number of shuffled samples from 3 seconds to 60 seconds but with a RVQ distribution favoring the higher levels, to see if it lobotomizes the AR and clean up the NAR's audio (it seems fine).
