int8 versus fp32 recall and the 24 hand-written fragments
#1
by guicybercode - opened
Recall drops from 0.999 on the synthetic test corpus to 0.956 on the 24 hand-written fragments. Which labels account for most of the misses there? Names in lowercase or inside long legal sentences are where I would expect trouble, especially with max length 128 during training. Did you compare the int8 per-channel export with the fp32 checkpoint on those same fragments? On a 109 MB model running on the user's own computer, knowing whether quantization or the training data causes the misses would help me decide how to use it.