File size: 572 Bytes
0d06934 | 1 2 3 4 5 6 7 8 9 | """Weight-only int8 (per-channel symmetric) of an encoder package: the 24-layer fp16 encoder does not fit the ANE
compiler on M1, the int8 one does (as FluidAudio's Parakeet encoders). Usage: python quantize.py <in.mlpackage> <out.mlpackage>"""
import sys
import coremltools as ct
import coremltools.optimize.coreml as cto
m = ct.models.MLModel(sys.argv[1], skip_model_load=True)
cfg = cto.OptimizationConfig(global_config=cto.OpLinearQuantizerConfig(mode="linear_symmetric", dtype="int8", granularity="per_channel"))
cto.linear_quantize_weights(m, cfg).save(sys.argv[2])
|