joytag-onnx-Q8_0-quantized

Q8_0 Quantized JoyTag Model, working well with merely 88MiB in size (74.9% smaller), using 200+ MiB less RAM and achieving ~10% performance improvement.

It's fully compatible with your existing ONNXRuntime code. Replace the model file and it will work.

Precision Loss is expected. You may get unwanted outcome when using this model.

Deploying as a Server

You can refer to joytag-node, a simple Node.js implementation for serving JoyTag by me.

It will use approximately 750MiB of RAM (when using this quantized model), which is relatively a small amount. And it's fairly easy to run.

Dynamic Quantizing Used

Pre-process the original model for quantizing:

python3 -m onnxruntime.quantization.preprocess --input original-model.onnx --output input.onnx

Quantize the pre-processed model using Q8_0:

import onnx
from onnxruntime.quantization import quantize_dynamic, QuantType

quantize_dynamic(
        model_input="input.onnx",
        model_output="output.onnx",
        weight_type=QuantType.QInt8
)

License

This project is licensed under Apache-2.0 License, as the same as the original model.