ONNX
#1
by cnmoro - opened
Any chance we can get an ONNX version with examples?
Would be awesome.
For context:
Spinning up a container with the "transformers" lib eats a lot of disk. Onnx and onnxruntime are very lightweight
Also, kudos, the models are awesome.
Well, I've managed to do it:
https://gist.github.com/cnmoro/3c66de4f92716e8cf044e550a23ee9d2
This gist contains code for converting to onnx, quantizing to 8bit and performing inference.
I will be uploading some of these in my profile.
cnmoro changed discussion status to closed
cnmoro changed discussion status to open
cnmoro changed discussion status to closed