Can be served?

by prudant - opened Oct 29, 2025

vllm, or something like that for production ready high demand scenarios?

Knowledgator Engineering org Nov 17, 2025

Hi @prudant , we are working on an easy way to serve these models. I`ll update you when it's ready.

•

@prudant , you can serve it on triton as an onnx model with a python backend ensemble. That is pretty fast. Need higher demand than that?

thanks! do you have a sample script for that in order to not start from zero? best regards!

Hi @prudant , we are working on an easy way to serve these models. I`ll update you when it's ready.

any news?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment