inclusionAI

/

Ling-3.0-tiny

Like 502

Follow

inclusionAI 3.14k

I built a docker image that is preset to run inference for Ling-3.0-tiny. These are intended to aid beginners, or just cut down on your workload. The current image is designed for sm_89 Lovelace GPU's. I can build images for other GPU families as well by request.

juiceb0xc0de/ling-3.0-tiny-cu130-torch213-py312-sm89

vllm chat --url http://localhost:30000/v1 --model-name auto --stats 

If you lack the VRAM for inference a RunPod template can be found here