Instructions to use vllm-sr/vela3-lite with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use vllm-sr/vela3-lite with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("zero-shot-classification", model="vllm-sr/vela3-lite", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("vllm-sr/vela3-lite", trust_remote_code=True) model = AutoModel.from_pretrained("vllm-sr/vela3-lite", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Keep each thread's adapter gate to itself
The adapter gate in vela3_adapters.py was one object for the whole process. active() set its adapter name and row mask, and every MultiLoRA.forward read them, so when two threads called system_one on the same model, one thread's passes ran with the other's adapter. On vela3-edge on CPU, with one thread asking span questions in a loop and another asking the same refund question 25 times, all 25 answers came back wrong: 0.736 to 0.820 against 0.956 when asked alone.
The gate is now a threading.local, so each thread sees only the adapter it switched on, and a new thread starts with every adapter off. Nothing else changes: active(), MultiLoRA.forward and the graph check in modeling_vela3.py read the gate as before, now on the calling thread.
Tested on vela3-edge on CPU. In the same two-thread run, 0 of 25 answers now differ from the answer asked alone. One request each over three states, with the nine router questions, a hazard question, an ordinary choice and yes/no question and three span questions (PII, relation, entities), gives byte-identical JSON before and after. The change is the same in all six sizes.
Thanks, merged.