Instructions to use vllm-sr/vela3-lite with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use vllm-sr/vela3-lite with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("zero-shot-classification", model="vllm-sr/vela3-lite", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("vllm-sr/vela3-lite", trust_remote_code=True) model = AutoModel.from_pretrained("vllm-sr/vela3-lite", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Return invalid_question for malformed span thresholds and instructions
Most malformed span questions get a per-question invalid_question answer while the rest of the request is answered. Two did not: "threshold": null reached float(None), and instructions given as an object reached the prompt builder's "\n".join. Both raised TypeError out of system_one, so the whole request failed, including the valid questions next to it.
span_answer now checks both before running the model. Instructions must be a string or absent, and a threshold must be a number or a string, since numeric strings were already accepted through float() and a non-numeric one already got invalid_question. Anything else gets invalid_question with "span instructions must be a string" or "threshold must be a number".
Tested on vela3-edge on CPU with a yes/no question next to each malformed span question. A null threshold and object instructions now return invalid_question and the yes/no question is still answered (0.962, as with the other malformed cases). Thresholds of 0.3, 1 and "0.3" and empty instructions give the same answers as before. The change is the same in all six sizes.
Thanks, merged.