Inference API
Python SDK
The qev distribution contains both the SDK and the English playground. With Python
3.12, install the release wheel or the published PyPI distribution, then:
import qev
from pathlib import Path
model = qev.load() # Download missing model files; select CUDA when available.
request = qev.DecisionRequest.from_json(open("request.json", encoding="utf-8").read())
result = model.predict(request, image_root=Path("."))
print(result["answers"])
The first load retrieves about 4.6 GB of pinned model files. Later loads reuse the cache.
qev.load(device="cpu", cache_dir="/path/to/hub", offline=True) selects CPU and requires
complete local files. QEV_HOME sets the checkpoint/cache root; QEV_CACHE_DIR overrides
the Hub cache. The lower-level qev.QEV.load(...) remains available for explicit checkpoint
management and retains the original inference contract.
Commands
qev playground
qev download
qev predict --request request.json
qev serve --image-root .
All inference commands prepare missing model files automatically. Use --offline to
require a complete cache. predict resolves image paths relative to the request JSON;
serve resolves them under --image-root. The API server defaults to
http://127.0.0.1:8000; the playground defaults to http://127.0.0.1:7860.
See playground installation, uv and GPU instructions.
Request and response
The public request schema is qev.DecisionRequest. Text and one local image can be
combined. Image paths are resolved under the configured image root; remote URLs are not
fetched by the inference API.
{
"state": {"text": "The customer was charged twice and asks for a refund."},
"questions": {
"department": {
"type": "choice",
"instructions": "Which department should handle the request?",
"criteria": {"billing": "Payments and refunds", "technical": "Software faults"}
}
}
}
Send JSON to POST /v1/systemone. score.criteria is an ordered list of descriptions;
the result is an expected zero-based level. noul.criteria has true and false strings.
The maximum is four questions, 16 alternatives per choice/score and one image. Processed
inputs exceeding the configured token budget fail instead of silently truncating.
Inspect answers, probabilities, abstention fields and usage in the returned object.
The model ID remains Qwen/Qwen3.5-2B in the validated contract to identify the actual base.
See src/veyra/schema.py, src/veyra/probability.py and examples/request.json.
The playground adds image-upload and resolution endpoints around the same model. A single resident worker serializes inference. It is a trusted local demo; uploads are shared within that server session. Local latency measurements do not imply concurrent serving capacity.