| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3-Omni-30B-A3B-Instruct |
| base_model_relation: finetune |
| library_name: transformers |
| pipeline_tag: any-to-any |
| language: |
| - multilingual |
| tags: |
| - multimodal |
| - text |
| - audio |
| - image |
| - video |
| - decision-model |
| - choice |
| - qwen3-omni |
| - lora |
| - custom-code |
| - safetensors |
| --- |
| |
| # SPEV |
|
|
| SPEV is a 30B parameter omnimodal decision model that evaluates questions over text, |
| audio, images and video. It scores the supplied choices in one forward pass and |
| returns an answer with a probability for each choice. |
|
|
| ## Model |
|
|
| - **Backbone:** Qwen3-Omni-30B-A3B Thinker, with a trained LoRA adapter. |
| - **Decision head:** an evidence-attention head that scores each question independently. |
| - **Output:** per-choice probabilities and the highest-probability answer for each question. |
|
|
| ## Quickstart |
|
|
| The easiest way to get started is with the hosted API on Sprag (free through October 2026). |
| The full API docs are available [here](https://sprag.ai/docs/decisions). |
|
|
| ## Usage |
|
|
| Install the dependencies listed in `requirements.txt`, then load the model and processor: |
|
|
| ```python |
| import torch |
| from transformers import AutoModel, AutoProcessor |
| |
| model_id = "SpragAI/SPEV" |
| processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True) |
| model = AutoModel.from_pretrained( |
| model_id, |
| trust_remote_code=True, |
| dtype=torch.bfloat16, |
| device_map="auto", |
| attn_implementation="sdpa", |
| ) |
| |
| questions = { |
| "decision": { |
| "instructions": ( |
| "At the end of this conversation audio, has the most recent " |
| "speaker yielded the floor or are they continuing their turn?" |
| ), |
| "criteria": { |
| "keep_listening": ( |
| "The most recent speaker is continuing their turn; keep listening." |
| ), |
| "respond": ( |
| "The most recent speaker has yielded the floor; " |
| "another speaker can respond." |
| ), |
| }, |
| } |
| } |
| inputs = processor(audio="conversation.wav", state={}, questions=questions) |
| inputs = inputs.to(model.device, dtype=torch.bfloat16) |
| with torch.inference_mode(): |
| outputs = model(**inputs) |
| result = processor.post_process(outputs, questions=questions) |
| |
| print(result["answers"]["decision"]["choice"]) |
| print(result["answers"]["decision"]["probabilities"]) |
| ``` |
|
|
| ### Inputs |
|
|
| | Argument | Description | |
| |---|---| |
| | `audio` | Audio file to evaluate | |
| | `image` | Image file | |
| | `video` | Video file; visual frames | |
| | `state` | JSON context for the decision | |
| | `questions` | Mapping of question IDs to instructions and named criteria | |
|
|
| Media arguments are optional and can be combined. For text-only questions, supply |
| text through `state`. The processor resamples audio to the expected 16 kHz; callers |
| do not supply a sample-rate argument. Supply a video's soundtrack separately as |
| `audio` when it is part of the evidence. |
|
|
| ## Files |
|
|
| | File | Purpose | |
| |---|---| |
| | `model-*.safetensors`, `model.safetensors.index.json` | Backbone, adapter and decision-head weights | |
| | `config.json`, `configuration_spev.py` | Model architecture and scoring configuration | |
| | `modeling_spev.py` | Transformers model implementation | |
| | `decision_head.py`, `decision_layout.py`, `candidate_tree.py`, `branch_attention.py`, `token_operations.py` | Decision scoring and question isolation | |
| | `processing_spev.py`, `decision_utils.py` | Media preparation, question rendering and answer formatting | |
| | Tokenizer, processor and chat-template files | Text, audio, image and video processing | |
| | `requirements.txt` | Inference dependencies | |
| | `LICENSE` | Apache-2.0 license | |
|
|
| ## License |
|
|
| Apache-2.0. See [LICENSE](LICENSE) and [third-party notices](THIRD_PARTY_NOTICES.md). |
|
|
