Hugging Face's logo

SpragAI
/
SPEV

Any-to-Any
Transformers
Safetensors
multilingual
spev
feature-extraction
multimodal
text
audio
image
video
decision-model
choice
qwen3-omni
lora
custom-code
custom_code
Model card Files
xet
 Community
SPEV
File size: 3,709 Bytes
7381480
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2e0601e
7381480
 
235e7a1
 
 
 
 
7381480
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2e0601e
7381480
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
---
license: apache-2.0
base_model: Qwen/Qwen3-Omni-30B-A3B-Instruct
base_model_relation: finetune
library_name: transformers
pipeline_tag: any-to-any
language:
- multilingual
tags:
- multimodal
- text
- audio
- image
- video
- decision-model
- choice
- qwen3-omni
- lora
- custom-code
- safetensors
---

# SPEV

SPEV is a 30B parameter omnimodal decision model that evaluates questions over text,
audio, images and video. It scores the supplied choices in one forward pass and
returns an answer with a probability for each choice.

## Model

- **Backbone:** Qwen3-Omni-30B-A3B Thinker, with a trained LoRA adapter.
- **Decision head:** an evidence-attention head that scores each question independently.
- **Output:** per-choice probabilities and the highest-probability answer for each question.

## Quickstart

The easiest way to get started is with the hosted API on Sprag (free through October 2026). 
The full API docs are available [here](https://sprag.ai/docs/decisions).

## Usage

Install the dependencies listed in `requirements.txt`, then load the model and processor:

```python
import torch
from transformers import AutoModel, AutoProcessor

model_id = "SpragAI/SPEV"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="sdpa",
)

questions = {
    "decision": {
        "instructions": (
            "At the end of this conversation audio, has the most recent "
            "speaker yielded the floor or are they continuing their turn?"
        ),
        "criteria": {
            "keep_listening": (
                "The most recent speaker is continuing their turn; keep listening."
            ),
            "respond": (
                "The most recent speaker has yielded the floor; "
                "another speaker can respond."
            ),
        },
    }
}
inputs = processor(audio="conversation.wav", state={}, questions=questions)
inputs = inputs.to(model.device, dtype=torch.bfloat16)
with torch.inference_mode():
    outputs = model(**inputs)
result = processor.post_process(outputs, questions=questions)

print(result["answers"]["decision"]["choice"])
print(result["answers"]["decision"]["probabilities"])
```

### Inputs

| Argument | Description |
|---|---|
| `audio` | Audio file to evaluate |
| `image` | Image file |
| `video` | Video file; visual frames |
| `state` | JSON context for the decision |
| `questions` | Mapping of question IDs to instructions and named criteria |

Media arguments are optional and can be combined. For text-only questions, supply
text through `state`. The processor resamples audio to the expected 16 kHz; callers
do not supply a sample-rate argument. Supply a video's soundtrack separately as
`audio` when it is part of the evidence.

## Files

| File | Purpose |
|---|---|
| `model-*.safetensors`, `model.safetensors.index.json` | Backbone, adapter and decision-head weights |
| `config.json`, `configuration_spev.py` | Model architecture and scoring configuration |
| `modeling_spev.py` | Transformers model implementation |
| `decision_head.py`, `decision_layout.py`, `candidate_tree.py`, `branch_attention.py`, `token_operations.py` | Decision scoring and question isolation |
| `processing_spev.py`, `decision_utils.py` | Media preparation, question rendering and answer formatting |
| Tokenizer, processor and chat-template files | Text, audio, image and video processing |
| `requirements.txt` | Inference dependencies |
| `LICENSE` | Apache-2.0 license |

## License

Apache-2.0. See [LICENSE](LICENSE) and [third-party notices](THIRD_PARTY_NOTICES.md).