---
license: apache-2.0
library_name: onnxruntime
base_model: iapp/OpenThai-SystemOne
tags:
  - onnx
  - onnxruntime
  - thai
  - qwen3.5
---

# OpenThai-SystemOne ONNX

This is a bundled ONNX Runtime conversion of `iapp/OpenThai-SystemOne` with
FP32, INT8, and INT4 versions.

The accepted structural limit is 262,144
tokens, inherited from the Qwen3.5 text backbone. PyTorch/ONNX numerical parity
is validated through 65,536 tokens only.

The model requires ONNX Runtime 1.30.0.

## Usage

Clone this model repository and run `uv sync --frozen` once to install its
locked helper package and runtime dependencies, then:

```python
from openthai_systemone_onnx import OnnxSystemOneClient, Choice

client = OnnxSystemOneClient("imtk/OpenThai-SystemOne-ONNX")
result = client.system_one(
    "บริการดีมาก",
    {"sentiment": Choice(instructions="Choose sentiment", criteria={"positive": None, "negative": None})},
)
```

Pass `precision="int8"` or `precision="int4"` to select a quantized graph.
* INT8 dynamically quantizes activations and uses per-channel signed weights.
* INT4 uses asymmetric 32-value weight blocks with FP32 activations.

The client subclasses the upstream `SystemOneClient`. Formatting, public methods,
response classes, decoding, confidence, permutations, and usage accounting are
inherited unchanged.

## Validation

See `artifact-manifest.json` for the tensor schema, operator domains, runtime
versions, source revisions, limits, and SHA-256 checksums.

### Benchmark on Thai Datasets

#### Result Summary

- **OpenThaiONNX and OpenThai produced identical predictions across all
  2,704 evaluated examples** (Cohen's kappa and raw agreement were both
  1.00). This benchmark only dips its toes in with up to 500 samples per task (204 for SIB-200). _Want to help it dive deeper? Larger and more thorough evaluation contributions are very welcome!_
  
- **That parity preserves OpenThai's accuracy advantage:** the OpenThai models
  averaged 76.5% accuracy across the six selected Thai CPU task configurations,
  compared with 60.6% for Laya. That is a gain of 15.9 percentage points, or a
  26.2% relative improvement.

- **OpenThaiONNX had 34.8% ± 4.4 percentage points lower CPU runtime than
  OpenThai** This makes the ONNX version a better fit for
  latency-sensitive, offline, and edge applications without a dedicated
  accelerator, provided the device has sufficient memory for the FP32 model. _Sadly, Laya still crosses the finish line about 10× sooner than both OpenThaiONNX and OpenThai. The good news: that leaves us a delightfully large optimization playground—contributions are very welcome!_lly
  large optimization playground, and contributions are very welcome!

#### CPU

| Dataset | Model | n | Type | Accuracy | ECE | Runtime (s) |
|---|---|---:|---|---:|---:|---:|
| xnli_th | ONNX | 500 | choice | 0.8100 | 0.0246 | 407.935 |
| xnli_th | OpenThai | 500 | choice | 0.8100 | 0.0246 | 686.813 |
| xnli_th | Laya | 500 | choice | 0.6740 | 0.2347 | 39.480 |
| | | | | | | |
| xnli_th | ONNX | 500 | noul | 0.8680 | 0.0247 | 407.935 |
| xnli_th | OpenThai | 500 | noul | 0.8680 | 0.0247 | 686.813 |
| xnli_th | Laya | 500 | noul | 0.8180 | 0.1511 | 39.480 |
| | | | | | | |
| sib200_th | ONNX | 204 | choice | 0.7843 | 0.0816 | 126.797 |
| sib200_th | OpenThai | 204 | choice | 0.7843 | 0.0816 | 199.093 |
| sib200_th | Laya | 204 | choice | 0.7500 | 0.1572 | 7.244 |
| | | | | | | |
| wongnai | ONNX | 500 | score | 0.6400 | 0.0428 | 2866.504 |
| wongnai | OpenThai | 500 | score | 0.6400 | 0.0428 | 4008.158 |
| wongnai | Laya | 500 | score | 0.2640 | 0.2493 | 167.471 |
| | | | | | | |
| wisesight | ONNX | 500 | choice | 0.4960 | 0.3603 | 1397.613 |
| wisesight | OpenThai | 500 | choice | 0.4960 | 0.3603 | 2125.524 |
| wisesight | Laya | 500 | choice | 0.2800 | 0.2753 | 91.182 |
| | | | | | | |
| xlam_tools | ONNX | 500 | choice | 0.9920 | 0.0072 | 448.700 |
| xlam_tools | OpenThai | 500 | choice | 0.9920 | 0.0072 | 682.976 |
| xlam_tools | Laya | 500 | choice | 0.8500 | 0.0598 | 80.015 |

#### GPU/MPS

| Dataset | Model | n | Type | Accuracy | ECE | Runtime (s) |
|---|---|---:|---|---:|---:|---:|
| xnli_th | ONNX | 500 | choice | ??? | ??? | ??? |
| xnli_th | OpenThai | 500 | choice | 0.8100 | 0.0246 | 165.242 |
| xnli_th | Laya | 500 | choice | 0.6700 | 0.2262 | 27.470 |
| | | | | | | |
| xnli_th | ONNX | 500 | noul | ??? | ??? | ??? |
| xnli_th | OpenThai | 500 | noul | 0.8680 | 0.0247 | 165.242 |
| xnli_th | Laya | 500 | noul | 0.7640 | 0.1072 | 27.470 |
| | | | | | | |
| sib200_th | OpenThai | 204 | choice | ??? | ??? | ??? |
| sib200_th | OpenThai | 204 | choice | 0.7843 | 0.0816 | 51.089 |
| sib200_th | Laya | 204 | choice | 0.6176 | 0.1412 | 5.103 |
| | | | | | | |
| wongnai | OpenThai | 500 | score | ??? | ??? | ??? |
| wongnai | OpenThai | 500 | score | 0.6400 | 0.0424 | 3178.868 |
| wongnai | Laya | 500 | score | 0.0340 | 0.0173 | 124.185 |
| | | | | | | |
| wisesight | OpenThai | 500 | choice | ??? | ??? | ??? |
| wisesight | OpenThai | 500 | choice | 0.4960 | 0.3603 | 2000.341 |
| wisesight | Laya | 500 | choice | 0.2500 | 0.2002 | 74.300 |
| | | | | | | |
| xlam_tools | OpenThai | 500 | choice | ??? | ??? | ??? |
| xlam_tools | OpenThai | 500 | choice | 0.9920 | 0.0072 | 609.601 |
| xlam_tools | Laya | 500 | choice | 0.8500 | 0.0598 | 45.588 |

### OpenThaiONNX vs OpenThai Agreement (Cohen's kappa)

| Task | n | Kappa |
|---|---:|---:|
| **Overall** | **2704** | **1.000000** |
| sib200_th / choice | 204 | 1.000000 |
| wisesight / choice | 500 | 1.000000 |
| wongnai / score | 500 | 1.000000 |
| xlam_tools / choice | 500 | 1.000000 |
| xnli_th / noul | 500 | 1.000000 |
| xnli_th / choice | 500 | 1.000000 |

## Limitations

- Quantized results are approximate and should be evaluated with the included
  held-out benchmark for the target workload.
- The model performs complete-request inference; generation caches are
  deliberately internal and reset for every call.
- 262,144 tokens is an accepted graph boundary, not a performance guarantee.