| --- |
| license: bsd-3-clause |
| language: |
| - en |
| library_name: transformers |
| pipeline_tag: text-classification |
| tags: |
| - bert |
| - bert-mini |
| - classification |
| - decision-model |
| - dynamic-classification |
| - calibrated-classification |
| - option-scoring |
| - oryn |
| datasets: |
| - zefang-liu/phishing-email-dataset |
| - fancyzhx/ag_news |
| - google/boolq |
| - SetFit/sst5 |
| metrics: |
| - accuracy |
| - brier_score |
| - expected_calibration_error |
| base_model: |
| - boltuix/bert-mini |
| --- |
| |
| # Oryn |
|
|
| Oryn is a compact, low-latency decision model built on |
| [`boltuix/bert-mini`](https://huggingface.co/boltuix/bert-mini). |
|
|
| Unlike a conventional classifier with a fixed output layer, Oryn receives the |
| classification question and candidate answers at inference time. This allows |
| the same model to handle binary decisions, multiclass classification, ordinal |
| scoring, and other option-based tasks. |
|
|
| Oryn is inspired by the publicly described architecture and training ideas |
| behind Laya. |
|
|
| ## Highlights |
|
|
| - Approximately 21 MB of trained weights |
| - Approximately 3 ms warmed-up single-request latency in the author's local test |
| - Dynamically supplied classification options |
| - Supports binary, categorical, and ordinal questions |
| - Uses one `[MASK]` marker for each candidate answer |
| - Scores all supplied options relative to one another |
| - Trained on phishing detection, BoolQ, AG News, and SST-5 |
| - Designed for local and CPU-friendly inference |
|
|
| Latency depends on the device, runtime, input length, and measurement method. |
| The reported 3 ms result should not be treated as a universal benchmark. |
|
|
| ## Architecture |
|
|
| Oryn consists of: |
|
|
| 1. A pretrained BERT Mini encoder |
| 2. Two additional Transformer encoder layers |
| 3. One `[MASK]` marker for each candidate answer |
| 4. A learned linear option scorer |
| 5. A softmax over the options provided with the request |
|
|
| A request is represented approximately as: |
|
|
| ```text |
| question: Which category best describes this text? |
| state: The company reported record quarterly revenue. |
| options: business [MASK], sports [MASK], technology [MASK], politics [MASK] |
| ``` |
|
|
| The contextual representation at each option's `[MASK]` token is passed through |
| the shared scoring layer. The resulting logits are normalized across the |
| candidate options. |
|
|
| Because the answer space is supplied at request time, it is not permanently |
| encoded into a fixed classification head. |
|
|
| ## Evaluation |
|
|
| Oryn was evaluated on a random sample of 1,000 held-out examples from its |
| multitask validation set. |
|
|
| | Metric | Result | |
| |---|---:| |
| | Overall accuracy | 71.7% | |
| | Brier score | 0.3468 | |
| | Negative log-likelihood | 0.6228 | |
| | Expected calibration error | 0.0381 | |
| | Average confidence | 0.7518 | |
| | Batched throughput | 129.9 examples/second | |
|
|
| ### Results by question type |
|
|
| | Question type | Examples | Accuracy | Brier score | |
| |---|---:|---:|---:| |
| | Binary / `noul` | 591 | 85.62% | 0.2083 | |
| | Categorical / `choice` | 196 | 79.59% | 0.2793 | |
| | Ordinal / `score` | 213 | 25.82% | 0.7933 | |
|
|
| The ordinal sentiment task is currently the model's main weakness. The model is |
| substantially stronger on binary and categorical decisions. |
|
|
| The benchmark was performed with batched GPU inference. Batched latency is not |
| equivalent to end-to-end single-request latency. |
|
|
| ## Training data |
|
|
| Oryn was trained on a mixture of: |
|
|
| - `zefang-liu/phishing-email-dataset` |
| - `fancyzhx/ag_news` |
| - `google/boolq` |
| - `SetFit/sst5` |
|
|
| These datasets expose the model to: |
|
|
| - Phishing and fraud detection |
| - Binary reading comprehension |
| - News topic classification |
| - Five-level ordinal sentiment classification |
|
|
| ## Training method |
|
|
| Training used two phases: |
|
|
| 1. Supervised cross-entropy warmup |
| 2. Calibration-oriented optimization using differentiable proper-scoring |
| objectives |
|
|
| For nominal decisions, the calibration objective combines cross-entropy with a |
| spherical scoring term. For ordinal decisions, cross-entropy is combined with |
| ranked probability score. |
|
|
| This training procedure is Laya-inspired, but it does not reproduce the |
| original RLCD/REINFORCE procedure exactly. |
|
|
| ## Intended uses |
|
|
| Oryn is suitable for experimentation with: |
|
|
| - Message triage |
| - Phishing detection |
| - Topic classification |
| - Sentiment classification |
| - Support-request routing |
| - Urgency classification |
| - Small local decision systems |
| - Dynamic classification research |
| - Low-latency edge or local inference |
|
|
| ## General classification |
|
|
| Oryn can technically receive any question and any set of options: |
|
|
| ```python |
| result = predict( |
| text="The company reported record quarterly revenue.", |
| question="Which category best describes this text?", |
| options=[ |
| "business", |
| "sports", |
| "technology", |
| "politics", |
| ], |
| ) |
| ``` |
|
|
| However, architectural support for arbitrary options does not guarantee |
| reliable knowledge of arbitrary domains. |
|
|
| Oryn is most reliable on tasks similar to its training mixture. New domains |
| should be evaluated and usually fine-tuned before production use. |
|
|
| ## Example output |
|
|
| ```python |
| { |
| "prediction": "business", |
| "confidence": 0.8421, |
| "probabilities": { |
| "business": 0.8421, |
| "sports": 0.0314, |
| "technology": 0.0987, |
| "politics": 0.0278 |
| } |
| } |
| ``` |
|
|
| ## Files |
|
|
|
|
| The custom `inference.py` implementation is required because Oryn uses a custom |
| option-scoring head rather than a standard Transformers sequence-classification |
| head. |
|
|
| ## Installation |
|
|
| ```bash |
| pip install torch transformers |
| ``` |
|
|
| ## Inference |
|
|
| ```bash |
| python inference.py \ |
| --text "The company reported record quarterly revenue." \ |
| --question "Which category best describes this text?" \ |
| --options business sports technology politics |
| ``` |
|
|
| ## Citation |
|
|
| If you use Oryn, cite this model repository: |
|
|
| ```bibtex |
| @misc{oryn_bert_mini, |
| title = {Oryn: A compact low-latency BERT Mini option-scoring model}, |
| author = {Timo Sarkar}, |
| year = {2026}, |
| howpublished = {Hugging Face}, |
| note = {A compact low-latency BERT Mini option-scoring model} |
| } |
| ``` |
