Mat-Classifier-0.5B (Qwen2.5-0.5B Decision Engine)

Mat-Classifier-0.5B is a lightweight, non-autoregressive multi-task decision-making model based on the Qwen/Qwen2.5-0.5B backbone. It is designed to act as a System 1 Decision Engine (Laya / Jev style) for math-related inputs, providing fast, single-pass structured evaluations instead of token-by-token generation.

Model Capabilities

The model features three parallel non-autoregressive decision heads attached directly to the pooled hidden state of the backbone:

1. Noul Head (Binary Flag): Determines whether a math problem requires multiple reasoning steps (Yes/No).
2. Choice Head (5-Class Categorical): Categorizes the math problem into domain-specific types (e.g., Arithmetic, Probability, Equations/Algebra, Fractions/Percentages, Geometry).
3. Score Head (5-Class Ordinal Scale): Evaluates and scales the estimated difficulty complexity of the task (0 to 4).

Performance & Evaluations

Evaluated on 7,473 preprocessed GSM8K problems:

• Noul Classification Accuracy: 99.75%
• Choice Classification Accuracy: 99.61%
• Score Classification Accuracy: 99.05%

Quick Usage (Inference)

import torch
import torch.nn as nn
from transformers import AutoModel, AutoTokenizer

class Qwen25_05B_DecisionEngine(nn.Module):
    def __init__(self, base_model_name="Qwen/Qwen2.5-0.5B", num_choices=5, num_scores=5):
        super().__init__()
        self.backbone = AutoModel.from_pretrained(base_model_name)
        hidden_size = self.backbone.config.hidden_size
        self.noul_head = nn.Linear(hidden_size, 1)
        self.choice_head = nn.Linear(hidden_size, num_choices)
        self.score_head = nn.Linear(hidden_size, num_scores)

    def forward(self, input_ids, attention_mask):
        outputs = self.backbone(input_ids=input_ids, attention_mask=attention_mask)
        pooled = torch.mean(outputs.last_hidden_state, dim=1).to(torch.float32)
        return {
            "noul_raw": self.noul_head(pooled).squeeze(-1),
            "choice_logits": self.choice_head(pooled),
            "score_logits": self.score_head(pooled)
        }

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("shimbaaa/mat-classifier-0.5b")