| ---
|
| library_name: setfit
|
| license: mit
|
| base_model: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
|
| tags:
|
| - setfit
|
| - onnx
|
| - attention-weights
|
| - context-compression
|
| - intent-classification
|
| - multilingual
|
| pipeline_tag: text-classification
|
| ---
|
|
|
| # SetFit Multilingual OVR Router (ONNX with Attentions)
|
|
|
| This is a State-of-the-Art **SetFit** model exported to **ONNX** format, specifically trained to classify LLM tasks into three semantic categories: **Needle** (Fact Retrieval), **Reasoning** (Logic/Analysis), and **Summary** (General Recap).
|
|
|
| The model is based on [paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) and has been modified to expose **all 12 layers of raw attention weights**.
|
|
|
| ## Key Features
|
|
|
| - **3-Class Classification:** High-precision separation of intents.
|
| - **Multilingual:** Native support for Russian, English, and 50+ other languages.
|
| - **Attention Output:** Every inference returns a full attention matrix `(batch, heads, seq_len, seq_len)` for all 12 layers.
|
| - **Dual Precision:** Both **FP32** (`model.onnx`) and **INT8 Quantized** (`model_quantized.onnx`) versions are available.
|
| - **Optimized for CPU:** Fast ONNX inference via `onnxruntime`.
|
|
|
| ## Classification Map
|
| - **Label 0:** Summary (Chatter, Recaps, TL;DR)
|
| - **Label 1:** Needle (Pinpoint facts, parameters, keys, IPs)
|
| - **Label 2:** Reasoning (Comparison, analysis, code debugging, logical chains)
|
|
|
| ## Project Origin
|
|
|
| This model is a core component of the **[WAMP-proxy](https://github.com/naranor/wamp-proxy)** project, an intelligent middleware for research into LLM context optimization.
|
|
|
| ## Quick Inference (Python)
|
|
|
| ```python
|
| import numpy as np
|
| import onnxruntime as ort
|
| from transformers import AutoTokenizer
|
| import json
|
|
|
| # 1. Load model and weights
|
| session = ort.InferenceSession("model.onnx")
|
| tokenizer = AutoTokenizer.from_pretrained(".")
|
| with open("router_weights_setfit.json", "r") as f:
|
| weights = json.load(f)
|
|
|
| # 2. Prepare Input
|
| text = "What is the database port?"
|
| inputs = tokenizer(text, return_tensors="np")
|
| onnx_inputs = {
|
| "input_ids": inputs["input_ids"].astype(np.int64),
|
| "attention_mask": inputs["attention_mask"].astype(np.int64)
|
| }
|
|
|
| # 3. Run
|
| outputs = session.run(None, onnx_inputs)
|
| embeddings = np.mean(outputs[0], axis=1) # Mean pooling
|
|
|
| # 4. Predict probabilities (LogReg Head)
|
| scores = np.dot(embeddings, np.array(weights["coef"]).T) + weights["intercept"]
|
| probs = np.exp(scores) / np.exp(scores).sum()
|
| print(f"Probabilities: {probs}")
|
| ```
|
|
|
| ## License
|
| MIT License.
|
|
|
