Instructions to use isaxtsystems/isaxt-quiz-embeddings-v6 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use isaxtsystems/isaxt-quiz-embeddings-v6 with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("isaxtsystems/isaxt-quiz-embeddings-v6") model = AutoModel.from_pretrained("isaxtsystems/isaxt-quiz-embeddings-v6", device_map="auto") - Notebooks
- Google Colab
- Kaggle
iSAXT Quiz Embeddings v6 β built by iSAXT Systems
Built by iSAXT Systems Β· Product Family: ISAXT Β· Repository: isaxtsystems/isaxt-quiz-embeddings-v6 Β· Manifest branding: brand: VELXT, studio: VELXT Studio, family: ISAXT, domain: quiz, version: 6.
What it is β and what it is not
This is a retrieval encoder. It is not a chat model and it does not generate text.
You give it a trivia question or a paraphrase of one, and it returns a single 128-dimension vector. You compare that vector against vectors you have already stored, and you serve the answer you stored with the nearest question. The model never writes an answer, never invents one, and never sees an option list β it cannot choose A, B, C or D.
If you are looking for something that answers a question directly, this is the wrong artifact. If you want to find the closest question in your own corpus and return its verified answer verbatim, use this model as the retrieval component.
Intended use
iSAXT Quiz Embeddings v6 is a fine-tune of the v5 production encoder with corpus-mined hard negatives. It is intended for the same semantic-retrieval pattern as v5: embed a user question, compare it with stored question embeddings, and return the verified answer stored with the nearest match.
The artifact is a ranking improvement, not a knowledge upgrade. It was trained
on the same corpus family as v5 and adds no new questions, entities, languages
or categories. The direct training parent is v5; the architecture base is
prajjwal1/bert-tiny.
Model details
| Architecture | BertModel (2 layers, 2 attention heads, hidden 128, intermediate 512) |
| Base architecture | prajjwal1/bert-tiny |
| Direct training parent | iSAXT Quiz Embeddings v5 |
| Parameters | ~4.39M (17,547,912-byte FP32 weights file) |
| Output | 128-dim, mean-pooled over the attention mask, L2-normalized |
| Objective | Contrastive InfoNCE, temperature 0.05 |
| Loss implementation | MultipleNegativesRankingLoss with synthetic entity swaps from mine_hard_negatives.py --extreme (same template, different entity, Dice β₯ 0.85) appended to the pool |
| Curriculum | Enabled (1 warm-up epoch) |
| Epochs | 2 |
| Batch size | 16 |
| Learning rate | 1e-05 (10% linear warm-up) |
| Seed | 20261006 |
| Early stopping | Patience 2 (best epoch 2 was the final epoch) |
| Training examples | 923 hard-band contrastive examples |
| Validation examples | 118 |
| Recorded losses | train 0.144143 β 0.109314; best validation loss 0.075807 |
| Max sequence length | 128 tokens |
| Training device | CPU only β the trainer has no GPU code path |
| Training wall time | 0.22 CPU minutes |
The weights file is byte-identical to the measured candidate v6b (SHA-256
db02cccd97297c45f74135bbfe9b036f9a2ae08afb2e49e1e1420c9fc2e78262). The
release name is v6 because v6b is the measured winner; v6a and v6.1 were
measured and rejected and are not part of this artifact.
Usage
import numpy as np
import torch
from transformers import AutoTokenizer, AutoModel
model_id = "isaxtsystems/isaxt-quiz-embeddings-v6"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModel.from_pretrained(model_id).eval()
def embed(text: str) -> np.ndarray:
"""Encode one complete question into a normalized 128-d vector."""
normalized = " ".join(text.lower().split())
encoded = tokenizer(
normalized,
truncation=True,
max_length=128,
padding="max_length",
return_tensors="pt",
)
with torch.no_grad():
hidden = model(**encoded)[0]
mask = encoded["attention_mask"].unsqueeze(-1).to(hidden.dtype)
pooled = (hidden * mask).sum(dim=1) / mask.sum(dim=1).clamp(min=1.0)
return torch.nn.functional.normalize(pooled, p=2, dim=1)[0].cpu().numpy()
query = embed("who wrote the play hamlet")
corpus = [embed(q) for q in stored_questions]
best = int(np.argmax(corpus @ query)) # cosine, because both sides are unit norm
answer = stored_answers[best] # your stored answer, returned verbatim
Warning: pass the whole normalized string to the tokenizer, never a list of words. A list is interpreted as a batch of separate sequences, and
[0]would return the embedding of the first word only, not the full question.
ONNX
The prepared onnx/ bundle holds model.onnx (FP32, with its external-data
sidecar model.onnx.data) and model.int8.onnx (dynamic INT8 weights).
import onnxruntime as ort
session = ort.InferenceSession("onnx/model.int8.onnx", providers=["CPUExecutionProvider"])
The v5 release accepted batch size 1 only. That limitation is not reproduced in this export: both graphs accepted batch sizes 1, 2 and 3 in the measured gate on 2026-10-21. That is a measured range, not a guarantee for every batch size; benchmark your own runtime before sending large batches.
Training data
| Corpus at training time | 11,167 dev-tenant rows (the snapshot v6b was fine-tuned on) |
| Current tracked corpus | 11,177 rows total, of which 11,172 belong to the dev tenant (measured 2026-10-21) |
| Evaluation corpus | The current 11,172-row dev-tenant snapshot |
| Corpus growth since training | +5 dev-tenant rows; the identity was re-measured on the current snapshot, not assumed |
| Categories | 30 |
| Difficulty (current dev rows) | easy 5,504 / medium 3,360 / hard 2,224 / unspecified 84 |
| Negative mining | synthetic entity swaps from mine_hard_negatives.py --extreme (same template, different entity, Dice >= 0.85) |
| New data | None. No questions were generated and no vendor API was called for this fine-tune |
| Dedup | Exact + fuzzy (Dice > 0.95) + semantic (cosine > 0.98), zero duplicates, per the repository protocol |
| Provenance | Per row in the tracked corpus; Open Trivia DB rows carry CC-BY-SA 4.0 |
The five-row drift matters for reproducibility and is stated rather than hidden: the v6b weights predate the last corpus growth, while every metric below was re-measured on the current snapshot. Hard-band held-out rows re-mine byte-identically from the tracked scripts, corpus and recorded seed.
Performance (measured only)
All rows compare v5 and v6b on identical held-out rows from the current corpus. Measurements were recorded on 2026-10-18 (Task 103) and reproduced on 2026-10-21 (Task 110 identity proof), with no metric drift beyond tolerance.
Scope caveat (unchanged from RELEASE_v5): every acc@1 / MRR figure below ranks each example's positive candidate against its own per-example hard-negative pool (3-7 negatives), not against the full serving corpus. Full-corpus retrieval quality is a separate hybrid-search surface (Task 28 probe battery) and is not re-measured here.
| metric | v5 (production) | v6b (this artifact) | how measured |
|---|---|---|---|
| acc@1 | 0.9985 | 0.9995 | fresh mined broad hold-out, 2,043 rows |
| acc@3 | 1.0000 | 1.0000 | same 2,043 rows (derived from recorded per-row rank) |
| acc@5 | 1.0000 | 1.0000 | same 2,043 rows (derived; widest rank observed is 2) |
| MRR | 0.9993 | 0.9998 | same 2,043 rows |
| acc@1 | 1.0000 | 1.0000 | hard-band hold-out, 118 rows (byte-identical split) |
| acc@3 | 1.0000 | 1.0000 | same 118 rows (derived from recorded per-row rank) |
| acc@5 | 1.0000 | 1.0000 | same 118 rows (derived; max rank 1) |
| MRR | 1.0000 | 1.0000 | same 118 rows |
| acc@1 / MRR | 1.0000 / 1.0000 | 1.0000 / 1.0000 | general split, 96 rows |
| acc@3 | 1.0000 | 1.0000 | general split, 96 rows (derived; max rank 1) |
| acc@5 | 1.0000 | 1.0000 | general split, 96 rows (derived; max rank 1) |
| acc@1 / MRR | 1.0000 / 1.0000 | 1.0000 / 1.0000 | ml/data/splits_v4 general, 1,223 rows |
| acc@3 | 1.0000 | 1.0000 | ml/data/splits_v4 general, 1,223 rows (derived; max rank 1) |
| acc@5 | 1.0000 | 1.0000 | ml/data/splits_v4 general, 1,223 rows (derived; max rank 1) |
| mean entity-ranking margin | 0.1864 | 0.2138 (+14.7%) | 10-probe confusability battery |
| Austria/Australia separation cosine | 0.2059 | 0.1461 (lower is better) | same battery |
| mean separation cosine | 0.5706 | 0.5163 (lower is better) | same battery |
| ranking probes with target above confusable | 10/10 | 10/10 | same battery |
| torch vs FP32 ONNX | β | min 1.000000 over 30 probes | verify_onnx_roundtrip.py, floor 0.999 |
| torch vs INT8 ONNX | β | min 0.994727, mean 0.996750 | same run, floor 0.970 |
| FP32 vs INT8 ONNX | β | min 0.994727, mean 0.996750 | same run, floor 0.950 |
| exporter FP32/INT8 agreement | β | 0.998451 | single sample recorded in onnx/manifest.json |
The acc@3 and acc@5 rows are derived, not separately recorded: the holdout
reports (audit_task103/holdout_*_v5_v6b.json) store a per-row rank for every
example, so acc@k is the fraction of rows whose positive landed in the top k of
that example's own hard-negative pool. The widest rank observed anywhere in these
runs is 2, which is why acc@3 and acc@5 are both 1.0000 on every split rather
than a rounded figure. Read them as "the positive is always in the top three",
not as full-corpus recall. acc@5 carries less information than it appears to:
where an example's pool holds fewer than five candidates, top-5 is the whole pool
and the metric saturates by definition.
Read 0.9995 as "this corpus is well separated under per-example hard negatives", not as full-corpus accuracy. The absolute values are near the ceiling on every surface; the informative differences are the entity confusability margins, where v6b separates Austria/Australia better than v5 and the mean margin improves by 14.7%.
v6 measurements (published 2026-11-15)
| Metric | v5 | v6 | Delta |
|---|---|---|---|
| INT8 top-1 | 0.9100 | 0.9560 | +4.60pp |
| INT8 top-3 | 1.0000 | 1.0000 | 0 |
| INT8 MRR | 0.9530 | 0.9780 | +2.50pp |
| INT8 artifact size | 4.44 MB | 4.76 MB | +7% |
| Bundle size | 22.07 MB | 22.40 MB | +1.5% |
Published model: https://huggingface.co/isaxtsystems/isaxt-quiz-embeddings-v6 Training corpus: https://huggingface.co/datasets/isaxtsystems/quiz-corpus
These are Layer-3 INT8-versus-FP32 rank-agreement figures over the 500-query
sample, the same surface as the v5 baseline in ml/scripts/onnx_baseline.py
(top-1 0.910, top-3 1.0, top-5 1.0, MRR 0.953). They are a different surface
from the holdout accuracy in the table above and the two must not be read as one
series. Sizes are the seven artifact files in MiB.
Artifact Breakdown
Two artifacts ship separately, so they are sized separately. The ONNX bundle is
what onnxruntime loads at serving time; model.safetensors is the torch
encoder used for training and corpus embedding, and it is not part of the
bundle total.
| File | Size (Bytes) | Size (MiB) |
|---|---|---|
onnx/model.onnx (FP32 graph) |
306,219 | 0.29 |
onnx/model.onnx.data (FP32 external data) |
17,475,584 | 16.67 |
onnx/model.int8.onnx (INT8, self-contained) |
4,992,954 | 4.76 |
onnx/tokenizer.json |
711,661 | 0.68 |
onnx/tokenizer_config.json |
568 | <0.01 |
onnx/config.json |
668 | <0.01 |
onnx/manifest.json |
626 | <0.01 |
| ONNX bundle total (7 files) | 23,488,280 | 22.40 |
model.safetensors (torch encoder, not in the bundle) |
17,547,912 | 16.73 |
Known limitations (honest)
- Retrieval only. No generation, no answer selection, no option ranking.
- Tiny model.
bert-tinywith a 128-dim output. Good at "find the nearest question", limited at fine-grained entity distinctions. - No knowledge growth. v6 is a ranking fine-tune of v5 on the same corpus family; it does not add facts, categories, languages or coverage.
- Near-ceiling benchmark. Distractors are near-duplicates from the same category pool. The held-out numbers are not full-corpus accuracy and are not evidence of general language understanding.
- Templated corpus. 34% of queries have two candidates within 0.02 cosine of each other, and 49% have no neighbour above 0.60 similarity. INT8 reorders near-tied pairs rather than losing retrievals, so a few points of top-1 agreement at this error magnitude are an artefact of the metric and the data, not evidence of a worse quantizer.
- No accent folding.
JΓΊpiterandJupiterare different strings to the tokenizer. Normalize upstream. - Lowercase only. The encoder was trained on
normalize_text()output: lowercased, whitespace-collapsed. Match that preprocessing or the vectors will drift. - Not calibrated as a standalone accept/reject threshold. In the serving
stack (
SEMANTIC_SEPARABLE = false) cosine contributes ranking signal through Reciprocal Rank Fusion and never accepts a candidate alone. Do not invent a cosine cut-off from these numbers. - CPU-only training lineage. 0.22 CPU minutes, no GPU code path. A GPU host buys no speedup without a trainer change.
- Single-encoder database.
embedding_meta.encoderis one global stamp. Moving a serving database from v5 to v6 is an all-tenant re-embed against a freshly built database (ADR-015); mixing encoders is refused by design. - Batch range measured to 3. The graphs accepted batch 1, 2 and 3 on the measured runtime; larger batches were not tested. Verify before relying on them.
- Full-corpus retrieval not measured. The hybrid-search probe battery is a separate surface and was not re-run here.
License & Usage
This model is released under the Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC 4.0) license, with an available Commercial Licensing path.
Free Tier (Non-Commercial)
You CAN use this model for free if:
- β Personal Projects & Hobbies: You are building a free XAT bot, Discord bot, or open-source tool.
- β Research & Education: You are using it for academic or non-profit purposes.
- β Modification: You can fine-tune or tweak the model, provided you credit iSAXT Systems.
Commercial Tier
You CANNOT use this model for free if:
- β SaaS / Paid Services: You are hosting this model behind a paywall.
- β Commercial Products: You are integrating it into a product that generates revenue.
- β Reselling: You cannot sell the weights or derivatives.
If you are buying the isaxt-knowledge-engine platform (Self-hosted or SaaS), your purchase ALREADY INCLUDES a commercial sub-license for these weights.
For custom commercial licensing inquiries, please contact iSAXT Systems.
Attribution: If you use this model for free, you must credit "iSAXT Systems / VELXT".
Citation
@misc{isaxt2026quiz,
author = {iSAXT Systems},
title = {iSAXT Quiz Embeddings v6},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/isaxtsystems/isaxt-quiz-embeddings-v6}}
}
Versioning policy
Versions are immutable: v6 is frozen at publication and never renamed. v5 remains frozen and immutable; a retrain ships as v7 with its own card, manifest and evaluation table. Publishing requires a green round-trip gate plus explicit owner go-ahead, and the manifest version must be set (a null version is unreleased and must never be uploaded).
Multi-tenant platform note
The embedding build is strictly tenant-scoped: every corpus read and vector write carries the tenant identifier, and the pipeline refuses to embed a set that spans two tenants. The serving database holds one global encoder stamp, so switching encoders is an atomic all-tenant operation, not a per-tenant mix.
- Downloads last month
- 139
Model tree for isaxtsystems/isaxt-quiz-embeddings-v6
Base model
prajjwal1/bert-tiny