iSAXT Quiz Embeddings v6 β€” built by iSAXT Systems

Built by iSAXT Systems Β· Product Family: ISAXT Β· Repository: isaxtsystems/isaxt-quiz-embeddings-v6 Β· Manifest branding: brand: VELXT, studio: VELXT Studio, family: ISAXT, domain: quiz, version: 6.

What it is β€” and what it is not

This is a retrieval encoder. It is not a chat model and it does not generate text.

You give it a trivia question or a paraphrase of one, and it returns a single 128-dimension vector. You compare that vector against vectors you have already stored, and you serve the answer you stored with the nearest question. The model never writes an answer, never invents one, and never sees an option list β€” it cannot choose A, B, C or D.

If you are looking for something that answers a question directly, this is the wrong artifact. If you want to find the closest question in your own corpus and return its verified answer verbatim, use this model as the retrieval component.

Intended use

iSAXT Quiz Embeddings v6 is a fine-tune of the v5 production encoder with corpus-mined hard negatives. It is intended for the same semantic-retrieval pattern as v5: embed a user question, compare it with stored question embeddings, and return the verified answer stored with the nearest match.

The artifact is a ranking improvement, not a knowledge upgrade. It was trained on the same corpus family as v5 and adds no new questions, entities, languages or categories. The direct training parent is v5; the architecture base is prajjwal1/bert-tiny.

Model details

Architecture BertModel (2 layers, 2 attention heads, hidden 128, intermediate 512)
Base architecture prajjwal1/bert-tiny
Direct training parent iSAXT Quiz Embeddings v5
Parameters ~4.39M (17,547,912-byte FP32 weights file)
Output 128-dim, mean-pooled over the attention mask, L2-normalized
Objective Contrastive InfoNCE, temperature 0.05
Loss implementation MultipleNegativesRankingLoss with synthetic entity swaps from mine_hard_negatives.py --extreme (same template, different entity, Dice β‰₯ 0.85) appended to the pool
Curriculum Enabled (1 warm-up epoch)
Epochs 2
Batch size 16
Learning rate 1e-05 (10% linear warm-up)
Seed 20261006
Early stopping Patience 2 (best epoch 2 was the final epoch)
Training examples 923 hard-band contrastive examples
Validation examples 118
Recorded losses train 0.144143 β†’ 0.109314; best validation loss 0.075807
Max sequence length 128 tokens
Training device CPU only β€” the trainer has no GPU code path
Training wall time 0.22 CPU minutes

The weights file is byte-identical to the measured candidate v6b (SHA-256 db02cccd97297c45f74135bbfe9b036f9a2ae08afb2e49e1e1420c9fc2e78262). The release name is v6 because v6b is the measured winner; v6a and v6.1 were measured and rejected and are not part of this artifact.

Usage

import numpy as np
import torch
from transformers import AutoTokenizer, AutoModel

model_id = "isaxtsystems/isaxt-quiz-embeddings-v6"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModel.from_pretrained(model_id).eval()

def embed(text: str) -> np.ndarray:
    """Encode one complete question into a normalized 128-d vector."""
    normalized = " ".join(text.lower().split())
    encoded = tokenizer(
        normalized,
        truncation=True,
        max_length=128,
        padding="max_length",
        return_tensors="pt",
    )
    with torch.no_grad():
        hidden = model(**encoded)[0]
    mask = encoded["attention_mask"].unsqueeze(-1).to(hidden.dtype)
    pooled = (hidden * mask).sum(dim=1) / mask.sum(dim=1).clamp(min=1.0)
    return torch.nn.functional.normalize(pooled, p=2, dim=1)[0].cpu().numpy()

query = embed("who wrote the play hamlet")
corpus = [embed(q) for q in stored_questions]
best = int(np.argmax(corpus @ query))          # cosine, because both sides are unit norm
answer = stored_answers[best]                  # your stored answer, returned verbatim

Warning: pass the whole normalized string to the tokenizer, never a list of words. A list is interpreted as a batch of separate sequences, and [0] would return the embedding of the first word only, not the full question.

ONNX

The prepared onnx/ bundle holds model.onnx (FP32, with its external-data sidecar model.onnx.data) and model.int8.onnx (dynamic INT8 weights).

import onnxruntime as ort
session = ort.InferenceSession("onnx/model.int8.onnx", providers=["CPUExecutionProvider"])

The v5 release accepted batch size 1 only. That limitation is not reproduced in this export: both graphs accepted batch sizes 1, 2 and 3 in the measured gate on 2026-10-21. That is a measured range, not a guarantee for every batch size; benchmark your own runtime before sending large batches.

Training data

Corpus at training time 11,167 dev-tenant rows (the snapshot v6b was fine-tuned on)
Current tracked corpus 11,177 rows total, of which 11,172 belong to the dev tenant (measured 2026-10-21)
Evaluation corpus The current 11,172-row dev-tenant snapshot
Corpus growth since training +5 dev-tenant rows; the identity was re-measured on the current snapshot, not assumed
Categories 30
Difficulty (current dev rows) easy 5,504 / medium 3,360 / hard 2,224 / unspecified 84
Negative mining synthetic entity swaps from mine_hard_negatives.py --extreme (same template, different entity, Dice >= 0.85)
New data None. No questions were generated and no vendor API was called for this fine-tune
Dedup Exact + fuzzy (Dice > 0.95) + semantic (cosine > 0.98), zero duplicates, per the repository protocol
Provenance Per row in the tracked corpus; Open Trivia DB rows carry CC-BY-SA 4.0

The five-row drift matters for reproducibility and is stated rather than hidden: the v6b weights predate the last corpus growth, while every metric below was re-measured on the current snapshot. Hard-band held-out rows re-mine byte-identically from the tracked scripts, corpus and recorded seed.

Performance (measured only)

All rows compare v5 and v6b on identical held-out rows from the current corpus. Measurements were recorded on 2026-10-18 (Task 103) and reproduced on 2026-10-21 (Task 110 identity proof), with no metric drift beyond tolerance.

Scope caveat (unchanged from RELEASE_v5): every acc@1 / MRR figure below ranks each example's positive candidate against its own per-example hard-negative pool (3-7 negatives), not against the full serving corpus. Full-corpus retrieval quality is a separate hybrid-search surface (Task 28 probe battery) and is not re-measured here.

metric v5 (production) v6b (this artifact) how measured
acc@1 0.9985 0.9995 fresh mined broad hold-out, 2,043 rows
acc@3 1.0000 1.0000 same 2,043 rows (derived from recorded per-row rank)
acc@5 1.0000 1.0000 same 2,043 rows (derived; widest rank observed is 2)
MRR 0.9993 0.9998 same 2,043 rows
acc@1 1.0000 1.0000 hard-band hold-out, 118 rows (byte-identical split)
acc@3 1.0000 1.0000 same 118 rows (derived from recorded per-row rank)
acc@5 1.0000 1.0000 same 118 rows (derived; max rank 1)
MRR 1.0000 1.0000 same 118 rows
acc@1 / MRR 1.0000 / 1.0000 1.0000 / 1.0000 general split, 96 rows
acc@3 1.0000 1.0000 general split, 96 rows (derived; max rank 1)
acc@5 1.0000 1.0000 general split, 96 rows (derived; max rank 1)
acc@1 / MRR 1.0000 / 1.0000 1.0000 / 1.0000 ml/data/splits_v4 general, 1,223 rows
acc@3 1.0000 1.0000 ml/data/splits_v4 general, 1,223 rows (derived; max rank 1)
acc@5 1.0000 1.0000 ml/data/splits_v4 general, 1,223 rows (derived; max rank 1)
mean entity-ranking margin 0.1864 0.2138 (+14.7%) 10-probe confusability battery
Austria/Australia separation cosine 0.2059 0.1461 (lower is better) same battery
mean separation cosine 0.5706 0.5163 (lower is better) same battery
ranking probes with target above confusable 10/10 10/10 same battery
torch vs FP32 ONNX β€” min 1.000000 over 30 probes verify_onnx_roundtrip.py, floor 0.999
torch vs INT8 ONNX β€” min 0.994727, mean 0.996750 same run, floor 0.970
FP32 vs INT8 ONNX β€” min 0.994727, mean 0.996750 same run, floor 0.950
exporter FP32/INT8 agreement β€” 0.998451 single sample recorded in onnx/manifest.json

The acc@3 and acc@5 rows are derived, not separately recorded: the holdout reports (audit_task103/holdout_*_v5_v6b.json) store a per-row rank for every example, so acc@k is the fraction of rows whose positive landed in the top k of that example's own hard-negative pool. The widest rank observed anywhere in these runs is 2, which is why acc@3 and acc@5 are both 1.0000 on every split rather than a rounded figure. Read them as "the positive is always in the top three", not as full-corpus recall. acc@5 carries less information than it appears to: where an example's pool holds fewer than five candidates, top-5 is the whole pool and the metric saturates by definition.

Read 0.9995 as "this corpus is well separated under per-example hard negatives", not as full-corpus accuracy. The absolute values are near the ceiling on every surface; the informative differences are the entity confusability margins, where v6b separates Austria/Australia better than v5 and the mean margin improves by 14.7%.

v6 measurements (published 2026-11-15)

Metric v5 v6 Delta
INT8 top-1 0.9100 0.9560 +4.60pp
INT8 top-3 1.0000 1.0000 0
INT8 MRR 0.9530 0.9780 +2.50pp
INT8 artifact size 4.44 MB 4.76 MB +7%
Bundle size 22.07 MB 22.40 MB +1.5%

Published model: https://huggingface.co/isaxtsystems/isaxt-quiz-embeddings-v6 Training corpus: https://huggingface.co/datasets/isaxtsystems/quiz-corpus

These are Layer-3 INT8-versus-FP32 rank-agreement figures over the 500-query sample, the same surface as the v5 baseline in ml/scripts/onnx_baseline.py (top-1 0.910, top-3 1.0, top-5 1.0, MRR 0.953). They are a different surface from the holdout accuracy in the table above and the two must not be read as one series. Sizes are the seven artifact files in MiB.

Artifact Breakdown

Two artifacts ship separately, so they are sized separately. The ONNX bundle is what onnxruntime loads at serving time; model.safetensors is the torch encoder used for training and corpus embedding, and it is not part of the bundle total.

File Size (Bytes) Size (MiB)
onnx/model.onnx (FP32 graph) 306,219 0.29
onnx/model.onnx.data (FP32 external data) 17,475,584 16.67
onnx/model.int8.onnx (INT8, self-contained) 4,992,954 4.76
onnx/tokenizer.json 711,661 0.68
onnx/tokenizer_config.json 568 <0.01
onnx/config.json 668 <0.01
onnx/manifest.json 626 <0.01
ONNX bundle total (7 files) 23,488,280 22.40
model.safetensors (torch encoder, not in the bundle) 17,547,912 16.73

Known limitations (honest)

  • Retrieval only. No generation, no answer selection, no option ranking.
  • Tiny model. bert-tiny with a 128-dim output. Good at "find the nearest question", limited at fine-grained entity distinctions.
  • No knowledge growth. v6 is a ranking fine-tune of v5 on the same corpus family; it does not add facts, categories, languages or coverage.
  • Near-ceiling benchmark. Distractors are near-duplicates from the same category pool. The held-out numbers are not full-corpus accuracy and are not evidence of general language understanding.
  • Templated corpus. 34% of queries have two candidates within 0.02 cosine of each other, and 49% have no neighbour above 0.60 similarity. INT8 reorders near-tied pairs rather than losing retrievals, so a few points of top-1 agreement at this error magnitude are an artefact of the metric and the data, not evidence of a worse quantizer.
  • No accent folding. JΓΊpiter and Jupiter are different strings to the tokenizer. Normalize upstream.
  • Lowercase only. The encoder was trained on normalize_text() output: lowercased, whitespace-collapsed. Match that preprocessing or the vectors will drift.
  • Not calibrated as a standalone accept/reject threshold. In the serving stack (SEMANTIC_SEPARABLE = false) cosine contributes ranking signal through Reciprocal Rank Fusion and never accepts a candidate alone. Do not invent a cosine cut-off from these numbers.
  • CPU-only training lineage. 0.22 CPU minutes, no GPU code path. A GPU host buys no speedup without a trainer change.
  • Single-encoder database. embedding_meta.encoder is one global stamp. Moving a serving database from v5 to v6 is an all-tenant re-embed against a freshly built database (ADR-015); mixing encoders is refused by design.
  • Batch range measured to 3. The graphs accepted batch 1, 2 and 3 on the measured runtime; larger batches were not tested. Verify before relying on them.
  • Full-corpus retrieval not measured. The hybrid-search probe battery is a separate surface and was not re-run here.

License & Usage

This model is released under the Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC 4.0) license, with an available Commercial Licensing path.

Free Tier (Non-Commercial)

You CAN use this model for free if:

  • βœ… Personal Projects & Hobbies: You are building a free XAT bot, Discord bot, or open-source tool.
  • βœ… Research & Education: You are using it for academic or non-profit purposes.
  • βœ… Modification: You can fine-tune or tweak the model, provided you credit iSAXT Systems.

Commercial Tier

You CANNOT use this model for free if:

  • ❌ SaaS / Paid Services: You are hosting this model behind a paywall.
  • ❌ Commercial Products: You are integrating it into a product that generates revenue.
  • ❌ Reselling: You cannot sell the weights or derivatives.

If you are buying the isaxt-knowledge-engine platform (Self-hosted or SaaS), your purchase ALREADY INCLUDES a commercial sub-license for these weights.

For custom commercial licensing inquiries, please contact iSAXT Systems.


Attribution: If you use this model for free, you must credit "iSAXT Systems / VELXT".

Citation

@misc{isaxt2026quiz,
  author = {iSAXT Systems},
  title = {iSAXT Quiz Embeddings v6},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/isaxtsystems/isaxt-quiz-embeddings-v6}}
}

Versioning policy

Versions are immutable: v6 is frozen at publication and never renamed. v5 remains frozen and immutable; a retrain ships as v7 with its own card, manifest and evaluation table. Publishing requires a green round-trip gate plus explicit owner go-ahead, and the manifest version must be set (a null version is unreleased and must never be uploaded).

Multi-tenant platform note

The embedding build is strictly tenant-scoped: every corpus read and vector write carries the tenant identifier, and the pipeline refuses to embed a set that spans two tenants. The serving database holds one global encoder stamp, so switching encoders is an atomic all-tenant operation, not a per-tenant mix.

Downloads last month
139
Safetensors
Model size
4.39M params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for isaxtsystems/isaxt-quiz-embeddings-v6

Quantized
(1)
this model

Dataset used to train isaxtsystems/isaxt-quiz-embeddings-v6