tesda-e5-small-v4

The matching model for TESDA-TRACK. It is a fine-tune of intfloat/multilingual-e5-small that matches learner goals, written in English, Filipino or Taglish, to TESDA qualifications. Its training data also has synthetic examples in Cebuano, Ilocano, Hiligaynon and — new in v4 — Kapampangan, plus a workplace-scenario query for every qualification in the catalog (see the limitations).

It follows tesda-e5-small-v3, which stays published for rollback, as does v2. The repo contains the ONNX model only, with no PyTorch weights. The TESDA-TRACK API runs it on CPU through fastembed.

Set up in TESDA-TRACK

From the repository root, download the model into backend/models/:

pip install -U huggingface_hub
hf download Nyanta19/tesda-e5-small-v4 --local-dir backend/models/tesda-e5-small-v4

Add both lines to the root .env, replacing any earlier EMBEDDING_MODEL* lines:

EMBEDDING_MODEL=tesda-track/tesda-e5-small-v4
EMBEDDING_MODEL_PATH=backend/models/tesda-e5-small-v4

Then embed the catalog with the new model, and restart the API:

cd backend
python -m tesda_track.embeddings sync

• Keep the name as given. EMBEDDING_MODEL must stay tesda-track/tesda-e5-small-v4, because stored embeddings are keyed by that name. Never set it to the base model's name.
• Rollback: keep the v3 folder. Point both lines back at tesda-e5-small-v3, run sync, and restart the API.
• Docker: download into the same folder before docker compose up -d --build. The build copies backend/models/ to /app/finetuned/, so set EMBEDDING_MODEL_PATH=/app/finetuned/tesda-e5-small-v4.
• Fallback: if neither variable is set, the API uses the base model. A missing or broken folder leaves the API on keyword matching.

calibration.json holds the score parameters tuned for this model: z_floor 3.5, z_ceiling 6.5, keyword_weight 0.3, min_score 15. The API reads them from the folder and uses them in place of the settings' values.

Files

File	Purpose
onnx/model.onnx	The model (470 MB, fp32)
tokenizer.json, tokenizer_config.json, config.json	Tokenizer and model config
modules.json, 1_Pooling/, 2_Normalize/, sentence_bert_config.json, config_sentence_transformers.json	Pooling (mean) and L2 normalisation
calibration.json	Score parameters for the TESDA-TRACK matcher
training.json, run_config.json	Training run record and settings
coverage.json	Training-data counts per language, example kind and qualification
checksums.json, bundle_sha256.txt	SHA-256 of the files produced by the training run, and of the training bundle

As with every E5 model, prefix goals with query: and qualification text with passage: . The API adds the prefixes itself. Embeddings have 384 dimensions, mean-pooled and L2-normalised; compare them with cosine similarity.

Training

• Data: TESDA catalog text only (qualification names, skills, jobs, keywords and core competencies), plus 14,650 synthetic learner-style queries for all 319 qualifications. The enrichment rows break down as 3,427 catalog-derived English rows and 6,002 drafts: 1,508 Ilocano, 1,498 each Cebuano/Bisaya, Hiligaynon/Ilonggo and Kapampangan. New in v4:

  • Kapampangan (pam), with the same example kinds the other regional languages already had.
  • 1,276 workplace-scenario queries — one practical situation for each of the 319 catalog qualifications, written in all four regional languages with English trade terms (query_style: code_switched_scenario).

  15% of the queries also got one typo or text-speak copy (2,166 pairs). No learner, worker, T2MIS or TSP records were used. Every evaluation goal was excluded before training and augmentation.

• Pairs: 45,674 in total: 43,395 for training and 2,279 held out. The same normalised query never appears on both sides. Held-out loss fell from 2.98 to 0.111.

• Settings: 3 epochs, learning rate 2e-5, batch 64 (mini-batch 8), CachedMultipleNegativesRankingLoss, seed 42, no mined hard negatives.

• Run: 2,037 steps, 52 minutes on a Colab GPU (torch 2.11.0+cu130, sentence-transformers 6.1.0).

• ONNX check: the export reproduces the PyTorch embeddings with a cosine of 0.999999.

Evaluation

Top-1 and top-3 count goals whose correct qualification is first, or in the first three. Rejection is the share of goals with no TVET match that correctly get no result. Score parameters are z_floor / z_ceiling / keyword_weight / min_score. All numbers come from the current TESDA-TRACK scoring code, which applies plain-language goal rules before the model.

v4 is not an accuracy win over v3 on English, Filipino or Taglish. It ties v3 on dev top-1, rejects more unrelated goals, and adds Kapampangan; v3 is marginally better on dev top-3 and MRR. Read the limitations before switching.

Dev set: 1,482 goals (1,283 with a TVET match, 199 without), including 7 Kapampangan goals added for v4. Both models were calibrated on this set with the same procedure (at least 90% rejection required), so both rows are optimistic in the same way.

Model	Score parameters	Top-1	Top-3	MRR	Rejection
tesda-e5-small-v3	3.5 / 5.5 / 0.3 / 20 (calibrated on dev)	0.779	0.838	0.807	0.955
tesda-e5-small-v4	3.5 / 6.5 / 0.3 / 15 (calibrated on dev)	0.779	0.829	0.802	0.980

Top-1 by language on dev, with the number of matchable goals behind each figure:

Model	English (456)	Taglish (408)	Filipino (399)	Kapampangan (5)	Cebuano (5)	Ilocano (5)	Hiligaynon (5)
v3	0.866	0.824	0.629	0.400	1.000	1.000	1.000
v4	0.882	0.819	0.614	0.600	1.000	1.000	1.000

Sealed test set: 499 goals (409 with a TVET match, 90 without), never used for training or tuning and run once for this candidate, after calibration. v3's figures below are the ones on its own card; they were measured before the five Kapampangan goals were added, so the two rows are not the identical set.

Model	Score parameters	Top-1	Top-3	MRR	Rejection
tesda-e5-small-v3 (494 goals)	3.5 / 5.5 / 0.3 / 20	0.805	0.877	0.840	0.932
tesda-e5-small-v4 (499 goals)	3.5 / 6.5 / 0.3 / 15	0.809	0.883	0.845	0.922

v4's sealed top-1 by language: English 0.855 (131 goals), Taglish 0.871 (140), Filipino 0.691 (123), Kapampangan 3 of 3, Cebuano 3 of 4, Ilocano 3 of 4, Hiligaynon 3 of 4.

v1 test set: 120 goals (99 with a TVET match, 21 without).

Model	Top-1	Top-3	MRR	Rejection
v3	0.788	0.818	0.800	0.905
v4	0.778	0.818	0.796	0.905

The TESDA-TRACK real-model test (8 goals with a known best match, 5 with none) passes with v4.

Limitations:

• Regional languages are not established, Kapampangan least of all. Every regional training row is synthetic and unreviewed by native speakers, including the 1,276 new scenario queries, and the four language variants of a scenario share one meaning. Kapampangan evaluation is 5 matchable goals in dev (3 right) and 3 in the sealed set (3 right) — far too few to claim anything. The other regional sets hold 5 matchable goals each in dev and 4 in the sealed set. Do not rely on Kapampangan, Cebuano, Ilocano or Hiligaynon results yet.
• Catalog coverage is not language coverage. One scenario per qualification gives every qualification an example, not broad conversational range; the queries use regional phrasing with English trade terms.
• Filipino-only goals stay the weakest major language: dev top-1 0.614 (0.629 for v3) and 0.450 on the v1 test set.
• Weak spots (sealed set): goals next to TVET but outside it 0.722 top-1 (0.580 on dev); indirect goals 0.750; vague goals 0.880 rejection; goals phrased with negation reject only half of their few unrelated cases.
• Scope: made for ranking TESDA qualifications in TESDA-TRACK; it is not a general-purpose embedding model.