VLUE: A New Benchmark and Multi-task Knowledge Transfer Learning for Vietnamese Natural Language Understanding
Paper • 2403.15882 • Published
Fast, calibrated decisions on Vietnamese and English text · 307M parameters · runs on CPU · v0.2-beta · research / non-commercial
Write a question and its options in plain language. VIYA reads your text and scores every option in one forward pass, with a confidence you can put a threshold on. It does not generate text: no GPU, no API calls, no output tokens.
What you can use it for
- Fact-check a claim against evidence: supports, refutes, or not enough information. Give it a whole article and it finds the evidence sentence itself.
- Find claims worth checking in a stream of sentences, before a person spends time on them.
- Check text against rules: missing conditions, exceptions, priorities, numbers.
- Route and triage messages: intent, topic, urgency, with labels you write yourself.
- Pre-screen content for human moderators. Use it as a first filter, not the final call (see Known weaknesses).
Options are free text, so you can add or rename them without retraining.
Highlights
- Top reported score on ViWikiFC (full fact-checking pipeline): 78.2, counting only answers with the right evidence and the right verdict. The best published system (InfoXLM large) reports 67.0.
- Reads a whole news article and beats Llama3 8B: 82.2 macro-F1 on ViFactCheck vs 79.7 (Gemini 1.5 Flash 76.3). Gemma 7B is still ahead at 85.9. VIYA is about 26× smaller than Llama3 8B and reads the article in one pass.
- Above the human reference when the evidence is given: 86.6 macro-F1 on ViFactCheck; the human reference in the ViFactCheck paper is 84.9. Larger fine-tuned models score higher (up to 89.9).
- Beats Laya and several larger LLM entries on the JevBench public hard split: 0.459 on the 111 hardest decisions. Laya scores 0.351, and VIYA is level with or above several larger LLM entries (kev 8B 0.450, Open-Jev 2B 0.414, kev 4B 0.369). The largest LLMs on the board score much higher (up to 0.964). On all 231 decisions VIYA ties Laya at 0.584.
- About 180 ms per decision on a 2016 laptop CPU (median; Intel i7-6820HQ, 4 threads, Kiểm Tin claim + evidence, ~110 input tokens).
- Calibrated confidence: median calibration error (ECE) 0.053 across 110 test sets; 0.023 on ViFactCheck. A score of 0.8 means roughly 80 % of such answers are right.
- Works without diacritics: Vietnamese typed without accents loses about 2 points (MASSIVE-vi intent 0.843 → 0.824).
Other models' scores come from their papers or the public leaderboard. VIYA's were measured by us on the official test sets. The Evaluation section has the full picture, including where v0.2-beta is weaker.
Quick start
pip install laya
import laya
viya = laya.load("hiepho/viya-base")
out = viya.predict(
{
"claim": "SAWACO ngưng cấp nước từ 12 giờ ngày 25-3.",
"evidence": "Thời gian thực hiện dự kiến từ 22 giờ ngày 25-3 đến 4 giờ ngày 26-3.",
},
{
"verdict": {
"type": "choice",
"instructions": "Đối chiếu `claim` với `evidence`: bằng chứng nói gì?",
"criteria": {
"supports": "bằng chứng xác nhận khẳng định là đúng",
"refutes": "bằng chứng cho thấy khẳng định là sai",
"nei": "bằng chứng không đủ để kết luận đúng hay sai",
},
},
},
)
print(out["answers"]["verdict"])
v0.2-beta runs on the open-source laya inference runtime (same input format as v0.1).
How it works
Every question has a type:
| Type | Returns | Example |
|---|---|---|
choice |
one of N options | Does the evidence support, refute, or not decide this claim? |
score |
a level on a scale | How urgent is this message? |
noul |
yes / no | Is this sentence a factual claim worth checking? |
[CLS] question [SEP] [MASK] option 1 [MASK] option 2 … [SEP] your text [SEP]
│ │
└── each [MASK] is scored ──► softmax ──► calibrated probabilities
- Encoder:
jhu-clsp/mmBERT-base(22 layers, 1,800+ languages), starting fromconvaiinnovations/laya-multilingual. - Decision head: two transformer layers and a scorer over option markers (Laya design). Trained on inputs of up to 1,024 tokens.
- Calibration: per-type temperature fitted on held-out data.
Training (internal checkpoint v08)
v0.2-beta is the internal checkpoint v08 (October 12, 2026). It restarts from the Laya checkpoint with cleaner data than v0.1, and one goal: follow the question and the rules instead of guessing from surface words.
- Mixed decision training from the Laya starting checkpoint: Vietnamese reading tasks, fact-checking, English typed decisions, public rule datasets and VIYA's own rule generator (K1–K13: missing conditions, exceptions, priorities, numbers in text, answer adequacy).
- Anti-shortcut data: every yes/no question also appears negated with the label flipped; options are shuffled; sentences appear with and without diacritics. Consistency losses keep the answers aligned.
- Teacher audit: older VIYA checkpoints were used as teachers only on tasks where they passed an accuracy and bias check.
- Bilingual masked-language modelling (3 hours, about 94 million tokens) followed by a final decision round.
Checkpoints are selected on a dev set, never on test sets or benchmarks.
Evaluation
VIYA is one model for all tasks. All VIYA numbers are measured by us on held-out test sets (macro-F1 unless noted).
The chart shows VIYA's strongest comparisons. The tables below include the ones where it is weaker.
Comparison with other models
Scores for other models come from the same sources as the chart: the Laya BENCHMARKS (Convai Innovations) for typed-decisions, VLUE (arXiv:2403.15882) for ViNLI, and the JevBench public leaderboard (October 2026).
typed-decisions (English business decisions, accuracy):
| Model | Accuracy |
|---|---|
| Laya fine-tune | 76.6 |
| VIYA v0.1-beta | 75.85 |
| VIYA v0.2-beta | 73.8 |
| Human annotators (reference) | 73.5 |
| TypeSafe Jev 1.13 | 72.7 |
| mmBERT base | 35.2 |
ViNLI (Vietnamese natural language inference, accuracy):
| Model | Accuracy |
|---|---|
| CafeBERT 550M | 86.11 |
| XLM-R large 550M | 85.99 |
| VIYA v0.1-beta | 81.75 |
| PhoBERT large 370M | 80.67 |
| VIYA v0.2-beta | 79.40 |
| mBERT 110M | 73.45 |
JevBench public (231 decisions, accuracy):
| Split | VIYA v0.2-beta | Laya |
|---|---|---|
| All 231 | 0.584 | 0.584 |
| Easy | 0.979 | 0.958 |
| Standard | 0.514 | 0.694 |
| Hard (111) | 0.459 | 0.351 |
VIYA is weaker than Laya on the standard split (0.514 vs 0.694), so overall the two tie.
Compared with v0.5 (internal)
The baseline here is v0.5 (consol5): our strongest checkpoint before this release. It is internal and was never published; it was trained after v0.1-beta. Both models were scored with the same scripts.
- Much better at rule-based decisions: 0.894 vs 0.461 on VIYA's own rule tests (missing conditions, exceptions, priorities, counting in text; mean of 13 test sets).
- Better in the Vietnamese fact-checking app "Kiểm Tin": offline cases 0.804 vs 0.754; "is this sentence worth checking?" 0.899 vs 0.800. These two sets are small (51 and 10 cases).
- Better on decision benchmarks: JevBench public 0.584 vs 0.524; ContractNLI 0.743 vs 0.581.
- Slightly better on an unseen Vietnamese task: UIT-VSMEC emotion 0.376 vs 0.333 (neither model trained on it).
- Trade-off: v0.2 is weaker than v0.5 on several classic Vietnamese classification sets (bottom of the table). If you only need intent, sentiment or NLI accuracy, v0.5 was better.
| Task | v0.5 (internal) | v0.2-beta |
|---|---|---|
| VIYA rule tests (RG, mean of 13) | 0.461 | 0.894 |
| Kiểm Tin offline cases (51) | 0.754 | 0.804 |
| Check-worthy sentence (Kiểm Tin, 10) | 0.800 | 0.899 |
| ContractNLI | 0.581 | 0.743 |
| OpenJev held-out | 0.413 | 0.508 |
| ViTHSD: is the given target mentioned? (v0.2 trained on its train split) | 0.549 | 0.784 |
| UIT-VSMEC emotion, never trained | 0.333 | 0.376 |
| ViOCD complaint, never trained (AUROC) | 0.919 | 0.917 |
| ViFactCheck verdict, gold evidence | 0.879 | 0.866 |
| ViWikiFC verdict, gold evidence | 0.853 | 0.831 |
| ViNLI | 0.813 | 0.794 |
| XNLI-vi | 0.768 | 0.752 |
| MASSIVE-vi intent (20 options) | 0.904 | 0.843 |
| … typed without diacritics | 0.901 | 0.824 |
| UIT-VSFC sentiment | 0.793 | 0.741 |
| UIT-ViHSD | 0.567 | 0.509 |
| Banking77 (77 options) | 0.711 | 0.640 |
Known weaknesses
- Says "yes" too often on some yes/no questions. Example: toxicity on ViCTSD, where it says "toxic" 44 % of the time while the true rate is about 10 %.
- Reads the option wording more than the text on some English decision types (extraction, ranking). We tested this by hiding the text: v08 still got many answers right without it.
- SMS scam detection does not work on real Vietnamese SMS (AUROC 0.50). Do not use it for that.
- Many options: choice questions work best with fewer than about 20 options.
- Text only. Images, video and deepfakes are out of scope. VIYA judges a claim against the evidence you give it, not from memory.
License, training data and disclaimer
License: CC BY-NC-SA 4.0 (research and non-commercial use only). v0.2-beta was trained on many public datasets. Some are licensed for research use only, some are share-alike (SA), and a few carry no license on their Hugging Face copy. The full list, with links and licenses, is in the Training data section below.
- Before you use VIYA (or anything derived from it) to train another model, or for any commercial purpose, check the license of every dataset in that list yourself. This license covers the weights only. It gives you no rights over the training data.
- Some sources (UIT-ViHSD, VLSP 2016 / 2018 / ReINTEL) ask users to request permission from their authors. A small share of the data (about 1–2 %) is text written by OpenAI models (WANLI, Prometheus feedback, MiniCheck, SPML, part of SODA), and those outputs fall under OpenAI's terms.
- Disclaimer. The model is provided "as is", without warranty of any kind. Its predictions can be wrong, biased or overconfident. The author and contributors accept no liability for any damage or loss arising from its use. Users are responsible for how they use it and for complying with the licenses of the model and of its training data. Do not use it as the only basis for decisions about people (moderation, finance, health, law).
The next major version (VIYA 1.0) is being trained only on data whose licenses allow commercial use.
How this model was made
VIYA is a student science-fair project. The author (a high-school student) designed the experiments and made the research decisions. An AI assistant (Claude, Anthropic) helped write code, run experiments and draft documents, including this model card. Every use of AI is logged in the project's AI-use declaration, as the contest rules require.
Credits
- mmBERT (JHU CLSP), the encoder.
- Laya (Convai Innovations, Apache-2.0): the decision head design, the starting checkpoint (
laya-multilingual) and the typed-decisions benchmark harness. - Jev (TypeSafe AI), which pioneered the typed-decision format (
choice/score/noul) that Laya and VIYA follow, and the JevBench authors for the public decision benchmark. - The authors of ViFactCheck, ViWikiFC, ViNLI, UIT-ViHSD, UIT-VSFC, UIT-ViSFD, UIT-ViQuAD 2.0, ViTHSD, UIT-VSMEC, ViOCD, ViCTSD, MASSIVE, XNLI, VitaminC, ClaimBuster, ReINTEL, ContractNLI, OpenJevData, typed-decisions and FineWeb-2.
Training data
v0.2-beta saw 675,961 training examples from the sources below (195 tasks). The License column is what each Hugging Face page stated on 19 October 2026 ("not stated" = no license given there). Notes flag what you should check before reusing anything: needs permission, no license, contains outputs of OpenAI models, NC (non-commercial / research only), SA (share-alike). This list is for information only and is not legal advice; check each source's own terms yourself.
Starting models
| Model | Used for | License | Notes |
|---|---|---|---|
| convaiinnovations/laya-multilingual | starting checkpoint (Laya decision format, encoder + heads) | apache-2.0 | |
| jhu-clsp/mmBERT-base | encoder architecture; MLM head for the reading-practice stage (viya_mlm.py) |
mit |
MLM reading practice (unlabeled text, viya_mlm.py, checkpoint viya_v06mlm that v08 started from)
| Dataset | Used for | License | Notes |
|---|---|---|---|
| HuggingFaceFW/fineweb-2 | MLM Vietnamese web text (vie_Latn) |
odc-by | |
| HuggingFaceFW/fineweb-edu | MLM English text (sample/10BT) |
odc-by | |
| wikimedia/wikipedia | MLM text (20231101.vi, 20231101.en) |
cc-by-sa-3.0, gfdl | SA |
| another-symato/tinhte-raw | MLM Vietnamese forum text | not stated | no license (forum posts) |
| pqthinh232/vietnamese-restaurant-review-sentiment-dataset | MLM comment text only (no labels used) | not stated | no license (not covered by the audit) |
Labeled training data
| Dataset | Used for (VIYA tasks) | License | Notes |
|---|---|---|---|
| mteb/amazon_massive_intent | massive_intent, vi_noul_intent, fc_checkworthy (negatives); MLM text | apache-2.0 | |
| mteb/amazon_massive_scenario | massive_scenario, vi_noul_scenario | apache-2.0 | |
| facebook/xnli | xnli, xnli_noul (vi) | not stated | NC |
| uitnlp/vietnamese_students_feedback | vsfc_sent, vsfc_topic, vi_noul_vsfc_sent, vi_noul_vsfc_topic, fc_checkworthy (negatives); MLM text | unknown | no license (UIT terms not checked) |
| presencesw/vinli_4_label | vinli, vinli_noul | not stated | no license (UIT terms not checked) |
| ura-hcmut/UIT-ViHSD | vihsd, vihsd_noul, vi_noul_polite, fc_checkworthy (negatives); MLM text | cc-by-nc-sa-4.0 | needs permission (card: "obtain a usage permission before using"); NC; SA |
| NamSyntax/vietnamese-news-classification | news_vne | apache-2.0 | |
| hiuman/vietnamese_classification | news_15 | not stated | no license |
| visolex/ViSFD | visfd_star, visfd_aspect, visfd_aspect_noul, vi_noul_aspect_mention, vi_noul_satisfied, absa_visfd; MLM text | not stated (card: "if unspecified, assume CC BY 4.0") | no license (UIT terms not checked) |
| iaiuet/banking_sentiment_vietnamese | bank_sent, bank_topic, vi_noul_bank_topic | not stated | no license |
| taidng/UIT-ViQuAD2.0 | viquad_noul | not stated | no license (UIT terms not checked) |
| visolex/VLSP2018-ABSA-Hotel | absa_hotel | not stated | needs permission (VLSP agreement, research only) |
| visolex/VLSP2018-ABSA-Restaurant | absa_rest | not stated | needs permission (VLSP agreement, research only) |
| visolex/ViSpamReviews | spam_bin, spam_kind | not stated (card: "if unspecified, assume CC BY 4.0") | no license (UIT terms not checked) |
| google/boolq | en_boolq | cc-by-sa-3.0 | SA |
| nyu-mll/glue | en_paraphrase (mrpc), en_sst2 (sst2), en_mnli (mnli) | other | |
| google-research-datasets/paws | en_paraphrase (labeled_final) | other (card: "may be freely used for any purpose") | |
| cornell-movie-review-data/rotten_tomatoes | en_rt | unknown | |
| SetFit/subj | en_subj | not stated | |
| fancyzhx/ag_news | en_topic, en_topic_noul, en_agnews | unknown | NC (research only) |
| fancyzhx/dbpedia_14 | en_topic, en_topic_noul | cc-by-sa-3.0 | |
| community-datasets/yahoo_answers_topics | en_topic, en_topic_noul | unknown | |
| clinc/clinc_oos | en_clinc_intent, en_clinc_scope | cc-by-3.0 | |
| allenai/ai2_arc | en_arc (ARC-Easy), en_arc_c (ARC-Challenge) | cc-by-sa-4.0 | SA |
| tau/commonsense_qa | en_csqa | mit | |
| Rowan/hellaswag | en_hellaswag | not stated in tags (card text: MIT) | |
| allenai/sciq | en_sciq | cc-by-nc-3.0 | NC |
| aps/super_glue | en_copa (copa) | other | |
| tranthaihoa/vifactcheck | fc_vifc, fc_vifc_noevid, fc_rel_vifc, fc_checkworthy (positives); source of the generated rows below | mit | |
| High-Will/ViWikiFC | fc_viwiki, fc_viwiki_noevid, fc_rel_viwiki, fc_checkworthy (positives); source of the generated rows below | cc-by-4.0 | |
| tals/vitaminc | en_vitaminc, en_vitaminc_noevid | cc-by-sa-3.0 | SA |
| Nithiwat/claimbuster | en_claimbuster | cc-by-sa-4.0 | SA |
| ReliableAI/ReINTEL | fc_reintel | not stated | needs permission (VLSP2020 agreement, research only; contains user_name) |
| LocalLLaMA/typed-decisions | en_typed_choice, en_typed_noul, en_typed_score | apache-2.0 | |
| SetFit/enron_spam | en_spam | not stated | |
| zefang-liu/phishing-email-dataset | en_phishing | lgpl-3.0 | |
| Tobi-Bueck/customer-support-tickets | en_tickets | cc-by-nc-4.0 | NC |
| microsoft/ms_marco | en_marco (v1.1) | not stated | NC (research only) |
| openai/gsm8k | en_routing (questions only) | mit | |
| google-research-datasets/mbpp | en_routing (prompts only) | cc-by-4.0 | |
| euclaise/writingprompts | en_routing (prompts only) | mit | |
| rajpurkar/squad | en_routing (questions only) | cc-by-sa-4.0 | SA |
| b-mc2/sql-create-context | en_routing (questions only) | cc-by-4.0 | |
| allenai/soda | en_routing (first dialogue turn only) | cc-by-4.0 | contains GPT-3.5 (InstructGPT) outputs |
| mteb/banking77 | en_banking77, en_banking77_full | mit | |
| facebook/anli | en_anli | cc-by-nc-4.0 | NC |
| allenai/winogrande | en_winogrande | not stated | |
| baber/piqa | en_piqa | not stated | |
| allenai/openbookqa | en_obqa | unknown | |
| dair-ai/emotion | en_emotion | other (card: "educational and research purposes only") | NC (research only) |
| uitnlp/ViANLI | vi_anli_h | mit tag, but card text says CC BY-NC-SA 4.0 | NC; SA; a model helped write sentences (which model not checked) |
| MoritzLaurer/multilingual-NLI-26lang-2mil7 | vi_fever, vi_fever_noevid (vi_fever), vi_wanli (vi_wanli), vi_ling (vi_ling) |
not stated | vi_wanli contains GPT-3 outputs (WANLI, human-revised) |
| tmnam20/ViGLUE | vi_qnli (qnli) | not stated | |
| FinnPham52/vietnamese-legal-claim-verification | bi_legal | not stated | no license, no card (audit: drop in later versions) |
| HiTZ/This-is-not-a-dataset | neg_tind | apache-2.0 | |
| tasksource/scone | neg_scone | cc0-1.0 | |
| lasha-nlp/CONDAQA | neg_condaqa | apache-2.0 | |
| tasksource/sharc | sharc | not stated | |
| tasksource/Boardgame-QA | boardgame | cc-by-4.0 | |
| tasksource/proofwriter | proofwriter | not stated | |
| presencesw/contractnli | contractnli | not stated | |
| tasksource/ConTRoL-nli | control | not stated | |
| tasksource/doc-nli | docnli | bsd | |
| tasksource/LogicNLI | logicnli | not stated | |
| shenjunhao/OpenJevData-140k | all 18 oj_* tasks | mit | its SOURCE_NOTICES list LogiQA (NC, SA), WANLI (GPT-3 outputs), PRM800K; part is model-generated (model not checked) |
| tasksource/procedural-typed-decisions | all 33 ptd_* tasks | apache-2.0 | |
| emozilla/quality | quality | not stated | |
| prometheus-eval/Feedback-Collection | feedback_rubric | cc-by-4.0 | contains GPT-4 outputs (stated on card) |
| lytang/C2D-and-D2C-MiniCheck | minicheck | mit | contains GPT-4 outputs |
| reshabhs/SPML_Chatbot_Prompt_Injection | spml_injection | mit | contains GPT-4 outputs |
| 5CD-AI/Vietnamese-cosmos-qa-gg-translated | cosmos_vi (Vietnamese columns), cosmos_en (original English columns) | not stated | |
| copenlu/fever_gold_evidence | fever_gold_en, fever_gold_en_noevid | cc-by-sa-3.0, gpl-3.0 | SA (and GPL-3.0) |
| rexarski/climate_fever_fixed | climate_fever | mit | |
| iai-group/clef2024_checkthat_task1_en | checkthat_en | cc-by-sa-4.0 | SA |
| visolex/VITHSD | vithsd_target | not stated (card: "if unspecified, assume CC BY 4.0") | no license (UIT terms not checked) |
| ura-hcmut/vlsp2016 | vlsp2016_sent | cc-by-nc-sa-4.0 | needs permission (VLSP agreement); NC; SA |
| ura-hcmut/PhoATIS | phoatis_intent | cc-by-nc-sa-4.0 | NC; SA |
| FastFit/hwu_64 | hwu64_intent | not stated | |
| benayas/snips | snips_intent | apache-2.0 |
Data generated by this project (no Hugging Face link)
| Source | Used for | License | Notes |
|---|---|---|---|
Rule-decision generator viya_rulegen.py (generated by this project from code templates; no external data; code written with Claude) |
all 26 rg_k1..rg_k13 _en/_vi tasks | project code | |
| Number-swap, negation and evidence-scope rows (generated by this project from tranthaihoa/vifactcheck and High-Will/ViWikiFC) | num_vifc, num_viwiki, numpara_vifc, numpara_viwiki, neg_vifc, neg_viwiki, scope_num_vifc, scope_num_viwiki, scope_question_vifc, scope_question_viwiki | inherits MIT (ViFactCheck) / CC BY 4.0 (ViWikiFC) | |
Bilingual cross pairs bilingual_pairs.json (generated by this project from tranthaihoa/vifactcheck and High-Will/ViWikiFC, translated with Helsinki-NLP/opus-mt-vi-en) |
bi_selftrans, bi_selftrans_noevid | inherits the source licenses |
Tiếng Việt
VIYA v0.2-beta là model AI nhỏ (307 triệu tham số, chạy trên CPU) ra quyết định trên văn bản tiếng Việt và tiếng Anh. Bạn viết câu hỏi và các lựa chọn bằng lời thường; VIYA đọc văn bản rồi chấm điểm từng lựa chọn trong một lượt, kèm độ tin cậy đã hiệu chỉnh. Có ba kiểu câu hỏi: chọn một trong nhiều lựa chọn, chấm mức độ, hoặc trả lời có/không.
Dùng để làm gì: kiểm chứng một khẳng định dựa trên bằng chứng (đưa cả bài báo, VIYA tự tìm câu làm bằng chứng); lọc ra câu cần kiểm chứng; đối chiếu văn bản với quy định; phân loại và chuyển tiếp tin nhắn; lọc sơ bộ nội dung trước khi người kiểm duyệt xem (không dùng làm quyết định cuối).
Điểm nổi bật:
- ViWikiFC (trọn quy trình): 78,2, cao nhất từng được báo cáo; hệ thống tốt nhất đã công bố đạt 67,0.
- Tự đọc cả bài báo ViFactCheck: 82,2, vượt Llama3 8B (79,7) dù nhỏ hơn khoảng 26 lần. Gemma 7B vẫn cao hơn (85,9).
- Có sẵn bằng chứng: 86,6 trên ViFactCheck, cao hơn mốc con người trong bài báo (84,9); các model lớn train riêng vẫn cao hơn (tới 89,9).
- JevBench phần khó: 0,459, vượt Laya (0,351) và ngang hoặc hơn vài LLM lớn hơn nhiều (kev 8B 0,450, Open-Jev 2B 0,414). LLM lớn nhất trên bảng đạt tới 0,964. Tính cả 231 câu, VIYA bằng Laya (0,584).
- Nhanh: khoảng 180 ms mỗi quyết định trên CPU laptop đời 2016.
- Độ tin cậy đã hiệu chỉnh: sai số ECE trung vị 0,053 trên 110 bộ test.
- Gõ không dấu chỉ tụt khoảng 2 điểm.
So với v0.5 (bản nội bộ mạnh nhất trước đó, chưa đăng, train sau v0.1):
- Làm theo luật tốt hơn hẳn: bài luật của VIYA đạt 0,894, so với 0,461 ở v0.5 (trung bình 13 bộ).
- Kiểm Tin tốt hơn: ca thật 0,804 (v0.5: 0,754); nhận ra câu cần kiểm chứng 0,899 (v0.5: 0,800). Hai bộ này nhỏ (51 và 10 ca).
- JevBench 0,584, bằng Laya; ContractNLI 0,743 (v0.5: 0,581).
- Đánh đổi: v0.2 yếu hơn v0.5 ở vài bài phân loại tiếng Việt quen thuộc (ý định, cảm xúc, NLI). Bảng đầy đủ ở mục Evaluation.
Điểm yếu đã biết: hay nói "có" quá nhiều; đôi khi chấm theo chữ của lựa chọn hơn là đọc văn bản; chưa dùng được để phát hiện tin nhắn SMS lừa đảo; chỉ xử lý văn bản.
Giấy phép: chỉ dùng cho nghiên cứu, phi thương mại (CC BY-NC-SA 4.0), vì v0.2 có học dữ liệu chỉ cho phép nghiên cứu. Ai muốn dùng VIYA để train model khác hoặc dùng thương mại thì tự kiểm giấy phép từng bộ dữ liệu ở mục Training data. Bản VIYA 1.0 đang được train chỉ bằng dữ liệu cho phép thương mại.
Miễn trừ trách nhiệm: model được cung cấp "nguyên trạng", không bảo đảm gì. Kết quả có thể sai, thiên lệch hoặc quá tự tin. Tác giả không chịu trách nhiệm về mọi thiệt hại phát sinh khi sử dụng; người dùng tự chịu trách nhiệm về cách dùng và việc tuân thủ giấy phép của model cũng như của dữ liệu. Không dùng model làm căn cứ duy nhất cho quyết định về con người.
Cách làm: đây là đề tài nghiên cứu khoa học của học sinh. Tác giả thiết kế thí nghiệm và ra các quyết định nghiên cứu; trợ lý AI (Claude) hỗ trợ viết code, chạy thí nghiệm và soạn tài liệu, kể cả model card này. Mọi việc dùng AI đều được ghi lại trong bản khai báo dùng AI theo quy định cuộc thi.
Model tree for hiepho/viya-base
Base model
convaiinnovations/laya-multilingual