๐Ÿ“Š Qwen3.5-4B-Financial-SQL (Vietnamese Financial Text-to-SQL)

Mรด hรฌnh ngรดn ngแปฏ chuyรชn biแป‡t cho bร i toรกn Text-to-SQL Bรกo Cรกo Tร i Chรญnh Viแป‡t Nam, ฤ‘ฦฐแปฃc huแบฅn luyแป‡n tแปซ base model Qwen/Qwen3.5-4B trรชn tแบญp dแปฏ liแป‡u 1.847 mแบซu Gold Standard kiแปƒm ฤ‘แป‹nh thแปฑc thi 100% trรชn cฦก sแปŸ dแปฏ liแป‡u SQLite BCTC cรกc doanh nghiแป‡p niรชm yแบฟt (HOSE, HNX, UPCoM).

๐ŸŒŸ ฤแบทc ฤ‘iแปƒm nแป•i bแบญt

  • Suy luแบญn chuแป—i tฦฐ duy (<think>): Trฦฐแป›c khi viแบฟt SQL, mรด hรฌnh tแปฑ ฤ‘แป™ng phรขn tรญch: khรกi niแป‡m kแบฟ toรกn, bแบฃng tร i chรญnh (balance_sheet, income_statement, cash_flow), cรดng thแปฉc tรญnh toรกn vร  ฤ‘ฦกn vแป‹ quy ฤ‘แป•i (tแปท ฤ‘แป“ng, triแป‡u ฤ‘แป“ng, %).
  • Chuแบฉn ANSI SQLite: 100% cรขu truy vแบฅn tuรขn thแปง cรบ phรกp SQLite Native, hแป— trแปฃ cรกc truy vแบฅn phแปฉc tแบกp (CTEs, Window functions, lแปc nhรฃn thแปi gian period_label, truy vแบฅn ฤ‘a kแปณ).
  • Tรญnh truy vแบฟt kiแปƒm toรกn (Audit Provenance): Mแปi cรขu truy vแบฅn trแบฃ vแป dแปฏ liแป‡u ฤ‘แปu kรจm nguแป“n gแป‘c rรต rร ng (raw_value, unit, page_no, source_doc).
  • Merged 16-bit Full Weights: Model ฤ‘แป™c lแบญp hoร n chแป‰nh, tฦฐฦกng thรญch tแป‘i ฤ‘a vแป›i vLLM, SGLang, Transformers vร  Ollama.

๐Ÿ—„๏ธ Cแบฅu trรบc Schema (financial_facts)

CREATE TABLE financial_facts (
    ticker TEXT,          -- Mรฃ chแปฉng khoรกn (VNM, FPT, HPG, VJC...)
    company_name TEXT,    -- Tรชn ฤ‘แบงy ฤ‘แปง doanh nghiแป‡p
    year INTEGER,         -- Nฤƒm tร i chรญnh (2016 - 2024)
    report_type TEXT,     -- 'consolidated' (hแปฃp nhแบฅt) hoแบทc 'separate' (cรดng ty mแบน)
    statement TEXT,       -- 'balance_sheet', 'income_statement', 'cash_flow', 'notes'
    item_name TEXT,       -- Tรชn chแป‰ tiรชu tiแบฟng Viแป‡t cรณ dแบฅu
    item_name_ascii TEXT, -- Tรชn chแป‰ tiรชu khรดng dแบฅu (dรนng vแป›i LIKE '%keyword%')
    period_label TEXT,    -- Nhรฃn kแปณ bรกo cรกo ('31/12/2023', 'Sแป‘ cuแป‘i nฤƒm'...)
    value_vnd REAL,       -- Giรก trแป‹ quy ฤ‘แป•i sang VND
    raw_value TEXT,       -- Sแป‘ liแป‡u nguyรชn bแบฃn trรชn BCTC PDF
    unit TEXT,            -- ฤฦกn vแป‹ tiแปn tแป‡
    page_no INTEGER,      -- Trang sแป‘ trรชn BCTC PDF
    source_doc TEXT       -- Tรชn file PDF nguแป“n
);

๐Ÿš€ Hฦฐแป›ng dแบซn suy luแบญn (Quickstart vแป›i Transformers)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "giangkh19/Qwen3.5-4B-Financial-SQL"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16,
    device_map="auto"
)

system_prompt = """You are a financial SQLite expert for Vietnamese corporate financial reports.
Given the database schema for table `financial_facts`, analyze the question and return:
1. A concise reasoning block enclosed in <think>...</think>.
2. The exact ANSI SQLite query inside ```sql ... ```."""

question = "Doanh thu thuแบงn nฤƒm 2023 cแปงa CTCP Sแปฏa Viแป‡t Nam (mรฃ VNM) lร  bao nhiรชu tแปท ฤ‘แป“ng?"

prompt = f"<|im_start|>system\n{system_prompt}<|im_end|>\n<|im_start|>user\n{question}<|im_end|>\n<|im_start|>assistant\n"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.01)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=False))

๐Ÿ“ˆ Thรดng sแป‘ huแบฅn luyแป‡n (Training Specs)

  • Base Model: Qwen/Qwen3.5-4B
  • Tแบญp dแปฏ liแป‡u: 1.847 mแบซu (1.665 Train / 182 Val)
  • Sแป‘ Epochs: 3 Epochs (627 steps, effective batch = 8)
  • Training Loss: 0.0978 (Hแป™i tแปฅ tแป‘i ฦฐu)
  • Gradient Norm: 0.41 (Cแปฑc kแปณ แป•n ฤ‘แป‹nh)
  • Cฦก chแบฟ Loss: Response-only loss masking (chแป‰ tรญnh phแบกt trรชn phแบงn suy luแบญn vร  cรขu SQL)
Downloads last month
27
Safetensors
Model size
5B params
Tensor type
BF16
ยท
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for giangkh19/Qwen3.5-4B-Financial-SQL

Finetuned
Qwen/Qwen3.5-4B
Finetuned
(1,528)
this model