๐ Qwen3.5-4B-Financial-SQL (Vietnamese Financial Text-to-SQL)
Mรด hรฌnh ngรดn ngแปฏ chuyรชn biแปt cho bร i toรกn Text-to-SQL Bรกo Cรกo Tร i Chรญnh Viแปt Nam, ฤฦฐแปฃc huแบฅn luyแปn tแปซ base model Qwen/Qwen3.5-4B trรชn tแบญp dแปฏ liแปu 1.847 mแบซu Gold Standard kiแปm ฤแปnh thแปฑc thi 100% trรชn cฦก sแป dแปฏ liแปu SQLite BCTC cรกc doanh nghiแปp niรชm yแบฟt (HOSE, HNX, UPCoM).
๐ ฤแบทc ฤiแปm nแปi bแบญt
- Suy luแบญn chuแปi tฦฐ duy (
<think>): Trฦฐแปc khi viแบฟt SQL, mรด hรฌnh tแปฑ ฤแปng phรขn tรญch: khรกi niแปm kแบฟ toรกn, bแบฃng tร i chรญnh (balance_sheet,income_statement,cash_flow), cรดng thแปฉc tรญnh toรกn vร ฤฦกn vแป quy ฤแปi (tแปท ฤแปng, triแปu ฤแปng, %). - Chuแบฉn ANSI SQLite: 100% cรขu truy vแบฅn tuรขn thแปง cรบ phรกp SQLite Native, hแป trแปฃ cรกc truy vแบฅn phแปฉc tแบกp (CTEs, Window functions, lแปc nhรฃn thแปi gian
period_label, truy vแบฅn ฤa kแปณ). - Tรญnh truy vแบฟt kiแปm toรกn (Audit Provenance): Mแปi cรขu truy vแบฅn trแบฃ vแป dแปฏ liแปu ฤแปu kรจm nguแปn gแปc rรต rร ng (
raw_value,unit,page_no,source_doc). - Merged 16-bit Full Weights: Model ฤแปc lแบญp hoร n chแปnh, tฦฐฦกng thรญch tแปi ฤa vแปi vLLM, SGLang, Transformers vร Ollama.
๐๏ธ Cแบฅu trรบc Schema (financial_facts)
CREATE TABLE financial_facts (
ticker TEXT, -- Mรฃ chแปฉng khoรกn (VNM, FPT, HPG, VJC...)
company_name TEXT, -- Tรชn ฤแบงy ฤแปง doanh nghiแปp
year INTEGER, -- Nฤm tร i chรญnh (2016 - 2024)
report_type TEXT, -- 'consolidated' (hแปฃp nhแบฅt) hoแบทc 'separate' (cรดng ty mแบน)
statement TEXT, -- 'balance_sheet', 'income_statement', 'cash_flow', 'notes'
item_name TEXT, -- Tรชn chแป tiรชu tiแบฟng Viแปt cรณ dแบฅu
item_name_ascii TEXT, -- Tรชn chแป tiรชu khรดng dแบฅu (dรนng vแปi LIKE '%keyword%')
period_label TEXT, -- Nhรฃn kแปณ bรกo cรกo ('31/12/2023', 'Sแป cuแปi nฤm'...)
value_vnd REAL, -- Giรก trแป quy ฤแปi sang VND
raw_value TEXT, -- Sแป liแปu nguyรชn bแบฃn trรชn BCTC PDF
unit TEXT, -- ฤฦกn vแป tiแปn tแป
page_no INTEGER, -- Trang sแป trรชn BCTC PDF
source_doc TEXT -- Tรชn file PDF nguแปn
);
๐ Hฦฐแปng dแบซn suy luแบญn (Quickstart vแปi Transformers)
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "giangkh19/Qwen3.5-4B-Financial-SQL"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16,
device_map="auto"
)
system_prompt = """You are a financial SQLite expert for Vietnamese corporate financial reports.
Given the database schema for table `financial_facts`, analyze the question and return:
1. A concise reasoning block enclosed in <think>...</think>.
2. The exact ANSI SQLite query inside ```sql ... ```."""
question = "Doanh thu thuแบงn nฤm 2023 cแปงa CTCP Sแปฏa Viแปt Nam (mรฃ VNM) lร bao nhiรชu tแปท ฤแปng?"
prompt = f"<|im_start|>system\n{system_prompt}<|im_end|>\n<|im_start|>user\n{question}<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.01)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=False))
๐ Thรดng sแป huแบฅn luyแปn (Training Specs)
- Base Model:
Qwen/Qwen3.5-4B - Tแบญp dแปฏ liแปu: 1.847 mแบซu (1.665 Train / 182 Val)
- Sแป Epochs: 3 Epochs (627 steps, effective batch = 8)
- Training Loss:
0.0978(Hแปi tแปฅ tแปi ฦฐu) - Gradient Norm:
0.41(Cแปฑc kแปณ แปn ฤแปnh) - Cฦก chแบฟ Loss: Response-only loss masking (chแป tรญnh phแบกt trรชn phแบงn suy luแบญn vร cรขu SQL)
- Downloads last month
- 27