| --- |
| language: vi |
| tags: |
| - ner |
| - sequence-labeling |
| - bilstm-crf |
| - pytorch |
| - vietnamese |
| - food-order |
| - cs221 |
| metrics: |
| - f1 |
| - precision |
| - recall |
| - accuracy |
| --- |
| |
| # BiLSTM-CRF-FoodNER: Deep Sequence Labeling Baseline for Vietnamese Food Orders |
|
|
| #### Table of contents |
| 1. [Introduction](#introduction) |
| 2. [Dataset Overview](#dataset) |
| 3. [Entity Label Space](#labels) |
| 4. [Empirical Evaluation](#evaluation) |
| 5. [Architecture Characteristics](#characteristics) |
| 6. [Using BiLSTM-CRF-FoodNER with PyTorch](#usage) |
| 7. [Authors & Citation](#citation) |
|
|
| --- |
|
|
| ## <a name="introduction"></a> 1. Introduction |
|
|
| **`CS221DoAn/Do_an_group_BiLSTM_CRF`** is a deep learning model based on the **Bidirectional LSTM + Conditional Random Field (BiLSTM-CRF)** architecture. It is specifically trained for **Named Entity Recognition (NER)** on domain-specific Vietnamese unstructured text: **Online Food Delivery Orders and Messages**. |
|
|
| In our research project for the CS221.Q21 course (Natural Language Processing), this model serves as a deep-learning sequence baseline. It was developed to overcome the limitations of traditional CRF (which relies heavily on manual feature engineering) by automating non-linear feature extraction through recurrent neural networks. |
|
|
| ## <a name="dataset"></a> 2. Dataset Overview |
|
|
| The model was trained on a custom, manually annotated dataset consisting of **2,325 real-world food ordering messages** extracted from Facebook comments (specifically from Sơn Nguyễn vegetarian restaurant). |
| * **Train Set:** 1,860 samples (80%) |
| * **Validation Set:** 232 samples (10%) |
| * **Test Set:** 233 samples (10%) |
|
|
| ## <a name="labels"></a> 3. Entity Label Space |
|
|
| The model utilizes the standard **BIO (Begin - Inside - Outside)** tagging scheme across **15 distinct entity labels**, mapping to **7 core information classes**: |
|
|
| | Tag | Entity Class | Description | Examples | |
| | :--- | :--- | :--- | :--- | |
| | `B-FOOD`, `I-FOOD` | **FOOD** | Names of dishes, drinks, toppings | *cơm sườn, trà sữa, trân châu* | |
| | `B-QUANTITY`, `I-QUANTITY` | **QUANTITY** | Portions, servings, item counts | *1p, 2 ly, một hộp, 3 suất* | |
| | `B-NOTE`, `I-NOTE` | **NOTE** | Special requests, flavor modifications | *không, ít ngọt, nhiều* | |
| | `B-PLACE`, `I-PLACE` | **PLACE** | Delivery location, addresses | *Ký túc xá khu A, tòa D6, rào b4* | |
| | `B-PHONE`, `I-PHONE` | **PHONE** | Receiver's contact number | *0794987xxx, 0903123xxx* | |
| | `B-TIME`, `I-TIME` | **TIME** | Expected/requested delivery time | *lúc 11h30, trưa nay, 18h* | |
| | `B-PRICE`, `I-PRICE` | **PRICE** | Monetary cost, item prices | *35k, 40000, 25 ngàn* | |
| | `O` | **OUTSIDE** | Non-entity words, syntax connecting words | *cho em, giao qua, với, ạ, nhé* | |
|
|
| ## <a name="evaluation"></a> 4. Empirical Evaluation |
|
|
| The model was evaluated on an unseen Test Set using strict **Entity-level F1-Scores** via the `seqeval` framework. |
|
|
| * **BiLSTM-CRF Macro F1-Score:** `0.9771` |
| * **Comparison:** While the BiLSTM-CRF automates feature extraction, it achieved the lowest Macro F1-score (0.9771) among our tested baselines (CRF: 0.9889, mBERT: 0.9880, PhoBERT: 0.9913). |
|
|
| ## <a name="characteristics"></a> 5. Architecture Characteristics |
|
|
| The data flows through 3 main layers in this architecture: |
| 1. **Embedding Layer:** Converts numerical vocabulary identifiers into continuous mathematical vectors. |
| 2. **BiLSTM Layer:** Processes the sequence in both directions (left-to-right and right-to-left). At each time step $t$, the hidden state is a concatenation of both directions: $h_{t}=[\vec{h}_{t};\vec{h_{t}}]$. |
| 3. **CRF Layer:** Receives emission scores from the BiLSTM and applies a transition matrix to find the most valid sequence of tags via Viterbi Decoding. |
| |
| **Limitation:** Our analysis showed that BiLSTM-CRF suffers from *Information Decay* when dealing with long food orders where related entities are far apart (e.g., `QUANTITY` at the beginning and `FOOD` at the end). The Self-Attention mechanism in Transformer models (like PhoBERT) resolves this limitation efficiently. |
| |
| ## <a name="usage"></a> 6. Using BiLSTM-CRF-FoodNER with PyTorch |
| |
| Because this is a custom PyTorch model, you must define the model architecture class in your local environment before loading the `.pth` weights. |
| |
| ### Installation |
| ```bash |
| pip install torch huggingface_hub |
|
|
| ``` |
| |
| ### Example usage (Inference Pipeline) |
| |
| > **⚠️ IMPORTANT:** You MUST replace the `BiLSTM_CRF` class placeholder below with the exact PyTorch class definition you used during the training phase. |
| |
| ```python |
| import torch |
| import torch.nn as nn |
| from huggingface_hub import hf_hub_download |
| |
| # 1. DEFINE YOUR MODEL ARCHITECTURE (MUST MATCH TRAINING CODE) |
| class BiLSTM_CRF(nn.Module): |
| def __init__(self, vocab_size, tagset_size, embedding_dim, hidden_dim): |
| super(BiLSTM_CRF, self).__init__() |
| # --- PASTE YOUR ACTUAL PYTORCH INIT CODE HERE --- |
| pass |
| |
| def forward(self, sentence): |
| # --- PASTE YOUR ACTUAL FORWARD PASS HERE --- |
| pass |
| |
| def decode(self, sentence): |
| # --- PASTE YOUR VITERBI DECODING HERE --- |
| pass |
| |
| # Initialize the model with your original hyperparameters |
| # model = BiLSTM_CRF(vocab_size=..., tagset_size=15, embedding_dim=..., hidden_dim=...) |
| |
| # 2. DOWNLOAD AND LOAD WEIGHTS |
| REPO_ID = "CS221DoAn/Do_an_group_BiLSTM_CRF" |
| FILENAME = "bilstm_crf.pth" |
| |
| print("Downloading BiLSTM-CRF weights...") |
| model_path = hf_hub_download(repo_id=REPO_ID, filename=FILENAME) |
|
|
| # Load state dict |
| # model.load_state_dict(torch.load(model_path, map_location=torch.device('cpu'))) |
| # model.eval() |
|
|
| print("Model loaded successfully! Ready for inference.") |
|
|
| # 3. PREDICT (Replace with your actual text processing pipeline) |
| def predict_food_order(raw_text): |
| print(f"\nInput: {raw_text}") |
| # 1. Preprocess & Tokenize text |
| # 2. Convert to Tensor |
| # 3. Pass through model.decode() |
| # 4. Map ID to Label |
| pass |
| |
| ``` |
| |
| ## 7. Authors & Citation |
| |
| This project was developed for the **Natural Language Processing (CS221.Q21)** course at the University of Information Technology (UIT) - VNU-HCM. |
| |
| * **Students:** Võ Thành Lộc (24520989), Nguyễn Anh Nguyên (24521185) |
| |
| |
| * **Instructor:** Ph.D. Nguyễn Trọng Chỉnh |
| |
| |
| * **Date:** July 2026 |
| |
| |
| |
| If you use this model in your academic research or projects, please cite our project: |
| |
| ```bibtex |
| @misc{cs221_food_order_ner_bilstm_crf, |
| author = {Vo Thanh Loc and Nguyen Anh Nguyen}, |
| title = {Food Order Extraction: BiLSTM-CRF Baseline for Vietnamese NER}, |
| year = {2026}, |
| publisher = {Hugging Face}, |
| howpublished = https://huggingface.co/CS221DoAn/Do_an_group_BiLSTM_CRF |
| } |
| |
| ``` |
| |
