README.md

---
language: vi
library_name: transformers
tags:
  - ner
  - token-classification
  - mbert
  - bert
  - vietnamese
  - food-order
  - cs221
metrics:
  - f1
  - precision
  - recall
  - accuracy
pipeline_tag: token-classification
---

# mBERT-FoodNER: Multilingual Baseline for Vietnamese Food Order Extraction

#### Table of contents
1. [Introduction](#introduction)
2. [Dataset Overview](#dataset)
3. [Entity Label Space](#labels)
4. [Empirical Evaluation](#evaluation)
5. [Training Hyperparameters](#hyperparameters)
6. [Using mBERT-FoodNER with `transformers`](#transformers)
7. [Architecture Limitations (mBERT vs PhoBERT)](#limitations)
8. [Authors & Citation](#citation)

---

## <a name="introduction"></a> 1. Introduction

**`CS221DoAn/Do_an_group_mBERT`** is a fine-tuned Named Entity Recognition (NER) model based on the pre-trained Multilingual BERT (mBERT) architecture. It is specifically trained for **Token Classification** on domain-specific Vietnamese unstructured text: **Online Food Delivery Orders and Messages**.

In our research project for the CS221.Q21 course (Natural Language Processing), this mBERT model serves as a strong deep-learning baseline. It is used to compare and evaluate the effectiveness of monolingual models (like PhoBERT) versus multilingual architectures in handling the complex morphology of the Vietnamese language.

## <a name="dataset"></a> 2. Dataset Overview

The model was fine-tuned on a custom, manually annotated dataset consisting of **2,325 real-world food ordering messages** extracted from Facebook comments (specifically from Sơn Nguyễn vegetarian restaurant). The dataset exhibits strong social media characteristics including teencode, abbreviations, and missing diacritics. 
* **Train Set:** 1,860 samples (80%)
* **Validation Set:** 232 samples (10%)
* **Test Set:** 233 samples (10%)

## <a name="labels"></a> 3. Entity Label Space

The model utilizes the standard **BIO (Begin - Inside - Outside)** tagging scheme across **15 distinct entity labels**, mapping to **7 core information classes**:

| Tag | Entity Class | Description | Examples |
| :--- | :--- | :--- | :--- |
| `B-FOOD`, `I-FOOD` | **FOOD** | Names of dishes, drinks, toppings | *cơm sườn, trà sữa, trân châu* |
| `B-QUANTITY`, `I-QUANTITY` | **QUANTITY** | Portions, servings, item counts | *1p, 2 ly, một hộp, 3 suất* |
| `B-NOTE`, `I-NOTE` | **NOTE** | Special requests, flavor modifications | *không, ít ngọt, nhiều* |
| `B-PLACE`, `I-PLACE` | **PLACE** | Delivery location, addresses | *Ký túc xá khu A, tòa D6, rào b4* |
| `B-PHONE`, `I-PHONE` | **PHONE** | Receiver's contact number | *0794987xxx, 0903123xxx* |
| `B-TIME`, `I-TIME` | **TIME** | Expected/requested delivery time | *lúc 11h30, trưa nay, 18h* |
| `B-PRICE`, `I-PRICE` | **PRICE** | Monetary cost, item prices | *35k, 40000, 25 ngàn* |
| `O` | **OUTSIDE** | Non-entity words, syntax connecting words | *cho em, giao qua, với, ạ, nhé* |

## <a name="evaluation"></a> 4. Empirical Evaluation

The model was rigorously evaluated on an unseen Test Set using strict **Entity-level F1-Scores** via the `seqeval` framework.

*   **mBERT Macro F1-Score:** `0.9880`
*   **Comparison:** While mBERT achieved an excellent Macro F1-score of 0.9880, it slightly underperformed compared to the monolingual PhoBERT architecture (0.9913) due to limitations in handling Vietnamese compound words during tokenization.

## <a name="hyperparameters"></a> 5. Training Hyperparameters

The model was fine-tuned with the following configuration:
* **Learning Rate:** `2e-5`
* **Batch Size (Train/Eval):** `16`
* **Epochs:** `10`
* **Weight Decay:** `0.01`
* **Optimizer:** AdamW 
* **Evaluation Strategy:** Epoch-based (loading the best model based on F1-score at the end)

## <a name="transformers"></a> 6. Using mBERT-FoodNER with `transformers`

> **Note on Inference:** Unlike PhoBERT, this mBERT pipeline does **NOT** require the `py_vncorenlp` word segmenter. It applies the WordPiece algorithm directly to raw syllables.

### Installation 
Install the required libraries using pip:
```bash
pip install -q transformers torch

```

### Example usage (Full Inference Pipeline)

```py3
import re
import torch
from transformers import AutoTokenizer, AutoModelForTokenClassification

# 1. Setup Device and Load Model from Hugging Face Hub
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
MODEL_ID = "CS221DoAn/Do_an_group_mBERT"

print("Loading Model and Tokenizer...")
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForTokenClassification.from_pretrained(MODEL_ID).to(device)
model.eval()

def predict_food_order(raw_text):
    print(f"\nInput: {raw_text}")
    
    # Preprocessing & Text Cleaning
    clean_text = re.sub(r'([.,()!?:+])', r' \1 ', raw_text.replace('_', ' '))
    clean_text = re.sub(r'\s+', ' ', clean_text).strip()
    
    # Syllable-level tokenization (Whitespace split for WordPiece)
    tokens = clean_text.split()              
    
    # Tokenization & Subword Alignment
    input_ids = [tokenizer.cls_token_id]
    word_ids = [None]
    
    for i, word in enumerate(tokens):
        sub_ids = tokenizer.encode(word, add_special_tokens=False)
        input_ids.extend(sub_ids)
        word_ids.extend([i] * len(sub_ids))
        
    input_ids.append(tokenizer.sep_token_id)
    word_ids.append(None)
    
    # Forward Pass / Inference
    inputs = torch.tensor([input_ids]).to(device)
    with torch.no_grad():
        logits = model(inputs).logits
        preds = logits.argmax(dim=-1)[0].tolist()
        
    # Display Extracted Entities
    prev_idx = None
    for i, idx in enumerate(word_ids):
        if idx is not None and idx != prev_idx:
            tag_id = preds[i]
            label = model.config.id2label.get(tag_id, "O")
            print(f"{tokens[idx]:<20} {label}")
            prev_idx = idx

# --- EXECUTE TEST CASES ---
test_cases = [
    "1p cải xào, gà lát chiên giòn, chả giò, cơm thêm, Start Cf, 0384293xxx, giao lúc 12h15 ạ",
    "em 1p sườn ngào, bầu xào, chả giò, rau củ kho + cơm thêm. giao rào b4, 11h30. 0773570xxx.",
    "1p 15k thập cẩm ( cơm thêm) giao rào D6 0585115xxx ạ"
]

for sample in test_cases:
    predict_food_order(sample)

```

## 7. Architecture Limitations (mBERT vs PhoBERT)

While mBERT is a powerful multilingual model trained on 104 languages, our empirical analysis reveals a specific limitation when applied to Vietnamese NER tasks compared to monolingual architectures:

* **WordPiece vs. Vietnamese Morphology:** mBERT utilizes the WordPiece tokenization algorithm and does not natively support Vietnamese word segmentation tools (like VnCoreNLP). As a result, it applies WordPiece directly to discrete syllables.
* **Boundary Breaking:** When processing compound food names (e.g., `cơm sườn`, `rau muống`), the lack of prior word segmentation causes the model to fragment these cohesive terms, which slightly degrades the ability to preserve semantic boundaries compared to PhoBERT.

## 8. Authors & Citation

This project was developed for the **Natural Language Processing (CS221.Q21)** course at the University of Information Technology (UIT) - VNU-HCM.

* **Students:** Võ Thành Lộc (24520989), Nguyễn Anh Nguyên (24521185)
* **Instructor:** Ph.D. Nguyễn Trọng Chỉnh
* **Date:** July 2026

If you use this model in your academic research or projects, please cite our project and the original BERT paper:

```bibtex
@misc{cs221_food_order_ner_mbert,
  author = {Vo Thanh Loc and Nguyen Anh Nguyen},
  title = {Food Order Extraction: Multilingual BERT Baseline for Vietnamese NER},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = https://huggingface.co/CS221DoAn/Do_an_group_mBERT
}

@inproceedings{bert,
  title = {BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding},
  author = {Jacob Devlin and Ming-Wei Chang and Kenton Lee and Kristina Toutanova},
  booktitle = {Proceedings of NAACL-HLT 2019},
  year = {2019},
  pages = {4171--4186}
}

```

```

```