CS221DoAn/vietnamese_food_order_extraction

PhoBERT-FoodNER: Vietnamese Food Order Named Entity Recognition

Table of contents

1. Introduction
2. Dataset Overview
3. Entity Label Space
4. Empirical Evaluation
5. Training Hyperparameters
6. Using PhoBERT-FoodNER with transformers
7. Notes on Word Segmentation
8. Authors & Citation

---

1. Introduction

CS221DoAn/vietnamese_food_order_extraction is a fine-tuned Named Entity Recognition (NER) model based on the pre-trained monolingual language model PhoBERT-base (vinai/phobert-base)[cite: 1]. It is specifically trained and optimized for Token Classification on domain-specific Vietnamese unstructured text: Online Food Delivery Orders and Messages.

The model acts as an automated information extraction engine capable of accurately resolving complex entities within conversational speech, abbreviations, and student slang commonly used in food delivery platforms. This project was developed as part of the CS221.Q21 course (Natural Language Processing).

2. Dataset Overview

The model was fine-tuned on a custom, manually annotated dataset consisting of 2,325 real-world food ordering messages extracted from Facebook comments (specifically from Sơn Nguyễn vegetarian restaurant). The dataset exhibits strong social media characteristics including teencode, abbreviations, and missing diacritics.

• Train Set: 1,860 samples (80%)
• Validation Set: 232 samples (10%)
• Test Set: 233 samples (10%)

3. Entity Label Space

The model utilizes the standard BIO (Begin - Inside - Outside) tagging scheme across 15 distinct entity labels, mapping to 7 core information classes:

Tag	Entity Class	Description	Examples
B-FOOD, I-FOOD	FOOD	Names of dishes, drinks, toppings	cơm sườn, trà sữa, trân châu
B-QUANTITY, I-QUANTITY	QUANTITY	Portions, servings, item counts	1p, 2 ly, một hộp, 3 suất
B-NOTE, I-NOTE	NOTE	Special requests, flavor modifications	không ..., ít ngọt, nhiều ...
B-PLACE, I-PLACE	PLACE	Delivery location, addresses	Ký túc xá khu A, tòa D6, rào b4
B-PHONE, I-PHONE	PHONE	Receiver's contact number	0794987xxx, 0903123xxx
B-TIME, I-TIME	TIME	Expected/requested delivery time	lúc 11h30, trưa nay, 18h
B-PRICE, I-PRICE	PRICE	Monetary cost, item prices	35k, 40000, 25 ngàn
O	OUTSIDE	Non-entity words, syntax connecting words	cho em, giao qua, với, ạ, nhé

4. Empirical Evaluation

The model's performance was evaluated on an unseen Test Set using strict Entity-level F1-Scores via the seqeval framework:

              precision    recall  f1-score   support

        FOOD     0.9921    0.9947    0.9934      1135
        NOTE     0.9938    0.9815    0.9876       162
       PHONE     1.0000    1.0000    1.0000       225
       PLACE     0.9926    0.9963    0.9945       270
       PRICE     1.0000    1.0000    1.0000        34
    QUANTITY     0.9965    0.9965    0.9965       282
        TIME     0.9593    0.9752    0.9672       121

   micro avg     0.9919    0.9937    0.9928      2229
   macro avg     0.9906    0.9920    0.9913      2229
weighted avg     0.9920    0.9937    0.9928      2229

5. Training Hyperparameters

The model was fine-tuned using the Trainer API from Hugging Face with the following configuration:

• Learning Rate: 2e-5
• Batch Size (Train/Eval): 16
• Epochs: 10
• Weight Decay: 0.01
• Optimizer: AdamW
• Evaluation Strategy: Epoch-based (loading the best model based on F1-score at the end)

6. Using PhoBERT-FoodNER with transformers

Installation

Install the required libraries using pip:

pip install -q transformers py_vncorenlp torch

Example usage (Full Inference Pipeline)

import os
import re
import torch
import urllib.request
import py_vncorenlp
from transformers import AutoTokenizer, AutoModelForTokenClassification

# 1. Setup Device and Load Model from Hugging Face Hub
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
MODEL_ID = "CS221DoAn/vietnamese_food_order_extraction"

print("Loading Model and Tokenizer...")
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForTokenClassification.from_pretrained(MODEL_ID).to(device)
model.eval()

# 2. Initialize VnCoreNLP (Robust setup for Windows/Linux/Colab)
vncorenlp_dir = os.path.abspath('./vncorenlp')
os.makedirs(os.path.join(vncorenlp_dir, "models", "wordsegmenter"), exist_ok=True)

if not os.path.exists(os.path.join(vncorenlp_dir, "VnCoreNLP-1.2.jar")):
    print("Downloading VnCoreNLP models...")
    base_url = "https://raw.githubusercontent.com/vncorenlp/VnCoreNLP/master/"
    files = ["VnCoreNLP-1.2.jar", "models/wordsegmenter/vi-vocab", "models/wordsegmenter/wordsegmenter.rdr"]
    for f in files:
        urllib.request.urlretrieve(base_url + f, os.path.join(vncorenlp_dir, f))

rdrsegmenter = py_vncorenlp.VnCoreNLP(annotators=["wseg"], save_dir=vncorenlp_dir)
user_dict = {"a", "giao"}  # Custom lexicon to prevent incorrect compounding

def predict_food_order(raw_text):
    print(f"\nInput: {raw_text}")
    
    # Preprocessing & Text Cleaning
    clean_text = re.sub(r'([.,()!?:+])', r' \1 ', raw_text.replace('_', ' '))
    clean_text = re.sub(r'\s+', ' ', clean_text).strip()
    
    # Word Segmentation
    tokens = []
    for token in rdrsegmenter.word_segment(clean_text)[0].split():
        parts = token.split('_')
        if any(p.lower() in user_dict for p in parts):
            tokens.extend(parts)  
        else:
            tokens.append(token)              
    
    # Tokenization & Subword Alignment
    input_ids = [tokenizer.cls_token_id]
    word_ids = [None]
    
    for i, word in enumerate(tokens):
        sub_ids = tokenizer.encode(word, add_special_tokens=False)
        input_ids.extend(sub_ids)
        word_ids.extend([i] * len(sub_ids))
        
    input_ids.append(tokenizer.sep_token_id)
    word_ids.append(None)
    
    # Forward Pass / Inference
    inputs = torch.tensor([input_ids]).to(device)
    with torch.no_grad():
        logits = model(inputs).logits
        preds = logits.argmax(dim=-1)[0].tolist()
        
    # Display Extracted Entities
    prev_idx = None
    for i, idx in enumerate(word_ids):
        if idx is not None and idx != prev_idx:
            tag_id = preds[i]
            label = model.config.id2label.get(tag_id, "O")
            print(f"{tokens[idx]:<20} {label}")
            prev_idx = idx

# --- EXECUTE TEST CASES ---
test_cases = [
    "1p cải xào, gà lát chiên giòn, chả giò, cơm thêm, Start Cf, 0384293xxx, giao lúc 12h15 ạ",
    "em 1p sườn ngào, bầu xào, chả giò, rau củ kho + cơm thêm. giao rào b4, 11h30. 0773570xxx.",
    "1p 15k thập cẩm ( cơm thêm) giao rào D6 0585115xxx ạ"
]

for sample in test_cases:
    predict_food_order(sample)

7. Notes on Word Segmentation

As PhoBERT employed the RDRSegmenter from VnCoreNLP to pre-process its pre-training data, input texts MUST be word-segmented before being fed into the tokenizer (e.g., separating syllables of compound words with underscores like cơm_sườn).

Feeding raw, unsegmented text directly into tokenizer.encode() will break word boundaries and significantly degrade NER accuracy. The example code above handles this pipeline automatically using py_vncorenlp.

8. Authors & Citation

This project was developed for the Natural Language Processing (CS221.Q21) course at the University of Information Technology (UIT) - VNU-HCM.

• Students: Võ Thành Lộc (24520989), Nguyễn Anh Nguyên (24521185)
• Instructor: Ph.D Nguyễn Trọng Chỉnh
• Date: July 2026

If you use this model in your academic research or projects, please cite our project and the original PhoBERT paper:

@misc{cs221_food_order_ner,
  author = {Vo Thanh Loc and Nguyen Anh Nguyen},
  title = {Food Order Extraction: Vietnamese NER using PhoBERT},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = https://huggingface.co/CS221DoAn/vietnamese_food_order_extraction
}

@inproceedings{phobert,
  title     = {{PhoBERT: Pre-trained language models for Vietnamese}},
  author    = {Dat Quoc Nguyen and Anh Tuan Nguyen},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
  year      = {2020},
  pages     = {1037--1042}
}