CS221DoAn/Do_an_group_CRF

CRF-FoodNER: Conditional Random Fields Baseline for Vietnamese Food Order Extraction

Table of contents

1. Introduction
2. Dataset Overview
3. Empirical Evaluation
4. Architecture Characteristics (CRF vs Deep Learning)
5. Using CRF-FoodNER with Python
6. Authors & Citation

---

1. Introduction

CS221DoAn/Do_an_group_CRF is a machine learning model based on Conditional Random Fields (CRF). It is specifically trained for Named Entity Recognition (NER) on domain-specific Vietnamese unstructured text: Online Food Delivery Orders and Messages.

In our research project for the CS221.Q21 course (Natural Language Processing), this CRF model serves as a traditional statistical baseline to compare against modern deep-learning Transformer architectures (PhoBERT, mBERT). It computes the conditional probability distribution of the entire output label sequence given the input observation sequence, relying on Emission and Transition matrices.

2. Dataset Overview

The model was trained on a custom, manually annotated dataset consisting of 2,325 real-world food ordering messages extracted from Facebook comments (specifically from Sơn Nguyễn vegetarian restaurant).

• Train Set: 1,860 samples (80%)
• Validation Set: 232 samples (10%)
• Test Set: 233 samples (10%)

3. Empirical Evaluation

The model was evaluated on an unseen Test Set using strict Entity-level F1-Scores via the seqeval framework.

• CRF Macro F1-Score: 0.9889
• Comparison: The CRF model achieved an impressively high Macro F1-score (0.9889), outperforming BiLSTM-CRF (0.9771) and slightly outperforming mBERT (0.9880). However, it remains lower than the PhoBERT architecture (0.9913).

4. Architecture Characteristics (CRF vs Deep Learning)

While CRF performs exceptionally well, our empirical analysis highlights its inherent limitations when processing social media text compared to architectures like PhoBERT:

• Transition Constraints: The Viterbi decoding algorithm allows CRF to strictly control the logical constraints of the BIO tagging scheme (e.g., the transition probability from O to I-FOOD is exactly 0).
• Out-of-Vocabulary (OOV) Vulnerability: CRF depends entirely on frequency matrices and manual Feature Engineering. When encountering social media text with heavy abbreviations, typos, or teencode, CRF struggles to find matching dictionary features.
• Unlike PhoBERT, which uses Byte-Pair Encoding (BPE) to break down OOV words into meaningful subwords, CRF lacks the ability to preserve vector representations for unknown linguistic noise.

5. Using CRF-FoodNER with Python

To use this model, you need to install the huggingface_hub and sklearn-crfsuite libraries.

Installation

pip install huggingface_hub sklearn-crfsuite

Example usage (Inference Pipeline)

⚠️ IMPORTANT: You MUST replace the word2features function below with the exact feature extraction function you used during the training phase. Otherwise, the model will not understand the input data format.

import pickle
from huggingface_hub import hf_hub_download

# 1. Download and load the .pkl model from Hugging Face Hub
REPO_ID = "CS221DoAn/Do_an_group_CRF"
FILENAME = "crf_model.pkl"

print("Downloading and loading the CRF model...")
model_path = hf_hub_download(repo_id=REPO_ID, filename=FILENAME)

with open(model_path, "rb") as f:
    crf_model = pickle.load(f)

# 2. Define the Feature Extraction Function (MUST MATCH YOUR TRAINING CODE)
def word2features(sent, i):
    word = sent[i]
    # --- REPLACE THIS BLOCK WITH YOUR ACTUAL FEATURE ENGINEERING LOGIC ---
    features = {
        'bias': 1.0,
        'word.lower()': word.lower(),
        'word.isupper()': word.isupper(),
        'word.istitle()': word.istitle(),
        'word.isdigit()': word.isdigit(),
        'word[:2]': word[:2] if len(word) > 2 else word,
        'word[-2:]': word[-2:] if len(word) > 2 else word,
    }
    # ---------------------------------------------------------------------
    return features

def sent2features(sent):
    return [word2features(sent, i) for i in range(len(sent))]

# 3. Predict Function
def predict_food_order(raw_text):
    print(f"\nInput: {raw_text}")
    print("=" * 60)
    
    # Simple whitespace tokenization (Replace with VnCoreNLP if you used it during training)
    tokens = raw_text.split()
    
    # Extract features
    features = [sent2features(tokens)]
    
    # Predict
    preds = crf_model.predict(features)[0]
    
    # Display Extracted Entities
    for token, label in zip(tokens, preds):
        print(f"{token:<20} {label}")

# --- EXECUTE TEST CASES ---
test_cases = [
    "1p cải xào, gà lát chiên giòn, chả giò, cơm thêm",
    "giao rào b4, 11h30. 0773570xxx."
]

for sample in test_cases:
    predict_food_order(sample)

6. Authors & Citation

This project was developed for the Natural Language Processing (CS221.Q21) course at the University of Information Technology (UIT) - VNU-HCM.

• Students: Võ Thành Lộc (24520989), Nguyễn Anh Nguyên (24521185)

• Instructor: Ph.D. Nguyễn Trọng Chỉnh

• Date: July 2026

If you use this model in your academic research or projects, please cite our project:

@misc{cs221_food_order_ner_crf,
  author = {Vo Thanh Loc and Nguyen Anh Nguyen},
  title = {Food Order Extraction: Conditional Random Fields Baseline for Vietnamese NER},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {(https://huggingface.co/CS221DoAn/Do_an_group_CRF)}
}