README.md

---
language: vi
tags:
  - ner
  - conditional-random-field
  - crf
  - sklearn-crfsuite
  - vietnamese
  - food-order
  - cs221
metrics:
  - f1
  - precision
  - recall
  - accuracy
---

# CRF-FoodNER: Conditional Random Fields Baseline for Vietnamese Food Order Extraction

#### Table of contents
1. [Introduction](#introduction)
2. [Dataset Overview](#dataset)
3. [Empirical Evaluation](#evaluation)
4. [Architecture Characteristics (CRF vs Deep Learning)](#characteristics)
5. [Using CRF-FoodNER with Python](#usage)
6. [Authors & Citation](#citation)

---

## <a name="introduction"></a> 1. Introduction

**`CS221DoAn/Do_an_group_CRF`** is a machine learning model based on **Conditional Random Fields (CRF)**. It is specifically trained for **Named Entity Recognition (NER)** on domain-specific Vietnamese unstructured text: **Online Food Delivery Orders and Messages**.

In our research project for the CS221.Q21 course (Natural Language Processing), this CRF model serves as a traditional statistical baseline to compare against modern deep-learning Transformer architectures (PhoBERT, mBERT). It computes the conditional probability distribution of the entire output label sequence given the input observation sequence, relying on Emission and Transition matrices.

## <a name="dataset"></a> 2. Dataset Overview

The model was trained on a custom, manually annotated dataset consisting of **2,325 real-world food ordering messages** extracted from Facebook comments (specifically from Sơn Nguyễn vegetarian restaurant). 
* **Train Set:** 1,860 samples (80%)
* **Validation Set:** 232 samples (10%)
* **Test Set:** 233 samples (10%)

## <a name="evaluation"></a> 3. Empirical Evaluation

The model was evaluated on an unseen Test Set using strict **Entity-level F1-Scores** via the `seqeval` framework.

*   **CRF Macro F1-Score:** `0.9889`
*   **Comparison:** The CRF model achieved an impressively high Macro F1-score (0.9889), outperforming BiLSTM-CRF (0.9771) and slightly outperforming mBERT (0.9880). However, it remains lower than the PhoBERT architecture (0.9913).

## <a name="characteristics"></a> 4. Architecture Characteristics (CRF vs Deep Learning)

While CRF performs exceptionally well, our empirical analysis highlights its inherent limitations when processing social media text compared to architectures like PhoBERT:

*   **Transition Constraints:** The Viterbi decoding algorithm allows CRF to strictly control the logical constraints of the BIO tagging scheme (e.g., the transition probability from `O` to `I-FOOD` is exactly 0).
*   **Out-of-Vocabulary (OOV) Vulnerability:** CRF depends entirely on frequency matrices and manual Feature Engineering. When encountering social media text with heavy abbreviations, typos, or teencode, CRF struggles to find matching dictionary features. 
*   Unlike PhoBERT, which uses Byte-Pair Encoding (BPE) to break down OOV words into meaningful subwords, CRF lacks the ability to preserve vector representations for unknown linguistic noise.

## <a name="usage"></a> 5. Using CRF-FoodNER with Python

To use this model, you need to install the `huggingface_hub` and `sklearn-crfsuite` libraries.

### Installation 
```bash
pip install huggingface_hub sklearn-crfsuite

```

### Example usage (Inference Pipeline)

> **⚠️ IMPORTANT:** You MUST replace the `word2features` function below with the exact feature extraction function you used during the training phase. Otherwise, the model will not understand the input data format.

```py3
import pickle
from huggingface_hub import hf_hub_download

# 1. Download and load the .pkl model from Hugging Face Hub
REPO_ID = "CS221DoAn/Do_an_group_CRF"
FILENAME = "crf_model.pkl"

print("Downloading and loading the CRF model...")
model_path = hf_hub_download(repo_id=REPO_ID, filename=FILENAME)

with open(model_path, "rb") as f:
    crf_model = pickle.load(f)

# 2. Define the Feature Extraction Function (MUST MATCH YOUR TRAINING CODE)
def word2features(sent, i):
    word = sent[i]
    # --- REPLACE THIS BLOCK WITH YOUR ACTUAL FEATURE ENGINEERING LOGIC ---
    features = {
        'bias': 1.0,
        'word.lower()': word.lower(),
        'word.isupper()': word.isupper(),
        'word.istitle()': word.istitle(),
        'word.isdigit()': word.isdigit(),
        'word[:2]': word[:2] if len(word) > 2 else word,
        'word[-2:]': word[-2:] if len(word) > 2 else word,
    }
    # ---------------------------------------------------------------------
    return features

def sent2features(sent):
    return [word2features(sent, i) for i in range(len(sent))]

# 3. Predict Function
def predict_food_order(raw_text):
    print(f"\nInput: {raw_text}")
    print("=" * 60)
    
    # Simple whitespace tokenization (Replace with VnCoreNLP if you used it during training)
    tokens = raw_text.split()
    
    # Extract features
    features = [sent2features(tokens)]
    
    # Predict
    preds = crf_model.predict(features)[0]
    
    # Display Extracted Entities
    for token, label in zip(tokens, preds):
        print(f"{token:<20} {label}")

# --- EXECUTE TEST CASES ---
test_cases = [
    "1p cải xào, gà lát chiên giòn, chả giò, cơm thêm",
    "giao rào b4, 11h30. 0773570xxx."
]

for sample in test_cases:
    predict_food_order(sample)

```

##  6. Authors & Citation

This project was developed for the **Natural Language Processing (CS221.Q21)** course at the University of Information Technology (UIT) - VNU-HCM.

* **Students:** Võ Thành Lộc (24520989), Nguyễn Anh Nguyên (24521185)


* **Instructor:** Ph.D. Nguyễn Trọng Chỉnh


* **Date:** July 2026



If you use this model in your academic research or projects, please cite our project:

```bibtex
@misc{cs221_food_order_ner_crf,
  author = {Vo Thanh Loc and Nguyen Anh Nguyen},
  title = {Food Order Extraction: Conditional Random Fields Baseline for Vietnamese NER},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {(https://huggingface.co/CS221DoAn/Do_an_group_CRF)}
}

```

```

```