[Init]

README.md

@@ -1,3 +1,349 @@
----
-license: mit
----
+# Improving Low-Resource Sequence Labeling with Knowledge Fusion and Contextual Label Explanations
+
+![workflow](docs/assets/workflow.png)
+
+## 🌍 Overview
+
+This repository provides the official implementation of our paper:
+
+> **Improving Low-Resource Sequence Labeling with Knowledge Fusion and Contextual Label Explanations**
+> [arXiv:2501.19093](https://arxiv.org/abs/2501.19093)
+
+Low-resource sequence labeling often suffers from data sparsity and limited contextual generalization.
+We propose **KnowFREE (Knowledge-Fused Representation Enhancement Framework)** — a framework that integrates **external linguistic knowledge** and **contextual label explanations** into the model’s representation space to enhance low-resource performance.
+
+**Key Highlights:**
+
+Combining an **LLM-based knowledge enhancement workflow** with a **span-based KnowFREE model** to effectively address these challenges.
+
+**Pipeline 1: Label Extension Annotation**
+* Objective: To leverage LLMs to generate extension entity labels, word segmentation tags, and POS tags for the original samples.
+* Effect:
+  * Enhances the model's understanding of fine-grained contextual semantics.
+  * Improves the ability to distinguish entity boundaries in character-dense languages.
+
+**Pipeline 2: Enriched Explanation Synthesis**
+
+* Objective: Using LLMs to generate detailed, context-aware explanations for target entities, thereby synthesizing new, high-quality training samples.
+* Effect:
+  * Effectively mitigates semantic distribution bias between synthetic samples and the target domain.
+  * Significantly expands the number of samples and improves model performance in extremely low-resource settings.
+
+
+
+---
+
+## 🔗 Quick Links
+
+  - [Model Checkpoints](#♠️-model-checkpoints)
+  - [Data Augmentation Workflow](#📊-data-augmentation-workflow)
+  - [Train KnowFREE](#🔥-run-knowfree-models)
+  - [Citation](#📚-citation)
+
+## ♠️ Model Checkpoints
+
+Due to the large number of experiments, the architectural differences between the initial and reconstructed models, and the limited practical value of low-resource checkpoints sampled from the full dataset, we only release a few representative checkpoints (e.g., weibo) on Hugging Face for reference, as shown below:
+
+| Model                                                                                                                |  F1   |
+| :------------------------------------------------------------------------------------------------------------------- | :---: |
+| [aleversn/KnowFREE-Weibo-BERT-base (Many shots 1000 with ChatGLM3)](https://huggingface.co/aleversn/GCSE-BERT-base)  | 76.78 |
+| [aleversn/KnowFREE-Youku-BERT-base (Many shots 1000 with ChatGLM3)](https://huggingface.co/aleversn/GCSE-BERT-large) | 84.50 |
+
+---
+
+## 🧩 KnowFREE Framework
+
+![KnowFREE](docs/assets/knowfree.png)
+
+**Architecture**: A Biaffine-based span model that supports **nested entity** annotation.
+
+**Core Innovations:**
+
+* Introduces a **Local Multi-head Attention Layer** to efficiently fuse the multi-type extension label features generated in Pipeline 1.
+* **No External Knowledge Needed for Inference:** The model learns to fuse knowledge during the training, the logits of extension labels will be masked during inference.
+
+---
+
+## ⚙️ Installation Guide
+
+### Core Dependencies
+
+Create an environment and install dependencies:
+
+```bash
+conda create -n knowfree python=3.8
+conda activate knowfree
+```
+
+```bash
+pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html
+pip install transformers==4.18.0 fastNLP==1.0.1 PrettyTable
+pip install torch-scatter==2.0.8 -f https://data.pyg.org/whl/torch-1.8.0+cu111.html
+```
+
+## 📊 Data Augmentation Workflow
+
+See the detailed data synthesis pipeline in [Syn_Pipelines](docs/Syn_Pipelines.md).
+
+In KnowFREE, we employ **contextual paraphrasing and label explanation synthesis** to augment low-resource datasets.
+For each entity label, LLMs generate descriptive explanations that are integrated into the learning process to mitigate label semantic sparsity.
+
+---
+
+## 🔥 Run KnowFREE Models
+
+### Training with `KnowFREE`
+
+#### Dataset Format
+
+Specify the dataset path using the `data_present_path` argument (`Default`: `./datasets/present.json`). The file should be a JSON object with the following format:
+
+```json
+{
+    "weibo": {
+        "train": "./datasets/weibo/train.jsonl",
+        "dev": "./datasets/weibo/dev.jsonl",
+        "test": "./datasets/weibo/test.jsonl",
+        "labels": "./datasets/weibo/labels.txt"
+    }
+}
+```
+
+**Train Samples of Different Languages:**
+
+- Chinese
+
+```jsonl
+{"text": ["科", "技", "全", "方", "位", "资", "讯", "智", "能", "，", "快", "捷", "的", "汽", "车", "生", "活", "需", "要", "有", "三", "屏", "一", "云", "爱", "你"], "label": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"], "entities": []}
+{"text": ["对", "，", "输", "给", "一", "个", "女", "人", "，", "的", "成", "绩", "。", "失", "望"], "label": ["O", "O", "O", "O", "O", "O", "B-PER.NOM", "E-PER.NOM", "O", "O", "O", "O", "O", "O", "O"], "entities": [{"start": 6, "entity": "PER.NOM", "end": 8, "text": ["女", "人"]}]}
+{"text": ["今", "天", "下", "午", "起", "来", "看", "到", "外", "面", "的", "太", "阳", "。", "。", "。", "。", "我", "第", "一", "反", "应", "竟", "然", "是", "强", "烈", "的", "想", "回", "家", "泪", "想", "我", "们", "一", "起", "在", "嘉", "鱼", "个", "时", "候", "了", "。", "。", "。", "。", "有", "好", "多", "好", "多", "的", "话", "想", "对", "你", "说", "李", "巾", "凡", "想", "要", "瘦", "瘦", "瘦", "成", "李", "帆", "我", "是", "想", "切", "开", "云", "朵", "的", "心"], "label": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-LOC.NAM", "E-LOC.NAM", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-PER.NAM", "I-PER.NAM", "E-PER.NAM", "O", "O", "O", "O", "O", "O", "B-PER.NAM", "E-PER.NAM", "O", "O", "O", "O", "O", "O", "O", "O", "O"], "entities": [{"start": 38, "entity": "LOC.NAM", "end": 40, "text": ["嘉", "鱼"]}, {"start": 59, "entity": "PER.NAM", "end": 62, "text": ["李", "巾", "凡"]}, {"start": 68, "entity": "PER.NAM", "end": 70, "text": ["李", "帆"]}]}
+```
+
+- English
+
+```jsonl
+{"text": ["im", "thinking", "of", "a", "comedy", "where", "a", "group", "of", "husbands", "receive", "one", "chance", "from", "their", "wives", "to", "engage", "with", "other", "women"], "entities": [{"start": 4, "end": 5, "entity": "GENRE", "text": ["comedy"]}, {"start": 6, "end": 21, "entity": "PLOT", "text": ["a", "group", "of", "husbands", "receive", "one", "chance", "from", "their", "wives", "to", "engage", "with", "other", "women"]}]}
+{"text": ["another", "sequel", "of", "an", "action", "movie", "about", "drag", "street", "car", "races", "alcohol", "and", "gun", "violence"], "entities": [{"start": 1, "end": 2, "entity": "RELATIONSHIP", "text": ["sequel"]}, {"start": 4, "end": 5, "entity": "GENRE", "text": ["action"]}, {"start": 7, "end": 15, "entity": "PLOT", "text": ["drag", "street", "car", "races", "alcohol", "and", "gun", "violence"]}]}
+{"text": ["what", "is", "the", "name", "of", "the", "movie", "in", "which", "a", "group", "of", "criminals", "begin", "to", "suspect", "that", "one", "of", "them", "is", "a", "police", "informant", "after", "a", "simple", "jewelery", "heist", "goes", "terribly", "wrong"], "entities": [{"start": 9, "end": 32, "entity": "PLOT", "text": ["a", "group", "of", "criminals", "begin", "to", "suspect", "that", "one", "of", "them", "is", "a", "police", "informant", "after", "a", "simple", "jewelery", "heist", "goes", "terribly", "wrong"]}]}
+{"text": ["a", "movie", "with", "vin", "diesel", "in", "world", "war", "2", "in", "a", "foreign", "country", "shooting", "people"], "entities": [{"start": 3, "end": 5, "entity": "ACTOR", "text": ["vin", "diesel"]}, {"start": 6, "end": 9, "entity": "GENRE", "text": ["world", "war", "2"]}, {"start": 11, "end": 15, "entity": "PLOT", "text": ["foreign", "country", "shooting", "people"]}]}
+{"text": ["what", "is", "the", "1991", "disney", "animated", "movie", "that", "featured", "angela", "lansbury", "as", "the", "voice", "of", "a", "teapot"], "entities": [{"start": 3, "end": 4, "entity": "YEAR", "text": ["1991"]}, {"start": 5, "end": 6, "entity": "GENRE", "text": ["animated"]}, {"start": 9, "end": 11, "entity": "ACTOR", "text": ["angela", "lansbury"]}, {"start": 16, "end": 17, "entity": "CHARACTER_NAME", "text": ["teapot"]}]}
+```
+
+- Japanese
+
+```jsonl
+{"text": ["I", "n", "f", "o", "r", "m", "i", "x", "の", "動", "き", "を", "み", "て", "、", "オ", "ラ", "ク", "ル", "と", "I", "B", "M", "も", "追", "随", "し", "た", "。"], "entities": [{"start": 0, "end": 8, "entity": "法人名", "text": ["I", "n", "f", "o", "r", "m", "i", "x"]}, {"start": 15, "end": 19, "entity": "法人名", "text": ["オ", "ラ", "ク", "ル"]}, {"start": 20, "end": 23, "entity": "法人名", "text": ["I", "B", "M"]}]}
+{"text": ["現", "在", "は", "ア", "ニ", "メ", "ー", "シ", "ョ", "ン", "業", "界", "か", "ら", "退", "い", "て", "お", "り", "、", "水", "彩", "画", "家", "と", "し", "て", "も", "活", "動", "し", "て", "い", "る", "。"], "entities": []}
+{"text": ["大", "野", "東", "イ", "ン", "タ", "ー", "チ", "ェ", "ン", "ジ", "は", "、", "大", "分", "県", "豊", "後", "大", "野", "市", "大", "野", "町", "後", "田", "に", "あ", "る", "中", "九", "州", "横", "断", "道", "路", "の", "イ", "ン", "タ", "ー", "チ", "ェ", "ン", "ジ", "で", "あ", "る", "。"], "entities": [{"start": 0, "end": 11, "entity": "施設名", "text": ["大", "野", "東", "イ", "ン", "タ", "ー", "チ", "ェ", "ン", "ジ"]}, {"start": 13, "end": 26, "entity": "地名", "text": ["大", "分", "県", "豊", "後", "大", "野", "市", "大", "野", "町", "後", "田"]}, {"start": 29, "end": 36, "entity": "施設名", "text": ["中", "九", "州", "横", "断", "道", "路"]}]}
+{"text": ["2", "0", "1", "4", "年", "1", "月", "1", "5", "日", "、", "マ", "バ", "タ", "は", "ミ", "ャ", "ン", "マ", "ー", "の", "上", "座", "部", "仏", "教", "を", "擁", "護", "す", "る", "使", "命", "を", "持", "っ", "て", "、", "マ", "ン", "ダ", "レ", "ー", "の", "仏", "教", "僧", "の", "大", "規", "模", "な", "会", "議", "で", "正", "式", "に", "設", "立", "さ", "れ", "た", "。"], "entities": [{"start": 11, "end": 14, "entity": "法人名", "text": ["マ", "バ", "タ"]}, {"start": 15, "end": 20, "entity": "地名", "text": ["ミ", "ャ", "ン", "マ", "ー"]}, {"start": 38, "end": 43, "entity": "地名", "text": ["マ", "ン", "ダ", "レ", "ー"]}]}
+{"text": ["永", "泰", "荘", "駅", "は", "、", "中", "華", "人", "民", "共", "和", "国", "北", "京", "市", "海", "淀", "区", "に", "位", "置", "す", "る", "北", "京", "地", "下", "鉄", "8", "号", "線", "の", "駅", "で", "あ", "る", "。"], "entities": [{"start": 0, "end": 4, "entity": "施設名", "text": ["永", "泰", "荘", "駅"]}, {"start": 6, "end": 19, "entity": "地名", "text": ["中", "華", "人", "民", "共", "和", "国", "北", "京", "市", "海", "淀", "区"]}]}
+```
+
+- Korean
+
+```jsonl
+{"text": ["그", "모습", "을", "보", "ㄴ", "민이", "는", "할아버지", "가", "마치", "전쟁터", "에서", "이기", "고", "돌아오", "ㄴ", "장군", "처럼", "의젓", "하", "아", "보이", "ㄴ다고", "생각", "하", "았", "습니다", "."], "entities": [{"start": 5, "end": 6, "entity": "PS", "text": ["민이"]}]}
+{"text": ["내달", "18", "일", "부터", "내년", "2", "월", "20", "일", "까지", "는", "서울역", "에서", "무주리조트", "부근", "까지", "스키관광", "열차", "를", "운행", "하", "ㄴ다", "."], "entities": [{"start": 0, "end": 10, "entity": "DT", "text": ["내달", "18", "일", "부터", "내년", "2", "월", "20", "일", "까지"]}, {"start": 11, "end": 12, "entity": "LC", "text": ["서울역"]}, {"start": 13, "end": 14, "entity": "OG", "text": ["무주리조트"]}]}
+{"text": ["호소력", "있", "고", "선동", "적", "이", "ㄴ", "주제", "를", "잡아내", "는", "데", "능하", "ㄴ", "즈윅", "이", "지만", "이", "영화", "에서", "는", "무엇", "이", "호소력", "이", "있", "을지", "결정", "하", "지", "못하", "고", "망설이", "ㄴ다", "."], "entities": [{"start": 14, "end": 15, "entity": "PS", "text": ["즈윅"]}]}
+{"text": ["그래서", "세호", "는", "밤", "이", "면", "친구", "네", "집", "을", "돌아다니", "며", "아버지", "몰래", "연습", "을", "하", "았", "습니다", "."], "entities": [{"start": 1, "end": 2, "entity": "PS", "text": ["세호"]}, {"start": 3, "end": 4, "entity": "TI", "text": ["밤"]}]}
+{"text": ["황씨", "는", "자신", "이", "어리", "어서", "듣", "은", "이", "이야기", "가", "어린이", "들", "에게", "소박", "하", "ㄴ", "효자", "의", "마음", "을", "전하", "아", "주", "ㄹ", "수", "있", "을", "것", "같", "아", "5", "분", "짜리", "구연동화", "로", "각색", "하", "았", "다고", "말", "하", "ㄴ다", "."], "entities": [{"start": 0, "end": 1, "entity": "PS", "text": ["황씨"]}, {"start": 31, "end": 33, "entity": "TI", "text": ["5", "분"]}]}
+{"text": ["아버지", "가", "돌아가", "시", "ㄴ", "뒤", "어머니", "의", "편애", "를", "배경", "으로", "승주", "는", "집안", "에서", "만", "은", "대단", "하", "ㄴ", "권세", "를", "누리", "었", "다", "."], "entities": [{"start": 12, "end": 13, "entity": "PS", "text": ["승주"]}]}
+```
+
+**Labels**
+
+- `.txt`
+
+```
+O
+GPE.NAM
+GPE.NOM
+LOC.NAM
+LOC.NOM
+ORG.NAM
+ORG.NOM
+PER.NAM
+PER.NOM
+```
+
+- `.json` / `.jsonl`
+
+```json
+{
+    "O": {
+        "idx": 0,
+        "count": -1,
+        "is_target": true
+    },
+    "GPE.NAM": {
+        "idx": 1,
+        "count": -1,
+        "is_target": true
+    },
+    "GPE.NOM": {
+        "idx": 2,
+        "count": -1,
+        "is_target": true
+    },
+    "LOC.NAM": {
+        "idx": 3,
+        "count": -1,
+        "is_target": true
+    },
+    "LOC.NOM": {
+        "idx": 4,
+        "count": -1,
+        "is_target": true
+    },
+    "ORG.NAM": {
+        "idx": 5,
+        "count": -1,
+        "is_target": true
+    },
+    "ORG.NOM": {
+        "idx": 6,
+        "count": -1,
+        "is_target": true
+    },
+    "PER.NAM": {
+        "idx": 7,
+        "count": -1,
+        "is_target": true
+    },
+    "PER.NOM": {
+        "idx": 8,
+        "count": -1,
+        "is_target": true
+    },
+    "ADJECTIVE": {
+        "idx": 9,
+        "count": 1008,
+        "is_target": false
+    },
+    "ADPOSITION": {
+        "idx": 10,
+        "count": 41,
+        "is_target": false
+    },
+    "ADVERB": {
+        "idx": 11,
+        "count": 1147,
+        "is_target": false
+    },
+    "APP": {
+        "idx": 12,
+        "count": 3,
+        "is_target": false
+    },
+    "AUXILIARY": {
+        "idx": 13,
+        "count": 4,
+        "is_target": false
+    },...
+}
+```
+
+* **Model**: BERT / RoBERTa
+
+```python
+from main.trainers.knowfree_trainer import Trainer
+from transformers import BertTokenizer, BertConfig
+
+MODEL_PATH = "<MODEL_PATH>"
+tokenizer = BertTokenizer.from_pretrained(MODEL_PATH)
+config = BertConfig.from_pretrained(MODEL_PATH)
+trainer = Trainer(tokenizer=tokenizer, config=config, from_pretrained=MODEL_PATH,
+                  data_name='<DATASET_NAME>',
+                  batch_size=4,
+                  batch_size_eval=8,
+                  task_name='<TASK_NAME>')
+
+for i in trainer(num_epochs=120, other_lr=1e-3, weight_decay=0.01, remove_clashed=True, nested=False, eval_call_step=lambda x: x % 125 == 0):
+    a = i
+```
+
+**Key Params**
+
+- `other_lr`: the learning rate of the non-PLM part.
+- `remove_clashed`: remove the label that exists overlap (only choose the label with min start position)
+- `nested`: whether support nested entities, when do sequence labeling like `CMeEE`, you should set it as true and disabled `remove_clashed`.
+- `eval_call_step`: determine evaluation with `x` steps, defined with a function call.
+
+#### Evaluation Only
+
+Comment out the training loop to evaluate directly:
+
+```python
+trainer.eval(0, is_eval=True)
+```
+
+### Train with `CNN Nested NER`
+
+```python
+from main.trainers.cnnner_trainer import Trainer
+from transformers import BertTokenizer, BertConfig
+
+MODEL_PATH = "<MODEL_PATH>"
+tokenizer = BertTokenizer.from_pretrained(MODEL_PATH)
+config = BertConfig.from_pretrained(MODEL_PATH)
+trainer = Trainer(tokenizer=tokenizer, config=config, from_pretrained=MODEL_PATH,
+                  data_name='<DATASET_NAME>',
+                  batch_size=4,
+                  batch_size_eval=8,
+                  task_name='<TASK_NAME>')
+
+for i in trainer(num_epochs=120, other_lr=1e-3, weight_decay=0.01, remove_clashed=True, nested=False, eval_call_step=lambda x: x % 125 == 0):
+    a = i
+```
+
+#### Prediction
+
+```python
+from main.predictor.knowfree_predictor import KnowFREEPredictor
+from transformers import BertTokenizer, BertConfig
+
+MODEL_PATH = "<MODEL_PATH>"
+LABEL_FILE = '<LABEL_PATH>'
+tokenizer = BertTokenizer.from_pretrained(MODEL_PATH)
+config = BertConfig.from_pretrained(MODEL_PATH)
+pred = KnowFREEPredictor(tokenizer=tokenizer, config=config, from_pretrained=MODEL_PATH, label_file=LABEL_FILE, batch_size=4)
+
+for entities in pred(['叶赟葆：全球时尚财运滚滚而来钱', '我要去我要去花心花心花心耶分手大师贵仔邓超四大名捕围观话筒转发邓超贴吧微博号外话筒望周知。邓超四大名捕']):
+    print(entities)
+```
+
+**Result**
+
+```json
+[
+    [
+        {'start': 0, 'end': 3, 'entity': 'PER.NAM', 'text': ['叶', '赟', '葆'
+            ]
+        }
+    ],
+    [
+        {'start': 45, 'end': 47, 'entity': 'PER.NAM', 'text': ['邓', '超'
+            ]
+        },
+        {'start': 19, 'end': 21, 'entity': 'PER.NAM', 'text': ['邓', '超'
+            ]
+        },
+        {'start': 31, 'end': 33, 'entity': 'PER.NAM', 'text': ['邓', '超'
+            ]
+        }
+    ]
+]
+```
+
+## 📚 Citation
+```bibtex
+@misc{lai2025improvinglowresourcesequencelabeling,
+      title={Improving Low-Resource Sequence Labeling with Knowledge Fusion and Contextual Label Explanations}, 
+      author={Peichao Lai and Jiaxin Gan and Feiyang Ye and Yilei Wang and Bin Cui},
+      year={2025},
+      eprint={2501.19093},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2501.19093}, 
+}
+```

config.json

@@ -0,0 +1,181 @@
+{
+  "_name_or_path": "/home/lpc/models/chinese-bert-wwm-ext/",
+  "architectures": [
+    "CNNNerv1"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "biaffine_size": 200,
+  "classifier_dropout": null,
+  "cnn_depth": 3,
+  "cnn_dim": 200,
+  "directionality": "bidi",
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1",
+    "2": "LABEL_2",
+    "3": "LABEL_3",
+    "4": "LABEL_4",
+    "5": "LABEL_5",
+    "6": "LABEL_6",
+    "7": "LABEL_7",
+    "8": "LABEL_8",
+    "9": "LABEL_9",
+    "10": "LABEL_10",
+    "11": "LABEL_11",
+    "12": "LABEL_12",
+    "13": "LABEL_13",
+    "14": "LABEL_14",
+    "15": "LABEL_15",
+    "16": "LABEL_16",
+    "17": "LABEL_17",
+    "18": "LABEL_18",
+    "19": "LABEL_19",
+    "20": "LABEL_20",
+    "21": "LABEL_21",
+    "22": "LABEL_22",
+    "23": "LABEL_23",
+    "24": "LABEL_24",
+    "25": "LABEL_25",
+    "26": "LABEL_26",
+    "27": "LABEL_27",
+    "28": "LABEL_28",
+    "29": "LABEL_29",
+    "30": "LABEL_30",
+    "31": "LABEL_31",
+    "32": "LABEL_32",
+    "33": "LABEL_33",
+    "34": "LABEL_34",
+    "35": "LABEL_35",
+    "36": "LABEL_36",
+    "37": "LABEL_37",
+    "38": "LABEL_38",
+    "39": "LABEL_39",
+    "40": "LABEL_40",
+    "41": "LABEL_41",
+    "42": "LABEL_42",
+    "43": "LABEL_43",
+    "44": "LABEL_44",
+    "45": "LABEL_45",
+    "46": "LABEL_46",
+    "47": "LABEL_47",
+    "48": "LABEL_48",
+    "49": "LABEL_49",
+    "50": "LABEL_50",
+    "51": "LABEL_51",
+    "52": "LABEL_52",
+    "53": "LABEL_53",
+    "54": "LABEL_54",
+    "55": "LABEL_55",
+    "56": "LABEL_56",
+    "57": "LABEL_57",
+    "58": "LABEL_58",
+    "59": "LABEL_59",
+    "60": "LABEL_60",
+    "61": "LABEL_61",
+    "62": "LABEL_62",
+    "63": "LABEL_63",
+    "64": "LABEL_64",
+    "65": "LABEL_65",
+    "66": "LABEL_66",
+    "67": "LABEL_67"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "kernel_size": 3,
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1,
+    "LABEL_10": 10,
+    "LABEL_11": 11,
+    "LABEL_12": 12,
+    "LABEL_13": 13,
+    "LABEL_14": 14,
+    "LABEL_15": 15,
+    "LABEL_16": 16,
+    "LABEL_17": 17,
+    "LABEL_18": 18,
+    "LABEL_19": 19,
+    "LABEL_2": 2,
+    "LABEL_20": 20,
+    "LABEL_21": 21,
+    "LABEL_22": 22,
+    "LABEL_23": 23,
+    "LABEL_24": 24,
+    "LABEL_25": 25,
+    "LABEL_26": 26,
+    "LABEL_27": 27,
+    "LABEL_28": 28,
+    "LABEL_29": 29,
+    "LABEL_3": 3,
+    "LABEL_30": 30,
+    "LABEL_31": 31,
+    "LABEL_32": 32,
+    "LABEL_33": 33,
+    "LABEL_34": 34,
+    "LABEL_35": 35,
+    "LABEL_36": 36,
+    "LABEL_37": 37,
+    "LABEL_38": 38,
+    "LABEL_39": 39,
+    "LABEL_4": 4,
+    "LABEL_40": 40,
+    "LABEL_41": 41,
+    "LABEL_42": 42,
+    "LABEL_43": 43,
+    "LABEL_44": 44,
+    "LABEL_45": 45,
+    "LABEL_46": 46,
+    "LABEL_47": 47,
+    "LABEL_48": 48,
+    "LABEL_49": 49,
+    "LABEL_5": 5,
+    "LABEL_50": 50,
+    "LABEL_51": 51,
+    "LABEL_52": 52,
+    "LABEL_53": 53,
+    "LABEL_54": 54,
+    "LABEL_55": 55,
+    "LABEL_56": 56,
+    "LABEL_57": 57,
+    "LABEL_58": 58,
+    "LABEL_59": 59,
+    "LABEL_6": 6,
+    "LABEL_60": 60,
+    "LABEL_61": 61,
+    "LABEL_62": 62,
+    "LABEL_63": 63,
+    "LABEL_64": 64,
+    "LABEL_65": 65,
+    "LABEL_66": 66,
+    "LABEL_67": 67,
+    "LABEL_7": 7,
+    "LABEL_8": 8,
+    "LABEL_9": 9
+  },
+  "layer_norm_eps": 1e-12,
+  "logit_drop": 0,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "n_head": 4,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "num_target_labels": 8,
+  "output_past": true,
+  "pad_token_id": 0,
+  "pooler_fc_size": 768,
+  "pooler_num_attention_heads": 12,
+  "pooler_num_fc_layers": 3,
+  "pooler_size_per_head": 128,
+  "pooler_type": "first_token_transform",
+  "position_embedding_type": "absolute",
+  "size_embed_dim": 25,
+  "span_threshold": 0.5,
+  "torch_dtype": "float32",
+  "transformers_version": "4.18.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 21128
+}

enhance_data_info/labels_fusion.json

@@ -0,0 +1,342 @@
+{
+    "O": {
+        "idx": 0,
+        "count": -1,
+        "is_target": true
+    },
+    "GPE.NAM": {
+        "idx": 1,
+        "count": -1,
+        "is_target": true
+    },
+    "GPE.NOM": {
+        "idx": 2,
+        "count": -1,
+        "is_target": true
+    },
+    "LOC.NAM": {
+        "idx": 3,
+        "count": -1,
+        "is_target": true
+    },
+    "LOC.NOM": {
+        "idx": 4,
+        "count": -1,
+        "is_target": true
+    },
+    "ORG.NAM": {
+        "idx": 5,
+        "count": -1,
+        "is_target": true
+    },
+    "ORG.NOM": {
+        "idx": 6,
+        "count": -1,
+        "is_target": true
+    },
+    "PER.NAM": {
+        "idx": 7,
+        "count": -1,
+        "is_target": true
+    },
+    "PER.NOM": {
+        "idx": 8,
+        "count": -1,
+        "is_target": true
+    },
+    "ADJECTIVE": {
+        "idx": 9,
+        "count": 425,
+        "is_target": false
+    },
+    "ADPOSITION": {
+        "idx": 10,
+        "count": 50,
+        "is_target": false
+    },
+    "ADVERB": {
+        "idx": 11,
+        "count": 523,
+        "is_target": false
+    },
+    "Action": {
+        "idx": 12,
+        "count": 25,
+        "is_target": false
+    },
+    "Activity": {
+        "idx": 13,
+        "count": 16,
+        "is_target": false
+    },
+    "CONJUNCTION": {
+        "idx": 14,
+        "count": 52,
+        "is_target": false
+    },
+    "Country": {
+        "idx": 15,
+        "count": 4,
+        "is_target": false
+    },
+    "DETERMINER": {
+        "idx": 16,
+        "count": 55,
+        "is_target": false
+    },
+    "EXCLAMATION": {
+        "idx": 17,
+        "count": 2,
+        "is_target": false
+    },
+    "Emotion": {
+        "idx": 18,
+        "count": 18,
+        "is_target": false
+    },
+    "Event": {
+        "idx": 19,
+        "count": 22,
+        "is_target": false
+    },
+    "INTERJECTION": {
+        "idx": 20,
+        "count": 2,
+        "is_target": false
+    },
+    "Location": {
+        "idx": 21,
+        "count": 144,
+        "is_target": false
+    },
+    "MODAL": {
+        "idx": 22,
+        "count": 2,
+        "is_target": false
+    },
+    "NOUN": {
+        "idx": 23,
+        "count": 1974,
+        "is_target": false
+    },
+    "OCCUPATION": {
+        "idx": 24,
+        "count": 40,
+        "is_target": false
+    },
+    "ONOMATOPOEIA": {
+        "idx": 25,
+        "count": 2,
+        "is_target": false
+    },
+    "OTHER": {
+        "idx": 26,
+        "count": 2439,
+        "is_target": false
+    },
+    "Object": {
+        "idx": 27,
+        "count": 40,
+        "is_target": false
+    },
+    "Organization": {
+        "idx": 28,
+        "count": 140,
+        "is_target": false
+    },
+    "PARTICLE": {
+        "idx": 29,
+        "count": 6,
+        "is_target": false
+    },
+    "PERSON": {
+        "idx": 30,
+        "count": 635,
+        "is_target": false
+    },
+    "PRONOUN": {
+        "idx": 31,
+        "count": 209,
+        "is_target": false
+    },
+    "PROPER_NOUN": {
+        "idx": 32,
+        "count": 476,
+        "is_target": false
+    },
+    "PUNCTUATION": {
+        "idx": 33,
+        "count": 47,
+        "is_target": false
+    },
+    "Product": {
+        "idx": 34,
+        "count": 2,
+        "is_target": false
+    },
+    "QUANTIFIER": {
+        "idx": 35,
+        "count": 234,
+        "is_target": false
+    },
+    "Sympton": {
+        "idx": 36,
+        "count": 1,
+        "is_target": false
+    },
+    "UNKNOWN": {
+        "idx": 37,
+        "count": 1,
+        "is_target": false
+    },
+    "URL": {
+        "idx": 38,
+        "count": 9,
+        "is_target": false
+    },
+    "VERB": {
+        "idx": 39,
+        "count": 993,
+        "is_target": false
+    },
+    "WORD": {
+        "idx": 40,
+        "count": 9999,
+        "is_target": false
+    },
+    "Weather": {
+        "idx": 41,
+        "count": 2,
+        "is_target": false
+    },
+    "animal": {
+        "idx": 42,
+        "count": 8,
+        "is_target": false
+    },
+    "award": {
+        "idx": 43,
+        "count": 1,
+        "is_target": false
+    },
+    "color": {
+        "idx": 44,
+        "count": 1,
+        "is_target": false
+    },
+    "description": {
+        "idx": 45,
+        "count": 1,
+        "is_target": false
+    },
+    "food": {
+        "idx": 46,
+        "count": 10,
+        "is_target": false
+    },
+    "time": {
+        "idx": 47,
+        "count": 43,
+        "is_target": false
+    },
+    "Media": {
+        "idx": 48,
+        "count": 6,
+        "is_target": false
+    },
+    "MEASURE": {
+        "idx": 49,
+        "count": 8,
+        "is_target": false
+    },
+    "APP": {
+        "idx": 50,
+        "count": 3,
+        "is_target": false
+    },
+    "AUXILIARY": {
+        "idx": 51,
+        "count": 4,
+        "is_target": false
+    },
+    "EMOJI": {
+        "idx": 52,
+        "count": 5,
+        "is_target": false
+    },
+    "Group": {
+        "idx": 53,
+        "count": 4,
+        "is_target": false
+    },
+    "INTERROGATIVE": {
+        "idx": 54,
+        "count": 3,
+        "is_target": false
+    },
+    "Media_Data": {
+        "idx": 55,
+        "count": 1,
+        "is_target": false
+    },
+    "NUMERAL": {
+        "idx": 56,
+        "count": 26,
+        "is_target": false
+    },
+    "SYMBOL": {
+        "idx": 57,
+        "count": 4,
+        "is_target": false
+    },
+    "Symbol": {
+        "idx": 58,
+        "count": 2,
+        "is_target": false
+    },
+    "article": {
+        "idx": 59,
+        "count": 2,
+        "is_target": false
+    },
+    "element": {
+        "idx": 60,
+        "count": 2,
+        "is_target": false
+    },
+    "price": {
+        "idx": 61,
+        "count": 5,
+        "is_target": false
+    },
+    "news": {
+        "idx": 62,
+        "count": 1000,
+        "is_target": false
+    },
+    "建筑": {
+        "idx": 63,
+        "count": 1000,
+        "is_target": false
+    },
+    "Feature": {
+        "idx": 64,
+        "count": 1000,
+        "is_target": false
+    },
+    "Art": {
+        "idx": 65,
+        "count": 1000,
+        "is_target": false
+    },
+    "Class": {
+        "idx": 66,
+        "count": 1000,
+        "is_target": false
+    },
+    "CLASSIFIER": {
+        "idx": 67,
+        "count": 1000,
+        "is_target": false
+    }
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ba0fb801977f08de152eca7c94b51506af50055492cab8a9eb8ecd0a8661e29
+size 418927535

special_tokens_map.json

@@ -0,0 +1 @@
+{"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}

tokenizer_config.json

@@ -0,0 +1 @@
+{"do_lower_case": true, "do_basic_tokenize": true, "never_split": null, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "tokenize_chinese_chars": true, "strip_accents": null, "special_tokens_map_file": "/home/lpc/models/chinese-bert-wwm-ext/special_tokens_map.json", "name_or_path": "/home/lpc/models/chinese-bert-wwm-ext/", "tokenizer_class": "BertTokenizer"}