Upload 8 files

README.md

@@ -0,0 +1,84 @@
+---
+datasets:
+- drcd
+tags:
+- question-generation
+widget:
+- text: "[HL]伊隆·里夫·馬斯克[HL]是一名企業家和商業大亨"
+---
+# Transformer QG on DRCD
+請參閱 https://github.com/p208p2002/Transformer-QG-on-DRCD 獲得更多細節
+
+The inputs of the model refers to 
+```
+we integrate C and A into a new C' in the following form.
+C' = [c1, c2, ..., [HL], a1, ..., a|A|, [HL], ..., c|C|]
+```
+> Proposed by [Ying-Hong Chan & Yao-Chung Fan. (2019). A Re-current BERT-based Model for Question Generation.](https://www.aclweb.org/anthology/D19-5821/)
+
+## Features
+- Fully pipline from fine-tune to evaluation
+- Support most of state of the art models
+- Fast deploy as a API server
+
+## DRCD dataset
+[台達閱讀理解資料集 Delta Reading Comprehension Dataset (DRCD)](https://github.com/DRCKnowledgeTeam/DRCD) 屬於通用領域繁體中文機器閱讀理解資料集。 DRCD資料集從2,108篇維基條目中整理出10,014篇段落，並從段落中標註出30,000多個問題。
+
+## Available models
+- BART (base on **[uer/bart-base-chinese-cluecorpussmall](https://huggingface.co/uer/bart-base-chinese-cluecorpussmall)**)
+
+## Expriments
+Model             |Bleu 1|Bleu 2|Bleu 3|Bleu 4|METEOR|ROUGE-L|
+------------------|------|------|------|------|------|-------|
+BART-HLSQG        |34.25 |27.70 |22.43 |18.13 |23.58 |36.88  |
+BART-HLSQG-v2     |39.30 |32.51 |26.72 |22.08 |24.94 |41.18  |
+
+## Environment requirements
+The hole development is based on Ubuntu system
+
+1. If you don't have pytorch 1.6+ please install or update first
+> https://pytorch.org/get-started/locally/
+
+2. Install packages `pip install -r requirements.txt`
+
+3. Setup scorer `python setup_scorer.py`
+
+5. Download dataset `python init_dataset.py`
+
+
+## Training
+### Seq2Seq LM
+```
+usage: train_seq2seq_lm.py [-h]
+                           [--base_model {facebook/bart-base,facebook/bart-large,t5-small,t5-base,t5-large}]
+                           [-d {squad,squad-nqg}] [--epoch EPOCH] [--lr LR]
+                           [--dev DEV] [--server] [--run_test]
+                           [-fc FROM_CHECKPOINT]
+
+optional arguments:
+  -h, --help            show this help message and exit
+  --base_model {facebook/bart-base,facebook/bart-large,t5-small,t5-base,t5-large}
+  -d {squad,squad-nqg}, --dataset {squad,squad-nqg}
+  --epoch EPOCH
+  --lr LR
+  --dev DEV
+  --server
+  --run_test
+  -fc FROM_CHECKPOINT, --from_checkpoint FROM_CHECKPOINT
+```
+
+## Deploy
+### Start up
+```
+python train_seq2seq_lm.py --server --base_model YOUR_BASE_MODEL --from_checkpoint FROM_CHECKPOINT
+```
+### Request example
+```
+curl --location --request POST 'http://127.0.0.1:5000/' \
+--header 'Content-Type: application/x-www-form-urlencoded' \
+--data-urlencode 'context=[HL]伊隆·里夫·馬斯克[HL]是一名企業家和商業大亨'
+```
+```json
+{"predict": "哪一個人是一名企業家和商業大亨?"}
+```
+

added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "[HL]": 21128
+}

config.json

@@ -0,0 +1,74 @@
+{
+  "_name_or_path": "fnlp/bart-base-chinese",
+  "activation_dropout": 0.1,
+  "activation_function": "gelu",
+  "add_bias_logits": false,
+  "add_final_layer_norm": false,
+  "architectures": [
+    "BartForConditionalGeneration"
+  ],
+  "attention_dropout": 0.1,
+  "bos_token_id": 101,
+  "classif_dropout": 0.1,
+  "classifier_dropout": 0.0,
+  "d_model": 768,
+  "decoder_attention_heads": 12,
+  "decoder_ffn_dim": 3072,
+  "decoder_layerdrop": 0.0,
+  "decoder_layers": 6,
+  "decoder_start_token_id": 102,
+  "dropout": 0.1,
+  "early_stopping": true,
+  "encoder_attention_heads": 12,
+  "encoder_ffn_dim": 3072,
+  "encoder_layerdrop": 0.0,
+  "encoder_layers": 6,
+  "eos_token_id": 102,
+  "forced_eos_token_id": 102,
+  "gradient_checkpointing": false,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1",
+    "2": "LABEL_2"
+  },
+  "init_std": 0.02,
+  "is_encoder_decoder": true,
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1,
+    "LABEL_2": 2
+  },
+  "max_position_embeddings": 512,
+  "model_type": "bart",
+  "no_repeat_ngram_size": 3,
+  "normalize_before": false,
+  "normalize_embedding": true,
+  "num_beams": 4,
+  "num_hidden_layers": 6,
+  "pad_token_id": 0,
+  "scale_embedding": false,
+  "task_specific_params": {
+    "summarization": {
+      "length_penalty": 1.0,
+      "max_length": 128,
+      "min_length": 12,
+      "num_beams": 4
+    },
+    "summarization_cnn": {
+      "length_penalty": 2.0,
+      "max_length": 142,
+      "min_length": 56,
+      "num_beams": 4
+    },
+    "summarization_xsum": {
+      "length_penalty": 1.0,
+      "max_length": 62,
+      "min_length": 11,
+      "num_beams": 6
+    }
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.24.0",
+  "use_cache": true,
+  "vocab_size": 21129
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:588acbdbe87688056472b537032a5faf86f355a9b642d1c4240ca11726290143
+size 465197369

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,15 @@
+{
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "name_or_path": "fnlp/bart-base-chinese",
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "special_tokens_map_file": "/remote-home/yfshao/.cache/huggingface/transformers/d521373fc7ac35f63d56cf303de74a202403dcf1aaa792cd01f653694be59563.dd8bd9bfd3664b530ea4e645105f557769387b3da9f79bdb55ed556bdd80611d",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}