Hugging Face's logo

alistairmcleay
/
UBAR-distilgpt2

Text Generation
Transformers
PyTorch
gpt2
text-generation-inference
Model card Files
xet
Community
1
UBAR-distilgpt2 / eval_log.json
alistairmcleay's picture
alistairmcleay
Upload eval_log.json
6c2afd3
raw
history blame contribute delete
25.7 kB
Invalid JSON: Unexpected token 'I', "INFO:root:"... is not valid JSON
 INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:inference time: 2.18 min
INFO:root:Scoring time: 0.02 min
INFO:root:validation [CTR] match: 93.00 success: 81.00 bleu: 14.31 score: 101.31
INFO:root:update eval results to logs_test/home-all_test.json
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:inference time: 0.62 min
INFO:root:Scoring time: 0.00 min
INFO:root:validation [CTR] match: 100.00 success: 100.00 bleu: 18.49 score: 118.49
INFO:root:update eval results to logs_test/experiments-all_test.json
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:inference time: 0.70 min
INFO:root:Scoring time: 0.01 min
INFO:root:validation [CTR] match: 100.00 success: 100.00 bleu: 18.49 score: 118.49
INFO:root:update eval results to logs_test/experiments-all_test.json
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:inference time: 4.61 min
INFO:root:Scoring time: 0.01 min
INFO:root:validation [CTR] match: 100.00 success: 90.00 bleu: 12.60 score: 107.60
INFO:root:update eval results to logs_test/experiments-all_test.json
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:inference time: 21.91 min
INFO:root:Scoring time: 0.17 min
INFO:root:validation [CTR] match: 87.90 success: 76.60 bleu: 15.30 score: 97.55
INFO:root:update eval results to logs_test/home-all_test.json
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:8434, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed_gen_usr_utts/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:14969, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed_gen_usr_utts/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:14969, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:Reading encoded data from ./data/preprocessed_gen_usr_utts/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:14969, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:Reading encoded data from ./data/preprocessed_gen_usr_utts/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:14969, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:inference time: 0.71 min
INFO:root:Reading encoded data from ./data/preprocessed_gen_usr_utts/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:14969, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:inference time: 0.89 min
INFO:root:Reading encoded data from ./data/preprocessed_gen_usr_utts/UBAR/multi-woz-processed/new_db_se_blank_encoded.data.json
INFO:root:train size:14969, dev size:999, test size:1000
INFO:root:Generate setting:
use true_prev_bspn=False
use true_prev_aspn=False
use true_db_pointer=False
use true_prev_resp=False
use true_curr_bspn=False
use true_curr_aspn=False
use_all_previous_context=True
INFO:root:Running eval on test
INFO:root:***** Running Evaluation *****
INFO:root: Num Turns = 7372
INFO:root:inference time: 0.10 min
INFO:root:Scoring time: 1.08 min
INFO:root:validation [CTR] match: 100.00 success: 100.00 bleu: 20.28 score: 120.28
INFO:root:update eval results to logs_test/UBAR-all_test.json
INFO:root:Evaluation finished