Upload 31 files

ncr_cclue/with_knowledge_without_select/final/README.md

@@ -0,0 +1,220 @@
+---
+library_name: peft
+base_model: /data0/maqi/huggingface_models/TechGPT-7B
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+## Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: bitsandbytes
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: True
+- bnb_4bit_compute_dtype: float16
+
+### Framework versions
+
+
+- PEFT 0.6.2

ncr_cclue/with_knowledge_without_select/final/adapter_config.json

@@ -0,0 +1,28 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data0/maqi/huggingface_models/TechGPT-7B",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "gate_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM"
+}

ncr_cclue/with_knowledge_without_select/final/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38ac689c1c2aeef1b8cd33782432326ff4aa37250df1a55b39d84db39c7ee20d
+size 1279424269

ncr_cclue/with_knowledge_without_select/final/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae27fd3611baf5c6c9ae69d69ae6f5b999fa1f67067c1270becc950da8064ecb
+size 4155

ncr_cclue/without_knowledge_chunk/all_results.json

@@ -0,0 +1,7 @@
+{
+    "epoch": 3.0,
+    "train_loss": 0.5164486801520791,
+    "train_runtime": 92863.651,
+    "train_samples_per_second": 0.61,
+    "train_steps_per_second": 0.051
+}

ncr_cclue/without_knowledge_chunk/final/README.md

@@ -0,0 +1,220 @@
+---
+library_name: peft
+base_model: /data0/maqi/huggingface_models/TechGPT-7B
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+## Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: bitsandbytes
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: True
+- bnb_4bit_compute_dtype: float16
+
+### Framework versions
+
+
+- PEFT 0.6.2

ncr_cclue/without_knowledge_chunk/final/adapter_config.json

@@ -0,0 +1,28 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data0/maqi/huggingface_models/TechGPT-7B",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "down_proj",
+    "v_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM"
+}

ncr_cclue/without_knowledge_chunk/final/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83d911889de361f50d96c224367c5d62e52376c66515a4aab711903143d5af6c
+size 1279424269

ncr_cclue/without_knowledge_chunk/final/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc6d8a464fc4fdef13ae69b3da2c9549544cd6abe2a8028c06902bd45f11e303
+size 4091

ncr_cclue/without_knowledge_chunk/train_results.json

@@ -0,0 +1,7 @@
+{
+    "epoch": 3.0,
+    "train_loss": 0.5164486801520791,
+    "train_runtime": 92863.651,
+    "train_samples_per_second": 0.61,
+    "train_steps_per_second": 0.051
+}

ncr_cclue/without_knowledge_chunk/trainer_state.json

@@ -0,0 +1,1680 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.999047316608447,
+  "eval_steps": 500,
+  "global_step": 4722,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.01,
+      "global_step": 20,
+      "learning_rate": 3.805496828752643e-06,
+      "loss": 5.993,
+      "step": 20
+    },
+    {
+      "epoch": 0.03,
+      "global_step": 40,
+      "learning_rate": 8.033826638477801e-06,
+      "loss": 5.4572,
+      "step": 40
+    },
+    {
+      "epoch": 0.04,
+      "global_step": 60,
+      "learning_rate": 1.226215644820296e-05,
+      "loss": 4.7881,
+      "step": 60
+    },
+    {
+      "epoch": 0.05,
+      "global_step": 80,
+      "learning_rate": 1.649048625792812e-05,
+      "loss": 1.5769,
+      "step": 80
+    },
+    {
+      "epoch": 0.06,
+      "global_step": 100,
+      "learning_rate": 2.0718816067653276e-05,
+      "loss": 0.7578,
+      "step": 100
+    },
+    {
+      "epoch": 0.08,
+      "global_step": 120,
+      "learning_rate": 2.4947145877378435e-05,
+      "loss": 0.7158,
+      "step": 120
+    },
+    {
+      "epoch": 0.09,
+      "global_step": 140,
+      "learning_rate": 2.9175475687103593e-05,
+      "loss": 0.7101,
+      "step": 140
+    },
+    {
+      "epoch": 0.1,
+      "global_step": 160,
+      "learning_rate": 3.3403805496828755e-05,
+      "loss": 0.7411,
+      "step": 160
+    },
+    {
+      "epoch": 0.11,
+      "global_step": 180,
+      "learning_rate": 3.763213530655391e-05,
+      "loss": 0.709,
+      "step": 180
+    },
+    {
+      "epoch": 0.13,
+      "global_step": 200,
+      "learning_rate": 4.186046511627907e-05,
+      "loss": 0.7433,
+      "step": 200
+    },
+    {
+      "epoch": 0.14,
+      "global_step": 220,
+      "learning_rate": 4.608879492600423e-05,
+      "loss": 0.7123,
+      "step": 220
+    },
+    {
+      "epoch": 0.15,
+      "global_step": 240,
+      "learning_rate": 5.0317124735729395e-05,
+      "loss": 0.6957,
+      "step": 240
+    },
+    {
+      "epoch": 0.17,
+      "global_step": 260,
+      "learning_rate": 5.4545454545454546e-05,
+      "loss": 0.7153,
+      "step": 260
+    },
+    {
+      "epoch": 0.18,
+      "global_step": 280,
+      "learning_rate": 5.8773784355179705e-05,
+      "loss": 0.67,
+      "step": 280
+    },
+    {
+      "epoch": 0.19,
+      "global_step": 300,
+      "learning_rate": 6.300211416490487e-05,
+      "loss": 0.7132,
+      "step": 300
+    },
+    {
+      "epoch": 0.2,
+      "global_step": 320,
+      "learning_rate": 6.723044397463001e-05,
+      "loss": 0.6879,
+      "step": 320
+    },
+    {
+      "epoch": 0.22,
+      "global_step": 340,
+      "learning_rate": 7.145877378435519e-05,
+      "loss": 0.6621,
+      "step": 340
+    },
+    {
+      "epoch": 0.23,
+      "global_step": 360,
+      "learning_rate": 7.568710359408034e-05,
+      "loss": 0.7223,
+      "step": 360
+    },
+    {
+      "epoch": 0.24,
+      "global_step": 380,
+      "learning_rate": 7.991543340380549e-05,
+      "loss": 0.6776,
+      "step": 380
+    },
+    {
+      "epoch": 0.25,
+      "global_step": 400,
+      "learning_rate": 8.414376321353066e-05,
+      "loss": 0.6865,
+      "step": 400
+    },
+    {
+      "epoch": 0.27,
+      "global_step": 420,
+      "learning_rate": 8.837209302325582e-05,
+      "loss": 0.687,
+      "step": 420
+    },
+    {
+      "epoch": 0.28,
+      "global_step": 440,
+      "learning_rate": 9.260042283298098e-05,
+      "loss": 0.6945,
+      "step": 440
+    },
+    {
+      "epoch": 0.29,
+      "global_step": 460,
+      "learning_rate": 9.682875264270614e-05,
+      "loss": 0.6864,
+      "step": 460
+    },
+    {
+      "epoch": 0.3,
+      "global_step": 480,
+      "learning_rate": 0.0001,
+      "loss": 0.7043,
+      "step": 480
+    },
+    {
+      "epoch": 0.32,
+      "global_step": 500,
+      "learning_rate": 0.0001,
+      "loss": 0.6746,
+      "step": 500
+    },
+    {
+      "epoch": 0.33,
+      "global_step": 520,
+      "learning_rate": 0.0001,
+      "loss": 0.7011,
+      "step": 520
+    },
+    {
+      "epoch": 0.34,
+      "global_step": 540,
+      "learning_rate": 0.0001,
+      "loss": 0.6918,
+      "step": 540
+    },
+    {
+      "epoch": 0.36,
+      "global_step": 560,
+      "learning_rate": 0.0001,
+      "loss": 0.691,
+      "step": 560
+    },
+    {
+      "epoch": 0.37,
+      "global_step": 580,
+      "learning_rate": 0.0001,
+      "loss": 0.7092,
+      "step": 580
+    },
+    {
+      "epoch": 0.38,
+      "global_step": 600,
+      "learning_rate": 0.0001,
+      "loss": 0.6908,
+      "step": 600
+    },
+    {
+      "epoch": 0.39,
+      "global_step": 620,
+      "learning_rate": 0.0001,
+      "loss": 0.7185,
+      "step": 620
+    },
+    {
+      "epoch": 0.41,
+      "global_step": 640,
+      "learning_rate": 0.0001,
+      "loss": 0.7019,
+      "step": 640
+    },
+    {
+      "epoch": 0.42,
+      "global_step": 660,
+      "learning_rate": 0.0001,
+      "loss": 0.6868,
+      "step": 660
+    },
+    {
+      "epoch": 0.43,
+      "global_step": 680,
+      "learning_rate": 0.0001,
+      "loss": 0.6615,
+      "step": 680
+    },
+    {
+      "epoch": 0.44,
+      "global_step": 700,
+      "learning_rate": 0.0001,
+      "loss": 0.6659,
+      "step": 700
+    },
+    {
+      "epoch": 0.46,
+      "global_step": 720,
+      "learning_rate": 0.0001,
+      "loss": 0.697,
+      "step": 720
+    },
+    {
+      "epoch": 0.47,
+      "global_step": 740,
+      "learning_rate": 0.0001,
+      "loss": 0.6979,
+      "step": 740
+    },
+    {
+      "epoch": 0.48,
+      "global_step": 760,
+      "learning_rate": 0.0001,
+      "loss": 0.7005,
+      "step": 760
+    },
+    {
+      "epoch": 0.5,
+      "global_step": 780,
+      "learning_rate": 0.0001,
+      "loss": 0.6435,
+      "step": 780
+    },
+    {
+      "epoch": 0.51,
+      "global_step": 800,
+      "learning_rate": 0.0001,
+      "loss": 0.6595,
+      "step": 800
+    },
+    {
+      "epoch": 0.52,
+      "global_step": 820,
+      "learning_rate": 0.0001,
+      "loss": 0.6852,
+      "step": 820
+    },
+    {
+      "epoch": 0.53,
+      "global_step": 840,
+      "learning_rate": 0.0001,
+      "loss": 0.6654,
+      "step": 840
+    },
+    {
+      "epoch": 0.55,
+      "global_step": 860,
+      "learning_rate": 0.0001,
+      "loss": 0.6548,
+      "step": 860
+    },
+    {
+      "epoch": 0.56,
+      "global_step": 880,
+      "learning_rate": 0.0001,
+      "loss": 0.6627,
+      "step": 880
+    },
+    {
+      "epoch": 0.57,
+      "global_step": 900,
+      "learning_rate": 0.0001,
+      "loss": 0.6684,
+      "step": 900
+    },
+    {
+      "epoch": 0.58,
+      "global_step": 920,
+      "learning_rate": 0.0001,
+      "loss": 0.6726,
+      "step": 920
+    },
+    {
+      "epoch": 0.6,
+      "global_step": 940,
+      "learning_rate": 0.0001,
+      "loss": 0.6564,
+      "step": 940
+    },
+    {
+      "epoch": 0.61,
+      "global_step": 960,
+      "learning_rate": 0.0001,
+      "loss": 0.6346,
+      "step": 960
+    },
+    {
+      "epoch": 0.62,
+      "global_step": 980,
+      "learning_rate": 0.0001,
+      "loss": 0.6418,
+      "step": 980
+    },
+    {
+      "epoch": 0.64,
+      "global_step": 1000,
+      "learning_rate": 0.0001,
+      "loss": 0.6413,
+      "step": 1000
+    },
+    {
+      "epoch": 0.65,
+      "global_step": 1020,
+      "learning_rate": 0.0001,
+      "loss": 0.6376,
+      "step": 1020
+    },
+    {
+      "epoch": 0.66,
+      "global_step": 1040,
+      "learning_rate": 0.0001,
+      "loss": 0.6478,
+      "step": 1040
+    },
+    {
+      "epoch": 0.67,
+      "global_step": 1060,
+      "learning_rate": 0.0001,
+      "loss": 0.6162,
+      "step": 1060
+    },
+    {
+      "epoch": 0.69,
+      "global_step": 1080,
+      "learning_rate": 0.0001,
+      "loss": 0.6376,
+      "step": 1080
+    },
+    {
+      "epoch": 0.7,
+      "global_step": 1100,
+      "learning_rate": 0.0001,
+      "loss": 0.6305,
+      "step": 1100
+    },
+    {
+      "epoch": 0.71,
+      "global_step": 1120,
+      "learning_rate": 0.0001,
+      "loss": 0.6242,
+      "step": 1120
+    },
+    {
+      "epoch": 0.72,
+      "global_step": 1140,
+      "learning_rate": 0.0001,
+      "loss": 0.6406,
+      "step": 1140
+    },
+    {
+      "epoch": 0.74,
+      "global_step": 1160,
+      "learning_rate": 0.0001,
+      "loss": 0.6121,
+      "step": 1160
+    },
+    {
+      "epoch": 0.75,
+      "global_step": 1180,
+      "learning_rate": 0.0001,
+      "loss": 0.6445,
+      "step": 1180
+    },
+    {
+      "epoch": 0.76,
+      "global_step": 1200,
+      "learning_rate": 0.0001,
+      "loss": 0.6508,
+      "step": 1200
+    },
+    {
+      "epoch": 0.77,
+      "global_step": 1220,
+      "learning_rate": 0.0001,
+      "loss": 0.6296,
+      "step": 1220
+    },
+    {
+      "epoch": 0.79,
+      "global_step": 1240,
+      "learning_rate": 0.0001,
+      "loss": 0.6214,
+      "step": 1240
+    },
+    {
+      "epoch": 0.8,
+      "global_step": 1260,
+      "learning_rate": 0.0001,
+      "loss": 0.6358,
+      "step": 1260
+    },
+    {
+      "epoch": 0.81,
+      "global_step": 1280,
+      "learning_rate": 0.0001,
+      "loss": 0.6312,
+      "step": 1280
+    },
+    {
+      "epoch": 0.83,
+      "global_step": 1300,
+      "learning_rate": 0.0001,
+      "loss": 0.5875,
+      "step": 1300
+    },
+    {
+      "epoch": 0.84,
+      "global_step": 1320,
+      "learning_rate": 0.0001,
+      "loss": 0.9667,
+      "step": 1320
+    },
+    {
+      "epoch": 0.85,
+      "global_step": 1340,
+      "learning_rate": 0.0001,
+      "loss": 0.6909,
+      "step": 1340
+    },
+    {
+      "epoch": 0.86,
+      "global_step": 1360,
+      "learning_rate": 0.0001,
+      "loss": 0.6983,
+      "step": 1360
+    },
+    {
+      "epoch": 0.88,
+      "global_step": 1380,
+      "learning_rate": 0.0001,
+      "loss": 0.612,
+      "step": 1380
+    },
+    {
+      "epoch": 0.89,
+      "global_step": 1400,
+      "learning_rate": 0.0001,
+      "loss": 0.7436,
+      "step": 1400
+    },
+    {
+      "epoch": 0.9,
+      "global_step": 1420,
+      "learning_rate": 0.0001,
+      "loss": 0.5849,
+      "step": 1420
+    },
+    {
+      "epoch": 0.91,
+      "global_step": 1440,
+      "learning_rate": 0.0001,
+      "loss": 0.5918,
+      "step": 1440
+    },
+    {
+      "epoch": 0.93,
+      "global_step": 1460,
+      "learning_rate": 0.0001,
+      "loss": 0.6022,
+      "step": 1460
+    },
+    {
+      "epoch": 0.94,
+      "global_step": 1480,
+      "learning_rate": 0.0001,
+      "loss": 0.6457,
+      "step": 1480
+    },
+    {
+      "epoch": 0.95,
+      "global_step": 1500,
+      "learning_rate": 0.0001,
+      "loss": 0.6052,
+      "step": 1500
+    },
+    {
+      "epoch": 0.97,
+      "global_step": 1520,
+      "learning_rate": 0.0001,
+      "loss": 0.6231,
+      "step": 1520
+    },
+    {
+      "epoch": 0.98,
+      "global_step": 1540,
+      "learning_rate": 0.0001,
+      "loss": 0.5982,
+      "step": 1540
+    },
+    {
+      "epoch": 0.99,
+      "global_step": 1560,
+      "learning_rate": 0.0001,
+      "loss": 0.5454,
+      "step": 1560
+    },
+    {
+      "epoch": 1.0,
+      "global_step": 1580,
+      "learning_rate": 0.0001,
+      "loss": 0.581,
+      "step": 1580
+    },
+    {
+      "epoch": 1.02,
+      "global_step": 1600,
+      "learning_rate": 0.0001,
+      "loss": 0.5395,
+      "step": 1600
+    },
+    {
+      "epoch": 1.03,
+      "global_step": 1620,
+      "learning_rate": 0.0001,
+      "loss": 0.5489,
+      "step": 1620
+    },
+    {
+      "epoch": 1.04,
+      "global_step": 1640,
+      "learning_rate": 0.0001,
+      "loss": 0.5468,
+      "step": 1640
+    },
+    {
+      "epoch": 1.05,
+      "global_step": 1660,
+      "learning_rate": 0.0001,
+      "loss": 0.5423,
+      "step": 1660
+    },
+    {
+      "epoch": 1.07,
+      "global_step": 1680,
+      "learning_rate": 0.0001,
+      "loss": 0.5167,
+      "step": 1680
+    },
+    {
+      "epoch": 1.08,
+      "global_step": 1700,
+      "learning_rate": 0.0001,
+      "loss": 0.492,
+      "step": 1700
+    },
+    {
+      "epoch": 1.09,
+      "global_step": 1720,
+      "learning_rate": 0.0001,
+      "loss": 0.5167,
+      "step": 1720
+    },
+    {
+      "epoch": 1.11,
+      "global_step": 1740,
+      "learning_rate": 0.0001,
+      "loss": 0.4856,
+      "step": 1740
+    },
+    {
+      "epoch": 1.12,
+      "global_step": 1760,
+      "learning_rate": 0.0001,
+      "loss": 0.5123,
+      "step": 1760
+    },
+    {
+      "epoch": 1.13,
+      "global_step": 1780,
+      "learning_rate": 0.0001,
+      "loss": 0.5565,
+      "step": 1780
+    },
+    {
+      "epoch": 1.14,
+      "global_step": 1800,
+      "learning_rate": 0.0001,
+      "loss": 0.5063,
+      "step": 1800
+    },
+    {
+      "epoch": 1.16,
+      "global_step": 1820,
+      "learning_rate": 0.0001,
+      "loss": 0.4997,
+      "step": 1820
+    },
+    {
+      "epoch": 1.17,
+      "global_step": 1840,
+      "learning_rate": 0.0001,
+      "loss": 0.5393,
+      "step": 1840
+    },
+    {
+      "epoch": 1.18,
+      "global_step": 1860,
+      "learning_rate": 0.0001,
+      "loss": 0.4765,
+      "step": 1860
+    },
+    {
+      "epoch": 1.19,
+      "global_step": 1880,
+      "learning_rate": 0.0001,
+      "loss": 0.5269,
+      "step": 1880
+    },
+    {
+      "epoch": 1.21,
+      "global_step": 1900,
+      "learning_rate": 0.0001,
+      "loss": 0.4741,
+      "step": 1900
+    },
+    {
+      "epoch": 1.22,
+      "global_step": 1920,
+      "learning_rate": 0.0001,
+      "loss": 0.4856,
+      "step": 1920
+    },
+    {
+      "epoch": 1.23,
+      "global_step": 1940,
+      "learning_rate": 0.0001,
+      "loss": 0.5203,
+      "step": 1940
+    },
+    {
+      "epoch": 1.24,
+      "global_step": 1960,
+      "learning_rate": 0.0001,
+      "loss": 0.5241,
+      "step": 1960
+    },
+    {
+      "epoch": 1.26,
+      "global_step": 1980,
+      "learning_rate": 0.0001,
+      "loss": 0.5349,
+      "step": 1980
+    },
+    {
+      "epoch": 1.27,
+      "global_step": 2000,
+      "learning_rate": 0.0001,
+      "loss": 0.4801,
+      "step": 2000
+    },
+    {
+      "epoch": 1.28,
+      "global_step": 2020,
+      "learning_rate": 0.0001,
+      "loss": 0.4689,
+      "step": 2020
+    },
+    {
+      "epoch": 1.3,
+      "global_step": 2040,
+      "learning_rate": 0.0001,
+      "loss": 0.48,
+      "step": 2040
+    },
+    {
+      "epoch": 1.31,
+      "global_step": 2060,
+      "learning_rate": 0.0001,
+      "loss": 0.5066,
+      "step": 2060
+    },
+    {
+      "epoch": 1.32,
+      "global_step": 2080,
+      "learning_rate": 0.0001,
+      "loss": 0.4727,
+      "step": 2080
+    },
+    {
+      "epoch": 1.33,
+      "global_step": 2100,
+      "learning_rate": 0.0001,
+      "loss": 0.4415,
+      "step": 2100
+    },
+    {
+      "epoch": 1.35,
+      "global_step": 2120,
+      "learning_rate": 0.0001,
+      "loss": 0.458,
+      "step": 2120
+    },
+    {
+      "epoch": 1.36,
+      "global_step": 2140,
+      "learning_rate": 0.0001,
+      "loss": 0.4982,
+      "step": 2140
+    },
+    {
+      "epoch": 1.37,
+      "global_step": 2160,
+      "learning_rate": 0.0001,
+      "loss": 0.5122,
+      "step": 2160
+    },
+    {
+      "epoch": 1.38,
+      "global_step": 2180,
+      "learning_rate": 0.0001,
+      "loss": 0.4713,
+      "step": 2180
+    },
+    {
+      "epoch": 1.4,
+      "global_step": 2200,
+      "learning_rate": 0.0001,
+      "loss": 0.4892,
+      "step": 2200
+    },
+    {
+      "epoch": 1.41,
+      "global_step": 2220,
+      "learning_rate": 0.0001,
+      "loss": 0.4899,
+      "step": 2220
+    },
+    {
+      "epoch": 1.42,
+      "global_step": 2240,
+      "learning_rate": 0.0001,
+      "loss": 0.4855,
+      "step": 2240
+    },
+    {
+      "epoch": 1.44,
+      "global_step": 2260,
+      "learning_rate": 0.0001,
+      "loss": 0.4571,
+      "step": 2260
+    },
+    {
+      "epoch": 1.45,
+      "global_step": 2280,
+      "learning_rate": 0.0001,
+      "loss": 0.4589,
+      "step": 2280
+    },
+    {
+      "epoch": 1.46,
+      "global_step": 2300,
+      "learning_rate": 0.0001,
+      "loss": 0.4836,
+      "step": 2300
+    },
+    {
+      "epoch": 1.47,
+      "global_step": 2320,
+      "learning_rate": 0.0001,
+      "loss": 0.4435,
+      "step": 2320
+    },
+    {
+      "epoch": 1.49,
+      "global_step": 2340,
+      "learning_rate": 0.0001,
+      "loss": 0.4419,
+      "step": 2340
+    },
+    {
+      "epoch": 1.5,
+      "global_step": 2360,
+      "learning_rate": 0.0001,
+      "loss": 0.4896,
+      "step": 2360
+    },
+    {
+      "epoch": 1.51,
+      "global_step": 2380,
+      "learning_rate": 0.0001,
+      "loss": 0.4791,
+      "step": 2380
+    },
+    {
+      "epoch": 1.52,
+      "global_step": 2400,
+      "learning_rate": 0.0001,
+      "loss": 0.4616,
+      "step": 2400
+    },
+    {
+      "epoch": 1.54,
+      "global_step": 2420,
+      "learning_rate": 0.0001,
+      "loss": 0.4625,
+      "step": 2420
+    },
+    {
+      "epoch": 1.55,
+      "global_step": 2440,
+      "learning_rate": 0.0001,
+      "loss": 0.4727,
+      "step": 2440
+    },
+    {
+      "epoch": 1.56,
+      "global_step": 2460,
+      "learning_rate": 0.0001,
+      "loss": 0.4993,
+      "step": 2460
+    },
+    {
+      "epoch": 1.58,
+      "global_step": 2480,
+      "learning_rate": 0.0001,
+      "loss": 0.4491,
+      "step": 2480
+    },
+    {
+      "epoch": 1.59,
+      "global_step": 2500,
+      "learning_rate": 0.0001,
+      "loss": 0.4888,
+      "step": 2500
+    },
+    {
+      "epoch": 1.6,
+      "global_step": 2520,
+      "learning_rate": 0.0001,
+      "loss": 0.4203,
+      "step": 2520
+    },
+    {
+      "epoch": 1.61,
+      "global_step": 2540,
+      "learning_rate": 0.0001,
+      "loss": 0.4747,
+      "step": 2540
+    },
+    {
+      "epoch": 1.63,
+      "global_step": 2560,
+      "learning_rate": 0.0001,
+      "loss": 0.4609,
+      "step": 2560
+    },
+    {
+      "epoch": 1.64,
+      "global_step": 2580,
+      "learning_rate": 0.0001,
+      "loss": 0.4463,
+      "step": 2580
+    },
+    {
+      "epoch": 1.65,
+      "global_step": 2600,
+      "learning_rate": 0.0001,
+      "loss": 0.4456,
+      "step": 2600
+    },
+    {
+      "epoch": 1.66,
+      "global_step": 2620,
+      "learning_rate": 0.0001,
+      "loss": 0.4743,
+      "step": 2620
+    },
+    {
+      "epoch": 1.68,
+      "global_step": 2640,
+      "learning_rate": 0.0001,
+      "loss": 0.4706,
+      "step": 2640
+    },
+    {
+      "epoch": 1.69,
+      "global_step": 2660,
+      "learning_rate": 0.0001,
+      "loss": 0.485,
+      "step": 2660
+    },
+    {
+      "epoch": 1.7,
+      "global_step": 2680,
+      "learning_rate": 0.0001,
+      "loss": 0.4465,
+      "step": 2680
+    },
+    {
+      "epoch": 1.71,
+      "global_step": 2700,
+      "learning_rate": 0.0001,
+      "loss": 0.4331,
+      "step": 2700
+    },
+    {
+      "epoch": 1.73,
+      "global_step": 2720,
+      "learning_rate": 0.0001,
+      "loss": 0.4463,
+      "step": 2720
+    },
+    {
+      "epoch": 1.74,
+      "global_step": 2740,
+      "learning_rate": 0.0001,
+      "loss": 0.4771,
+      "step": 2740
+    },
+    {
+      "epoch": 1.75,
+      "global_step": 2760,
+      "learning_rate": 0.0001,
+      "loss": 0.4744,
+      "step": 2760
+    },
+    {
+      "epoch": 1.77,
+      "global_step": 2780,
+      "learning_rate": 0.0001,
+      "loss": 0.4438,
+      "step": 2780
+    },
+    {
+      "epoch": 1.78,
+      "global_step": 2800,
+      "learning_rate": 0.0001,
+      "loss": 0.444,
+      "step": 2800
+    },
+    {
+      "epoch": 1.79,
+      "global_step": 2820,
+      "learning_rate": 0.0001,
+      "loss": 0.41,
+      "step": 2820
+    },
+    {
+      "epoch": 1.8,
+      "global_step": 2840,
+      "learning_rate": 0.0001,
+      "loss": 0.4543,
+      "step": 2840
+    },
+    {
+      "epoch": 1.82,
+      "global_step": 2860,
+      "learning_rate": 0.0001,
+      "loss": 0.4458,
+      "step": 2860
+    },
+    {
+      "epoch": 1.83,
+      "global_step": 2880,
+      "learning_rate": 0.0001,
+      "loss": 0.4244,
+      "step": 2880
+    },
+    {
+      "epoch": 1.84,
+      "global_step": 2900,
+      "learning_rate": 0.0001,
+      "loss": 0.4155,
+      "step": 2900
+    },
+    {
+      "epoch": 1.85,
+      "global_step": 2920,
+      "learning_rate": 0.0001,
+      "loss": 0.4295,
+      "step": 2920
+    },
+    {
+      "epoch": 1.87,
+      "global_step": 2940,
+      "learning_rate": 0.0001,
+      "loss": 0.3695,
+      "step": 2940
+    },
+    {
+      "epoch": 1.88,
+      "global_step": 2960,
+      "learning_rate": 0.0001,
+      "loss": 0.3673,
+      "step": 2960
+    },
+    {
+      "epoch": 1.89,
+      "global_step": 2980,
+      "learning_rate": 0.0001,
+      "loss": 0.3788,
+      "step": 2980
+    },
+    {
+      "epoch": 1.91,
+      "global_step": 3000,
+      "learning_rate": 0.0001,
+      "loss": 0.4567,
+      "step": 3000
+    },
+    {
+      "epoch": 1.92,
+      "global_step": 3020,
+      "learning_rate": 0.0001,
+      "loss": 0.3972,
+      "step": 3020
+    },
+    {
+      "epoch": 1.93,
+      "global_step": 3040,
+      "learning_rate": 0.0001,
+      "loss": 0.3837,
+      "step": 3040
+    },
+    {
+      "epoch": 1.94,
+      "global_step": 3060,
+      "learning_rate": 0.0001,
+      "loss": 0.4288,
+      "step": 3060
+    },
+    {
+      "epoch": 1.96,
+      "global_step": 3080,
+      "learning_rate": 0.0001,
+      "loss": 0.3926,
+      "step": 3080
+    },
+    {
+      "epoch": 1.97,
+      "global_step": 3100,
+      "learning_rate": 0.0001,
+      "loss": 0.3972,
+      "step": 3100
+    },
+    {
+      "epoch": 1.98,
+      "global_step": 3120,
+      "learning_rate": 0.0001,
+      "loss": 0.3765,
+      "step": 3120
+    },
+    {
+      "epoch": 1.99,
+      "global_step": 3140,
+      "learning_rate": 0.0001,
+      "loss": 0.3855,
+      "step": 3140
+    },
+    {
+      "epoch": 2.01,
+      "global_step": 3160,
+      "learning_rate": 0.0001,
+      "loss": 0.3017,
+      "step": 3160
+    },
+    {
+      "epoch": 2.02,
+      "global_step": 3180,
+      "learning_rate": 0.0001,
+      "loss": 0.1814,
+      "step": 3180
+    },
+    {
+      "epoch": 2.03,
+      "global_step": 3200,
+      "learning_rate": 0.0001,
+      "loss": 0.2362,
+      "step": 3200
+    },
+    {
+      "epoch": 2.05,
+      "global_step": 3220,
+      "learning_rate": 0.0001,
+      "loss": 0.1673,
+      "step": 3220
+    },
+    {
+      "epoch": 2.06,
+      "global_step": 3240,
+      "learning_rate": 0.0001,
+      "loss": 0.2179,
+      "step": 3240
+    },
+    {
+      "epoch": 2.07,
+      "global_step": 3260,
+      "learning_rate": 0.0001,
+      "loss": 0.1813,
+      "step": 3260
+    },
+    {
+      "epoch": 2.08,
+      "global_step": 3280,
+      "learning_rate": 0.0001,
+      "loss": 0.228,
+      "step": 3280
+    },
+    {
+      "epoch": 2.1,
+      "global_step": 3300,
+      "learning_rate": 0.0001,
+      "loss": 0.2059,
+      "step": 3300
+    },
+    {
+      "epoch": 2.11,
+      "global_step": 3320,
+      "learning_rate": 0.0001,
+      "loss": 0.2615,
+      "step": 3320
+    },
+    {
+      "epoch": 2.12,
+      "global_step": 3340,
+      "learning_rate": 0.0001,
+      "loss": 0.2087,
+      "step": 3340
+    },
+    {
+      "epoch": 2.13,
+      "global_step": 3360,
+      "learning_rate": 0.0001,
+      "loss": 0.1891,
+      "step": 3360
+    },
+    {
+      "epoch": 2.15,
+      "global_step": 3380,
+      "learning_rate": 0.0001,
+      "loss": 0.2607,
+      "step": 3380
+    },
+    {
+      "epoch": 2.16,
+      "global_step": 3400,
+      "learning_rate": 0.0001,
+      "loss": 0.1799,
+      "step": 3400
+    },
+    {
+      "epoch": 2.17,
+      "global_step": 3420,
+      "learning_rate": 0.0001,
+      "loss": 0.1713,
+      "step": 3420
+    },
+    {
+      "epoch": 2.18,
+      "global_step": 3440,
+      "learning_rate": 0.0001,
+      "loss": 0.2722,
+      "step": 3440
+    },
+    {
+      "epoch": 2.2,
+      "global_step": 3460,
+      "learning_rate": 0.0001,
+      "loss": 0.2159,
+      "step": 3460
+    },
+    {
+      "epoch": 2.21,
+      "global_step": 3480,
+      "learning_rate": 0.0001,
+      "loss": 0.2537,
+      "step": 3480
+    },
+    {
+      "epoch": 2.22,
+      "global_step": 3500,
+      "learning_rate": 0.0001,
+      "loss": 0.252,
+      "step": 3500
+    },
+    {
+      "epoch": 2.24,
+      "global_step": 3520,
+      "learning_rate": 0.0001,
+      "loss": 0.253,
+      "step": 3520
+    },
+    {
+      "epoch": 2.25,
+      "global_step": 3540,
+      "learning_rate": 0.0001,
+      "loss": 0.2197,
+      "step": 3540
+    },
+    {
+      "epoch": 2.26,
+      "global_step": 3560,
+      "learning_rate": 0.0001,
+      "loss": 0.2381,
+      "step": 3560
+    },
+    {
+      "epoch": 2.27,
+      "global_step": 3580,
+      "learning_rate": 0.0001,
+      "loss": 0.2066,
+      "step": 3580
+    },
+    {
+      "epoch": 2.29,
+      "global_step": 3600,
+      "learning_rate": 0.0001,
+      "loss": 0.2329,
+      "step": 3600
+    },
+    {
+      "epoch": 2.3,
+      "global_step": 3620,
+      "learning_rate": 0.0001,
+      "loss": 0.2196,
+      "step": 3620
+    },
+    {
+      "epoch": 2.31,
+      "global_step": 3640,
+      "learning_rate": 0.0001,
+      "loss": 0.1825,
+      "step": 3640
+    },
+    {
+      "epoch": 2.32,
+      "global_step": 3660,
+      "learning_rate": 0.0001,
+      "loss": 0.2117,
+      "step": 3660
+    },
+    {
+      "epoch": 2.34,
+      "global_step": 3680,
+      "learning_rate": 0.0001,
+      "loss": 0.2319,
+      "step": 3680
+    },
+    {
+      "epoch": 2.35,
+      "global_step": 3700,
+      "learning_rate": 0.0001,
+      "loss": 0.2473,
+      "step": 3700
+    },
+    {
+      "epoch": 2.36,
+      "global_step": 3720,
+      "learning_rate": 0.0001,
+      "loss": 0.2855,
+      "step": 3720
+    },
+    {
+      "epoch": 2.38,
+      "global_step": 3740,
+      "learning_rate": 0.0001,
+      "loss": 0.2518,
+      "step": 3740
+    },
+    {
+      "epoch": 2.39,
+      "global_step": 3760,
+      "learning_rate": 0.0001,
+      "loss": 0.2141,
+      "step": 3760
+    },
+    {
+      "epoch": 2.4,
+      "global_step": 3780,
+      "learning_rate": 0.0001,
+      "loss": 0.2368,
+      "step": 3780
+    },
+    {
+      "epoch": 2.41,
+      "global_step": 3800,
+      "learning_rate": 0.0001,
+      "loss": 0.2136,
+      "step": 3800
+    },
+    {
+      "epoch": 2.43,
+      "global_step": 3820,
+      "learning_rate": 0.0001,
+      "loss": 0.2024,
+      "step": 3820
+    },
+    {
+      "epoch": 2.44,
+      "global_step": 3840,
+      "learning_rate": 0.0001,
+      "loss": 0.2169,
+      "step": 3840
+    },
+    {
+      "epoch": 2.45,
+      "global_step": 3860,
+      "learning_rate": 0.0001,
+      "loss": 0.1803,
+      "step": 3860
+    },
+    {
+      "epoch": 2.46,
+      "global_step": 3880,
+      "learning_rate": 0.0001,
+      "loss": 0.2033,
+      "step": 3880
+    },
+    {
+      "epoch": 2.48,
+      "global_step": 3900,
+      "learning_rate": 0.0001,
+      "loss": 0.2119,
+      "step": 3900
+    },
+    {
+      "epoch": 2.49,
+      "global_step": 3920,
+      "learning_rate": 0.0001,
+      "loss": 0.2318,
+      "step": 3920
+    },
+    {
+      "epoch": 2.5,
+      "global_step": 3940,
+      "learning_rate": 0.0001,
+      "loss": 0.2026,
+      "step": 3940
+    },
+    {
+      "epoch": 2.52,
+      "global_step": 3960,
+      "learning_rate": 0.0001,
+      "loss": 0.2032,
+      "step": 3960
+    },
+    {
+      "epoch": 2.53,
+      "global_step": 3980,
+      "learning_rate": 0.0001,
+      "loss": 0.231,
+      "step": 3980
+    },
+    {
+      "epoch": 2.54,
+      "global_step": 4000,
+      "learning_rate": 0.0001,
+      "loss": 0.1719,
+      "step": 4000
+    },
+    {
+      "epoch": 2.55,
+      "global_step": 4020,
+      "learning_rate": 0.0001,
+      "loss": 0.2496,
+      "step": 4020
+    },
+    {
+      "epoch": 2.57,
+      "global_step": 4040,
+      "learning_rate": 0.0001,
+      "loss": 0.2273,
+      "step": 4040
+    },
+    {
+      "epoch": 2.58,
+      "global_step": 4060,
+      "learning_rate": 0.0001,
+      "loss": 0.2142,
+      "step": 4060
+    },
+    {
+      "epoch": 2.59,
+      "global_step": 4080,
+      "learning_rate": 0.0001,
+      "loss": 0.2144,
+      "step": 4080
+    },
+    {
+      "epoch": 2.6,
+      "global_step": 4100,
+      "learning_rate": 0.0001,
+      "loss": 0.2501,
+      "step": 4100
+    },
+    {
+      "epoch": 2.62,
+      "global_step": 4120,
+      "learning_rate": 0.0001,
+      "loss": 0.2253,
+      "step": 4120
+    },
+    {
+      "epoch": 2.63,
+      "global_step": 4140,
+      "learning_rate": 0.0001,
+      "loss": 0.2109,
+      "step": 4140
+    },
+    {
+      "epoch": 2.64,
+      "global_step": 4160,
+      "learning_rate": 0.0001,
+      "loss": 0.1697,
+      "step": 4160
+    },
+    {
+      "epoch": 2.65,
+      "global_step": 4180,
+      "learning_rate": 0.0001,
+      "loss": 0.2384,
+      "step": 4180
+    },
+    {
+      "epoch": 2.67,
+      "global_step": 4200,
+      "learning_rate": 0.0001,
+      "loss": 0.2827,
+      "step": 4200
+    },
+    {
+      "epoch": 2.68,
+      "global_step": 4220,
+      "learning_rate": 0.0001,
+      "loss": 0.1777,
+      "step": 4220
+    },
+    {
+      "epoch": 2.69,
+      "global_step": 4240,
+      "learning_rate": 0.0001,
+      "loss": 0.2163,
+      "step": 4240
+    },
+    {
+      "epoch": 2.71,
+      "global_step": 4260,
+      "learning_rate": 0.0001,
+      "loss": 0.2495,
+      "step": 4260
+    },
+    {
+      "epoch": 2.72,
+      "global_step": 4280,
+      "learning_rate": 0.0001,
+      "loss": 0.2451,
+      "step": 4280
+    },
+    {
+      "epoch": 2.73,
+      "global_step": 4300,
+      "learning_rate": 0.0001,
+      "loss": 0.2092,
+      "step": 4300
+    },
+    {
+      "epoch": 2.74,
+      "global_step": 4320,
+      "learning_rate": 0.0001,
+      "loss": 0.252,
+      "step": 4320
+    },
+    {
+      "epoch": 2.76,
+      "global_step": 4340,
+      "learning_rate": 0.0001,
+      "loss": 0.224,
+      "step": 4340
+    },
+    {
+      "epoch": 2.77,
+      "global_step": 4360,
+      "learning_rate": 0.0001,
+      "loss": 0.1834,
+      "step": 4360
+    },
+    {
+      "epoch": 2.78,
+      "global_step": 4380,
+      "learning_rate": 0.0001,
+      "loss": 0.2695,
+      "step": 4380
+    },
+    {
+      "epoch": 2.79,
+      "global_step": 4400,
+      "learning_rate": 0.0001,
+      "loss": 0.2274,
+      "step": 4400
+    },
+    {
+      "epoch": 2.81,
+      "global_step": 4420,
+      "learning_rate": 0.0001,
+      "loss": 0.261,
+      "step": 4420
+    },
+    {
+      "epoch": 2.82,
+      "global_step": 4440,
+      "learning_rate": 0.0001,
+      "loss": 0.2441,
+      "step": 4440
+    },
+    {
+      "epoch": 2.83,
+      "global_step": 4460,
+      "learning_rate": 0.0001,
+      "loss": 0.24,
+      "step": 4460
+    },
+    {
+      "epoch": 2.85,
+      "global_step": 4480,
+      "learning_rate": 0.0001,
+      "loss": 0.1677,
+      "step": 4480
+    },
+    {
+      "epoch": 2.86,
+      "global_step": 4500,
+      "learning_rate": 0.0001,
+      "loss": 0.2072,
+      "step": 4500
+    },
+    {
+      "epoch": 2.87,
+      "global_step": 4520,
+      "learning_rate": 0.0001,
+      "loss": 0.1982,
+      "step": 4520
+    },
+    {
+      "epoch": 2.88,
+      "global_step": 4540,
+      "learning_rate": 0.0001,
+      "loss": 0.1978,
+      "step": 4540
+    },
+    {
+      "epoch": 2.9,
+      "global_step": 4560,
+      "learning_rate": 0.0001,
+      "loss": 0.2224,
+      "step": 4560
+    },
+    {
+      "epoch": 2.91,
+      "global_step": 4580,
+      "learning_rate": 0.0001,
+      "loss": 0.2078,
+      "step": 4580
+    },
+    {
+      "epoch": 2.92,
+      "global_step": 4600,
+      "learning_rate": 0.0001,
+      "loss": 0.1945,
+      "step": 4600
+    },
+    {
+      "epoch": 2.93,
+      "global_step": 4620,
+      "learning_rate": 0.0001,
+      "loss": 0.1798,
+      "step": 4620
+    },
+    {
+      "epoch": 2.95,
+      "global_step": 4640,
+      "learning_rate": 0.0001,
+      "loss": 0.2276,
+      "step": 4640
+    },
+    {
+      "epoch": 2.96,
+      "global_step": 4660,
+      "learning_rate": 0.0001,
+      "loss": 0.1654,
+      "step": 4660
+    },
+    {
+      "epoch": 2.97,
+      "global_step": 4680,
+      "learning_rate": 0.0001,
+      "loss": 0.2142,
+      "step": 4680
+    },
+    {
+      "epoch": 2.99,
+      "global_step": 4700,
+      "learning_rate": 0.0001,
+      "loss": 0.272,
+      "step": 4700
+    },
+    {
+      "epoch": 3.0,
+      "global_step": 4720,
+      "learning_rate": 0.0001,
+      "loss": 0.2313,
+      "step": 4720
+    },
+    {
+      "epoch": 3.0,
+      "step": 4722,
+      "total_flos": 1.659745790093951e+18,
+      "train_loss": 0.5164486801520791,
+      "train_runtime": 92863.651,
+      "train_samples_per_second": 0.61,
+      "train_steps_per_second": 0.051
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4722,
+  "num_train_epochs": 3,
+  "save_steps": 2000,
+  "total_flos": 1.659745790093951e+18,
+  "trial_name": null,
+  "trial_params": null
+}

ncr_cclue/without_knowledge_chunk/without_knowledge_chunk.json

@@ -0,0 +1,31 @@
+{
+    "desc": "不做知识选择，使用NCR+CCLUE数据集，截断至2048",
+    "output_dir": "output/ablation_study/option-2/NCR/without_knowledge_chunk",
+    "model_name_or_path": "/data0/maqi/huggingface_models/TechGPT-7B",
+    "train_file": "/data0/maqi/KGLQA-data/datasets/merge/random_select/without_knowledge_chunk_instruction/train.jsonl",
+    "num_train_epochs": 3,
+    "per_device_train_batch_size":6,
+    "gradient_accumulation_steps": 2,
+    "learning_rate": 1e-4,
+    "max_seq_length": 2048,
+    "logging_steps": 20,
+    "save_steps": 2000,
+    "save_total_limit": 1,
+    "lr_scheduler_type": "constant_with_warmup",
+    "warmup_ratio": 0.1,
+    "lora_rank": 128,
+    "lora_alpha": 64,
+    "lora_dropout": 0.05,
+
+    "gradient_checkpointing": true,
+    "disable_tqdm": false,
+    "optim": "paged_adamw_32bit",
+    "seed": 318,
+    "fp16": true,
+    "report_to": "tensorboard",
+    "dataloader_num_workers": 10,
+    "save_strategy": "steps",
+    "weight_decay": 0,
+    "max_grad_norm": 0.3,
+    "remove_unused_columns": false
+}

ncr_cclue/without_knowledge_random/all_results.json

@@ -0,0 +1,7 @@
+{
+    "epoch": 3.0,
+    "train_loss": 0.5372816354023708,
+    "train_runtime": 164669.988,
+    "train_samples_per_second": 0.344,
+    "train_steps_per_second": 0.029
+}

ncr_cclue/without_knowledge_random/final/README.md

@@ -0,0 +1,220 @@
+---
+library_name: peft
+base_model: /data0/maqi/huggingface_models/TechGPT-7B
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+## Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: bitsandbytes
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: True
+- bnb_4bit_compute_dtype: float16
+
+### Framework versions
+
+
+- PEFT 0.6.2

ncr_cclue/without_knowledge_random/final/adapter_config.json

@@ -0,0 +1,28 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data0/maqi/huggingface_models/TechGPT-7B",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "k_proj",
+    "down_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM"
+}

ncr_cclue/without_knowledge_random/final/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a000dc6e08ca8f8b2c5a37ef64fd06cc825495d5fb2cf1a201d22054e9c208eb
+size 1279424269

ncr_cclue/without_knowledge_random/final/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b6c17d7a9449fbdbdf1f468bab59252647db3c2258fc773d57b67e19ed14a63d
+size 4091

ncr_cclue/without_knowledge_random/train_results.json

@@ -0,0 +1,7 @@
+{
+    "epoch": 3.0,
+    "train_loss": 0.5372816354023708,
+    "train_runtime": 164669.988,
+    "train_samples_per_second": 0.344,
+    "train_steps_per_second": 0.029
+}

ncr_cclue/without_knowledge_random/trainer_state.json

@@ -0,0 +1,1680 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.9993648105018,
+  "eval_steps": 500,
+  "global_step": 4722,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.01,
+      "global_step": 20,
+      "learning_rate": 4.016913319238901e-06,
+      "loss": 6.0331,
+      "step": 20
+    },
+    {
+      "epoch": 0.03,
+      "global_step": 40,
+      "learning_rate": 8.24524312896406e-06,
+      "loss": 5.4374,
+      "step": 40
+    },
+    {
+      "epoch": 0.04,
+      "global_step": 60,
+      "learning_rate": 1.2473572938689217e-05,
+      "loss": 4.6579,
+      "step": 60
+    },
+    {
+      "epoch": 0.05,
+      "global_step": 80,
+      "learning_rate": 1.6701902748414377e-05,
+      "loss": 1.4229,
+      "step": 80
+    },
+    {
+      "epoch": 0.06,
+      "global_step": 100,
+      "learning_rate": 2.0718816067653276e-05,
+      "loss": 0.7216,
+      "step": 100
+    },
+    {
+      "epoch": 0.08,
+      "global_step": 120,
+      "learning_rate": 2.473572938689218e-05,
+      "loss": 0.7585,
+      "step": 120
+    },
+    {
+      "epoch": 0.09,
+      "global_step": 140,
+      "learning_rate": 2.8964059196617337e-05,
+      "loss": 0.7468,
+      "step": 140
+    },
+    {
+      "epoch": 0.1,
+      "global_step": 160,
+      "learning_rate": 3.2346723044397465e-05,
+      "loss": 0.8176,
+      "step": 160
+    },
+    {
+      "epoch": 0.11,
+      "global_step": 180,
+      "learning_rate": 3.6575052854122623e-05,
+      "loss": 0.7115,
+      "step": 180
+    },
+    {
+      "epoch": 0.13,
+      "global_step": 200,
+      "learning_rate": 4.080338266384778e-05,
+      "loss": 0.742,
+      "step": 200
+    },
+    {
+      "epoch": 0.14,
+      "global_step": 220,
+      "learning_rate": 4.503171247357294e-05,
+      "loss": 0.7049,
+      "step": 220
+    },
+    {
+      "epoch": 0.15,
+      "global_step": 240,
+      "learning_rate": 4.9260042283298105e-05,
+      "loss": 0.7075,
+      "step": 240
+    },
+    {
+      "epoch": 0.17,
+      "global_step": 260,
+      "learning_rate": 5.348837209302326e-05,
+      "loss": 0.7125,
+      "step": 260
+    },
+    {
+      "epoch": 0.18,
+      "global_step": 280,
+      "learning_rate": 5.7716701902748415e-05,
+      "loss": 0.7052,
+      "step": 280
+    },
+    {
+      "epoch": 0.19,
+      "global_step": 300,
+      "learning_rate": 6.194503171247358e-05,
+      "loss": 0.7236,
+      "step": 300
+    },
+    {
+      "epoch": 0.2,
+      "global_step": 320,
+      "learning_rate": 6.617336152219874e-05,
+      "loss": 0.7166,
+      "step": 320
+    },
+    {
+      "epoch": 0.22,
+      "global_step": 340,
+      "learning_rate": 7.04016913319239e-05,
+      "loss": 0.7356,
+      "step": 340
+    },
+    {
+      "epoch": 0.23,
+      "global_step": 360,
+      "learning_rate": 7.463002114164905e-05,
+      "loss": 0.766,
+      "step": 360
+    },
+    {
+      "epoch": 0.24,
+      "global_step": 380,
+      "learning_rate": 7.885835095137421e-05,
+      "loss": 0.7219,
+      "step": 380
+    },
+    {
+      "epoch": 0.25,
+      "global_step": 400,
+      "learning_rate": 8.308668076109937e-05,
+      "loss": 0.7014,
+      "step": 400
+    },
+    {
+      "epoch": 0.27,
+      "global_step": 420,
+      "learning_rate": 8.731501057082453e-05,
+      "loss": 0.7175,
+      "step": 420
+    },
+    {
+      "epoch": 0.28,
+      "global_step": 440,
+      "learning_rate": 9.154334038054969e-05,
+      "loss": 0.7089,
+      "step": 440
+    },
+    {
+      "epoch": 0.29,
+      "global_step": 460,
+      "learning_rate": 9.577167019027485e-05,
+      "loss": 0.7039,
+      "step": 460
+    },
+    {
+      "epoch": 0.3,
+      "global_step": 480,
+      "learning_rate": 0.0001,
+      "loss": 0.702,
+      "step": 480
+    },
+    {
+      "epoch": 0.32,
+      "global_step": 500,
+      "learning_rate": 0.0001,
+      "loss": 0.7014,
+      "step": 500
+    },
+    {
+      "epoch": 0.33,
+      "global_step": 520,
+      "learning_rate": 0.0001,
+      "loss": 0.6886,
+      "step": 520
+    },
+    {
+      "epoch": 0.34,
+      "global_step": 540,
+      "learning_rate": 0.0001,
+      "loss": 0.7174,
+      "step": 540
+    },
+    {
+      "epoch": 0.36,
+      "global_step": 560,
+      "learning_rate": 0.0001,
+      "loss": 0.7239,
+      "step": 560
+    },
+    {
+      "epoch": 0.37,
+      "global_step": 580,
+      "learning_rate": 0.0001,
+      "loss": 0.6954,
+      "step": 580
+    },
+    {
+      "epoch": 0.38,
+      "global_step": 600,
+      "learning_rate": 0.0001,
+      "loss": 0.6941,
+      "step": 600
+    },
+    {
+      "epoch": 0.39,
+      "global_step": 620,
+      "learning_rate": 0.0001,
+      "loss": 0.737,
+      "step": 620
+    },
+    {
+      "epoch": 0.41,
+      "global_step": 640,
+      "learning_rate": 0.0001,
+      "loss": 0.7092,
+      "step": 640
+    },
+    {
+      "epoch": 0.42,
+      "global_step": 660,
+      "learning_rate": 0.0001,
+      "loss": 0.6924,
+      "step": 660
+    },
+    {
+      "epoch": 0.43,
+      "global_step": 680,
+      "learning_rate": 0.0001,
+      "loss": 0.685,
+      "step": 680
+    },
+    {
+      "epoch": 0.44,
+      "global_step": 700,
+      "learning_rate": 0.0001,
+      "loss": 0.6935,
+      "step": 700
+    },
+    {
+      "epoch": 0.46,
+      "global_step": 720,
+      "learning_rate": 0.0001,
+      "loss": 0.7088,
+      "step": 720
+    },
+    {
+      "epoch": 0.47,
+      "global_step": 740,
+      "learning_rate": 0.0001,
+      "loss": 0.6861,
+      "step": 740
+    },
+    {
+      "epoch": 0.48,
+      "global_step": 760,
+      "learning_rate": 0.0001,
+      "loss": 0.691,
+      "step": 760
+    },
+    {
+      "epoch": 0.5,
+      "global_step": 780,
+      "learning_rate": 0.0001,
+      "loss": 0.6685,
+      "step": 780
+    },
+    {
+      "epoch": 0.51,
+      "global_step": 800,
+      "learning_rate": 0.0001,
+      "loss": 0.6819,
+      "step": 800
+    },
+    {
+      "epoch": 0.52,
+      "global_step": 820,
+      "learning_rate": 0.0001,
+      "loss": 0.6847,
+      "step": 820
+    },
+    {
+      "epoch": 0.53,
+      "global_step": 840,
+      "learning_rate": 0.0001,
+      "loss": 0.6785,
+      "step": 840
+    },
+    {
+      "epoch": 0.55,
+      "global_step": 860,
+      "learning_rate": 0.0001,
+      "loss": 0.6951,
+      "step": 860
+    },
+    {
+      "epoch": 0.56,
+      "global_step": 880,
+      "learning_rate": 0.0001,
+      "loss": 0.6787,
+      "step": 880
+    },
+    {
+      "epoch": 0.57,
+      "global_step": 900,
+      "learning_rate": 0.0001,
+      "loss": 0.6704,
+      "step": 900
+    },
+    {
+      "epoch": 0.58,
+      "global_step": 920,
+      "learning_rate": 0.0001,
+      "loss": 0.6836,
+      "step": 920
+    },
+    {
+      "epoch": 0.6,
+      "global_step": 940,
+      "learning_rate": 0.0001,
+      "loss": 0.6727,
+      "step": 940
+    },
+    {
+      "epoch": 0.61,
+      "global_step": 960,
+      "learning_rate": 0.0001,
+      "loss": 0.6839,
+      "step": 960
+    },
+    {
+      "epoch": 0.62,
+      "global_step": 980,
+      "learning_rate": 0.0001,
+      "loss": 0.6351,
+      "step": 980
+    },
+    {
+      "epoch": 0.64,
+      "global_step": 1000,
+      "learning_rate": 0.0001,
+      "loss": 0.6772,
+      "step": 1000
+    },
+    {
+      "epoch": 0.65,
+      "global_step": 1020,
+      "learning_rate": 0.0001,
+      "loss": 0.6626,
+      "step": 1020
+    },
+    {
+      "epoch": 0.66,
+      "global_step": 1040,
+      "learning_rate": 0.0001,
+      "loss": 0.6675,
+      "step": 1040
+    },
+    {
+      "epoch": 0.67,
+      "global_step": 1060,
+      "learning_rate": 0.0001,
+      "loss": 0.6329,
+      "step": 1060
+    },
+    {
+      "epoch": 0.69,
+      "global_step": 1080,
+      "learning_rate": 0.0001,
+      "loss": 0.6878,
+      "step": 1080
+    },
+    {
+      "epoch": 0.7,
+      "global_step": 1100,
+      "learning_rate": 0.0001,
+      "loss": 0.6659,
+      "step": 1100
+    },
+    {
+      "epoch": 0.71,
+      "global_step": 1120,
+      "learning_rate": 0.0001,
+      "loss": 0.6665,
+      "step": 1120
+    },
+    {
+      "epoch": 0.72,
+      "global_step": 1140,
+      "learning_rate": 0.0001,
+      "loss": 0.6391,
+      "step": 1140
+    },
+    {
+      "epoch": 0.74,
+      "global_step": 1160,
+      "learning_rate": 0.0001,
+      "loss": 0.64,
+      "step": 1160
+    },
+    {
+      "epoch": 0.75,
+      "global_step": 1180,
+      "learning_rate": 0.0001,
+      "loss": 0.6322,
+      "step": 1180
+    },
+    {
+      "epoch": 0.76,
+      "global_step": 1200,
+      "learning_rate": 0.0001,
+      "loss": 0.671,
+      "step": 1200
+    },
+    {
+      "epoch": 0.77,
+      "global_step": 1220,
+      "learning_rate": 0.0001,
+      "loss": 0.6406,
+      "step": 1220
+    },
+    {
+      "epoch": 0.79,
+      "global_step": 1240,
+      "learning_rate": 0.0001,
+      "loss": 0.6248,
+      "step": 1240
+    },
+    {
+      "epoch": 0.8,
+      "global_step": 1260,
+      "learning_rate": 0.0001,
+      "loss": 0.6691,
+      "step": 1260
+    },
+    {
+      "epoch": 0.81,
+      "global_step": 1280,
+      "learning_rate": 0.0001,
+      "loss": 0.6671,
+      "step": 1280
+    },
+    {
+      "epoch": 0.83,
+      "global_step": 1300,
+      "learning_rate": 0.0001,
+      "loss": 0.6482,
+      "step": 1300
+    },
+    {
+      "epoch": 0.84,
+      "global_step": 1320,
+      "learning_rate": 0.0001,
+      "loss": 0.615,
+      "step": 1320
+    },
+    {
+      "epoch": 0.85,
+      "global_step": 1340,
+      "learning_rate": 0.0001,
+      "loss": 0.6038,
+      "step": 1340
+    },
+    {
+      "epoch": 0.86,
+      "global_step": 1360,
+      "learning_rate": 0.0001,
+      "loss": 0.6402,
+      "step": 1360
+    },
+    {
+      "epoch": 0.88,
+      "global_step": 1380,
+      "learning_rate": 0.0001,
+      "loss": 0.6416,
+      "step": 1380
+    },
+    {
+      "epoch": 0.89,
+      "global_step": 1400,
+      "learning_rate": 0.0001,
+      "loss": 0.664,
+      "step": 1400
+    },
+    {
+      "epoch": 0.9,
+      "global_step": 1420,
+      "learning_rate": 0.0001,
+      "loss": 0.6518,
+      "step": 1420
+    },
+    {
+      "epoch": 0.91,
+      "global_step": 1440,
+      "learning_rate": 0.0001,
+      "loss": 0.6877,
+      "step": 1440
+    },
+    {
+      "epoch": 0.93,
+      "global_step": 1460,
+      "learning_rate": 0.0001,
+      "loss": 0.6685,
+      "step": 1460
+    },
+    {
+      "epoch": 0.94,
+      "global_step": 1480,
+      "learning_rate": 0.0001,
+      "loss": 0.6522,
+      "step": 1480
+    },
+    {
+      "epoch": 0.95,
+      "global_step": 1500,
+      "learning_rate": 0.0001,
+      "loss": 0.6292,
+      "step": 1500
+    },
+    {
+      "epoch": 0.97,
+      "global_step": 1520,
+      "learning_rate": 0.0001,
+      "loss": 0.6319,
+      "step": 1520
+    },
+    {
+      "epoch": 0.98,
+      "global_step": 1540,
+      "learning_rate": 0.0001,
+      "loss": 0.6283,
+      "step": 1540
+    },
+    {
+      "epoch": 0.99,
+      "global_step": 1560,
+      "learning_rate": 0.0001,
+      "loss": 0.5989,
+      "step": 1560
+    },
+    {
+      "epoch": 1.0,
+      "global_step": 1580,
+      "learning_rate": 0.0001,
+      "loss": 0.6319,
+      "step": 1580
+    },
+    {
+      "epoch": 1.02,
+      "global_step": 1600,
+      "learning_rate": 0.0001,
+      "loss": 0.6013,
+      "step": 1600
+    },
+    {
+      "epoch": 1.03,
+      "global_step": 1620,
+      "learning_rate": 0.0001,
+      "loss": 0.523,
+      "step": 1620
+    },
+    {
+      "epoch": 1.04,
+      "global_step": 1640,
+      "learning_rate": 0.0001,
+      "loss": 0.5803,
+      "step": 1640
+    },
+    {
+      "epoch": 1.05,
+      "global_step": 1660,
+      "learning_rate": 0.0001,
+      "loss": 0.5543,
+      "step": 1660
+    },
+    {
+      "epoch": 1.07,
+      "global_step": 1680,
+      "learning_rate": 0.0001,
+      "loss": 0.5372,
+      "step": 1680
+    },
+    {
+      "epoch": 1.08,
+      "global_step": 1700,
+      "learning_rate": 0.0001,
+      "loss": 0.5814,
+      "step": 1700
+    },
+    {
+      "epoch": 1.09,
+      "global_step": 1720,
+      "learning_rate": 0.0001,
+      "loss": 0.5101,
+      "step": 1720
+    },
+    {
+      "epoch": 1.11,
+      "global_step": 1740,
+      "learning_rate": 0.0001,
+      "loss": 0.5555,
+      "step": 1740
+    },
+    {
+      "epoch": 1.12,
+      "global_step": 1760,
+      "learning_rate": 0.0001,
+      "loss": 0.5373,
+      "step": 1760
+    },
+    {
+      "epoch": 1.13,
+      "global_step": 1780,
+      "learning_rate": 0.0001,
+      "loss": 0.54,
+      "step": 1780
+    },
+    {
+      "epoch": 1.14,
+      "global_step": 1800,
+      "learning_rate": 0.0001,
+      "loss": 0.5472,
+      "step": 1800
+    },
+    {
+      "epoch": 1.16,
+      "global_step": 1820,
+      "learning_rate": 0.0001,
+      "loss": 0.567,
+      "step": 1820
+    },
+    {
+      "epoch": 1.17,
+      "global_step": 1840,
+      "learning_rate": 0.0001,
+      "loss": 0.5342,
+      "step": 1840
+    },
+    {
+      "epoch": 1.18,
+      "global_step": 1860,
+      "learning_rate": 0.0001,
+      "loss": 0.5612,
+      "step": 1860
+    },
+    {
+      "epoch": 1.19,
+      "global_step": 1880,
+      "learning_rate": 0.0001,
+      "loss": 0.5275,
+      "step": 1880
+    },
+    {
+      "epoch": 1.21,
+      "global_step": 1900,
+      "learning_rate": 0.0001,
+      "loss": 0.5375,
+      "step": 1900
+    },
+    {
+      "epoch": 1.22,
+      "global_step": 1920,
+      "learning_rate": 0.0001,
+      "loss": 0.554,
+      "step": 1920
+    },
+    {
+      "epoch": 1.23,
+      "global_step": 1940,
+      "learning_rate": 0.0001,
+      "loss": 0.5071,
+      "step": 1940
+    },
+    {
+      "epoch": 1.24,
+      "global_step": 1960,
+      "learning_rate": 0.0001,
+      "loss": 0.5844,
+      "step": 1960
+    },
+    {
+      "epoch": 1.26,
+      "global_step": 1980,
+      "learning_rate": 0.0001,
+      "loss": 0.4972,
+      "step": 1980
+    },
+    {
+      "epoch": 1.27,
+      "global_step": 2000,
+      "learning_rate": 0.0001,
+      "loss": 0.514,
+      "step": 2000
+    },
+    {
+      "epoch": 1.28,
+      "global_step": 2020,
+      "learning_rate": 0.0001,
+      "loss": 0.5176,
+      "step": 2020
+    },
+    {
+      "epoch": 1.3,
+      "global_step": 2040,
+      "learning_rate": 0.0001,
+      "loss": 0.5269,
+      "step": 2040
+    },
+    {
+      "epoch": 1.31,
+      "global_step": 2060,
+      "learning_rate": 0.0001,
+      "loss": 0.5243,
+      "step": 2060
+    },
+    {
+      "epoch": 1.32,
+      "global_step": 2080,
+      "learning_rate": 0.0001,
+      "loss": 0.5316,
+      "step": 2080
+    },
+    {
+      "epoch": 1.33,
+      "global_step": 2100,
+      "learning_rate": 0.0001,
+      "loss": 0.4833,
+      "step": 2100
+    },
+    {
+      "epoch": 1.35,
+      "global_step": 2120,
+      "learning_rate": 0.0001,
+      "loss": 0.4978,
+      "step": 2120
+    },
+    {
+      "epoch": 1.36,
+      "global_step": 2140,
+      "learning_rate": 0.0001,
+      "loss": 0.4742,
+      "step": 2140
+    },
+    {
+      "epoch": 1.37,
+      "global_step": 2160,
+      "learning_rate": 0.0001,
+      "loss": 0.5633,
+      "step": 2160
+    },
+    {
+      "epoch": 1.38,
+      "global_step": 2180,
+      "learning_rate": 0.0001,
+      "loss": 0.5113,
+      "step": 2180
+    },
+    {
+      "epoch": 1.4,
+      "global_step": 2200,
+      "learning_rate": 0.0001,
+      "loss": 0.4774,
+      "step": 2200
+    },
+    {
+      "epoch": 1.41,
+      "global_step": 2220,
+      "learning_rate": 0.0001,
+      "loss": 0.4837,
+      "step": 2220
+    },
+    {
+      "epoch": 1.42,
+      "global_step": 2240,
+      "learning_rate": 0.0001,
+      "loss": 0.5512,
+      "step": 2240
+    },
+    {
+      "epoch": 1.44,
+      "global_step": 2260,
+      "learning_rate": 0.0001,
+      "loss": 0.4818,
+      "step": 2260
+    },
+    {
+      "epoch": 1.45,
+      "global_step": 2280,
+      "learning_rate": 0.0001,
+      "loss": 0.5282,
+      "step": 2280
+    },
+    {
+      "epoch": 1.46,
+      "global_step": 2300,
+      "learning_rate": 0.0001,
+      "loss": 0.468,
+      "step": 2300
+    },
+    {
+      "epoch": 1.47,
+      "global_step": 2320,
+      "learning_rate": 0.0001,
+      "loss": 0.5227,
+      "step": 2320
+    },
+    {
+      "epoch": 1.49,
+      "global_step": 2340,
+      "learning_rate": 0.0001,
+      "loss": 0.4959,
+      "step": 2340
+    },
+    {
+      "epoch": 1.5,
+      "global_step": 2360,
+      "learning_rate": 0.0001,
+      "loss": 0.5313,
+      "step": 2360
+    },
+    {
+      "epoch": 1.51,
+      "global_step": 2380,
+      "learning_rate": 0.0001,
+      "loss": 0.4999,
+      "step": 2380
+    },
+    {
+      "epoch": 1.52,
+      "global_step": 2400,
+      "learning_rate": 0.0001,
+      "loss": 0.5336,
+      "step": 2400
+    },
+    {
+      "epoch": 1.54,
+      "global_step": 2420,
+      "learning_rate": 0.0001,
+      "loss": 0.4878,
+      "step": 2420
+    },
+    {
+      "epoch": 1.55,
+      "global_step": 2440,
+      "learning_rate": 0.0001,
+      "loss": 0.4692,
+      "step": 2440
+    },
+    {
+      "epoch": 1.56,
+      "global_step": 2460,
+      "learning_rate": 0.0001,
+      "loss": 0.4919,
+      "step": 2460
+    },
+    {
+      "epoch": 1.58,
+      "global_step": 2480,
+      "learning_rate": 0.0001,
+      "loss": 0.5449,
+      "step": 2480
+    },
+    {
+      "epoch": 1.59,
+      "global_step": 2500,
+      "learning_rate": 0.0001,
+      "loss": 0.4466,
+      "step": 2500
+    },
+    {
+      "epoch": 1.6,
+      "global_step": 2520,
+      "learning_rate": 0.0001,
+      "loss": 0.4335,
+      "step": 2520
+    },
+    {
+      "epoch": 1.61,
+      "global_step": 2540,
+      "learning_rate": 0.0001,
+      "loss": 0.4979,
+      "step": 2540
+    },
+    {
+      "epoch": 1.63,
+      "global_step": 2560,
+      "learning_rate": 0.0001,
+      "loss": 0.4886,
+      "step": 2560
+    },
+    {
+      "epoch": 1.64,
+      "global_step": 2580,
+      "learning_rate": 0.0001,
+      "loss": 0.4613,
+      "step": 2580
+    },
+    {
+      "epoch": 1.65,
+      "global_step": 2600,
+      "learning_rate": 0.0001,
+      "loss": 0.4633,
+      "step": 2600
+    },
+    {
+      "epoch": 1.66,
+      "global_step": 2620,
+      "learning_rate": 0.0001,
+      "loss": 0.4782,
+      "step": 2620
+    },
+    {
+      "epoch": 1.68,
+      "global_step": 2640,
+      "learning_rate": 0.0001,
+      "loss": 0.4189,
+      "step": 2640
+    },
+    {
+      "epoch": 1.69,
+      "global_step": 2660,
+      "learning_rate": 0.0001,
+      "loss": 0.4752,
+      "step": 2660
+    },
+    {
+      "epoch": 1.7,
+      "global_step": 2680,
+      "learning_rate": 0.0001,
+      "loss": 0.4688,
+      "step": 2680
+    },
+    {
+      "epoch": 1.72,
+      "global_step": 2700,
+      "learning_rate": 0.0001,
+      "loss": 0.5238,
+      "step": 2700
+    },
+    {
+      "epoch": 1.73,
+      "global_step": 2720,
+      "learning_rate": 0.0001,
+      "loss": 0.5027,
+      "step": 2720
+    },
+    {
+      "epoch": 1.74,
+      "global_step": 2740,
+      "learning_rate": 0.0001,
+      "loss": 0.4318,
+      "step": 2740
+    },
+    {
+      "epoch": 1.75,
+      "global_step": 2760,
+      "learning_rate": 0.0001,
+      "loss": 0.4194,
+      "step": 2760
+    },
+    {
+      "epoch": 1.77,
+      "global_step": 2780,
+      "learning_rate": 0.0001,
+      "loss": 0.4614,
+      "step": 2780
+    },
+    {
+      "epoch": 1.78,
+      "global_step": 2800,
+      "learning_rate": 0.0001,
+      "loss": 0.4679,
+      "step": 2800
+    },
+    {
+      "epoch": 1.79,
+      "global_step": 2820,
+      "learning_rate": 0.0001,
+      "loss": 0.4475,
+      "step": 2820
+    },
+    {
+      "epoch": 1.8,
+      "global_step": 2840,
+      "learning_rate": 0.0001,
+      "loss": 0.4803,
+      "step": 2840
+    },
+    {
+      "epoch": 1.82,
+      "global_step": 2860,
+      "learning_rate": 0.0001,
+      "loss": 0.4865,
+      "step": 2860
+    },
+    {
+      "epoch": 1.83,
+      "global_step": 2880,
+      "learning_rate": 0.0001,
+      "loss": 0.4418,
+      "step": 2880
+    },
+    {
+      "epoch": 1.84,
+      "global_step": 2900,
+      "learning_rate": 0.0001,
+      "loss": 0.4205,
+      "step": 2900
+    },
+    {
+      "epoch": 1.85,
+      "global_step": 2920,
+      "learning_rate": 0.0001,
+      "loss": 0.4326,
+      "step": 2920
+    },
+    {
+      "epoch": 1.87,
+      "global_step": 2940,
+      "learning_rate": 0.0001,
+      "loss": 0.4419,
+      "step": 2940
+    },
+    {
+      "epoch": 1.88,
+      "global_step": 2960,
+      "learning_rate": 0.0001,
+      "loss": 0.4743,
+      "step": 2960
+    },
+    {
+      "epoch": 1.89,
+      "global_step": 2980,
+      "learning_rate": 0.0001,
+      "loss": 0.4763,
+      "step": 2980
+    },
+    {
+      "epoch": 1.91,
+      "global_step": 3000,
+      "learning_rate": 0.0001,
+      "loss": 0.4424,
+      "step": 3000
+    },
+    {
+      "epoch": 1.92,
+      "global_step": 3020,
+      "learning_rate": 0.0001,
+      "loss": 0.4982,
+      "step": 3020
+    },
+    {
+      "epoch": 1.93,
+      "global_step": 3040,
+      "learning_rate": 0.0001,
+      "loss": 0.433,
+      "step": 3040
+    },
+    {
+      "epoch": 1.94,
+      "global_step": 3060,
+      "learning_rate": 0.0001,
+      "loss": 0.4642,
+      "step": 3060
+    },
+    {
+      "epoch": 1.96,
+      "global_step": 3080,
+      "learning_rate": 0.0001,
+      "loss": 0.4504,
+      "step": 3080
+    },
+    {
+      "epoch": 1.97,
+      "global_step": 3100,
+      "learning_rate": 0.0001,
+      "loss": 0.4278,
+      "step": 3100
+    },
+    {
+      "epoch": 1.98,
+      "global_step": 3120,
+      "learning_rate": 0.0001,
+      "loss": 0.4197,
+      "step": 3120
+    },
+    {
+      "epoch": 1.99,
+      "global_step": 3140,
+      "learning_rate": 0.0001,
+      "loss": 0.4454,
+      "step": 3140
+    },
+    {
+      "epoch": 2.01,
+      "global_step": 3160,
+      "learning_rate": 0.0001,
+      "loss": 0.3011,
+      "step": 3160
+    },
+    {
+      "epoch": 2.02,
+      "global_step": 3180,
+      "learning_rate": 0.0001,
+      "loss": 0.2277,
+      "step": 3180
+    },
+    {
+      "epoch": 2.03,
+      "global_step": 3200,
+      "learning_rate": 0.0001,
+      "loss": 0.2198,
+      "step": 3200
+    },
+    {
+      "epoch": 2.05,
+      "global_step": 3220,
+      "learning_rate": 0.0001,
+      "loss": 0.2302,
+      "step": 3220
+    },
+    {
+      "epoch": 2.06,
+      "global_step": 3240,
+      "learning_rate": 0.0001,
+      "loss": 0.3219,
+      "step": 3240
+    },
+    {
+      "epoch": 2.07,
+      "global_step": 3260,
+      "learning_rate": 0.0001,
+      "loss": 0.2044,
+      "step": 3260
+    },
+    {
+      "epoch": 2.08,
+      "global_step": 3280,
+      "learning_rate": 0.0001,
+      "loss": 0.265,
+      "step": 3280
+    },
+    {
+      "epoch": 2.1,
+      "global_step": 3300,
+      "learning_rate": 0.0001,
+      "loss": 0.2372,
+      "step": 3300
+    },
+    {
+      "epoch": 2.11,
+      "global_step": 3320,
+      "learning_rate": 0.0001,
+      "loss": 0.2402,
+      "step": 3320
+    },
+    {
+      "epoch": 2.12,
+      "global_step": 3340,
+      "learning_rate": 0.0001,
+      "loss": 0.2581,
+      "step": 3340
+    },
+    {
+      "epoch": 2.13,
+      "global_step": 3360,
+      "learning_rate": 0.0001,
+      "loss": 0.2237,
+      "step": 3360
+    },
+    {
+      "epoch": 2.15,
+      "global_step": 3380,
+      "learning_rate": 0.0001,
+      "loss": 0.2123,
+      "step": 3380
+    },
+    {
+      "epoch": 2.16,
+      "global_step": 3400,
+      "learning_rate": 0.0001,
+      "loss": 0.2064,
+      "step": 3400
+    },
+    {
+      "epoch": 2.17,
+      "global_step": 3420,
+      "learning_rate": 0.0001,
+      "loss": 0.234,
+      "step": 3420
+    },
+    {
+      "epoch": 2.19,
+      "global_step": 3440,
+      "learning_rate": 0.0001,
+      "loss": 0.2014,
+      "step": 3440
+    },
+    {
+      "epoch": 2.2,
+      "global_step": 3460,
+      "learning_rate": 0.0001,
+      "loss": 0.2686,
+      "step": 3460
+    },
+    {
+      "epoch": 2.21,
+      "global_step": 3480,
+      "learning_rate": 0.0001,
+      "loss": 0.1797,
+      "step": 3480
+    },
+    {
+      "epoch": 2.22,
+      "global_step": 3500,
+      "learning_rate": 0.0001,
+      "loss": 0.221,
+      "step": 3500
+    },
+    {
+      "epoch": 2.24,
+      "global_step": 3520,
+      "learning_rate": 0.0001,
+      "loss": 0.2378,
+      "step": 3520
+    },
+    {
+      "epoch": 2.25,
+      "global_step": 3540,
+      "learning_rate": 0.0001,
+      "loss": 0.307,
+      "step": 3540
+    },
+    {
+      "epoch": 2.26,
+      "global_step": 3560,
+      "learning_rate": 0.0001,
+      "loss": 0.1915,
+      "step": 3560
+    },
+    {
+      "epoch": 2.27,
+      "global_step": 3580,
+      "learning_rate": 0.0001,
+      "loss": 0.2686,
+      "step": 3580
+    },
+    {
+      "epoch": 2.29,
+      "global_step": 3600,
+      "learning_rate": 0.0001,
+      "loss": 0.2719,
+      "step": 3600
+    },
+    {
+      "epoch": 2.3,
+      "global_step": 3620,
+      "learning_rate": 0.0001,
+      "loss": 0.2416,
+      "step": 3620
+    },
+    {
+      "epoch": 2.31,
+      "global_step": 3640,
+      "learning_rate": 0.0001,
+      "loss": 0.2409,
+      "step": 3640
+    },
+    {
+      "epoch": 2.32,
+      "global_step": 3660,
+      "learning_rate": 0.0001,
+      "loss": 0.2486,
+      "step": 3660
+    },
+    {
+      "epoch": 2.34,
+      "global_step": 3680,
+      "learning_rate": 0.0001,
+      "loss": 0.2249,
+      "step": 3680
+    },
+    {
+      "epoch": 2.35,
+      "global_step": 3700,
+      "learning_rate": 0.0001,
+      "loss": 0.2383,
+      "step": 3700
+    },
+    {
+      "epoch": 2.36,
+      "global_step": 3720,
+      "learning_rate": 0.0001,
+      "loss": 0.2195,
+      "step": 3720
+    },
+    {
+      "epoch": 2.38,
+      "global_step": 3740,
+      "learning_rate": 0.0001,
+      "loss": 0.2611,
+      "step": 3740
+    },
+    {
+      "epoch": 2.39,
+      "global_step": 3760,
+      "learning_rate": 0.0001,
+      "loss": 0.2851,
+      "step": 3760
+    },
+    {
+      "epoch": 2.4,
+      "global_step": 3780,
+      "learning_rate": 0.0001,
+      "loss": 0.1962,
+      "step": 3780
+    },
+    {
+      "epoch": 2.41,
+      "global_step": 3800,
+      "learning_rate": 0.0001,
+      "loss": 0.2459,
+      "step": 3800
+    },
+    {
+      "epoch": 2.43,
+      "global_step": 3820,
+      "learning_rate": 0.0001,
+      "loss": 0.3119,
+      "step": 3820
+    },
+    {
+      "epoch": 2.44,
+      "global_step": 3840,
+      "learning_rate": 0.0001,
+      "loss": 0.2237,
+      "step": 3840
+    },
+    {
+      "epoch": 2.45,
+      "global_step": 3860,
+      "learning_rate": 0.0001,
+      "loss": 0.2777,
+      "step": 3860
+    },
+    {
+      "epoch": 2.46,
+      "global_step": 3880,
+      "learning_rate": 0.0001,
+      "loss": 0.2371,
+      "step": 3880
+    },
+    {
+      "epoch": 2.48,
+      "global_step": 3900,
+      "learning_rate": 0.0001,
+      "loss": 0.2181,
+      "step": 3900
+    },
+    {
+      "epoch": 2.49,
+      "global_step": 3920,
+      "learning_rate": 0.0001,
+      "loss": 0.2603,
+      "step": 3920
+    },
+    {
+      "epoch": 2.5,
+      "global_step": 3940,
+      "learning_rate": 0.0001,
+      "loss": 0.2764,
+      "step": 3940
+    },
+    {
+      "epoch": 2.52,
+      "global_step": 3960,
+      "learning_rate": 0.0001,
+      "loss": 0.2124,
+      "step": 3960
+    },
+    {
+      "epoch": 2.53,
+      "global_step": 3980,
+      "learning_rate": 0.0001,
+      "loss": 0.2239,
+      "step": 3980
+    },
+    {
+      "epoch": 2.54,
+      "global_step": 4000,
+      "learning_rate": 0.0001,
+      "loss": 0.2812,
+      "step": 4000
+    },
+    {
+      "epoch": 2.55,
+      "global_step": 4020,
+      "learning_rate": 0.0001,
+      "loss": 0.1968,
+      "step": 4020
+    },
+    {
+      "epoch": 2.57,
+      "global_step": 4040,
+      "learning_rate": 0.0001,
+      "loss": 0.2648,
+      "step": 4040
+    },
+    {
+      "epoch": 2.58,
+      "global_step": 4060,
+      "learning_rate": 0.0001,
+      "loss": 0.2645,
+      "step": 4060
+    },
+    {
+      "epoch": 2.59,
+      "global_step": 4080,
+      "learning_rate": 0.0001,
+      "loss": 0.1914,
+      "step": 4080
+    },
+    {
+      "epoch": 2.6,
+      "global_step": 4100,
+      "learning_rate": 0.0001,
+      "loss": 0.2733,
+      "step": 4100
+    },
+    {
+      "epoch": 2.62,
+      "global_step": 4120,
+      "learning_rate": 0.0001,
+      "loss": 0.2374,
+      "step": 4120
+    },
+    {
+      "epoch": 2.63,
+      "global_step": 4140,
+      "learning_rate": 0.0001,
+      "loss": 0.208,
+      "step": 4140
+    },
+    {
+      "epoch": 2.64,
+      "global_step": 4160,
+      "learning_rate": 0.0001,
+      "loss": 0.2691,
+      "step": 4160
+    },
+    {
+      "epoch": 2.66,
+      "global_step": 4180,
+      "learning_rate": 0.0001,
+      "loss": 0.2268,
+      "step": 4180
+    },
+    {
+      "epoch": 2.67,
+      "global_step": 4200,
+      "learning_rate": 0.0001,
+      "loss": 0.2565,
+      "step": 4200
+    },
+    {
+      "epoch": 2.68,
+      "global_step": 4220,
+      "learning_rate": 0.0001,
+      "loss": 0.2758,
+      "step": 4220
+    },
+    {
+      "epoch": 2.69,
+      "global_step": 4240,
+      "learning_rate": 0.0001,
+      "loss": 0.2662,
+      "step": 4240
+    },
+    {
+      "epoch": 2.71,
+      "global_step": 4260,
+      "learning_rate": 0.0001,
+      "loss": 0.2026,
+      "step": 4260
+    },
+    {
+      "epoch": 2.72,
+      "global_step": 4280,
+      "learning_rate": 0.0001,
+      "loss": 0.2575,
+      "step": 4280
+    },
+    {
+      "epoch": 2.73,
+      "global_step": 4300,
+      "learning_rate": 0.0001,
+      "loss": 0.2477,
+      "step": 4300
+    },
+    {
+      "epoch": 2.74,
+      "global_step": 4320,
+      "learning_rate": 0.0001,
+      "loss": 0.2333,
+      "step": 4320
+    },
+    {
+      "epoch": 2.76,
+      "global_step": 4340,
+      "learning_rate": 0.0001,
+      "loss": 0.2689,
+      "step": 4340
+    },
+    {
+      "epoch": 2.77,
+      "global_step": 4360,
+      "learning_rate": 0.0001,
+      "loss": 0.2046,
+      "step": 4360
+    },
+    {
+      "epoch": 2.78,
+      "global_step": 4380,
+      "learning_rate": 0.0001,
+      "loss": 0.2546,
+      "step": 4380
+    },
+    {
+      "epoch": 2.79,
+      "global_step": 4400,
+      "learning_rate": 0.0001,
+      "loss": 0.2691,
+      "step": 4400
+    },
+    {
+      "epoch": 2.81,
+      "global_step": 4420,
+      "learning_rate": 0.0001,
+      "loss": 0.2444,
+      "step": 4420
+    },
+    {
+      "epoch": 2.82,
+      "global_step": 4440,
+      "learning_rate": 0.0001,
+      "loss": 0.2667,
+      "step": 4440
+    },
+    {
+      "epoch": 2.83,
+      "global_step": 4460,
+      "learning_rate": 0.0001,
+      "loss": 0.2469,
+      "step": 4460
+    },
+    {
+      "epoch": 2.85,
+      "global_step": 4480,
+      "learning_rate": 0.0001,
+      "loss": 0.2684,
+      "step": 4480
+    },
+    {
+      "epoch": 2.86,
+      "global_step": 4500,
+      "learning_rate": 0.0001,
+      "loss": 0.2077,
+      "step": 4500
+    },
+    {
+      "epoch": 2.87,
+      "global_step": 4520,
+      "learning_rate": 0.0001,
+      "loss": 0.2208,
+      "step": 4520
+    },
+    {
+      "epoch": 2.88,
+      "global_step": 4540,
+      "learning_rate": 0.0001,
+      "loss": 0.2401,
+      "step": 4540
+    },
+    {
+      "epoch": 2.9,
+      "global_step": 4560,
+      "learning_rate": 0.0001,
+      "loss": 0.2773,
+      "step": 4560
+    },
+    {
+      "epoch": 2.91,
+      "global_step": 4580,
+      "learning_rate": 0.0001,
+      "loss": 0.1959,
+      "step": 4580
+    },
+    {
+      "epoch": 2.92,
+      "global_step": 4600,
+      "learning_rate": 0.0001,
+      "loss": 0.2761,
+      "step": 4600
+    },
+    {
+      "epoch": 2.93,
+      "global_step": 4620,
+      "learning_rate": 0.0001,
+      "loss": 0.2821,
+      "step": 4620
+    },
+    {
+      "epoch": 2.95,
+      "global_step": 4640,
+      "learning_rate": 0.0001,
+      "loss": 0.2815,
+      "step": 4640
+    },
+    {
+      "epoch": 2.96,
+      "global_step": 4660,
+      "learning_rate": 0.0001,
+      "loss": 0.1933,
+      "step": 4660
+    },
+    {
+      "epoch": 2.97,
+      "global_step": 4680,
+      "learning_rate": 0.0001,
+      "loss": 0.2199,
+      "step": 4680
+    },
+    {
+      "epoch": 2.99,
+      "global_step": 4700,
+      "learning_rate": 0.0001,
+      "loss": 0.3124,
+      "step": 4700
+    },
+    {
+      "epoch": 3.0,
+      "global_step": 4720,
+      "learning_rate": 0.0001,
+      "loss": 0.2222,
+      "step": 4720
+    },
+    {
+      "epoch": 3.0,
+      "step": 4722,
+      "total_flos": 1.3718268437321482e+18,
+      "train_loss": 0.5372816354023708,
+      "train_runtime": 164669.988,
+      "train_samples_per_second": 0.344,
+      "train_steps_per_second": 0.029
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4722,
+  "num_train_epochs": 3,
+  "save_steps": 300,
+  "total_flos": 1.3718268437321482e+18,
+  "trial_name": null,
+  "trial_params": null
+}

ncr_cclue/without_knowledge_random/without_knowledge_random.json

@@ -0,0 +1,31 @@
+{
+    "desc": "不做知识选择，使用NCR+CCLUE数据集，随机选择的关键句训练",
+    "output_dir": "output/ablation_study/option-2/NCR/without_knowledge_random",
+    "model_name_or_path": "/data0/maqi/huggingface_models/TechGPT-7B",
+    "train_file": "/data0/maqi/KGLQA-data/datasets/merge/random_select/without_knowledge_random_instruction/train.jsonl",
+    "num_train_epochs": 3,
+    "per_device_train_batch_size": 2,
+    "gradient_accumulation_steps": 6,
+    "learning_rate": 1e-4,
+    "max_seq_length": 2048,
+    "logging_steps": 20,
+    "save_steps": 300,
+    "save_total_limit": 1,
+    "lr_scheduler_type": "constant_with_warmup",
+    "warmup_ratio": 0.1,
+    "lora_rank": 128,
+    "lora_alpha": 64,
+    "lora_dropout": 0.05,
+
+    "gradient_checkpointing": true,
+    "disable_tqdm": false,
+    "optim": "paged_adamw_32bit",
+    "seed": 318,
+    "fp16": true,
+    "report_to": "tensorboard",
+    "dataloader_num_workers": 10,
+    "save_strategy": "steps",
+    "weight_decay": 0,
+    "max_grad_norm": 0.3,
+    "remove_unused_columns": false
+}

ncr_cclue/without_knowledge_select/all_results.json

@@ -0,0 +1,7 @@
+{
+    "epoch": 3.0,
+    "train_loss": 0.4998251678882035,
+    "train_runtime": 78696.4679,
+    "train_samples_per_second": 0.72,
+    "train_steps_per_second": 0.06
+}

ncr_cclue/without_knowledge_select/final/README.md

@@ -0,0 +1,21 @@
+---
+library_name: peft
+---
+## Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: bitsandbytes
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: True
+- bnb_4bit_compute_dtype: float16
+### Framework versions
+
+
+- PEFT 0.5.0

ncr_cclue/without_knowledge_select/final/adapter_config.json

@@ -0,0 +1,26 @@
+{
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data0/maqi/huggingface_models/TechGPT-7B",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj",
+    "up_proj",
+    "down_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM"
+}

ncr_cclue/without_knowledge_select/final/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:894596690b52df40e0aad38771689697ea329822501789e768ea346e895f7854
+size 1279424269

ncr_cclue/without_knowledge_select/final/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:716036ed7f3421139f3b86de3c45a71a7481541db8c59ccaf8809befede91227
+size 4091

ncr_cclue/without_knowledge_select/train_results.json

@@ -0,0 +1,7 @@
+{
+    "epoch": 3.0,
+    "train_loss": 0.4998251678882035,
+    "train_runtime": 78696.4679,
+    "train_samples_per_second": 0.72,
+    "train_steps_per_second": 0.06
+}

ncr_cclue/without_knowledge_select/trainer_state.json

@@ -0,0 +1,1680 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.999047316608447,
+  "eval_steps": 500,
+  "global_step": 4722,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.01,
+      "global_step": 20,
+      "learning_rate": 4.016913319238901e-06,
+      "loss": 6.0272,
+      "step": 20
+    },
+    {
+      "epoch": 0.03,
+      "global_step": 40,
+      "learning_rate": 8.24524312896406e-06,
+      "loss": 5.4216,
+      "step": 40
+    },
+    {
+      "epoch": 0.04,
+      "global_step": 60,
+      "learning_rate": 1.2473572938689217e-05,
+      "loss": 4.646,
+      "step": 60
+    },
+    {
+      "epoch": 0.05,
+      "global_step": 80,
+      "learning_rate": 1.6701902748414377e-05,
+      "loss": 1.4871,
+      "step": 80
+    },
+    {
+      "epoch": 0.06,
+      "global_step": 100,
+      "learning_rate": 2.0930232558139536e-05,
+      "loss": 0.7485,
+      "step": 100
+    },
+    {
+      "epoch": 0.08,
+      "global_step": 120,
+      "learning_rate": 2.5158562367864697e-05,
+      "loss": 0.7163,
+      "step": 120
+    },
+    {
+      "epoch": 0.09,
+      "global_step": 140,
+      "learning_rate": 2.9386892177589852e-05,
+      "loss": 0.7384,
+      "step": 140
+    },
+    {
+      "epoch": 0.1,
+      "global_step": 160,
+      "learning_rate": 3.361522198731501e-05,
+      "loss": 0.7763,
+      "step": 160
+    },
+    {
+      "epoch": 0.11,
+      "global_step": 180,
+      "learning_rate": 3.784355179704017e-05,
+      "loss": 0.7499,
+      "step": 180
+    },
+    {
+      "epoch": 0.13,
+      "global_step": 200,
+      "learning_rate": 4.207188160676533e-05,
+      "loss": 0.7006,
+      "step": 200
+    },
+    {
+      "epoch": 0.14,
+      "global_step": 220,
+      "learning_rate": 4.630021141649049e-05,
+      "loss": 0.7016,
+      "step": 220
+    },
+    {
+      "epoch": 0.15,
+      "global_step": 240,
+      "learning_rate": 5.052854122621564e-05,
+      "loss": 0.7406,
+      "step": 240
+    },
+    {
+      "epoch": 0.17,
+      "global_step": 260,
+      "learning_rate": 5.4756871035940805e-05,
+      "loss": 0.7206,
+      "step": 260
+    },
+    {
+      "epoch": 0.18,
+      "global_step": 280,
+      "learning_rate": 5.8985200845665964e-05,
+      "loss": 0.7188,
+      "step": 280
+    },
+    {
+      "epoch": 0.19,
+      "global_step": 300,
+      "learning_rate": 6.257928118393235e-05,
+      "loss": 0.7429,
+      "step": 300
+    },
+    {
+      "epoch": 0.2,
+      "global_step": 320,
+      "learning_rate": 6.659619450317124e-05,
+      "loss": 0.9766,
+      "step": 320
+    },
+    {
+      "epoch": 0.22,
+      "global_step": 340,
+      "learning_rate": 7.082452431289642e-05,
+      "loss": 0.7143,
+      "step": 340
+    },
+    {
+      "epoch": 0.23,
+      "global_step": 360,
+      "learning_rate": 7.505285412262156e-05,
+      "loss": 0.7145,
+      "step": 360
+    },
+    {
+      "epoch": 0.24,
+      "global_step": 380,
+      "learning_rate": 7.928118393234673e-05,
+      "loss": 0.7015,
+      "step": 380
+    },
+    {
+      "epoch": 0.25,
+      "global_step": 400,
+      "learning_rate": 8.350951374207189e-05,
+      "loss": 0.692,
+      "step": 400
+    },
+    {
+      "epoch": 0.27,
+      "global_step": 420,
+      "learning_rate": 8.773784355179705e-05,
+      "loss": 0.6956,
+      "step": 420
+    },
+    {
+      "epoch": 0.28,
+      "global_step": 440,
+      "learning_rate": 9.19661733615222e-05,
+      "loss": 0.6791,
+      "step": 440
+    },
+    {
+      "epoch": 0.29,
+      "global_step": 460,
+      "learning_rate": 9.619450317124737e-05,
+      "loss": 0.713,
+      "step": 460
+    },
+    {
+      "epoch": 0.3,
+      "global_step": 480,
+      "learning_rate": 0.0001,
+      "loss": 0.7247,
+      "step": 480
+    },
+    {
+      "epoch": 0.32,
+      "global_step": 500,
+      "learning_rate": 0.0001,
+      "loss": 0.6834,
+      "step": 500
+    },
+    {
+      "epoch": 0.33,
+      "global_step": 520,
+      "learning_rate": 0.0001,
+      "loss": 0.697,
+      "step": 520
+    },
+    {
+      "epoch": 0.34,
+      "global_step": 540,
+      "learning_rate": 0.0001,
+      "loss": 0.6864,
+      "step": 540
+    },
+    {
+      "epoch": 0.36,
+      "global_step": 560,
+      "learning_rate": 0.0001,
+      "loss": 0.7196,
+      "step": 560
+    },
+    {
+      "epoch": 0.37,
+      "global_step": 580,
+      "learning_rate": 0.0001,
+      "loss": 0.9249,
+      "step": 580
+    },
+    {
+      "epoch": 0.38,
+      "global_step": 600,
+      "learning_rate": 0.0001,
+      "loss": 0.6809,
+      "step": 600
+    },
+    {
+      "epoch": 0.39,
+      "global_step": 620,
+      "learning_rate": 0.0001,
+      "loss": 0.7093,
+      "step": 620
+    },
+    {
+      "epoch": 0.41,
+      "global_step": 640,
+      "learning_rate": 0.0001,
+      "loss": 0.7105,
+      "step": 640
+    },
+    {
+      "epoch": 0.42,
+      "global_step": 660,
+      "learning_rate": 0.0001,
+      "loss": 0.6874,
+      "step": 660
+    },
+    {
+      "epoch": 0.43,
+      "global_step": 680,
+      "learning_rate": 0.0001,
+      "loss": 0.6884,
+      "step": 680
+    },
+    {
+      "epoch": 0.44,
+      "global_step": 700,
+      "learning_rate": 0.0001,
+      "loss": 0.6619,
+      "step": 700
+    },
+    {
+      "epoch": 0.46,
+      "global_step": 720,
+      "learning_rate": 0.0001,
+      "loss": 0.6527,
+      "step": 720
+    },
+    {
+      "epoch": 0.47,
+      "global_step": 740,
+      "learning_rate": 0.0001,
+      "loss": 0.6646,
+      "step": 740
+    },
+    {
+      "epoch": 0.48,
+      "global_step": 760,
+      "learning_rate": 0.0001,
+      "loss": 0.6953,
+      "step": 760
+    },
+    {
+      "epoch": 0.5,
+      "global_step": 780,
+      "learning_rate": 0.0001,
+      "loss": 0.6712,
+      "step": 780
+    },
+    {
+      "epoch": 0.51,
+      "global_step": 800,
+      "learning_rate": 0.0001,
+      "loss": 0.6592,
+      "step": 800
+    },
+    {
+      "epoch": 0.52,
+      "global_step": 820,
+      "learning_rate": 0.0001,
+      "loss": 0.6703,
+      "step": 820
+    },
+    {
+      "epoch": 0.53,
+      "global_step": 840,
+      "learning_rate": 0.0001,
+      "loss": 0.674,
+      "step": 840
+    },
+    {
+      "epoch": 0.55,
+      "global_step": 860,
+      "learning_rate": 0.0001,
+      "loss": 0.6376,
+      "step": 860
+    },
+    {
+      "epoch": 0.56,
+      "global_step": 880,
+      "learning_rate": 0.0001,
+      "loss": 0.669,
+      "step": 880
+    },
+    {
+      "epoch": 0.57,
+      "global_step": 900,
+      "learning_rate": 0.0001,
+      "loss": 0.6423,
+      "step": 900
+    },
+    {
+      "epoch": 0.58,
+      "global_step": 920,
+      "learning_rate": 0.0001,
+      "loss": 0.6659,
+      "step": 920
+    },
+    {
+      "epoch": 0.6,
+      "global_step": 940,
+      "learning_rate": 0.0001,
+      "loss": 0.645,
+      "step": 940
+    },
+    {
+      "epoch": 0.61,
+      "global_step": 960,
+      "learning_rate": 0.0001,
+      "loss": 0.6713,
+      "step": 960
+    },
+    {
+      "epoch": 0.62,
+      "global_step": 980,
+      "learning_rate": 0.0001,
+      "loss": 0.6273,
+      "step": 980
+    },
+    {
+      "epoch": 0.64,
+      "global_step": 1000,
+      "learning_rate": 0.0001,
+      "loss": 0.653,
+      "step": 1000
+    },
+    {
+      "epoch": 0.65,
+      "global_step": 1020,
+      "learning_rate": 0.0001,
+      "loss": 0.6166,
+      "step": 1020
+    },
+    {
+      "epoch": 0.66,
+      "global_step": 1040,
+      "learning_rate": 0.0001,
+      "loss": 0.6679,
+      "step": 1040
+    },
+    {
+      "epoch": 0.67,
+      "global_step": 1060,
+      "learning_rate": 0.0001,
+      "loss": 0.6208,
+      "step": 1060
+    },
+    {
+      "epoch": 0.69,
+      "global_step": 1080,
+      "learning_rate": 0.0001,
+      "loss": 0.6481,
+      "step": 1080
+    },
+    {
+      "epoch": 0.7,
+      "global_step": 1100,
+      "learning_rate": 0.0001,
+      "loss": 0.6166,
+      "step": 1100
+    },
+    {
+      "epoch": 0.71,
+      "global_step": 1120,
+      "learning_rate": 0.0001,
+      "loss": 0.6489,
+      "step": 1120
+    },
+    {
+      "epoch": 0.72,
+      "global_step": 1140,
+      "learning_rate": 0.0001,
+      "loss": 0.6087,
+      "step": 1140
+    },
+    {
+      "epoch": 0.74,
+      "global_step": 1160,
+      "learning_rate": 0.0001,
+      "loss": 0.6045,
+      "step": 1160
+    },
+    {
+      "epoch": 0.75,
+      "global_step": 1180,
+      "learning_rate": 0.0001,
+      "loss": 0.6593,
+      "step": 1180
+    },
+    {
+      "epoch": 0.76,
+      "global_step": 1200,
+      "learning_rate": 0.0001,
+      "loss": 0.6348,
+      "step": 1200
+    },
+    {
+      "epoch": 0.77,
+      "global_step": 1220,
+      "learning_rate": 0.0001,
+      "loss": 0.5896,
+      "step": 1220
+    },
+    {
+      "epoch": 0.79,
+      "global_step": 1240,
+      "learning_rate": 0.0001,
+      "loss": 0.631,
+      "step": 1240
+    },
+    {
+      "epoch": 0.8,
+      "global_step": 1260,
+      "learning_rate": 0.0001,
+      "loss": 0.5845,
+      "step": 1260
+    },
+    {
+      "epoch": 0.81,
+      "global_step": 1280,
+      "learning_rate": 0.0001,
+      "loss": 0.6301,
+      "step": 1280
+    },
+    {
+      "epoch": 0.83,
+      "global_step": 1300,
+      "learning_rate": 0.0001,
+      "loss": 0.617,
+      "step": 1300
+    },
+    {
+      "epoch": 0.84,
+      "global_step": 1320,
+      "learning_rate": 0.0001,
+      "loss": 0.6049,
+      "step": 1320
+    },
+    {
+      "epoch": 0.85,
+      "global_step": 1340,
+      "learning_rate": 0.0001,
+      "loss": 0.6058,
+      "step": 1340
+    },
+    {
+      "epoch": 0.86,
+      "global_step": 1360,
+      "learning_rate": 0.0001,
+      "loss": 0.601,
+      "step": 1360
+    },
+    {
+      "epoch": 0.88,
+      "global_step": 1380,
+      "learning_rate": 0.0001,
+      "loss": 0.5956,
+      "step": 1380
+    },
+    {
+      "epoch": 0.89,
+      "global_step": 1400,
+      "learning_rate": 0.0001,
+      "loss": 0.6179,
+      "step": 1400
+    },
+    {
+      "epoch": 0.9,
+      "global_step": 1420,
+      "learning_rate": 0.0001,
+      "loss": 0.6244,
+      "step": 1420
+    },
+    {
+      "epoch": 0.91,
+      "global_step": 1440,
+      "learning_rate": 0.0001,
+      "loss": 0.5867,
+      "step": 1440
+    },
+    {
+      "epoch": 0.93,
+      "global_step": 1460,
+      "learning_rate": 0.0001,
+      "loss": 0.6267,
+      "step": 1460
+    },
+    {
+      "epoch": 0.94,
+      "global_step": 1480,
+      "learning_rate": 0.0001,
+      "loss": 0.5942,
+      "step": 1480
+    },
+    {
+      "epoch": 0.95,
+      "global_step": 1500,
+      "learning_rate": 0.0001,
+      "loss": 0.5941,
+      "step": 1500
+    },
+    {
+      "epoch": 0.97,
+      "global_step": 1520,
+      "learning_rate": 0.0001,
+      "loss": 0.565,
+      "step": 1520
+    },
+    {
+      "epoch": 0.98,
+      "global_step": 1540,
+      "learning_rate": 0.0001,
+      "loss": 0.5815,
+      "step": 1540
+    },
+    {
+      "epoch": 0.99,
+      "global_step": 1560,
+      "learning_rate": 0.0001,
+      "loss": 0.6256,
+      "step": 1560
+    },
+    {
+      "epoch": 1.0,
+      "global_step": 1580,
+      "learning_rate": 0.0001,
+      "loss": 0.5428,
+      "step": 1580
+    },
+    {
+      "epoch": 1.02,
+      "global_step": 1600,
+      "learning_rate": 0.0001,
+      "loss": 0.4787,
+      "step": 1600
+    },
+    {
+      "epoch": 1.03,
+      "global_step": 1620,
+      "learning_rate": 0.0001,
+      "loss": 0.4868,
+      "step": 1620
+    },
+    {
+      "epoch": 1.04,
+      "global_step": 1640,
+      "learning_rate": 0.0001,
+      "loss": 0.482,
+      "step": 1640
+    },
+    {
+      "epoch": 1.05,
+      "global_step": 1660,
+      "learning_rate": 0.0001,
+      "loss": 0.5174,
+      "step": 1660
+    },
+    {
+      "epoch": 1.07,
+      "global_step": 1680,
+      "learning_rate": 0.0001,
+      "loss": 0.494,
+      "step": 1680
+    },
+    {
+      "epoch": 1.08,
+      "global_step": 1700,
+      "learning_rate": 0.0001,
+      "loss": 0.4921,
+      "step": 1700
+    },
+    {
+      "epoch": 1.09,
+      "global_step": 1720,
+      "learning_rate": 0.0001,
+      "loss": 0.5036,
+      "step": 1720
+    },
+    {
+      "epoch": 1.11,
+      "global_step": 1740,
+      "learning_rate": 0.0001,
+      "loss": 0.5123,
+      "step": 1740
+    },
+    {
+      "epoch": 1.12,
+      "global_step": 1760,
+      "learning_rate": 0.0001,
+      "loss": 0.4705,
+      "step": 1760
+    },
+    {
+      "epoch": 1.13,
+      "global_step": 1780,
+      "learning_rate": 0.0001,
+      "loss": 0.5217,
+      "step": 1780
+    },
+    {
+      "epoch": 1.14,
+      "global_step": 1800,
+      "learning_rate": 0.0001,
+      "loss": 0.4837,
+      "step": 1800
+    },
+    {
+      "epoch": 1.16,
+      "global_step": 1820,
+      "learning_rate": 0.0001,
+      "loss": 0.5294,
+      "step": 1820
+    },
+    {
+      "epoch": 1.17,
+      "global_step": 1840,
+      "learning_rate": 0.0001,
+      "loss": 0.5013,
+      "step": 1840
+    },
+    {
+      "epoch": 1.18,
+      "global_step": 1860,
+      "learning_rate": 0.0001,
+      "loss": 0.5218,
+      "step": 1860
+    },
+    {
+      "epoch": 1.19,
+      "global_step": 1880,
+      "learning_rate": 0.0001,
+      "loss": 0.4609,
+      "step": 1880
+    },
+    {
+      "epoch": 1.21,
+      "global_step": 1900,
+      "learning_rate": 0.0001,
+      "loss": 0.4848,
+      "step": 1900
+    },
+    {
+      "epoch": 1.22,
+      "global_step": 1920,
+      "learning_rate": 0.0001,
+      "loss": 0.49,
+      "step": 1920
+    },
+    {
+      "epoch": 1.23,
+      "global_step": 1940,
+      "learning_rate": 0.0001,
+      "loss": 0.4473,
+      "step": 1940
+    },
+    {
+      "epoch": 1.24,
+      "global_step": 1960,
+      "learning_rate": 0.0001,
+      "loss": 0.4259,
+      "step": 1960
+    },
+    {
+      "epoch": 1.26,
+      "global_step": 1980,
+      "learning_rate": 0.0001,
+      "loss": 0.4392,
+      "step": 1980
+    },
+    {
+      "epoch": 1.27,
+      "global_step": 2000,
+      "learning_rate": 0.0001,
+      "loss": 0.4838,
+      "step": 2000
+    },
+    {
+      "epoch": 1.28,
+      "global_step": 2020,
+      "learning_rate": 0.0001,
+      "loss": 0.4985,
+      "step": 2020
+    },
+    {
+      "epoch": 1.3,
+      "global_step": 2040,
+      "learning_rate": 0.0001,
+      "loss": 0.4694,
+      "step": 2040
+    },
+    {
+      "epoch": 1.31,
+      "global_step": 2060,
+      "learning_rate": 0.0001,
+      "loss": 0.4078,
+      "step": 2060
+    },
+    {
+      "epoch": 1.32,
+      "global_step": 2080,
+      "learning_rate": 0.0001,
+      "loss": 0.4588,
+      "step": 2080
+    },
+    {
+      "epoch": 1.33,
+      "global_step": 2100,
+      "learning_rate": 0.0001,
+      "loss": 0.4392,
+      "step": 2100
+    },
+    {
+      "epoch": 1.35,
+      "global_step": 2120,
+      "learning_rate": 0.0001,
+      "loss": 0.4355,
+      "step": 2120
+    },
+    {
+      "epoch": 1.36,
+      "global_step": 2140,
+      "learning_rate": 0.0001,
+      "loss": 0.5047,
+      "step": 2140
+    },
+    {
+      "epoch": 1.37,
+      "global_step": 2160,
+      "learning_rate": 0.0001,
+      "loss": 0.4923,
+      "step": 2160
+    },
+    {
+      "epoch": 1.38,
+      "global_step": 2180,
+      "learning_rate": 0.0001,
+      "loss": 0.397,
+      "step": 2180
+    },
+    {
+      "epoch": 1.4,
+      "global_step": 2200,
+      "learning_rate": 0.0001,
+      "loss": 0.4434,
+      "step": 2200
+    },
+    {
+      "epoch": 1.41,
+      "global_step": 2220,
+      "learning_rate": 0.0001,
+      "loss": 0.3956,
+      "step": 2220
+    },
+    {
+      "epoch": 1.42,
+      "global_step": 2240,
+      "learning_rate": 0.0001,
+      "loss": 0.4646,
+      "step": 2240
+    },
+    {
+      "epoch": 1.44,
+      "global_step": 2260,
+      "learning_rate": 0.0001,
+      "loss": 0.4445,
+      "step": 2260
+    },
+    {
+      "epoch": 1.45,
+      "global_step": 2280,
+      "learning_rate": 0.0001,
+      "loss": 0.4538,
+      "step": 2280
+    },
+    {
+      "epoch": 1.46,
+      "global_step": 2300,
+      "learning_rate": 0.0001,
+      "loss": 0.4295,
+      "step": 2300
+    },
+    {
+      "epoch": 1.47,
+      "global_step": 2320,
+      "learning_rate": 0.0001,
+      "loss": 0.5049,
+      "step": 2320
+    },
+    {
+      "epoch": 1.49,
+      "global_step": 2340,
+      "learning_rate": 0.0001,
+      "loss": 0.425,
+      "step": 2340
+    },
+    {
+      "epoch": 1.5,
+      "global_step": 2360,
+      "learning_rate": 0.0001,
+      "loss": 0.4057,
+      "step": 2360
+    },
+    {
+      "epoch": 1.51,
+      "global_step": 2380,
+      "learning_rate": 0.0001,
+      "loss": 0.4622,
+      "step": 2380
+    },
+    {
+      "epoch": 1.52,
+      "global_step": 2400,
+      "learning_rate": 0.0001,
+      "loss": 0.4465,
+      "step": 2400
+    },
+    {
+      "epoch": 1.54,
+      "global_step": 2420,
+      "learning_rate": 0.0001,
+      "loss": 0.4446,
+      "step": 2420
+    },
+    {
+      "epoch": 1.55,
+      "global_step": 2440,
+      "learning_rate": 0.0001,
+      "loss": 0.4403,
+      "step": 2440
+    },
+    {
+      "epoch": 1.56,
+      "global_step": 2460,
+      "learning_rate": 0.0001,
+      "loss": 0.412,
+      "step": 2460
+    },
+    {
+      "epoch": 1.58,
+      "global_step": 2480,
+      "learning_rate": 0.0001,
+      "loss": 0.3916,
+      "step": 2480
+    },
+    {
+      "epoch": 1.59,
+      "global_step": 2500,
+      "learning_rate": 0.0001,
+      "loss": 0.4018,
+      "step": 2500
+    },
+    {
+      "epoch": 1.6,
+      "global_step": 2520,
+      "learning_rate": 0.0001,
+      "loss": 0.3805,
+      "step": 2520
+    },
+    {
+      "epoch": 1.61,
+      "global_step": 2540,
+      "learning_rate": 0.0001,
+      "loss": 0.3839,
+      "step": 2540
+    },
+    {
+      "epoch": 1.63,
+      "global_step": 2560,
+      "learning_rate": 0.0001,
+      "loss": 0.4083,
+      "step": 2560
+    },
+    {
+      "epoch": 1.64,
+      "global_step": 2580,
+      "learning_rate": 0.0001,
+      "loss": 0.4723,
+      "step": 2580
+    },
+    {
+      "epoch": 1.65,
+      "global_step": 2600,
+      "learning_rate": 0.0001,
+      "loss": 0.3934,
+      "step": 2600
+    },
+    {
+      "epoch": 1.66,
+      "global_step": 2620,
+      "learning_rate": 0.0001,
+      "loss": 0.4418,
+      "step": 2620
+    },
+    {
+      "epoch": 1.68,
+      "global_step": 2640,
+      "learning_rate": 0.0001,
+      "loss": 0.438,
+      "step": 2640
+    },
+    {
+      "epoch": 1.69,
+      "global_step": 2660,
+      "learning_rate": 0.0001,
+      "loss": 0.4364,
+      "step": 2660
+    },
+    {
+      "epoch": 1.7,
+      "global_step": 2680,
+      "learning_rate": 0.0001,
+      "loss": 0.4225,
+      "step": 2680
+    },
+    {
+      "epoch": 1.71,
+      "global_step": 2700,
+      "learning_rate": 0.0001,
+      "loss": 0.4432,
+      "step": 2700
+    },
+    {
+      "epoch": 1.73,
+      "global_step": 2720,
+      "learning_rate": 0.0001,
+      "loss": 0.4017,
+      "step": 2720
+    },
+    {
+      "epoch": 1.74,
+      "global_step": 2740,
+      "learning_rate": 0.0001,
+      "loss": 0.4591,
+      "step": 2740
+    },
+    {
+      "epoch": 1.75,
+      "global_step": 2760,
+      "learning_rate": 0.0001,
+      "loss": 0.4125,
+      "step": 2760
+    },
+    {
+      "epoch": 1.77,
+      "global_step": 2780,
+      "learning_rate": 0.0001,
+      "loss": 0.4457,
+      "step": 2780
+    },
+    {
+      "epoch": 1.78,
+      "global_step": 2800,
+      "learning_rate": 0.0001,
+      "loss": 0.4422,
+      "step": 2800
+    },
+    {
+      "epoch": 1.79,
+      "global_step": 2820,
+      "learning_rate": 0.0001,
+      "loss": 0.4259,
+      "step": 2820
+    },
+    {
+      "epoch": 1.8,
+      "global_step": 2840,
+      "learning_rate": 0.0001,
+      "loss": 0.4357,
+      "step": 2840
+    },
+    {
+      "epoch": 1.82,
+      "global_step": 2860,
+      "learning_rate": 0.0001,
+      "loss": 0.4169,
+      "step": 2860
+    },
+    {
+      "epoch": 1.83,
+      "global_step": 2880,
+      "learning_rate": 0.0001,
+      "loss": 0.4214,
+      "step": 2880
+    },
+    {
+      "epoch": 1.84,
+      "global_step": 2900,
+      "learning_rate": 0.0001,
+      "loss": 0.4353,
+      "step": 2900
+    },
+    {
+      "epoch": 1.85,
+      "global_step": 2920,
+      "learning_rate": 0.0001,
+      "loss": 0.4325,
+      "step": 2920
+    },
+    {
+      "epoch": 1.87,
+      "global_step": 2940,
+      "learning_rate": 0.0001,
+      "loss": 0.4011,
+      "step": 2940
+    },
+    {
+      "epoch": 1.88,
+      "global_step": 2960,
+      "learning_rate": 0.0001,
+      "loss": 0.3682,
+      "step": 2960
+    },
+    {
+      "epoch": 1.89,
+      "global_step": 2980,
+      "learning_rate": 0.0001,
+      "loss": 0.3371,
+      "step": 2980
+    },
+    {
+      "epoch": 1.91,
+      "global_step": 3000,
+      "learning_rate": 0.0001,
+      "loss": 0.4074,
+      "step": 3000
+    },
+    {
+      "epoch": 1.92,
+      "global_step": 3020,
+      "learning_rate": 0.0001,
+      "loss": 0.348,
+      "step": 3020
+    },
+    {
+      "epoch": 1.93,
+      "global_step": 3040,
+      "learning_rate": 0.0001,
+      "loss": 0.3886,
+      "step": 3040
+    },
+    {
+      "epoch": 1.94,
+      "global_step": 3060,
+      "learning_rate": 0.0001,
+      "loss": 0.3677,
+      "step": 3060
+    },
+    {
+      "epoch": 1.96,
+      "global_step": 3080,
+      "learning_rate": 0.0001,
+      "loss": 0.3696,
+      "step": 3080
+    },
+    {
+      "epoch": 1.97,
+      "global_step": 3100,
+      "learning_rate": 0.0001,
+      "loss": 0.3335,
+      "step": 3100
+    },
+    {
+      "epoch": 1.98,
+      "global_step": 3120,
+      "learning_rate": 0.0001,
+      "loss": 0.3569,
+      "step": 3120
+    },
+    {
+      "epoch": 1.99,
+      "global_step": 3140,
+      "learning_rate": 0.0001,
+      "loss": 0.3725,
+      "step": 3140
+    },
+    {
+      "epoch": 2.01,
+      "global_step": 3160,
+      "learning_rate": 0.0001,
+      "loss": 0.3084,
+      "step": 3160
+    },
+    {
+      "epoch": 2.02,
+      "global_step": 3180,
+      "learning_rate": 0.0001,
+      "loss": 0.2052,
+      "step": 3180
+    },
+    {
+      "epoch": 2.03,
+      "global_step": 3200,
+      "learning_rate": 0.0001,
+      "loss": 0.1521,
+      "step": 3200
+    },
+    {
+      "epoch": 2.05,
+      "global_step": 3220,
+      "learning_rate": 0.0001,
+      "loss": 0.1778,
+      "step": 3220
+    },
+    {
+      "epoch": 2.06,
+      "global_step": 3240,
+      "learning_rate": 0.0001,
+      "loss": 0.2081,
+      "step": 3240
+    },
+    {
+      "epoch": 2.07,
+      "global_step": 3260,
+      "learning_rate": 0.0001,
+      "loss": 0.1696,
+      "step": 3260
+    },
+    {
+      "epoch": 2.08,
+      "global_step": 3280,
+      "learning_rate": 0.0001,
+      "loss": 0.1653,
+      "step": 3280
+    },
+    {
+      "epoch": 2.1,
+      "global_step": 3300,
+      "learning_rate": 0.0001,
+      "loss": 0.1713,
+      "step": 3300
+    },
+    {
+      "epoch": 2.11,
+      "global_step": 3320,
+      "learning_rate": 0.0001,
+      "loss": 0.1614,
+      "step": 3320
+    },
+    {
+      "epoch": 2.12,
+      "global_step": 3340,
+      "learning_rate": 0.0001,
+      "loss": 0.1813,
+      "step": 3340
+    },
+    {
+      "epoch": 2.13,
+      "global_step": 3360,
+      "learning_rate": 0.0001,
+      "loss": 0.202,
+      "step": 3360
+    },
+    {
+      "epoch": 2.15,
+      "global_step": 3380,
+      "learning_rate": 0.0001,
+      "loss": 0.2307,
+      "step": 3380
+    },
+    {
+      "epoch": 2.16,
+      "global_step": 3400,
+      "learning_rate": 0.0001,
+      "loss": 0.1663,
+      "step": 3400
+    },
+    {
+      "epoch": 2.17,
+      "global_step": 3420,
+      "learning_rate": 0.0001,
+      "loss": 0.1635,
+      "step": 3420
+    },
+    {
+      "epoch": 2.18,
+      "global_step": 3440,
+      "learning_rate": 0.0001,
+      "loss": 0.1487,
+      "step": 3440
+    },
+    {
+      "epoch": 2.2,
+      "global_step": 3460,
+      "learning_rate": 0.0001,
+      "loss": 0.1952,
+      "step": 3460
+    },
+    {
+      "epoch": 2.21,
+      "global_step": 3480,
+      "learning_rate": 0.0001,
+      "loss": 0.158,
+      "step": 3480
+    },
+    {
+      "epoch": 2.22,
+      "global_step": 3500,
+      "learning_rate": 0.0001,
+      "loss": 0.2287,
+      "step": 3500
+    },
+    {
+      "epoch": 2.24,
+      "global_step": 3520,
+      "learning_rate": 0.0001,
+      "loss": 0.1666,
+      "step": 3520
+    },
+    {
+      "epoch": 2.25,
+      "global_step": 3540,
+      "learning_rate": 0.0001,
+      "loss": 0.1694,
+      "step": 3540
+    },
+    {
+      "epoch": 2.26,
+      "global_step": 3560,
+      "learning_rate": 0.0001,
+      "loss": 0.1856,
+      "step": 3560
+    },
+    {
+      "epoch": 2.27,
+      "global_step": 3580,
+      "learning_rate": 0.0001,
+      "loss": 0.2015,
+      "step": 3580
+    },
+    {
+      "epoch": 2.29,
+      "global_step": 3600,
+      "learning_rate": 0.0001,
+      "loss": 0.1732,
+      "step": 3600
+    },
+    {
+      "epoch": 2.3,
+      "global_step": 3620,
+      "learning_rate": 0.0001,
+      "loss": 0.2336,
+      "step": 3620
+    },
+    {
+      "epoch": 2.31,
+      "global_step": 3640,
+      "learning_rate": 0.0001,
+      "loss": 0.1811,
+      "step": 3640
+    },
+    {
+      "epoch": 2.32,
+      "global_step": 3660,
+      "learning_rate": 0.0001,
+      "loss": 0.246,
+      "step": 3660
+    },
+    {
+      "epoch": 2.34,
+      "global_step": 3680,
+      "learning_rate": 0.0001,
+      "loss": 0.2835,
+      "step": 3680
+    },
+    {
+      "epoch": 2.35,
+      "global_step": 3700,
+      "learning_rate": 0.0001,
+      "loss": 0.1667,
+      "step": 3700
+    },
+    {
+      "epoch": 2.36,
+      "global_step": 3720,
+      "learning_rate": 0.0001,
+      "loss": 0.2501,
+      "step": 3720
+    },
+    {
+      "epoch": 2.38,
+      "global_step": 3740,
+      "learning_rate": 0.0001,
+      "loss": 0.1996,
+      "step": 3740
+    },
+    {
+      "epoch": 2.39,
+      "global_step": 3760,
+      "learning_rate": 0.0001,
+      "loss": 0.233,
+      "step": 3760
+    },
+    {
+      "epoch": 2.4,
+      "global_step": 3780,
+      "learning_rate": 0.0001,
+      "loss": 0.1755,
+      "step": 3780
+    },
+    {
+      "epoch": 2.41,
+      "global_step": 3800,
+      "learning_rate": 0.0001,
+      "loss": 0.2196,
+      "step": 3800
+    },
+    {
+      "epoch": 2.43,
+      "global_step": 3820,
+      "learning_rate": 0.0001,
+      "loss": 0.1767,
+      "step": 3820
+    },
+    {
+      "epoch": 2.44,
+      "global_step": 3840,
+      "learning_rate": 0.0001,
+      "loss": 0.1443,
+      "step": 3840
+    },
+    {
+      "epoch": 2.45,
+      "global_step": 3860,
+      "learning_rate": 0.0001,
+      "loss": 0.2324,
+      "step": 3860
+    },
+    {
+      "epoch": 2.46,
+      "global_step": 3880,
+      "learning_rate": 0.0001,
+      "loss": 0.2332,
+      "step": 3880
+    },
+    {
+      "epoch": 2.48,
+      "global_step": 3900,
+      "learning_rate": 0.0001,
+      "loss": 0.2032,
+      "step": 3900
+    },
+    {
+      "epoch": 2.49,
+      "global_step": 3920,
+      "learning_rate": 0.0001,
+      "loss": 0.1638,
+      "step": 3920
+    },
+    {
+      "epoch": 2.5,
+      "global_step": 3940,
+      "learning_rate": 0.0001,
+      "loss": 0.2386,
+      "step": 3940
+    },
+    {
+      "epoch": 2.52,
+      "global_step": 3960,
+      "learning_rate": 0.0001,
+      "loss": 0.1971,
+      "step": 3960
+    },
+    {
+      "epoch": 2.53,
+      "global_step": 3980,
+      "learning_rate": 0.0001,
+      "loss": 0.2,
+      "step": 3980
+    },
+    {
+      "epoch": 2.54,
+      "global_step": 4000,
+      "learning_rate": 0.0001,
+      "loss": 0.2215,
+      "step": 4000
+    },
+    {
+      "epoch": 2.55,
+      "global_step": 4020,
+      "learning_rate": 0.0001,
+      "loss": 0.1947,
+      "step": 4020
+    },
+    {
+      "epoch": 2.57,
+      "global_step": 4040,
+      "learning_rate": 0.0001,
+      "loss": 0.2255,
+      "step": 4040
+    },
+    {
+      "epoch": 2.58,
+      "global_step": 4060,
+      "learning_rate": 0.0001,
+      "loss": 0.1673,
+      "step": 4060
+    },
+    {
+      "epoch": 2.59,
+      "global_step": 4080,
+      "learning_rate": 0.0001,
+      "loss": 0.2353,
+      "step": 4080
+    },
+    {
+      "epoch": 2.6,
+      "global_step": 4100,
+      "learning_rate": 0.0001,
+      "loss": 0.2311,
+      "step": 4100
+    },
+    {
+      "epoch": 2.62,
+      "global_step": 4120,
+      "learning_rate": 0.0001,
+      "loss": 0.2066,
+      "step": 4120
+    },
+    {
+      "epoch": 2.63,
+      "global_step": 4140,
+      "learning_rate": 0.0001,
+      "loss": 0.2318,
+      "step": 4140
+    },
+    {
+      "epoch": 2.64,
+      "global_step": 4160,
+      "learning_rate": 0.0001,
+      "loss": 0.2399,
+      "step": 4160
+    },
+    {
+      "epoch": 2.65,
+      "global_step": 4180,
+      "learning_rate": 0.0001,
+      "loss": 0.1898,
+      "step": 4180
+    },
+    {
+      "epoch": 2.67,
+      "global_step": 4200,
+      "learning_rate": 0.0001,
+      "loss": 0.239,
+      "step": 4200
+    },
+    {
+      "epoch": 2.68,
+      "global_step": 4220,
+      "learning_rate": 0.0001,
+      "loss": 0.179,
+      "step": 4220
+    },
+    {
+      "epoch": 2.69,
+      "global_step": 4240,
+      "learning_rate": 0.0001,
+      "loss": 0.1383,
+      "step": 4240
+    },
+    {
+      "epoch": 2.71,
+      "global_step": 4260,
+      "learning_rate": 0.0001,
+      "loss": 0.2512,
+      "step": 4260
+    },
+    {
+      "epoch": 2.72,
+      "global_step": 4280,
+      "learning_rate": 0.0001,
+      "loss": 0.2032,
+      "step": 4280
+    },
+    {
+      "epoch": 2.73,
+      "global_step": 4300,
+      "learning_rate": 0.0001,
+      "loss": 0.2055,
+      "step": 4300
+    },
+    {
+      "epoch": 2.74,
+      "global_step": 4320,
+      "learning_rate": 0.0001,
+      "loss": 0.1862,
+      "step": 4320
+    },
+    {
+      "epoch": 2.76,
+      "global_step": 4340,
+      "learning_rate": 0.0001,
+      "loss": 0.2174,
+      "step": 4340
+    },
+    {
+      "epoch": 2.77,
+      "global_step": 4360,
+      "learning_rate": 0.0001,
+      "loss": 0.2088,
+      "step": 4360
+    },
+    {
+      "epoch": 2.78,
+      "global_step": 4380,
+      "learning_rate": 0.0001,
+      "loss": 0.1589,
+      "step": 4380
+    },
+    {
+      "epoch": 2.79,
+      "global_step": 4400,
+      "learning_rate": 0.0001,
+      "loss": 0.2764,
+      "step": 4400
+    },
+    {
+      "epoch": 2.81,
+      "global_step": 4420,
+      "learning_rate": 0.0001,
+      "loss": 0.2035,
+      "step": 4420
+    },
+    {
+      "epoch": 2.82,
+      "global_step": 4440,
+      "learning_rate": 0.0001,
+      "loss": 0.2427,
+      "step": 4440
+    },
+    {
+      "epoch": 2.83,
+      "global_step": 4460,
+      "learning_rate": 0.0001,
+      "loss": 0.1672,
+      "step": 4460
+    },
+    {
+      "epoch": 2.85,
+      "global_step": 4480,
+      "learning_rate": 0.0001,
+      "loss": 0.1658,
+      "step": 4480
+    },
+    {
+      "epoch": 2.86,
+      "global_step": 4500,
+      "learning_rate": 0.0001,
+      "loss": 0.2497,
+      "step": 4500
+    },
+    {
+      "epoch": 2.87,
+      "global_step": 4520,
+      "learning_rate": 0.0001,
+      "loss": 0.1743,
+      "step": 4520
+    },
+    {
+      "epoch": 2.88,
+      "global_step": 4540,
+      "learning_rate": 0.0001,
+      "loss": 0.1921,
+      "step": 4540
+    },
+    {
+      "epoch": 2.9,
+      "global_step": 4560,
+      "learning_rate": 0.0001,
+      "loss": 0.2023,
+      "step": 4560
+    },
+    {
+      "epoch": 2.91,
+      "global_step": 4580,
+      "learning_rate": 0.0001,
+      "loss": 0.2288,
+      "step": 4580
+    },
+    {
+      "epoch": 2.92,
+      "global_step": 4600,
+      "learning_rate": 0.0001,
+      "loss": 0.2006,
+      "step": 4600
+    },
+    {
+      "epoch": 2.93,
+      "global_step": 4620,
+      "learning_rate": 0.0001,
+      "loss": 0.2346,
+      "step": 4620
+    },
+    {
+      "epoch": 2.95,
+      "global_step": 4640,
+      "learning_rate": 0.0001,
+      "loss": 0.2218,
+      "step": 4640
+    },
+    {
+      "epoch": 2.96,
+      "global_step": 4660,
+      "learning_rate": 0.0001,
+      "loss": 0.1904,
+      "step": 4660
+    },
+    {
+      "epoch": 2.97,
+      "global_step": 4680,
+      "learning_rate": 0.0001,
+      "loss": 0.1838,
+      "step": 4680
+    },
+    {
+      "epoch": 2.99,
+      "global_step": 4700,
+      "learning_rate": 0.0001,
+      "loss": 0.1881,
+      "step": 4700
+    },
+    {
+      "epoch": 3.0,
+      "global_step": 4720,
+      "learning_rate": 0.0001,
+      "loss": 0.1947,
+      "step": 4720
+    },
+    {
+      "epoch": 3.0,
+      "step": 4722,
+      "total_flos": 1.6390180492835553e+18,
+      "train_loss": 0.4998251678882035,
+      "train_runtime": 78696.4679,
+      "train_samples_per_second": 0.72,
+      "train_steps_per_second": 0.06
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4722,
+  "num_train_epochs": 3,
+  "save_steps": 300,
+  "total_flos": 1.6390180492835553e+18,
+  "trial_name": null,
+  "trial_params": null
+}

ncr_cclue/without_knowledge_select/without_knowledge_select.json

@@ -0,0 +1,31 @@
+{
+    "desc": "不做知识选择，使用NCR+CCLUE数据集，算法选择的关键句训练",
+    "output_dir": "output/ablation_study/option-2/NCR/without_knowledge_select",
+    "model_name_or_path": "/data0/maqi/huggingface_models/TechGPT-7B",
+    "train_file": "/data0/maqi/KGLQA-data/datasets/merge/random_select/without_knowledge_select_instruction/train.jsonl",
+    "num_train_epochs": 3,
+    "per_device_train_batch_size": 6,
+    "gradient_accumulation_steps": 2,
+    "learning_rate": 1e-4,
+    "max_seq_length": 2048,
+    "logging_steps": 20,
+    "save_steps": 300,
+    "save_total_limit": 1,
+    "lr_scheduler_type": "constant_with_warmup",
+    "warmup_ratio": 0.1,
+    "lora_rank": 128,
+    "lora_alpha": 64,
+    "lora_dropout": 0.05,
+
+    "gradient_checkpointing": true,
+    "disable_tqdm": false,
+    "optim": "paged_adamw_32bit",
+    "seed": 318,
+    "fp16": true,
+    "report_to": "tensorboard",
+    "dataloader_num_workers": 10,
+    "save_strategy": "steps",
+    "weight_decay": 0,
+    "max_grad_norm": 0.3,
+    "remove_unused_columns": false
+}