Hugging Face's logo

lapp0
/
rloo_exp_pre-refactor

TensorBoard
Safetensors
Model card Files
xet
 Metrics Community
lapp0 commited on
Commit
c670c1d
·
verified ·
1 Parent(s): 02b57ed

Upload folder using huggingface_hub

Browse files
Files changed (27) hide show
  1. checkpoint-40/README.md +202 -0
  2. checkpoint-40/adapter_config.json +34 -0
  3. checkpoint-40/adapter_model.safetensors +3 -0
  4. checkpoint-40/added_tokens.json +3 -0
  5. checkpoint-40/optimizer.pt +3 -0
  6. checkpoint-40/rng_state.pth +3 -0
  7. checkpoint-40/scheduler.pt +3 -0
  8. checkpoint-40/special_tokens_map.json +30 -0
  9. checkpoint-40/tokenizer.json +0 -0
  10. checkpoint-40/tokenizer.model +3 -0
  11. checkpoint-40/tokenizer_config.json +52 -0
  12. checkpoint-40/trainer_state.json +981 -0
  13. checkpoint-40/training_args.bin +3 -0
  14. checkpoint-48/README.md +202 -0
  15. checkpoint-48/adapter_config.json +34 -0
  16. checkpoint-48/adapter_model.safetensors +3 -0
  17. checkpoint-48/added_tokens.json +3 -0
  18. checkpoint-48/optimizer.pt +3 -0
  19. checkpoint-48/rng_state.pth +3 -0
  20. checkpoint-48/scheduler.pt +3 -0
  21. checkpoint-48/special_tokens_map.json +30 -0
  22. checkpoint-48/tokenizer.json +0 -0
  23. checkpoint-48/tokenizer.model +3 -0
  24. checkpoint-48/tokenizer_config.json +52 -0
  25. checkpoint-48/trainer_state.json +1173 -0
  26. checkpoint-48/training_args.bin +3 -0
  27. runs/May03_18-30-59_2efa04e46d11/events.out.tfevents.1714761060.2efa04e46d11 +2 -2
checkpoint-40/README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ base_model: mistralai/Mistral-7B-Instruct-v0.2
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.10.0
checkpoint-40/adapter_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.2",
5
+ "bias": "none",
6
+ "fan_in_fan_out": false,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layer_replication": null,
10
+ "layers_pattern": null,
11
+ "layers_to_transform": null,
12
+ "loftq_config": {},
13
+ "lora_alpha": 64,
14
+ "lora_dropout": 0,
15
+ "megatron_config": null,
16
+ "megatron_core": "megatron.core",
17
+ "modules_to_save": null,
18
+ "peft_type": "LORA",
19
+ "r": 16,
20
+ "rank_pattern": {},
21
+ "revision": null,
22
+ "target_modules": [
23
+ "down_proj",
24
+ "k_proj",
25
+ "o_proj",
26
+ "v_proj",
27
+ "up_proj",
28
+ "gate_proj",
29
+ "q_proj"
30
+ ],
31
+ "task_type": "CAUSAL_LM",
32
+ "use_dora": false,
33
+ "use_rslora": false
34
+ }
checkpoint-40/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:70044cab9467765af41352e7fd5443ea240d3eadc40c02253a075926bcc45196
3
+ size 167832240
checkpoint-40/added_tokens.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "[PAD]": 32000
3
+ }
checkpoint-40/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9289c8203209cfb2eb8074550b768f63d949e47a1cb7688216b5d8295e490344
3
+ size 335812858
checkpoint-40/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:261c4e49ac26581fa11ec8d056defd925ad6868f506fcb838fdead487c42a1ff
3
+ size 14244
checkpoint-40/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5edd5b0bd0a1e427c19238b01ec197fb6618bc62ad31251d7cae4fcc55b11eee
3
+ size 1064
checkpoint-40/special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "</s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "unk_token": {
24
+ "content": "<unk>",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
checkpoint-40/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-40/tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
3
+ size 493443
checkpoint-40/tokenizer_config.json ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "added_tokens_decoder": {
5
+ "0": {
6
+ "content": "<unk>",
7
+ "lstrip": false,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false,
11
+ "special": true
12
+ },
13
+ "1": {
14
+ "content": "<s>",
15
+ "lstrip": false,
16
+ "normalized": false,
17
+ "rstrip": false,
18
+ "single_word": false,
19
+ "special": true
20
+ },
21
+ "2": {
22
+ "content": "</s>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": false,
26
+ "single_word": false,
27
+ "special": true
28
+ },
29
+ "32000": {
30
+ "content": "[PAD]",
31
+ "lstrip": false,
32
+ "normalized": false,
33
+ "rstrip": false,
34
+ "single_word": false,
35
+ "special": true
36
+ }
37
+ },
38
+ "additional_special_tokens": [],
39
+ "bos_token": "<s>",
40
+ "chat_template": "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}",
41
+ "clean_up_tokenization_spaces": false,
42
+ "eos_token": "</s>",
43
+ "legacy": true,
44
+ "model_max_length": 1000000000000000019884624838656,
45
+ "pad_token": "[PAD]",
46
+ "padding_side": "left",
47
+ "sp_model_kwargs": {},
48
+ "spaces_between_special_tokens": false,
49
+ "tokenizer_class": "LlamaTokenizer",
50
+ "unk_token": "<unk>",
51
+ "use_default_system_prompt": false
52
+ }
checkpoint-40/trainer_state.json ADDED
@@ -0,0 +1,981 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 0.006337512130394312,
5
+ "eval_steps": 500,
6
+ "global_step": 40,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0001584378032598578,
13
+ "grad_norm": 4.8242447519442067e-05,
14
+ "learning_rate": 0.0029998415465061004,
15
+ "loss": 0.0,
16
+ "loss/policy_avg": 7.299240678548813e-08,
17
+ "objective/entropy": 67.43842315673828,
18
+ "objective/kl": 0.0,
19
+ "objective/rlhf_reward": 1.5219240188598633,
20
+ "objective/scores": 1.52203369140625,
21
+ "policy/approxkl_avg": 0.0,
22
+ "policy/clipfrac_avg": 0.0,
23
+ "step": 1,
24
+ "timer/active_model_forward": 0.7594873309135437,
25
+ "timer/calc_advantages": 31.33674430847168,
26
+ "timer/calc_loss": 0.774461030960083,
27
+ "timer/generate": 28.869869232177734,
28
+ "timer/get_reward": 0.5197552442550659,
29
+ "timer/model_forward": 0.7690438628196716,
30
+ "timer/ref_model_forward": 0.7777600884437561,
31
+ "timer/training_step": 34.545597076416016,
32
+ "val/num_eos_tokens": 0.0,
33
+ "val/ratio": 1.0
34
+ },
35
+ {
36
+ "epoch": 0.0003168756065197156,
37
+ "grad_norm": 0.00026670348597690463,
38
+ "learning_rate": 0.002999683093012201,
39
+ "loss": -0.0,
40
+ "loss/policy_avg": -1.7043203115463257e-07,
41
+ "objective/entropy": 118.22715759277344,
42
+ "objective/kl": 0.0,
43
+ "objective/rlhf_reward": -3.824268341064453,
44
+ "objective/scores": -3.82421875,
45
+ "policy/approxkl_avg": 0.0,
46
+ "policy/clipfrac_avg": 0.0,
47
+ "step": 2,
48
+ "timer/active_model_forward": 0.6569761037826538,
49
+ "timer/calc_advantages": 30.551788330078125,
50
+ "timer/calc_loss": 0.6658673286437988,
51
+ "timer/generate": 28.41728973388672,
52
+ "timer/get_reward": 0.43157756328582764,
53
+ "timer/model_forward": 0.6641299724578857,
54
+ "timer/ref_model_forward": 0.67349773645401,
55
+ "timer/training_step": 33.37031936645508,
56
+ "val/num_eos_tokens": 0.0,
57
+ "val/ratio": 1.0
58
+ },
59
+ {
60
+ "epoch": 0.0004753134097795734,
61
+ "grad_norm": 0.00013700829003937542,
62
+ "learning_rate": 0.0029995246395183015,
63
+ "loss": -0.0,
64
+ "loss/policy_avg": -2.9243528842926025e-07,
65
+ "objective/entropy": 1081.7860107421875,
66
+ "objective/kl": 0.0,
67
+ "objective/rlhf_reward": -5.476104736328125,
68
+ "objective/scores": -5.476806640625,
69
+ "policy/approxkl_avg": 0.0,
70
+ "policy/clipfrac_avg": 0.0,
71
+ "step": 3,
72
+ "timer/active_model_forward": 0.9020705223083496,
73
+ "timer/calc_advantages": 32.11817169189453,
74
+ "timer/calc_loss": 0.9118615984916687,
75
+ "timer/generate": 29.300296783447266,
76
+ "timer/get_reward": 0.5959209203720093,
77
+ "timer/model_forward": 0.9100804924964905,
78
+ "timer/ref_model_forward": 0.9205952882766724,
79
+ "timer/training_step": 35.83306121826172,
80
+ "val/num_eos_tokens": 0.0,
81
+ "val/ratio": 1.0
82
+ },
83
+ {
84
+ "epoch": 0.0006337512130394313,
85
+ "grad_norm": 7.692693907301873e-05,
86
+ "learning_rate": 0.002999366186024402,
87
+ "loss": -0.0,
88
+ "loss/policy_avg": -2.0489096641540527e-07,
89
+ "objective/entropy": 1181.2274169921875,
90
+ "objective/kl": 0.0,
91
+ "objective/rlhf_reward": -5.0478363037109375,
92
+ "objective/scores": -5.04833984375,
93
+ "policy/approxkl_avg": 0.0,
94
+ "policy/clipfrac_avg": 0.0,
95
+ "step": 4,
96
+ "timer/active_model_forward": 0.8427364826202393,
97
+ "timer/calc_advantages": 31.744182586669922,
98
+ "timer/calc_loss": 0.8533791899681091,
99
+ "timer/generate": 29.096416473388672,
100
+ "timer/get_reward": 0.5531487464904785,
101
+ "timer/model_forward": 0.851642906665802,
102
+ "timer/ref_model_forward": 0.8617820143699646,
103
+ "timer/training_step": 35.25868225097656,
104
+ "val/num_eos_tokens": 0.0,
105
+ "val/ratio": 1.0
106
+ },
107
+ {
108
+ "epoch": 0.000792189016299289,
109
+ "grad_norm": 3.911158273695037e-05,
110
+ "learning_rate": 0.0029992077325305022,
111
+ "loss": -0.0,
112
+ "loss/policy_avg": -2.1513551473617554e-07,
113
+ "objective/entropy": 1185.438232421875,
114
+ "objective/kl": 0.0,
115
+ "objective/rlhf_reward": -4.198909759521484,
116
+ "objective/scores": -4.198486328125,
117
+ "policy/approxkl_avg": 0.0,
118
+ "policy/clipfrac_avg": 0.0,
119
+ "step": 5,
120
+ "timer/active_model_forward": 0.791987955570221,
121
+ "timer/calc_advantages": 31.324127197265625,
122
+ "timer/calc_loss": 0.8030194044113159,
123
+ "timer/generate": 28.832427978515625,
124
+ "timer/get_reward": 0.5202938914299011,
125
+ "timer/model_forward": 0.8012962937355042,
126
+ "timer/ref_model_forward": 0.8093616366386414,
127
+ "timer/training_step": 34.65721130371094,
128
+ "val/num_eos_tokens": 0.0,
129
+ "val/ratio": 1.0
130
+ },
131
+ {
132
+ "epoch": 0.0009506268195591468,
133
+ "grad_norm": 2.5278823159169406e-05,
134
+ "learning_rate": 0.002999049279036603,
135
+ "loss": -0.0,
136
+ "loss/policy_avg": -1.2619420886039734e-07,
137
+ "objective/entropy": 528.8932495117188,
138
+ "objective/kl": 0.0,
139
+ "objective/rlhf_reward": -3.1182708740234375,
140
+ "objective/scores": -3.1180419921875,
141
+ "policy/approxkl_avg": 0.0,
142
+ "policy/clipfrac_avg": 0.0,
143
+ "step": 6,
144
+ "timer/active_model_forward": 0.7996934652328491,
145
+ "timer/calc_advantages": 31.30746078491211,
146
+ "timer/calc_loss": 0.8107499480247498,
147
+ "timer/generate": 28.788341522216797,
148
+ "timer/get_reward": 0.5261460542678833,
149
+ "timer/model_forward": 0.8090219497680664,
150
+ "timer/ref_model_forward": 0.8171602487564087,
151
+ "timer/training_step": 34.65579605102539,
152
+ "val/num_eos_tokens": 0.0,
153
+ "val/ratio": 1.0
154
+ },
155
+ {
156
+ "epoch": 0.0011090646228190046,
157
+ "grad_norm": 1.0207173772869282e-06,
158
+ "learning_rate": 0.0029988908255427034,
159
+ "loss": -0.0,
160
+ "loss/policy_avg": -3.1781382858753204e-08,
161
+ "objective/entropy": 1.5396817922592163,
162
+ "objective/kl": 0.0,
163
+ "objective/rlhf_reward": -1.2192144393920898,
164
+ "objective/scores": -1.2193603515625,
165
+ "policy/approxkl_avg": 0.0,
166
+ "policy/clipfrac_avg": 0.0,
167
+ "step": 7,
168
+ "timer/active_model_forward": 1.0039308071136475,
169
+ "timer/calc_advantages": 32.610069274902344,
170
+ "timer/calc_loss": 1.0154139995574951,
171
+ "timer/generate": 29.52051544189453,
172
+ "timer/get_reward": 0.6604768633842468,
173
+ "timer/model_forward": 1.0136762857437134,
174
+ "timer/ref_model_forward": 1.0217972993850708,
175
+ "timer/training_step": 36.71036911010742,
176
+ "val/num_eos_tokens": 0.0,
177
+ "val/ratio": 1.0
178
+ },
179
+ {
180
+ "epoch": 0.0012675024260788625,
181
+ "grad_norm": 1.4357982536239433e-06,
182
+ "learning_rate": 0.0029987323720488037,
183
+ "loss": -0.0,
184
+ "loss/policy_avg": -7.380731403827667e-08,
185
+ "objective/entropy": 2.126983404159546,
186
+ "objective/kl": 0.0,
187
+ "objective/rlhf_reward": -1.3588581085205078,
188
+ "objective/scores": -1.35894775390625,
189
+ "policy/approxkl_avg": 0.0,
190
+ "policy/clipfrac_avg": 0.0,
191
+ "step": 8,
192
+ "timer/active_model_forward": 1.0782428979873657,
193
+ "timer/calc_advantages": 33.10707092285156,
194
+ "timer/calc_loss": 1.0924396514892578,
195
+ "timer/generate": 29.794893264770508,
196
+ "timer/get_reward": 0.7108417749404907,
197
+ "timer/model_forward": 1.0906702280044556,
198
+ "timer/ref_model_forward": 1.0941966772079468,
199
+ "timer/training_step": 37.46529769897461,
200
+ "val/num_eos_tokens": 0.0,
201
+ "val/ratio": 1.0
202
+ },
203
+ {
204
+ "epoch": 0.0014259402293387202,
205
+ "grad_norm": 1.58626687607466e-06,
206
+ "learning_rate": 0.002998573918554904,
207
+ "loss": -0.0,
208
+ "loss/policy_avg": -5.851325113326311e-08,
209
+ "objective/entropy": 5.8357086181640625,
210
+ "objective/kl": 0.0,
211
+ "objective/rlhf_reward": -1.3656387329101562,
212
+ "objective/scores": -1.36566162109375,
213
+ "policy/approxkl_avg": 0.0,
214
+ "policy/clipfrac_avg": 0.0,
215
+ "step": 9,
216
+ "timer/active_model_forward": 0.9783480167388916,
217
+ "timer/calc_advantages": 32.70989990234375,
218
+ "timer/calc_loss": 0.9892486333847046,
219
+ "timer/generate": 29.658161163330078,
220
+ "timer/get_reward": 0.6437245011329651,
221
+ "timer/model_forward": 0.9874842166900635,
222
+ "timer/ref_model_forward": 0.9964737892150879,
223
+ "timer/training_step": 36.70344924926758,
224
+ "val/num_eos_tokens": 0.0,
225
+ "val/ratio": 1.0
226
+ },
227
+ {
228
+ "epoch": 0.001584378032598578,
229
+ "grad_norm": 1.8057213537758798e-06,
230
+ "learning_rate": 0.002998415465061005,
231
+ "loss": -0.0,
232
+ "loss/policy_avg": -4.391768015921116e-08,
233
+ "objective/entropy": 37.01044464111328,
234
+ "objective/kl": 0.0,
235
+ "objective/rlhf_reward": -1.1269941329956055,
236
+ "objective/scores": -1.126983642578125,
237
+ "policy/approxkl_avg": 0.0,
238
+ "policy/clipfrac_avg": 0.0,
239
+ "step": 10,
240
+ "timer/active_model_forward": 0.7927595973014832,
241
+ "timer/calc_advantages": 31.501224517822266,
242
+ "timer/calc_loss": 0.8029042482376099,
243
+ "timer/generate": 28.97046661376953,
244
+ "timer/get_reward": 0.5210747718811035,
245
+ "timer/model_forward": 0.8011468052864075,
246
+ "timer/ref_model_forward": 0.8100461363792419,
247
+ "timer/training_step": 34.83835983276367,
248
+ "val/num_eos_tokens": 0.0,
249
+ "val/ratio": 1.0
250
+ },
251
+ {
252
+ "epoch": 0.0017428158358584358,
253
+ "grad_norm": 1.3606485254058498e-06,
254
+ "learning_rate": 0.002998257011567105,
255
+ "loss": -0.0,
256
+ "loss/policy_avg": -6.2282197177410126e-09,
257
+ "objective/entropy": 82.50379943847656,
258
+ "objective/kl": 0.0,
259
+ "objective/rlhf_reward": -0.20352506637573242,
260
+ "objective/scores": -0.20351409912109375,
261
+ "policy/approxkl_avg": 0.0,
262
+ "policy/clipfrac_avg": 0.0,
263
+ "step": 11,
264
+ "timer/active_model_forward": 0.8447611331939697,
265
+ "timer/calc_advantages": 31.68113899230957,
266
+ "timer/calc_loss": 0.8556842803955078,
267
+ "timer/generate": 29.030838012695312,
268
+ "timer/get_reward": 0.5556706786155701,
269
+ "timer/model_forward": 0.853936493396759,
270
+ "timer/ref_model_forward": 0.8623838424682617,
271
+ "timer/training_step": 35.189205169677734,
272
+ "val/num_eos_tokens": 0.0,
273
+ "val/ratio": 1.0
274
+ },
275
+ {
276
+ "epoch": 0.0019012536391182935,
277
+ "grad_norm": 1.5168055824688054e-06,
278
+ "learning_rate": 0.0029980985580732055,
279
+ "loss": -0.0,
280
+ "loss/policy_avg": -5.324545782059431e-08,
281
+ "objective/entropy": 43.37924575805664,
282
+ "objective/kl": 0.0,
283
+ "objective/rlhf_reward": -1.273842453956604,
284
+ "objective/scores": -1.2738037109375,
285
+ "policy/approxkl_avg": 0.0,
286
+ "policy/clipfrac_avg": 0.0,
287
+ "step": 12,
288
+ "timer/active_model_forward": 0.6771436929702759,
289
+ "timer/calc_advantages": 30.566415786743164,
290
+ "timer/calc_loss": 0.6872404217720032,
291
+ "timer/generate": 28.381263732910156,
292
+ "timer/get_reward": 0.4441335201263428,
293
+ "timer/model_forward": 0.6854845881462097,
294
+ "timer/ref_model_forward": 0.6946568489074707,
295
+ "timer/training_step": 33.479454040527344,
296
+ "val/num_eos_tokens": 0.0,
297
+ "val/ratio": 1.0
298
+ },
299
+ {
300
+ "epoch": 0.0020596914423781513,
301
+ "grad_norm": 9.032095817929076e-07,
302
+ "learning_rate": 0.0029979401045793063,
303
+ "loss": -0.0,
304
+ "loss/policy_avg": -5.2401446737349033e-08,
305
+ "objective/entropy": 43.8469123840332,
306
+ "objective/kl": 0.0,
307
+ "objective/rlhf_reward": -1.2306889295578003,
308
+ "objective/scores": -1.230804443359375,
309
+ "policy/approxkl_avg": 0.0,
310
+ "policy/clipfrac_avg": 0.0,
311
+ "step": 13,
312
+ "timer/active_model_forward": 0.8871802091598511,
313
+ "timer/calc_advantages": 31.831775665283203,
314
+ "timer/calc_loss": 0.8963775634765625,
315
+ "timer/generate": 29.063201904296875,
316
+ "timer/get_reward": 0.5812451839447021,
317
+ "timer/model_forward": 0.8946273326873779,
318
+ "timer/ref_model_forward": 0.9052593111991882,
319
+ "timer/training_step": 35.49396514892578,
320
+ "val/num_eos_tokens": 0.0,
321
+ "val/ratio": 1.0
322
+ },
323
+ {
324
+ "epoch": 0.002218129245638009,
325
+ "grad_norm": 1.8163491404266097e-06,
326
+ "learning_rate": 0.0029977816510854067,
327
+ "loss": -0.0,
328
+ "loss/policy_avg": -6.629852578043938e-08,
329
+ "objective/entropy": 46.85642623901367,
330
+ "objective/kl": 0.0,
331
+ "objective/rlhf_reward": -1.4318056106567383,
332
+ "objective/scores": -1.431884765625,
333
+ "policy/approxkl_avg": 0.0,
334
+ "policy/clipfrac_avg": 0.0,
335
+ "step": 14,
336
+ "timer/active_model_forward": 0.7645549774169922,
337
+ "timer/calc_advantages": 31.06265640258789,
338
+ "timer/calc_loss": 0.775362491607666,
339
+ "timer/generate": 28.632740020751953,
340
+ "timer/get_reward": 0.5032914876937866,
341
+ "timer/model_forward": 0.7736347317695618,
342
+ "timer/ref_model_forward": 0.7821637988090515,
343
+ "timer/training_step": 34.282867431640625,
344
+ "val/num_eos_tokens": 0.0,
345
+ "val/ratio": 1.0
346
+ },
347
+ {
348
+ "epoch": 0.002376567048897867,
349
+ "grad_norm": 1.1021771797459223e-06,
350
+ "learning_rate": 0.002997623197591507,
351
+ "loss": -0.0,
352
+ "loss/policy_avg": -4.4819898903369904e-08,
353
+ "objective/entropy": 83.53251647949219,
354
+ "objective/kl": 0.0,
355
+ "objective/rlhf_reward": -0.6715226173400879,
356
+ "objective/scores": -0.671539306640625,
357
+ "policy/approxkl_avg": 0.0,
358
+ "policy/clipfrac_avg": 0.0,
359
+ "step": 15,
360
+ "timer/active_model_forward": 1.1367318630218506,
361
+ "timer/calc_advantages": 33.59859848022461,
362
+ "timer/calc_loss": 1.2175476551055908,
363
+ "timer/generate": 30.091690063476562,
364
+ "timer/get_reward": 0.7473501563072205,
365
+ "timer/model_forward": 1.2155828475952148,
366
+ "timer/ref_model_forward": 1.1532230377197266,
367
+ "timer/training_step": 38.26678466796875,
368
+ "val/num_eos_tokens": 0.0,
369
+ "val/ratio": 1.0
370
+ },
371
+ {
372
+ "epoch": 0.002535004852157725,
373
+ "grad_norm": 1.2331845482549397e-06,
374
+ "learning_rate": 0.0029974647440976073,
375
+ "loss": -0.0,
376
+ "loss/policy_avg": -1.6909325495362282e-08,
377
+ "objective/entropy": 119.47362518310547,
378
+ "objective/kl": 0.0,
379
+ "objective/rlhf_reward": -0.5125551223754883,
380
+ "objective/scores": -0.5126724243164062,
381
+ "policy/approxkl_avg": 0.0,
382
+ "policy/clipfrac_avg": 0.0,
383
+ "step": 16,
384
+ "timer/active_model_forward": 0.9046459197998047,
385
+ "timer/calc_advantages": 32.02373123168945,
386
+ "timer/calc_loss": 0.914932906627655,
387
+ "timer/generate": 29.190690994262695,
388
+ "timer/get_reward": 0.5946121215820312,
389
+ "timer/model_forward": 0.9131810665130615,
390
+ "timer/ref_model_forward": 0.922136127948761,
391
+ "timer/training_step": 35.75535202026367,
392
+ "val/num_eos_tokens": 0.0,
393
+ "val/ratio": 1.0
394
+ },
395
+ {
396
+ "epoch": 0.0026934426554175825,
397
+ "grad_norm": 2.2155286387715023e-06,
398
+ "learning_rate": 0.002997306290603708,
399
+ "loss": -0.0,
400
+ "loss/policy_avg": -4.8712536226958036e-08,
401
+ "objective/entropy": 121.45540618896484,
402
+ "objective/kl": 0.0,
403
+ "objective/rlhf_reward": -1.4430209398269653,
404
+ "objective/scores": -1.443115234375,
405
+ "policy/approxkl_avg": 0.0,
406
+ "policy/clipfrac_avg": 0.0,
407
+ "step": 17,
408
+ "timer/active_model_forward": 0.7191606163978577,
409
+ "timer/calc_advantages": 31.075271606445312,
410
+ "timer/calc_loss": 0.7287229895591736,
411
+ "timer/generate": 28.78363037109375,
412
+ "timer/get_reward": 0.4706084430217743,
413
+ "timer/model_forward": 0.7269375324249268,
414
+ "timer/ref_model_forward": 0.737448513507843,
415
+ "timer/training_step": 34.119632720947266,
416
+ "val/num_eos_tokens": 0.0,
417
+ "val/ratio": 1.0
418
+ },
419
+ {
420
+ "epoch": 0.0028518804586774404,
421
+ "grad_norm": 8.308816177304834e-07,
422
+ "learning_rate": 0.0029971478371098085,
423
+ "loss": -0.0,
424
+ "loss/policy_avg": -1.1129304766654968e-07,
425
+ "objective/entropy": 96.97429656982422,
426
+ "objective/kl": 0.0,
427
+ "objective/rlhf_reward": -2.8110389709472656,
428
+ "objective/scores": -2.811279296875,
429
+ "policy/approxkl_avg": 0.0,
430
+ "policy/clipfrac_avg": 0.0,
431
+ "step": 18,
432
+ "timer/active_model_forward": 0.8409894704818726,
433
+ "timer/calc_advantages": 31.73868179321289,
434
+ "timer/calc_loss": 0.8524103164672852,
435
+ "timer/generate": 29.05769920349121,
436
+ "timer/get_reward": 0.5517525672912598,
437
+ "timer/model_forward": 0.8506196737289429,
438
+ "timer/ref_model_forward": 0.8585314154624939,
439
+ "timer/training_step": 35.25176239013672,
440
+ "val/num_eos_tokens": 0.0,
441
+ "val/ratio": 1.0
442
+ },
443
+ {
444
+ "epoch": 0.0030103182619372983,
445
+ "grad_norm": 3.799691228323354e-07,
446
+ "learning_rate": 0.002996989383615909,
447
+ "loss": -0.0,
448
+ "loss/policy_avg": -9.569339454174042e-08,
449
+ "objective/entropy": 85.54360961914062,
450
+ "objective/kl": 0.0,
451
+ "objective/rlhf_reward": -2.591951370239258,
452
+ "objective/scores": -2.591796875,
453
+ "policy/approxkl_avg": 0.0,
454
+ "policy/clipfrac_avg": 0.0,
455
+ "step": 19,
456
+ "timer/active_model_forward": 0.8125832080841064,
457
+ "timer/calc_advantages": 31.4530029296875,
458
+ "timer/calc_loss": 0.8229714632034302,
459
+ "timer/generate": 28.860492706298828,
460
+ "timer/get_reward": 0.5346251130104065,
461
+ "timer/model_forward": 0.8212224245071411,
462
+ "timer/ref_model_forward": 0.8305649757385254,
463
+ "timer/training_step": 34.838218688964844,
464
+ "val/num_eos_tokens": 0.0,
465
+ "val/ratio": 1.0
466
+ },
467
+ {
468
+ "epoch": 0.003168756065197156,
469
+ "grad_norm": 3.7777222132717725e-06,
470
+ "learning_rate": 0.002996830930122009,
471
+ "loss": -0.0,
472
+ "loss/policy_avg": -7.898779585957527e-08,
473
+ "objective/entropy": 78.87588500976562,
474
+ "objective/kl": 0.0,
475
+ "objective/rlhf_reward": -2.1325230598449707,
476
+ "objective/scores": -2.13275146484375,
477
+ "policy/approxkl_avg": 0.0,
478
+ "policy/clipfrac_avg": 0.0,
479
+ "step": 20,
480
+ "timer/active_model_forward": 1.0649389028549194,
481
+ "timer/calc_advantages": 33.12159729003906,
482
+ "timer/calc_loss": 1.0808004140853882,
483
+ "timer/generate": 29.827653884887695,
484
+ "timer/get_reward": 0.7013706564903259,
485
+ "timer/model_forward": 1.0790588855743408,
486
+ "timer/ref_model_forward": 1.0825612545013428,
487
+ "timer/training_step": 37.4581413269043,
488
+ "val/num_eos_tokens": 0.0,
489
+ "val/ratio": 1.0
490
+ },
491
+ {
492
+ "epoch": 0.0033271938684570138,
493
+ "grad_norm": 2.958485652015952e-07,
494
+ "learning_rate": 0.00299667247662811,
495
+ "loss": -0.0,
496
+ "loss/policy_avg": -6.577465683221817e-08,
497
+ "objective/entropy": 26.52663803100586,
498
+ "objective/kl": 0.0,
499
+ "objective/rlhf_reward": -1.5283641815185547,
500
+ "objective/scores": -1.52838134765625,
501
+ "policy/approxkl_avg": 0.0,
502
+ "policy/clipfrac_avg": 0.0,
503
+ "step": 21,
504
+ "timer/active_model_forward": 0.8408429622650146,
505
+ "timer/calc_advantages": 31.583477020263672,
506
+ "timer/calc_loss": 0.8517074584960938,
507
+ "timer/generate": 28.94709014892578,
508
+ "timer/get_reward": 0.5524436235427856,
509
+ "timer/model_forward": 0.8499797582626343,
510
+ "timer/ref_model_forward": 0.859445333480835,
511
+ "timer/training_step": 35.0816535949707,
512
+ "val/num_eos_tokens": 0.0,
513
+ "val/ratio": 1.0
514
+ },
515
+ {
516
+ "epoch": 0.0034856316717168717,
517
+ "grad_norm": 2.1688055085178348e-07,
518
+ "learning_rate": 0.0029965140231342103,
519
+ "loss": -0.0,
520
+ "loss/policy_avg": -7.275957614183426e-08,
521
+ "objective/entropy": 5.097399711608887,
522
+ "objective/kl": 0.0,
523
+ "objective/rlhf_reward": -1.3907923698425293,
524
+ "objective/scores": -1.390869140625,
525
+ "policy/approxkl_avg": 0.0,
526
+ "policy/clipfrac_avg": 0.0,
527
+ "step": 22,
528
+ "timer/active_model_forward": 0.7821312546730042,
529
+ "timer/calc_advantages": 31.17966079711914,
530
+ "timer/calc_loss": 0.7932944297790527,
531
+ "timer/generate": 28.714157104492188,
532
+ "timer/get_reward": 0.5111904144287109,
533
+ "timer/model_forward": 0.7915641069412231,
534
+ "timer/ref_model_forward": 0.8004368543624878,
535
+ "timer/training_step": 34.46521759033203,
536
+ "val/num_eos_tokens": 0.0,
537
+ "val/ratio": 1.0
538
+ },
539
+ {
540
+ "epoch": 0.0036440694749767296,
541
+ "grad_norm": 1.0821660367810182e-07,
542
+ "learning_rate": 0.0029963555696403106,
543
+ "loss": -0.0,
544
+ "loss/policy_avg": -5.2619725465774536e-08,
545
+ "objective/entropy": 0.7932516932487488,
546
+ "objective/kl": 0.0,
547
+ "objective/rlhf_reward": -1.3286209106445312,
548
+ "objective/scores": -1.3287353515625,
549
+ "policy/approxkl_avg": 0.0,
550
+ "policy/clipfrac_avg": 0.0,
551
+ "step": 23,
552
+ "timer/active_model_forward": 0.7814183235168457,
553
+ "timer/calc_advantages": 31.230453491210938,
554
+ "timer/calc_loss": 0.7906038761138916,
555
+ "timer/generate": 28.76888656616211,
556
+ "timer/get_reward": 0.511726438999176,
557
+ "timer/model_forward": 0.7888429164886475,
558
+ "timer/ref_model_forward": 0.7980221509933472,
559
+ "timer/training_step": 34.51930236816406,
560
+ "val/num_eos_tokens": 0.0,
561
+ "val/ratio": 1.0
562
+ },
563
+ {
564
+ "epoch": 0.003802507278236587,
565
+ "grad_norm": 7.358872267104744e-08,
566
+ "learning_rate": 0.0029961971161464114,
567
+ "loss": -0.0,
568
+ "loss/policy_avg": -4.1211023926734924e-08,
569
+ "objective/entropy": 0.42043036222457886,
570
+ "objective/kl": 0.0,
571
+ "objective/rlhf_reward": -1.3438358306884766,
572
+ "objective/scores": -1.34381103515625,
573
+ "policy/approxkl_avg": 0.0,
574
+ "policy/clipfrac_avg": 0.0,
575
+ "step": 24,
576
+ "timer/active_model_forward": 1.0947436094284058,
577
+ "timer/calc_advantages": 33.27772903442383,
578
+ "timer/calc_loss": 1.1070483922958374,
579
+ "timer/generate": 29.915428161621094,
580
+ "timer/get_reward": 0.7210176587104797,
581
+ "timer/model_forward": 1.1053131818771362,
582
+ "timer/ref_model_forward": 1.1129379272460938,
583
+ "timer/training_step": 37.71331024169922,
584
+ "val/num_eos_tokens": 0.0,
585
+ "val/ratio": 1.0
586
+ },
587
+ {
588
+ "epoch": 0.0039609450814964454,
589
+ "grad_norm": 2.472104299044986e-08,
590
+ "learning_rate": 0.0029960386626525118,
591
+ "loss": -0.0,
592
+ "loss/policy_avg": -5.727633833885193e-08,
593
+ "objective/entropy": 0.012148905545473099,
594
+ "objective/kl": 0.0,
595
+ "objective/rlhf_reward": -0.9438667297363281,
596
+ "objective/scores": -0.943817138671875,
597
+ "policy/approxkl_avg": 0.0,
598
+ "policy/clipfrac_avg": 0.0,
599
+ "step": 25,
600
+ "timer/active_model_forward": 0.8634592890739441,
601
+ "timer/calc_advantages": 31.91621971130371,
602
+ "timer/calc_loss": 0.8720701932907104,
603
+ "timer/generate": 29.20632553100586,
604
+ "timer/get_reward": 0.5676878094673157,
605
+ "timer/model_forward": 0.8703302145004272,
606
+ "timer/ref_model_forward": 0.8802404999732971,
607
+ "timer/training_step": 35.50330352783203,
608
+ "val/num_eos_tokens": 0.0,
609
+ "val/ratio": 1.0
610
+ },
611
+ {
612
+ "epoch": 0.0041193828847563025,
613
+ "grad_norm": 7.710570848473708e-09,
614
+ "learning_rate": 0.002995880209158612,
615
+ "loss": -0.0,
616
+ "loss/policy_avg": -5.185938789509237e-08,
617
+ "objective/entropy": 0.004173352383077145,
618
+ "objective/kl": 0.0,
619
+ "objective/rlhf_reward": -1.0292143821716309,
620
+ "objective/scores": -1.02923583984375,
621
+ "policy/approxkl_avg": 0.0,
622
+ "policy/clipfrac_avg": 0.0,
623
+ "step": 26,
624
+ "timer/active_model_forward": 1.1470246315002441,
625
+ "timer/calc_advantages": 33.62145233154297,
626
+ "timer/calc_loss": 1.1597700119018555,
627
+ "timer/generate": 30.103939056396484,
628
+ "timer/get_reward": 0.7558026313781738,
629
+ "timer/model_forward": 1.1580355167388916,
630
+ "timer/ref_model_forward": 1.1651380062103271,
631
+ "timer/training_step": 38.25925064086914,
632
+ "val/num_eos_tokens": 0.0,
633
+ "val/ratio": 1.0
634
+ },
635
+ {
636
+ "epoch": 0.00427782068801616,
637
+ "grad_norm": 3.3511178276768305e-09,
638
+ "learning_rate": 0.0029957217556647125,
639
+ "loss": -0.0,
640
+ "loss/policy_avg": -5.727633833885193e-08,
641
+ "objective/entropy": 0.0016420490574091673,
642
+ "objective/kl": 0.0,
643
+ "objective/rlhf_reward": -1.0198352336883545,
644
+ "objective/scores": -1.0199356079101562,
645
+ "policy/approxkl_avg": 0.0,
646
+ "policy/clipfrac_avg": 0.0,
647
+ "step": 27,
648
+ "timer/active_model_forward": 0.8805452585220337,
649
+ "timer/calc_advantages": 31.86100196838379,
650
+ "timer/calc_loss": 0.8916431665420532,
651
+ "timer/generate": 29.115583419799805,
652
+ "timer/get_reward": 0.5775084495544434,
653
+ "timer/model_forward": 0.8899136185646057,
654
+ "timer/ref_model_forward": 0.8984355926513672,
655
+ "timer/training_step": 35.50636291503906,
656
+ "val/num_eos_tokens": 0.0,
657
+ "val/ratio": 1.0
658
+ },
659
+ {
660
+ "epoch": 0.004436258491276018,
661
+ "grad_norm": 1.413538508465706e-09,
662
+ "learning_rate": 0.0029955633021708132,
663
+ "loss": -0.0,
664
+ "loss/policy_avg": -5.634501576423645e-08,
665
+ "objective/entropy": 0.0007059703930281103,
666
+ "objective/kl": 0.0,
667
+ "objective/rlhf_reward": -0.7993221282958984,
668
+ "objective/scores": -0.79931640625,
669
+ "policy/approxkl_avg": 0.0,
670
+ "policy/clipfrac_avg": 0.0,
671
+ "step": 28,
672
+ "timer/active_model_forward": 1.1332266330718994,
673
+ "timer/calc_advantages": 33.521759033203125,
674
+ "timer/calc_loss": 1.1450132131576538,
675
+ "timer/generate": 30.048032760620117,
676
+ "timer/get_reward": 0.7485961318016052,
677
+ "timer/model_forward": 1.1432639360427856,
678
+ "timer/ref_model_forward": 1.1509253978729248,
679
+ "timer/training_step": 38.09806442260742,
680
+ "val/num_eos_tokens": 0.0,
681
+ "val/ratio": 1.0
682
+ },
683
+ {
684
+ "epoch": 0.004594696294535876,
685
+ "grad_norm": 5.78209535806451e-10,
686
+ "learning_rate": 0.0029954048486769136,
687
+ "loss": -0.0,
688
+ "loss/policy_avg": -4.7963112592697144e-08,
689
+ "objective/entropy": 0.00033563331817276776,
690
+ "objective/kl": 0.0,
691
+ "objective/rlhf_reward": -0.8303084373474121,
692
+ "objective/scores": -0.8302459716796875,
693
+ "policy/approxkl_avg": 0.0,
694
+ "policy/clipfrac_avg": 0.0,
695
+ "step": 29,
696
+ "timer/active_model_forward": 0.5042444467544556,
697
+ "timer/calc_advantages": 29.47503662109375,
698
+ "timer/calc_loss": 0.5117393136024475,
699
+ "timer/generate": 27.804088592529297,
700
+ "timer/get_reward": 0.32876861095428467,
701
+ "timer/model_forward": 0.5100182294845581,
702
+ "timer/ref_model_forward": 0.5201169848442078,
703
+ "timer/training_step": 31.75719451904297,
704
+ "val/num_eos_tokens": 0.0,
705
+ "val/ratio": 1.0
706
+ },
707
+ {
708
+ "epoch": 0.004753134097795734,
709
+ "grad_norm": 1.9897845870975317e-10,
710
+ "learning_rate": 0.002995246395183014,
711
+ "loss": -0.0,
712
+ "loss/policy_avg": -3.341119736433029e-08,
713
+ "objective/entropy": 0.00016836810391396284,
714
+ "objective/kl": 0.0,
715
+ "objective/rlhf_reward": -1.0440139770507812,
716
+ "objective/scores": -1.0440673828125,
717
+ "policy/approxkl_avg": 0.0,
718
+ "policy/clipfrac_avg": 0.0,
719
+ "step": 30,
720
+ "timer/active_model_forward": 0.8818363547325134,
721
+ "timer/calc_advantages": 31.895645141601562,
722
+ "timer/calc_loss": 0.8967288732528687,
723
+ "timer/generate": 29.129695892333984,
724
+ "timer/get_reward": 0.5798584222793579,
725
+ "timer/model_forward": 0.8949993848800659,
726
+ "timer/ref_model_forward": 0.8993439674377441,
727
+ "timer/training_step": 35.555747985839844,
728
+ "val/num_eos_tokens": 0.0,
729
+ "val/ratio": 1.0
730
+ },
731
+ {
732
+ "epoch": 0.004911571901055592,
733
+ "grad_norm": 1.4891138322425945e-10,
734
+ "learning_rate": 0.0029950879416891143,
735
+ "loss": -0.0,
736
+ "loss/policy_avg": -5.029141902923584e-08,
737
+ "objective/entropy": 8.117404649965465e-05,
738
+ "objective/kl": 0.0,
739
+ "objective/rlhf_reward": -1.2238502502441406,
740
+ "objective/scores": -1.22381591796875,
741
+ "policy/approxkl_avg": 0.0,
742
+ "policy/clipfrac_avg": 0.0,
743
+ "step": 31,
744
+ "timer/active_model_forward": 0.9351563453674316,
745
+ "timer/calc_advantages": 32.229801177978516,
746
+ "timer/calc_loss": 0.946864128112793,
747
+ "timer/generate": 29.3177490234375,
748
+ "timer/get_reward": 0.6167161464691162,
749
+ "timer/model_forward": 0.9451220035552979,
750
+ "timer/ref_model_forward": 0.9523330330848694,
751
+ "timer/training_step": 36.09205627441406,
752
+ "val/num_eos_tokens": 0.0,
753
+ "val/ratio": 1.0
754
+ },
755
+ {
756
+ "epoch": 0.00507000970431545,
757
+ "grad_norm": 8.760232816928948e-11,
758
+ "learning_rate": 0.0029949294881952146,
759
+ "loss": -0.0,
760
+ "loss/policy_avg": -4.237517714500427e-08,
761
+ "objective/entropy": 6.1035148974042386e-05,
762
+ "objective/kl": 0.0,
763
+ "objective/rlhf_reward": -1.0421943664550781,
764
+ "objective/scores": -1.042266845703125,
765
+ "policy/approxkl_avg": 0.0,
766
+ "policy/clipfrac_avg": 0.0,
767
+ "step": 32,
768
+ "timer/active_model_forward": 0.829505205154419,
769
+ "timer/calc_advantages": 31.583553314208984,
770
+ "timer/calc_loss": 0.8393364548683167,
771
+ "timer/generate": 28.969213485717773,
772
+ "timer/get_reward": 0.5460556745529175,
773
+ "timer/model_forward": 0.8376072645187378,
774
+ "timer/ref_model_forward": 0.8457875847816467,
775
+ "timer/training_step": 35.03805160522461,
776
+ "val/num_eos_tokens": 0.0,
777
+ "val/ratio": 1.0
778
+ },
779
+ {
780
+ "epoch": 0.005228447507575307,
781
+ "grad_norm": 5.447125389634877e-11,
782
+ "learning_rate": 0.002994771034701315,
783
+ "loss": -0.0,
784
+ "loss/policy_avg": -4.797766450792551e-08,
785
+ "objective/entropy": 3.0517576306010596e-05,
786
+ "objective/kl": 0.0,
787
+ "objective/rlhf_reward": -1.0386168956756592,
788
+ "objective/scores": -1.03875732421875,
789
+ "policy/approxkl_avg": 0.0,
790
+ "policy/clipfrac_avg": 0.0,
791
+ "step": 33,
792
+ "timer/active_model_forward": 0.7481950521469116,
793
+ "timer/calc_advantages": 31.263355255126953,
794
+ "timer/calc_loss": 0.7592823505401611,
795
+ "timer/generate": 28.858570098876953,
796
+ "timer/get_reward": 0.4890865087509155,
797
+ "timer/model_forward": 0.7575281262397766,
798
+ "timer/ref_model_forward": 0.7657535672187805,
799
+ "timer/training_step": 34.43185806274414,
800
+ "val/num_eos_tokens": 0.0,
801
+ "val/ratio": 1.0
802
+ },
803
+ {
804
+ "epoch": 0.005386885310835165,
805
+ "grad_norm": 3.8278300668048715e-11,
806
+ "learning_rate": 0.0029946125812074153,
807
+ "loss": -0.0,
808
+ "loss/policy_avg": -3.8475263863801956e-08,
809
+ "objective/entropy": 3.0517576306010596e-05,
810
+ "objective/kl": 0.0,
811
+ "objective/rlhf_reward": -1.1492676734924316,
812
+ "objective/scores": -1.14923095703125,
813
+ "policy/approxkl_avg": 0.0,
814
+ "policy/clipfrac_avg": 0.0,
815
+ "step": 34,
816
+ "timer/active_model_forward": 0.9886236190795898,
817
+ "timer/calc_advantages": 32.65524673461914,
818
+ "timer/calc_loss": 1.0020769834518433,
819
+ "timer/generate": 29.594369888305664,
820
+ "timer/get_reward": 0.6510753631591797,
821
+ "timer/model_forward": 1.0003337860107422,
822
+ "timer/ref_model_forward": 1.0066032409667969,
823
+ "timer/training_step": 36.705928802490234,
824
+ "val/num_eos_tokens": 0.0,
825
+ "val/ratio": 1.0
826
+ },
827
+ {
828
+ "epoch": 0.005545323114095023,
829
+ "grad_norm": 3.6770659433971176e-11,
830
+ "learning_rate": 0.002994454127713516,
831
+ "loss": -0.0,
832
+ "loss/policy_avg": -6.775371730327606e-08,
833
+ "objective/entropy": 1.5258788153005298e-05,
834
+ "objective/kl": 0.0,
835
+ "objective/rlhf_reward": -1.0323657989501953,
836
+ "objective/scores": -1.03228759765625,
837
+ "policy/approxkl_avg": 0.0,
838
+ "policy/clipfrac_avg": 0.0,
839
+ "step": 35,
840
+ "timer/active_model_forward": 0.8282456398010254,
841
+ "timer/calc_advantages": 31.575471878051758,
842
+ "timer/calc_loss": 0.8373382687568665,
843
+ "timer/generate": 28.9643611907959,
844
+ "timer/get_reward": 0.542504072189331,
845
+ "timer/model_forward": 0.8355766534805298,
846
+ "timer/ref_model_forward": 0.8458244204521179,
847
+ "timer/training_step": 35.05115509033203,
848
+ "val/num_eos_tokens": 0.0,
849
+ "val/ratio": 1.0
850
+ },
851
+ {
852
+ "epoch": 0.005703760917354881,
853
+ "grad_norm": 1.9524635380685496e-11,
854
+ "learning_rate": 0.0029942956742196165,
855
+ "loss": -0.0,
856
+ "loss/policy_avg": -3.958120942115784e-08,
857
+ "objective/entropy": 1.5258788153005298e-05,
858
+ "objective/kl": 0.0,
859
+ "objective/rlhf_reward": -1.149190902709961,
860
+ "objective/scores": -1.1492919921875,
861
+ "policy/approxkl_avg": 0.0,
862
+ "policy/clipfrac_avg": 0.0,
863
+ "step": 36,
864
+ "timer/active_model_forward": 0.948405385017395,
865
+ "timer/calc_advantages": 32.343502044677734,
866
+ "timer/calc_loss": 0.9663164615631104,
867
+ "timer/generate": 29.394954681396484,
868
+ "timer/get_reward": 0.622231662273407,
869
+ "timer/model_forward": 0.9645813703536987,
870
+ "timer/ref_model_forward": 0.9657047390937805,
871
+ "timer/training_step": 36.252403259277344,
872
+ "val/num_eos_tokens": 0.0,
873
+ "val/ratio": 1.0
874
+ },
875
+ {
876
+ "epoch": 0.005862198720614739,
877
+ "grad_norm": 3.229160688844601e-11,
878
+ "learning_rate": 0.002994137220725717,
879
+ "loss": -0.0,
880
+ "loss/policy_avg": -6.612390279769897e-08,
881
+ "objective/entropy": 1.5258788153005298e-05,
882
+ "objective/kl": 0.0,
883
+ "objective/rlhf_reward": -1.2540512084960938,
884
+ "objective/scores": -1.25396728515625,
885
+ "policy/approxkl_avg": 0.0,
886
+ "policy/clipfrac_avg": 0.0,
887
+ "step": 37,
888
+ "timer/active_model_forward": 1.062816858291626,
889
+ "timer/calc_advantages": 33.06248474121094,
890
+ "timer/calc_loss": 1.074930191040039,
891
+ "timer/generate": 29.784706115722656,
892
+ "timer/get_reward": 0.7008613348007202,
893
+ "timer/model_forward": 1.0731862783432007,
894
+ "timer/ref_model_forward": 1.0801432132720947,
895
+ "timer/training_step": 37.3824348449707,
896
+ "val/num_eos_tokens": 0.0,
897
+ "val/ratio": 1.0
898
+ },
899
+ {
900
+ "epoch": 0.006020636523874597,
901
+ "grad_norm": 7.194660232162642e-12,
902
+ "learning_rate": 0.0029939787672318176,
903
+ "loss": -0.0,
904
+ "loss/policy_avg": -3.9639417082071304e-08,
905
+ "objective/entropy": 3.0249354949773988e-06,
906
+ "objective/kl": 0.0,
907
+ "objective/rlhf_reward": -1.0819144248962402,
908
+ "objective/scores": -1.082000732421875,
909
+ "policy/approxkl_avg": 0.0,
910
+ "policy/clipfrac_avg": 0.0,
911
+ "step": 38,
912
+ "timer/active_model_forward": 0.9271302223205566,
913
+ "timer/calc_advantages": 32.216827392578125,
914
+ "timer/calc_loss": 0.9388477802276611,
915
+ "timer/generate": 29.325376510620117,
916
+ "timer/get_reward": 0.6095032095909119,
917
+ "timer/model_forward": 0.9370960593223572,
918
+ "timer/ref_model_forward": 0.9444317817687988,
919
+ "timer/training_step": 36.04484558105469,
920
+ "val/num_eos_tokens": 0.0,
921
+ "val/ratio": 1.0
922
+ },
923
+ {
924
+ "epoch": 0.006179074327134455,
925
+ "grad_norm": 7.418058355079804e-12,
926
+ "learning_rate": 0.002993820313737918,
927
+ "loss": -0.0,
928
+ "loss/policy_avg": -4.330695446697064e-08,
929
+ "objective/entropy": 0.0,
930
+ "objective/kl": 0.0,
931
+ "objective/rlhf_reward": -1.1705617904663086,
932
+ "objective/scores": -1.170654296875,
933
+ "policy/approxkl_avg": 0.0,
934
+ "policy/clipfrac_avg": 0.0,
935
+ "step": 39,
936
+ "timer/active_model_forward": 0.7922002077102661,
937
+ "timer/calc_advantages": 31.311412811279297,
938
+ "timer/calc_loss": 0.8026509284973145,
939
+ "timer/generate": 28.80554962158203,
940
+ "timer/get_reward": 0.5210146903991699,
941
+ "timer/model_forward": 0.800923764705658,
942
+ "timer/ref_model_forward": 0.8090575933456421,
943
+ "timer/training_step": 34.634342193603516,
944
+ "val/num_eos_tokens": 0.0,
945
+ "val/ratio": 1.0
946
+ },
947
+ {
948
+ "epoch": 0.006337512130394312,
949
+ "grad_norm": 4.4267285150001445e-12,
950
+ "learning_rate": 0.0029936618602440183,
951
+ "loss": -0.0,
952
+ "loss/policy_avg": -3.7136487662792206e-08,
953
+ "objective/entropy": 0.0,
954
+ "objective/kl": 0.0,
955
+ "objective/rlhf_reward": -0.7537899017333984,
956
+ "objective/scores": -0.7537307739257812,
957
+ "policy/approxkl_avg": 0.0,
958
+ "policy/clipfrac_avg": 0.0,
959
+ "step": 40,
960
+ "timer/active_model_forward": 0.7792460322380066,
961
+ "timer/calc_advantages": 31.258018493652344,
962
+ "timer/calc_loss": 0.7891390323638916,
963
+ "timer/generate": 28.808034896850586,
964
+ "timer/get_reward": 0.51292884349823,
965
+ "timer/model_forward": 0.7873988151550293,
966
+ "timer/ref_model_forward": 0.795951247215271,
967
+ "timer/training_step": 34.5465087890625,
968
+ "val/num_eos_tokens": 0.0,
969
+ "val/ratio": 1.0
970
+ }
971
+ ],
972
+ "logging_steps": 1,
973
+ "max_steps": 18933,
974
+ "num_input_tokens_seen": 0,
975
+ "num_train_epochs": 3,
976
+ "save_steps": 8,
977
+ "total_flos": 2.096376480419021e+16,
978
+ "train_batch_size": 4,
979
+ "trial_name": null,
980
+ "trial_params": null
981
+ }
checkpoint-40/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6dee1690a11bb3358d66a6ba11dd88327a96654a51c9007c2a9830994ea29f7c
3
+ size 5240
checkpoint-48/README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ base_model: mistralai/Mistral-7B-Instruct-v0.2
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.10.0
checkpoint-48/adapter_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.2",
5
+ "bias": "none",
6
+ "fan_in_fan_out": false,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layer_replication": null,
10
+ "layers_pattern": null,
11
+ "layers_to_transform": null,
12
+ "loftq_config": {},
13
+ "lora_alpha": 64,
14
+ "lora_dropout": 0,
15
+ "megatron_config": null,
16
+ "megatron_core": "megatron.core",
17
+ "modules_to_save": null,
18
+ "peft_type": "LORA",
19
+ "r": 16,
20
+ "rank_pattern": {},
21
+ "revision": null,
22
+ "target_modules": [
23
+ "down_proj",
24
+ "k_proj",
25
+ "o_proj",
26
+ "v_proj",
27
+ "up_proj",
28
+ "gate_proj",
29
+ "q_proj"
30
+ ],
31
+ "task_type": "CAUSAL_LM",
32
+ "use_dora": false,
33
+ "use_rslora": false
34
+ }
checkpoint-48/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1c80e5f9cca04ea5fbe739f8090f498928754f3842d6bca18c108bb397a42b2b
3
+ size 167832240
checkpoint-48/added_tokens.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "[PAD]": 32000
3
+ }
checkpoint-48/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4a1d33e5acdf2c5425627dd9a582b9cd05f08626afe8e42186ea4fe58886c8cd
3
+ size 335812858
checkpoint-48/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cfde5cfd41e977c0022f4d066ab8406289fb4748063b5b12d17e139d1eee0a32
3
+ size 14244
checkpoint-48/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:96cda3cf4f2905fb5c9fb106d560cd4f955600b95476592c7ead77586b331fb8
3
+ size 1064
checkpoint-48/special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "</s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "unk_token": {
24
+ "content": "<unk>",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
checkpoint-48/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-48/tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
3
+ size 493443
checkpoint-48/tokenizer_config.json ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "added_tokens_decoder": {
5
+ "0": {
6
+ "content": "<unk>",
7
+ "lstrip": false,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false,
11
+ "special": true
12
+ },
13
+ "1": {
14
+ "content": "<s>",
15
+ "lstrip": false,
16
+ "normalized": false,
17
+ "rstrip": false,
18
+ "single_word": false,
19
+ "special": true
20
+ },
21
+ "2": {
22
+ "content": "</s>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": false,
26
+ "single_word": false,
27
+ "special": true
28
+ },
29
+ "32000": {
30
+ "content": "[PAD]",
31
+ "lstrip": false,
32
+ "normalized": false,
33
+ "rstrip": false,
34
+ "single_word": false,
35
+ "special": true
36
+ }
37
+ },
38
+ "additional_special_tokens": [],
39
+ "bos_token": "<s>",
40
+ "chat_template": "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}",
41
+ "clean_up_tokenization_spaces": false,
42
+ "eos_token": "</s>",
43
+ "legacy": true,
44
+ "model_max_length": 1000000000000000019884624838656,
45
+ "pad_token": "[PAD]",
46
+ "padding_side": "left",
47
+ "sp_model_kwargs": {},
48
+ "spaces_between_special_tokens": false,
49
+ "tokenizer_class": "LlamaTokenizer",
50
+ "unk_token": "<unk>",
51
+ "use_default_system_prompt": false
52
+ }
checkpoint-48/trainer_state.json ADDED
@@ -0,0 +1,1173 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 0.007605014556473174,
5
+ "eval_steps": 500,
6
+ "global_step": 48,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0001584378032598578,
13
+ "grad_norm": 4.8242447519442067e-05,
14
+ "learning_rate": 0.0029998415465061004,
15
+ "loss": 0.0,
16
+ "loss/policy_avg": 7.299240678548813e-08,
17
+ "objective/entropy": 67.43842315673828,
18
+ "objective/kl": 0.0,
19
+ "objective/rlhf_reward": 1.5219240188598633,
20
+ "objective/scores": 1.52203369140625,
21
+ "policy/approxkl_avg": 0.0,
22
+ "policy/clipfrac_avg": 0.0,
23
+ "step": 1,
24
+ "timer/active_model_forward": 0.7594873309135437,
25
+ "timer/calc_advantages": 31.33674430847168,
26
+ "timer/calc_loss": 0.774461030960083,
27
+ "timer/generate": 28.869869232177734,
28
+ "timer/get_reward": 0.5197552442550659,
29
+ "timer/model_forward": 0.7690438628196716,
30
+ "timer/ref_model_forward": 0.7777600884437561,
31
+ "timer/training_step": 34.545597076416016,
32
+ "val/num_eos_tokens": 0.0,
33
+ "val/ratio": 1.0
34
+ },
35
+ {
36
+ "epoch": 0.0003168756065197156,
37
+ "grad_norm": 0.00026670348597690463,
38
+ "learning_rate": 0.002999683093012201,
39
+ "loss": -0.0,
40
+ "loss/policy_avg": -1.7043203115463257e-07,
41
+ "objective/entropy": 118.22715759277344,
42
+ "objective/kl": 0.0,
43
+ "objective/rlhf_reward": -3.824268341064453,
44
+ "objective/scores": -3.82421875,
45
+ "policy/approxkl_avg": 0.0,
46
+ "policy/clipfrac_avg": 0.0,
47
+ "step": 2,
48
+ "timer/active_model_forward": 0.6569761037826538,
49
+ "timer/calc_advantages": 30.551788330078125,
50
+ "timer/calc_loss": 0.6658673286437988,
51
+ "timer/generate": 28.41728973388672,
52
+ "timer/get_reward": 0.43157756328582764,
53
+ "timer/model_forward": 0.6641299724578857,
54
+ "timer/ref_model_forward": 0.67349773645401,
55
+ "timer/training_step": 33.37031936645508,
56
+ "val/num_eos_tokens": 0.0,
57
+ "val/ratio": 1.0
58
+ },
59
+ {
60
+ "epoch": 0.0004753134097795734,
61
+ "grad_norm": 0.00013700829003937542,
62
+ "learning_rate": 0.0029995246395183015,
63
+ "loss": -0.0,
64
+ "loss/policy_avg": -2.9243528842926025e-07,
65
+ "objective/entropy": 1081.7860107421875,
66
+ "objective/kl": 0.0,
67
+ "objective/rlhf_reward": -5.476104736328125,
68
+ "objective/scores": -5.476806640625,
69
+ "policy/approxkl_avg": 0.0,
70
+ "policy/clipfrac_avg": 0.0,
71
+ "step": 3,
72
+ "timer/active_model_forward": 0.9020705223083496,
73
+ "timer/calc_advantages": 32.11817169189453,
74
+ "timer/calc_loss": 0.9118615984916687,
75
+ "timer/generate": 29.300296783447266,
76
+ "timer/get_reward": 0.5959209203720093,
77
+ "timer/model_forward": 0.9100804924964905,
78
+ "timer/ref_model_forward": 0.9205952882766724,
79
+ "timer/training_step": 35.83306121826172,
80
+ "val/num_eos_tokens": 0.0,
81
+ "val/ratio": 1.0
82
+ },
83
+ {
84
+ "epoch": 0.0006337512130394313,
85
+ "grad_norm": 7.692693907301873e-05,
86
+ "learning_rate": 0.002999366186024402,
87
+ "loss": -0.0,
88
+ "loss/policy_avg": -2.0489096641540527e-07,
89
+ "objective/entropy": 1181.2274169921875,
90
+ "objective/kl": 0.0,
91
+ "objective/rlhf_reward": -5.0478363037109375,
92
+ "objective/scores": -5.04833984375,
93
+ "policy/approxkl_avg": 0.0,
94
+ "policy/clipfrac_avg": 0.0,
95
+ "step": 4,
96
+ "timer/active_model_forward": 0.8427364826202393,
97
+ "timer/calc_advantages": 31.744182586669922,
98
+ "timer/calc_loss": 0.8533791899681091,
99
+ "timer/generate": 29.096416473388672,
100
+ "timer/get_reward": 0.5531487464904785,
101
+ "timer/model_forward": 0.851642906665802,
102
+ "timer/ref_model_forward": 0.8617820143699646,
103
+ "timer/training_step": 35.25868225097656,
104
+ "val/num_eos_tokens": 0.0,
105
+ "val/ratio": 1.0
106
+ },
107
+ {
108
+ "epoch": 0.000792189016299289,
109
+ "grad_norm": 3.911158273695037e-05,
110
+ "learning_rate": 0.0029992077325305022,
111
+ "loss": -0.0,
112
+ "loss/policy_avg": -2.1513551473617554e-07,
113
+ "objective/entropy": 1185.438232421875,
114
+ "objective/kl": 0.0,
115
+ "objective/rlhf_reward": -4.198909759521484,
116
+ "objective/scores": -4.198486328125,
117
+ "policy/approxkl_avg": 0.0,
118
+ "policy/clipfrac_avg": 0.0,
119
+ "step": 5,
120
+ "timer/active_model_forward": 0.791987955570221,
121
+ "timer/calc_advantages": 31.324127197265625,
122
+ "timer/calc_loss": 0.8030194044113159,
123
+ "timer/generate": 28.832427978515625,
124
+ "timer/get_reward": 0.5202938914299011,
125
+ "timer/model_forward": 0.8012962937355042,
126
+ "timer/ref_model_forward": 0.8093616366386414,
127
+ "timer/training_step": 34.65721130371094,
128
+ "val/num_eos_tokens": 0.0,
129
+ "val/ratio": 1.0
130
+ },
131
+ {
132
+ "epoch": 0.0009506268195591468,
133
+ "grad_norm": 2.5278823159169406e-05,
134
+ "learning_rate": 0.002999049279036603,
135
+ "loss": -0.0,
136
+ "loss/policy_avg": -1.2619420886039734e-07,
137
+ "objective/entropy": 528.8932495117188,
138
+ "objective/kl": 0.0,
139
+ "objective/rlhf_reward": -3.1182708740234375,
140
+ "objective/scores": -3.1180419921875,
141
+ "policy/approxkl_avg": 0.0,
142
+ "policy/clipfrac_avg": 0.0,
143
+ "step": 6,
144
+ "timer/active_model_forward": 0.7996934652328491,
145
+ "timer/calc_advantages": 31.30746078491211,
146
+ "timer/calc_loss": 0.8107499480247498,
147
+ "timer/generate": 28.788341522216797,
148
+ "timer/get_reward": 0.5261460542678833,
149
+ "timer/model_forward": 0.8090219497680664,
150
+ "timer/ref_model_forward": 0.8171602487564087,
151
+ "timer/training_step": 34.65579605102539,
152
+ "val/num_eos_tokens": 0.0,
153
+ "val/ratio": 1.0
154
+ },
155
+ {
156
+ "epoch": 0.0011090646228190046,
157
+ "grad_norm": 1.0207173772869282e-06,
158
+ "learning_rate": 0.0029988908255427034,
159
+ "loss": -0.0,
160
+ "loss/policy_avg": -3.1781382858753204e-08,
161
+ "objective/entropy": 1.5396817922592163,
162
+ "objective/kl": 0.0,
163
+ "objective/rlhf_reward": -1.2192144393920898,
164
+ "objective/scores": -1.2193603515625,
165
+ "policy/approxkl_avg": 0.0,
166
+ "policy/clipfrac_avg": 0.0,
167
+ "step": 7,
168
+ "timer/active_model_forward": 1.0039308071136475,
169
+ "timer/calc_advantages": 32.610069274902344,
170
+ "timer/calc_loss": 1.0154139995574951,
171
+ "timer/generate": 29.52051544189453,
172
+ "timer/get_reward": 0.6604768633842468,
173
+ "timer/model_forward": 1.0136762857437134,
174
+ "timer/ref_model_forward": 1.0217972993850708,
175
+ "timer/training_step": 36.71036911010742,
176
+ "val/num_eos_tokens": 0.0,
177
+ "val/ratio": 1.0
178
+ },
179
+ {
180
+ "epoch": 0.0012675024260788625,
181
+ "grad_norm": 1.4357982536239433e-06,
182
+ "learning_rate": 0.0029987323720488037,
183
+ "loss": -0.0,
184
+ "loss/policy_avg": -7.380731403827667e-08,
185
+ "objective/entropy": 2.126983404159546,
186
+ "objective/kl": 0.0,
187
+ "objective/rlhf_reward": -1.3588581085205078,
188
+ "objective/scores": -1.35894775390625,
189
+ "policy/approxkl_avg": 0.0,
190
+ "policy/clipfrac_avg": 0.0,
191
+ "step": 8,
192
+ "timer/active_model_forward": 1.0782428979873657,
193
+ "timer/calc_advantages": 33.10707092285156,
194
+ "timer/calc_loss": 1.0924396514892578,
195
+ "timer/generate": 29.794893264770508,
196
+ "timer/get_reward": 0.7108417749404907,
197
+ "timer/model_forward": 1.0906702280044556,
198
+ "timer/ref_model_forward": 1.0941966772079468,
199
+ "timer/training_step": 37.46529769897461,
200
+ "val/num_eos_tokens": 0.0,
201
+ "val/ratio": 1.0
202
+ },
203
+ {
204
+ "epoch": 0.0014259402293387202,
205
+ "grad_norm": 1.58626687607466e-06,
206
+ "learning_rate": 0.002998573918554904,
207
+ "loss": -0.0,
208
+ "loss/policy_avg": -5.851325113326311e-08,
209
+ "objective/entropy": 5.8357086181640625,
210
+ "objective/kl": 0.0,
211
+ "objective/rlhf_reward": -1.3656387329101562,
212
+ "objective/scores": -1.36566162109375,
213
+ "policy/approxkl_avg": 0.0,
214
+ "policy/clipfrac_avg": 0.0,
215
+ "step": 9,
216
+ "timer/active_model_forward": 0.9783480167388916,
217
+ "timer/calc_advantages": 32.70989990234375,
218
+ "timer/calc_loss": 0.9892486333847046,
219
+ "timer/generate": 29.658161163330078,
220
+ "timer/get_reward": 0.6437245011329651,
221
+ "timer/model_forward": 0.9874842166900635,
222
+ "timer/ref_model_forward": 0.9964737892150879,
223
+ "timer/training_step": 36.70344924926758,
224
+ "val/num_eos_tokens": 0.0,
225
+ "val/ratio": 1.0
226
+ },
227
+ {
228
+ "epoch": 0.001584378032598578,
229
+ "grad_norm": 1.8057213537758798e-06,
230
+ "learning_rate": 0.002998415465061005,
231
+ "loss": -0.0,
232
+ "loss/policy_avg": -4.391768015921116e-08,
233
+ "objective/entropy": 37.01044464111328,
234
+ "objective/kl": 0.0,
235
+ "objective/rlhf_reward": -1.1269941329956055,
236
+ "objective/scores": -1.126983642578125,
237
+ "policy/approxkl_avg": 0.0,
238
+ "policy/clipfrac_avg": 0.0,
239
+ "step": 10,
240
+ "timer/active_model_forward": 0.7927595973014832,
241
+ "timer/calc_advantages": 31.501224517822266,
242
+ "timer/calc_loss": 0.8029042482376099,
243
+ "timer/generate": 28.97046661376953,
244
+ "timer/get_reward": 0.5210747718811035,
245
+ "timer/model_forward": 0.8011468052864075,
246
+ "timer/ref_model_forward": 0.8100461363792419,
247
+ "timer/training_step": 34.83835983276367,
248
+ "val/num_eos_tokens": 0.0,
249
+ "val/ratio": 1.0
250
+ },
251
+ {
252
+ "epoch": 0.0017428158358584358,
253
+ "grad_norm": 1.3606485254058498e-06,
254
+ "learning_rate": 0.002998257011567105,
255
+ "loss": -0.0,
256
+ "loss/policy_avg": -6.2282197177410126e-09,
257
+ "objective/entropy": 82.50379943847656,
258
+ "objective/kl": 0.0,
259
+ "objective/rlhf_reward": -0.20352506637573242,
260
+ "objective/scores": -0.20351409912109375,
261
+ "policy/approxkl_avg": 0.0,
262
+ "policy/clipfrac_avg": 0.0,
263
+ "step": 11,
264
+ "timer/active_model_forward": 0.8447611331939697,
265
+ "timer/calc_advantages": 31.68113899230957,
266
+ "timer/calc_loss": 0.8556842803955078,
267
+ "timer/generate": 29.030838012695312,
268
+ "timer/get_reward": 0.5556706786155701,
269
+ "timer/model_forward": 0.853936493396759,
270
+ "timer/ref_model_forward": 0.8623838424682617,
271
+ "timer/training_step": 35.189205169677734,
272
+ "val/num_eos_tokens": 0.0,
273
+ "val/ratio": 1.0
274
+ },
275
+ {
276
+ "epoch": 0.0019012536391182935,
277
+ "grad_norm": 1.5168055824688054e-06,
278
+ "learning_rate": 0.0029980985580732055,
279
+ "loss": -0.0,
280
+ "loss/policy_avg": -5.324545782059431e-08,
281
+ "objective/entropy": 43.37924575805664,
282
+ "objective/kl": 0.0,
283
+ "objective/rlhf_reward": -1.273842453956604,
284
+ "objective/scores": -1.2738037109375,
285
+ "policy/approxkl_avg": 0.0,
286
+ "policy/clipfrac_avg": 0.0,
287
+ "step": 12,
288
+ "timer/active_model_forward": 0.6771436929702759,
289
+ "timer/calc_advantages": 30.566415786743164,
290
+ "timer/calc_loss": 0.6872404217720032,
291
+ "timer/generate": 28.381263732910156,
292
+ "timer/get_reward": 0.4441335201263428,
293
+ "timer/model_forward": 0.6854845881462097,
294
+ "timer/ref_model_forward": 0.6946568489074707,
295
+ "timer/training_step": 33.479454040527344,
296
+ "val/num_eos_tokens": 0.0,
297
+ "val/ratio": 1.0
298
+ },
299
+ {
300
+ "epoch": 0.0020596914423781513,
301
+ "grad_norm": 9.032095817929076e-07,
302
+ "learning_rate": 0.0029979401045793063,
303
+ "loss": -0.0,
304
+ "loss/policy_avg": -5.2401446737349033e-08,
305
+ "objective/entropy": 43.8469123840332,
306
+ "objective/kl": 0.0,
307
+ "objective/rlhf_reward": -1.2306889295578003,
308
+ "objective/scores": -1.230804443359375,
309
+ "policy/approxkl_avg": 0.0,
310
+ "policy/clipfrac_avg": 0.0,
311
+ "step": 13,
312
+ "timer/active_model_forward": 0.8871802091598511,
313
+ "timer/calc_advantages": 31.831775665283203,
314
+ "timer/calc_loss": 0.8963775634765625,
315
+ "timer/generate": 29.063201904296875,
316
+ "timer/get_reward": 0.5812451839447021,
317
+ "timer/model_forward": 0.8946273326873779,
318
+ "timer/ref_model_forward": 0.9052593111991882,
319
+ "timer/training_step": 35.49396514892578,
320
+ "val/num_eos_tokens": 0.0,
321
+ "val/ratio": 1.0
322
+ },
323
+ {
324
+ "epoch": 0.002218129245638009,
325
+ "grad_norm": 1.8163491404266097e-06,
326
+ "learning_rate": 0.0029977816510854067,
327
+ "loss": -0.0,
328
+ "loss/policy_avg": -6.629852578043938e-08,
329
+ "objective/entropy": 46.85642623901367,
330
+ "objective/kl": 0.0,
331
+ "objective/rlhf_reward": -1.4318056106567383,
332
+ "objective/scores": -1.431884765625,
333
+ "policy/approxkl_avg": 0.0,
334
+ "policy/clipfrac_avg": 0.0,
335
+ "step": 14,
336
+ "timer/active_model_forward": 0.7645549774169922,
337
+ "timer/calc_advantages": 31.06265640258789,
338
+ "timer/calc_loss": 0.775362491607666,
339
+ "timer/generate": 28.632740020751953,
340
+ "timer/get_reward": 0.5032914876937866,
341
+ "timer/model_forward": 0.7736347317695618,
342
+ "timer/ref_model_forward": 0.7821637988090515,
343
+ "timer/training_step": 34.282867431640625,
344
+ "val/num_eos_tokens": 0.0,
345
+ "val/ratio": 1.0
346
+ },
347
+ {
348
+ "epoch": 0.002376567048897867,
349
+ "grad_norm": 1.1021771797459223e-06,
350
+ "learning_rate": 0.002997623197591507,
351
+ "loss": -0.0,
352
+ "loss/policy_avg": -4.4819898903369904e-08,
353
+ "objective/entropy": 83.53251647949219,
354
+ "objective/kl": 0.0,
355
+ "objective/rlhf_reward": -0.6715226173400879,
356
+ "objective/scores": -0.671539306640625,
357
+ "policy/approxkl_avg": 0.0,
358
+ "policy/clipfrac_avg": 0.0,
359
+ "step": 15,
360
+ "timer/active_model_forward": 1.1367318630218506,
361
+ "timer/calc_advantages": 33.59859848022461,
362
+ "timer/calc_loss": 1.2175476551055908,
363
+ "timer/generate": 30.091690063476562,
364
+ "timer/get_reward": 0.7473501563072205,
365
+ "timer/model_forward": 1.2155828475952148,
366
+ "timer/ref_model_forward": 1.1532230377197266,
367
+ "timer/training_step": 38.26678466796875,
368
+ "val/num_eos_tokens": 0.0,
369
+ "val/ratio": 1.0
370
+ },
371
+ {
372
+ "epoch": 0.002535004852157725,
373
+ "grad_norm": 1.2331845482549397e-06,
374
+ "learning_rate": 0.0029974647440976073,
375
+ "loss": -0.0,
376
+ "loss/policy_avg": -1.6909325495362282e-08,
377
+ "objective/entropy": 119.47362518310547,
378
+ "objective/kl": 0.0,
379
+ "objective/rlhf_reward": -0.5125551223754883,
380
+ "objective/scores": -0.5126724243164062,
381
+ "policy/approxkl_avg": 0.0,
382
+ "policy/clipfrac_avg": 0.0,
383
+ "step": 16,
384
+ "timer/active_model_forward": 0.9046459197998047,
385
+ "timer/calc_advantages": 32.02373123168945,
386
+ "timer/calc_loss": 0.914932906627655,
387
+ "timer/generate": 29.190690994262695,
388
+ "timer/get_reward": 0.5946121215820312,
389
+ "timer/model_forward": 0.9131810665130615,
390
+ "timer/ref_model_forward": 0.922136127948761,
391
+ "timer/training_step": 35.75535202026367,
392
+ "val/num_eos_tokens": 0.0,
393
+ "val/ratio": 1.0
394
+ },
395
+ {
396
+ "epoch": 0.0026934426554175825,
397
+ "grad_norm": 2.2155286387715023e-06,
398
+ "learning_rate": 0.002997306290603708,
399
+ "loss": -0.0,
400
+ "loss/policy_avg": -4.8712536226958036e-08,
401
+ "objective/entropy": 121.45540618896484,
402
+ "objective/kl": 0.0,
403
+ "objective/rlhf_reward": -1.4430209398269653,
404
+ "objective/scores": -1.443115234375,
405
+ "policy/approxkl_avg": 0.0,
406
+ "policy/clipfrac_avg": 0.0,
407
+ "step": 17,
408
+ "timer/active_model_forward": 0.7191606163978577,
409
+ "timer/calc_advantages": 31.075271606445312,
410
+ "timer/calc_loss": 0.7287229895591736,
411
+ "timer/generate": 28.78363037109375,
412
+ "timer/get_reward": 0.4706084430217743,
413
+ "timer/model_forward": 0.7269375324249268,
414
+ "timer/ref_model_forward": 0.737448513507843,
415
+ "timer/training_step": 34.119632720947266,
416
+ "val/num_eos_tokens": 0.0,
417
+ "val/ratio": 1.0
418
+ },
419
+ {
420
+ "epoch": 0.0028518804586774404,
421
+ "grad_norm": 8.308816177304834e-07,
422
+ "learning_rate": 0.0029971478371098085,
423
+ "loss": -0.0,
424
+ "loss/policy_avg": -1.1129304766654968e-07,
425
+ "objective/entropy": 96.97429656982422,
426
+ "objective/kl": 0.0,
427
+ "objective/rlhf_reward": -2.8110389709472656,
428
+ "objective/scores": -2.811279296875,
429
+ "policy/approxkl_avg": 0.0,
430
+ "policy/clipfrac_avg": 0.0,
431
+ "step": 18,
432
+ "timer/active_model_forward": 0.8409894704818726,
433
+ "timer/calc_advantages": 31.73868179321289,
434
+ "timer/calc_loss": 0.8524103164672852,
435
+ "timer/generate": 29.05769920349121,
436
+ "timer/get_reward": 0.5517525672912598,
437
+ "timer/model_forward": 0.8506196737289429,
438
+ "timer/ref_model_forward": 0.8585314154624939,
439
+ "timer/training_step": 35.25176239013672,
440
+ "val/num_eos_tokens": 0.0,
441
+ "val/ratio": 1.0
442
+ },
443
+ {
444
+ "epoch": 0.0030103182619372983,
445
+ "grad_norm": 3.799691228323354e-07,
446
+ "learning_rate": 0.002996989383615909,
447
+ "loss": -0.0,
448
+ "loss/policy_avg": -9.569339454174042e-08,
449
+ "objective/entropy": 85.54360961914062,
450
+ "objective/kl": 0.0,
451
+ "objective/rlhf_reward": -2.591951370239258,
452
+ "objective/scores": -2.591796875,
453
+ "policy/approxkl_avg": 0.0,
454
+ "policy/clipfrac_avg": 0.0,
455
+ "step": 19,
456
+ "timer/active_model_forward": 0.8125832080841064,
457
+ "timer/calc_advantages": 31.4530029296875,
458
+ "timer/calc_loss": 0.8229714632034302,
459
+ "timer/generate": 28.860492706298828,
460
+ "timer/get_reward": 0.5346251130104065,
461
+ "timer/model_forward": 0.8212224245071411,
462
+ "timer/ref_model_forward": 0.8305649757385254,
463
+ "timer/training_step": 34.838218688964844,
464
+ "val/num_eos_tokens": 0.0,
465
+ "val/ratio": 1.0
466
+ },
467
+ {
468
+ "epoch": 0.003168756065197156,
469
+ "grad_norm": 3.7777222132717725e-06,
470
+ "learning_rate": 0.002996830930122009,
471
+ "loss": -0.0,
472
+ "loss/policy_avg": -7.898779585957527e-08,
473
+ "objective/entropy": 78.87588500976562,
474
+ "objective/kl": 0.0,
475
+ "objective/rlhf_reward": -2.1325230598449707,
476
+ "objective/scores": -2.13275146484375,
477
+ "policy/approxkl_avg": 0.0,
478
+ "policy/clipfrac_avg": 0.0,
479
+ "step": 20,
480
+ "timer/active_model_forward": 1.0649389028549194,
481
+ "timer/calc_advantages": 33.12159729003906,
482
+ "timer/calc_loss": 1.0808004140853882,
483
+ "timer/generate": 29.827653884887695,
484
+ "timer/get_reward": 0.7013706564903259,
485
+ "timer/model_forward": 1.0790588855743408,
486
+ "timer/ref_model_forward": 1.0825612545013428,
487
+ "timer/training_step": 37.4581413269043,
488
+ "val/num_eos_tokens": 0.0,
489
+ "val/ratio": 1.0
490
+ },
491
+ {
492
+ "epoch": 0.0033271938684570138,
493
+ "grad_norm": 2.958485652015952e-07,
494
+ "learning_rate": 0.00299667247662811,
495
+ "loss": -0.0,
496
+ "loss/policy_avg": -6.577465683221817e-08,
497
+ "objective/entropy": 26.52663803100586,
498
+ "objective/kl": 0.0,
499
+ "objective/rlhf_reward": -1.5283641815185547,
500
+ "objective/scores": -1.52838134765625,
501
+ "policy/approxkl_avg": 0.0,
502
+ "policy/clipfrac_avg": 0.0,
503
+ "step": 21,
504
+ "timer/active_model_forward": 0.8408429622650146,
505
+ "timer/calc_advantages": 31.583477020263672,
506
+ "timer/calc_loss": 0.8517074584960938,
507
+ "timer/generate": 28.94709014892578,
508
+ "timer/get_reward": 0.5524436235427856,
509
+ "timer/model_forward": 0.8499797582626343,
510
+ "timer/ref_model_forward": 0.859445333480835,
511
+ "timer/training_step": 35.0816535949707,
512
+ "val/num_eos_tokens": 0.0,
513
+ "val/ratio": 1.0
514
+ },
515
+ {
516
+ "epoch": 0.0034856316717168717,
517
+ "grad_norm": 2.1688055085178348e-07,
518
+ "learning_rate": 0.0029965140231342103,
519
+ "loss": -0.0,
520
+ "loss/policy_avg": -7.275957614183426e-08,
521
+ "objective/entropy": 5.097399711608887,
522
+ "objective/kl": 0.0,
523
+ "objective/rlhf_reward": -1.3907923698425293,
524
+ "objective/scores": -1.390869140625,
525
+ "policy/approxkl_avg": 0.0,
526
+ "policy/clipfrac_avg": 0.0,
527
+ "step": 22,
528
+ "timer/active_model_forward": 0.7821312546730042,
529
+ "timer/calc_advantages": 31.17966079711914,
530
+ "timer/calc_loss": 0.7932944297790527,
531
+ "timer/generate": 28.714157104492188,
532
+ "timer/get_reward": 0.5111904144287109,
533
+ "timer/model_forward": 0.7915641069412231,
534
+ "timer/ref_model_forward": 0.8004368543624878,
535
+ "timer/training_step": 34.46521759033203,
536
+ "val/num_eos_tokens": 0.0,
537
+ "val/ratio": 1.0
538
+ },
539
+ {
540
+ "epoch": 0.0036440694749767296,
541
+ "grad_norm": 1.0821660367810182e-07,
542
+ "learning_rate": 0.0029963555696403106,
543
+ "loss": -0.0,
544
+ "loss/policy_avg": -5.2619725465774536e-08,
545
+ "objective/entropy": 0.7932516932487488,
546
+ "objective/kl": 0.0,
547
+ "objective/rlhf_reward": -1.3286209106445312,
548
+ "objective/scores": -1.3287353515625,
549
+ "policy/approxkl_avg": 0.0,
550
+ "policy/clipfrac_avg": 0.0,
551
+ "step": 23,
552
+ "timer/active_model_forward": 0.7814183235168457,
553
+ "timer/calc_advantages": 31.230453491210938,
554
+ "timer/calc_loss": 0.7906038761138916,
555
+ "timer/generate": 28.76888656616211,
556
+ "timer/get_reward": 0.511726438999176,
557
+ "timer/model_forward": 0.7888429164886475,
558
+ "timer/ref_model_forward": 0.7980221509933472,
559
+ "timer/training_step": 34.51930236816406,
560
+ "val/num_eos_tokens": 0.0,
561
+ "val/ratio": 1.0
562
+ },
563
+ {
564
+ "epoch": 0.003802507278236587,
565
+ "grad_norm": 7.358872267104744e-08,
566
+ "learning_rate": 0.0029961971161464114,
567
+ "loss": -0.0,
568
+ "loss/policy_avg": -4.1211023926734924e-08,
569
+ "objective/entropy": 0.42043036222457886,
570
+ "objective/kl": 0.0,
571
+ "objective/rlhf_reward": -1.3438358306884766,
572
+ "objective/scores": -1.34381103515625,
573
+ "policy/approxkl_avg": 0.0,
574
+ "policy/clipfrac_avg": 0.0,
575
+ "step": 24,
576
+ "timer/active_model_forward": 1.0947436094284058,
577
+ "timer/calc_advantages": 33.27772903442383,
578
+ "timer/calc_loss": 1.1070483922958374,
579
+ "timer/generate": 29.915428161621094,
580
+ "timer/get_reward": 0.7210176587104797,
581
+ "timer/model_forward": 1.1053131818771362,
582
+ "timer/ref_model_forward": 1.1129379272460938,
583
+ "timer/training_step": 37.71331024169922,
584
+ "val/num_eos_tokens": 0.0,
585
+ "val/ratio": 1.0
586
+ },
587
+ {
588
+ "epoch": 0.0039609450814964454,
589
+ "grad_norm": 2.472104299044986e-08,
590
+ "learning_rate": 0.0029960386626525118,
591
+ "loss": -0.0,
592
+ "loss/policy_avg": -5.727633833885193e-08,
593
+ "objective/entropy": 0.012148905545473099,
594
+ "objective/kl": 0.0,
595
+ "objective/rlhf_reward": -0.9438667297363281,
596
+ "objective/scores": -0.943817138671875,
597
+ "policy/approxkl_avg": 0.0,
598
+ "policy/clipfrac_avg": 0.0,
599
+ "step": 25,
600
+ "timer/active_model_forward": 0.8634592890739441,
601
+ "timer/calc_advantages": 31.91621971130371,
602
+ "timer/calc_loss": 0.8720701932907104,
603
+ "timer/generate": 29.20632553100586,
604
+ "timer/get_reward": 0.5676878094673157,
605
+ "timer/model_forward": 0.8703302145004272,
606
+ "timer/ref_model_forward": 0.8802404999732971,
607
+ "timer/training_step": 35.50330352783203,
608
+ "val/num_eos_tokens": 0.0,
609
+ "val/ratio": 1.0
610
+ },
611
+ {
612
+ "epoch": 0.0041193828847563025,
613
+ "grad_norm": 7.710570848473708e-09,
614
+ "learning_rate": 0.002995880209158612,
615
+ "loss": -0.0,
616
+ "loss/policy_avg": -5.185938789509237e-08,
617
+ "objective/entropy": 0.004173352383077145,
618
+ "objective/kl": 0.0,
619
+ "objective/rlhf_reward": -1.0292143821716309,
620
+ "objective/scores": -1.02923583984375,
621
+ "policy/approxkl_avg": 0.0,
622
+ "policy/clipfrac_avg": 0.0,
623
+ "step": 26,
624
+ "timer/active_model_forward": 1.1470246315002441,
625
+ "timer/calc_advantages": 33.62145233154297,
626
+ "timer/calc_loss": 1.1597700119018555,
627
+ "timer/generate": 30.103939056396484,
628
+ "timer/get_reward": 0.7558026313781738,
629
+ "timer/model_forward": 1.1580355167388916,
630
+ "timer/ref_model_forward": 1.1651380062103271,
631
+ "timer/training_step": 38.25925064086914,
632
+ "val/num_eos_tokens": 0.0,
633
+ "val/ratio": 1.0
634
+ },
635
+ {
636
+ "epoch": 0.00427782068801616,
637
+ "grad_norm": 3.3511178276768305e-09,
638
+ "learning_rate": 0.0029957217556647125,
639
+ "loss": -0.0,
640
+ "loss/policy_avg": -5.727633833885193e-08,
641
+ "objective/entropy": 0.0016420490574091673,
642
+ "objective/kl": 0.0,
643
+ "objective/rlhf_reward": -1.0198352336883545,
644
+ "objective/scores": -1.0199356079101562,
645
+ "policy/approxkl_avg": 0.0,
646
+ "policy/clipfrac_avg": 0.0,
647
+ "step": 27,
648
+ "timer/active_model_forward": 0.8805452585220337,
649
+ "timer/calc_advantages": 31.86100196838379,
650
+ "timer/calc_loss": 0.8916431665420532,
651
+ "timer/generate": 29.115583419799805,
652
+ "timer/get_reward": 0.5775084495544434,
653
+ "timer/model_forward": 0.8899136185646057,
654
+ "timer/ref_model_forward": 0.8984355926513672,
655
+ "timer/training_step": 35.50636291503906,
656
+ "val/num_eos_tokens": 0.0,
657
+ "val/ratio": 1.0
658
+ },
659
+ {
660
+ "epoch": 0.004436258491276018,
661
+ "grad_norm": 1.413538508465706e-09,
662
+ "learning_rate": 0.0029955633021708132,
663
+ "loss": -0.0,
664
+ "loss/policy_avg": -5.634501576423645e-08,
665
+ "objective/entropy": 0.0007059703930281103,
666
+ "objective/kl": 0.0,
667
+ "objective/rlhf_reward": -0.7993221282958984,
668
+ "objective/scores": -0.79931640625,
669
+ "policy/approxkl_avg": 0.0,
670
+ "policy/clipfrac_avg": 0.0,
671
+ "step": 28,
672
+ "timer/active_model_forward": 1.1332266330718994,
673
+ "timer/calc_advantages": 33.521759033203125,
674
+ "timer/calc_loss": 1.1450132131576538,
675
+ "timer/generate": 30.048032760620117,
676
+ "timer/get_reward": 0.7485961318016052,
677
+ "timer/model_forward": 1.1432639360427856,
678
+ "timer/ref_model_forward": 1.1509253978729248,
679
+ "timer/training_step": 38.09806442260742,
680
+ "val/num_eos_tokens": 0.0,
681
+ "val/ratio": 1.0
682
+ },
683
+ {
684
+ "epoch": 0.004594696294535876,
685
+ "grad_norm": 5.78209535806451e-10,
686
+ "learning_rate": 0.0029954048486769136,
687
+ "loss": -0.0,
688
+ "loss/policy_avg": -4.7963112592697144e-08,
689
+ "objective/entropy": 0.00033563331817276776,
690
+ "objective/kl": 0.0,
691
+ "objective/rlhf_reward": -0.8303084373474121,
692
+ "objective/scores": -0.8302459716796875,
693
+ "policy/approxkl_avg": 0.0,
694
+ "policy/clipfrac_avg": 0.0,
695
+ "step": 29,
696
+ "timer/active_model_forward": 0.5042444467544556,
697
+ "timer/calc_advantages": 29.47503662109375,
698
+ "timer/calc_loss": 0.5117393136024475,
699
+ "timer/generate": 27.804088592529297,
700
+ "timer/get_reward": 0.32876861095428467,
701
+ "timer/model_forward": 0.5100182294845581,
702
+ "timer/ref_model_forward": 0.5201169848442078,
703
+ "timer/training_step": 31.75719451904297,
704
+ "val/num_eos_tokens": 0.0,
705
+ "val/ratio": 1.0
706
+ },
707
+ {
708
+ "epoch": 0.004753134097795734,
709
+ "grad_norm": 1.9897845870975317e-10,
710
+ "learning_rate": 0.002995246395183014,
711
+ "loss": -0.0,
712
+ "loss/policy_avg": -3.341119736433029e-08,
713
+ "objective/entropy": 0.00016836810391396284,
714
+ "objective/kl": 0.0,
715
+ "objective/rlhf_reward": -1.0440139770507812,
716
+ "objective/scores": -1.0440673828125,
717
+ "policy/approxkl_avg": 0.0,
718
+ "policy/clipfrac_avg": 0.0,
719
+ "step": 30,
720
+ "timer/active_model_forward": 0.8818363547325134,
721
+ "timer/calc_advantages": 31.895645141601562,
722
+ "timer/calc_loss": 0.8967288732528687,
723
+ "timer/generate": 29.129695892333984,
724
+ "timer/get_reward": 0.5798584222793579,
725
+ "timer/model_forward": 0.8949993848800659,
726
+ "timer/ref_model_forward": 0.8993439674377441,
727
+ "timer/training_step": 35.555747985839844,
728
+ "val/num_eos_tokens": 0.0,
729
+ "val/ratio": 1.0
730
+ },
731
+ {
732
+ "epoch": 0.004911571901055592,
733
+ "grad_norm": 1.4891138322425945e-10,
734
+ "learning_rate": 0.0029950879416891143,
735
+ "loss": -0.0,
736
+ "loss/policy_avg": -5.029141902923584e-08,
737
+ "objective/entropy": 8.117404649965465e-05,
738
+ "objective/kl": 0.0,
739
+ "objective/rlhf_reward": -1.2238502502441406,
740
+ "objective/scores": -1.22381591796875,
741
+ "policy/approxkl_avg": 0.0,
742
+ "policy/clipfrac_avg": 0.0,
743
+ "step": 31,
744
+ "timer/active_model_forward": 0.9351563453674316,
745
+ "timer/calc_advantages": 32.229801177978516,
746
+ "timer/calc_loss": 0.946864128112793,
747
+ "timer/generate": 29.3177490234375,
748
+ "timer/get_reward": 0.6167161464691162,
749
+ "timer/model_forward": 0.9451220035552979,
750
+ "timer/ref_model_forward": 0.9523330330848694,
751
+ "timer/training_step": 36.09205627441406,
752
+ "val/num_eos_tokens": 0.0,
753
+ "val/ratio": 1.0
754
+ },
755
+ {
756
+ "epoch": 0.00507000970431545,
757
+ "grad_norm": 8.760232816928948e-11,
758
+ "learning_rate": 0.0029949294881952146,
759
+ "loss": -0.0,
760
+ "loss/policy_avg": -4.237517714500427e-08,
761
+ "objective/entropy": 6.1035148974042386e-05,
762
+ "objective/kl": 0.0,
763
+ "objective/rlhf_reward": -1.0421943664550781,
764
+ "objective/scores": -1.042266845703125,
765
+ "policy/approxkl_avg": 0.0,
766
+ "policy/clipfrac_avg": 0.0,
767
+ "step": 32,
768
+ "timer/active_model_forward": 0.829505205154419,
769
+ "timer/calc_advantages": 31.583553314208984,
770
+ "timer/calc_loss": 0.8393364548683167,
771
+ "timer/generate": 28.969213485717773,
772
+ "timer/get_reward": 0.5460556745529175,
773
+ "timer/model_forward": 0.8376072645187378,
774
+ "timer/ref_model_forward": 0.8457875847816467,
775
+ "timer/training_step": 35.03805160522461,
776
+ "val/num_eos_tokens": 0.0,
777
+ "val/ratio": 1.0
778
+ },
779
+ {
780
+ "epoch": 0.005228447507575307,
781
+ "grad_norm": 5.447125389634877e-11,
782
+ "learning_rate": 0.002994771034701315,
783
+ "loss": -0.0,
784
+ "loss/policy_avg": -4.797766450792551e-08,
785
+ "objective/entropy": 3.0517576306010596e-05,
786
+ "objective/kl": 0.0,
787
+ "objective/rlhf_reward": -1.0386168956756592,
788
+ "objective/scores": -1.03875732421875,
789
+ "policy/approxkl_avg": 0.0,
790
+ "policy/clipfrac_avg": 0.0,
791
+ "step": 33,
792
+ "timer/active_model_forward": 0.7481950521469116,
793
+ "timer/calc_advantages": 31.263355255126953,
794
+ "timer/calc_loss": 0.7592823505401611,
795
+ "timer/generate": 28.858570098876953,
796
+ "timer/get_reward": 0.4890865087509155,
797
+ "timer/model_forward": 0.7575281262397766,
798
+ "timer/ref_model_forward": 0.7657535672187805,
799
+ "timer/training_step": 34.43185806274414,
800
+ "val/num_eos_tokens": 0.0,
801
+ "val/ratio": 1.0
802
+ },
803
+ {
804
+ "epoch": 0.005386885310835165,
805
+ "grad_norm": 3.8278300668048715e-11,
806
+ "learning_rate": 0.0029946125812074153,
807
+ "loss": -0.0,
808
+ "loss/policy_avg": -3.8475263863801956e-08,
809
+ "objective/entropy": 3.0517576306010596e-05,
810
+ "objective/kl": 0.0,
811
+ "objective/rlhf_reward": -1.1492676734924316,
812
+ "objective/scores": -1.14923095703125,
813
+ "policy/approxkl_avg": 0.0,
814
+ "policy/clipfrac_avg": 0.0,
815
+ "step": 34,
816
+ "timer/active_model_forward": 0.9886236190795898,
817
+ "timer/calc_advantages": 32.65524673461914,
818
+ "timer/calc_loss": 1.0020769834518433,
819
+ "timer/generate": 29.594369888305664,
820
+ "timer/get_reward": 0.6510753631591797,
821
+ "timer/model_forward": 1.0003337860107422,
822
+ "timer/ref_model_forward": 1.0066032409667969,
823
+ "timer/training_step": 36.705928802490234,
824
+ "val/num_eos_tokens": 0.0,
825
+ "val/ratio": 1.0
826
+ },
827
+ {
828
+ "epoch": 0.005545323114095023,
829
+ "grad_norm": 3.6770659433971176e-11,
830
+ "learning_rate": 0.002994454127713516,
831
+ "loss": -0.0,
832
+ "loss/policy_avg": -6.775371730327606e-08,
833
+ "objective/entropy": 1.5258788153005298e-05,
834
+ "objective/kl": 0.0,
835
+ "objective/rlhf_reward": -1.0323657989501953,
836
+ "objective/scores": -1.03228759765625,
837
+ "policy/approxkl_avg": 0.0,
838
+ "policy/clipfrac_avg": 0.0,
839
+ "step": 35,
840
+ "timer/active_model_forward": 0.8282456398010254,
841
+ "timer/calc_advantages": 31.575471878051758,
842
+ "timer/calc_loss": 0.8373382687568665,
843
+ "timer/generate": 28.9643611907959,
844
+ "timer/get_reward": 0.542504072189331,
845
+ "timer/model_forward": 0.8355766534805298,
846
+ "timer/ref_model_forward": 0.8458244204521179,
847
+ "timer/training_step": 35.05115509033203,
848
+ "val/num_eos_tokens": 0.0,
849
+ "val/ratio": 1.0
850
+ },
851
+ {
852
+ "epoch": 0.005703760917354881,
853
+ "grad_norm": 1.9524635380685496e-11,
854
+ "learning_rate": 0.0029942956742196165,
855
+ "loss": -0.0,
856
+ "loss/policy_avg": -3.958120942115784e-08,
857
+ "objective/entropy": 1.5258788153005298e-05,
858
+ "objective/kl": 0.0,
859
+ "objective/rlhf_reward": -1.149190902709961,
860
+ "objective/scores": -1.1492919921875,
861
+ "policy/approxkl_avg": 0.0,
862
+ "policy/clipfrac_avg": 0.0,
863
+ "step": 36,
864
+ "timer/active_model_forward": 0.948405385017395,
865
+ "timer/calc_advantages": 32.343502044677734,
866
+ "timer/calc_loss": 0.9663164615631104,
867
+ "timer/generate": 29.394954681396484,
868
+ "timer/get_reward": 0.622231662273407,
869
+ "timer/model_forward": 0.9645813703536987,
870
+ "timer/ref_model_forward": 0.9657047390937805,
871
+ "timer/training_step": 36.252403259277344,
872
+ "val/num_eos_tokens": 0.0,
873
+ "val/ratio": 1.0
874
+ },
875
+ {
876
+ "epoch": 0.005862198720614739,
877
+ "grad_norm": 3.229160688844601e-11,
878
+ "learning_rate": 0.002994137220725717,
879
+ "loss": -0.0,
880
+ "loss/policy_avg": -6.612390279769897e-08,
881
+ "objective/entropy": 1.5258788153005298e-05,
882
+ "objective/kl": 0.0,
883
+ "objective/rlhf_reward": -1.2540512084960938,
884
+ "objective/scores": -1.25396728515625,
885
+ "policy/approxkl_avg": 0.0,
886
+ "policy/clipfrac_avg": 0.0,
887
+ "step": 37,
888
+ "timer/active_model_forward": 1.062816858291626,
889
+ "timer/calc_advantages": 33.06248474121094,
890
+ "timer/calc_loss": 1.074930191040039,
891
+ "timer/generate": 29.784706115722656,
892
+ "timer/get_reward": 0.7008613348007202,
893
+ "timer/model_forward": 1.0731862783432007,
894
+ "timer/ref_model_forward": 1.0801432132720947,
895
+ "timer/training_step": 37.3824348449707,
896
+ "val/num_eos_tokens": 0.0,
897
+ "val/ratio": 1.0
898
+ },
899
+ {
900
+ "epoch": 0.006020636523874597,
901
+ "grad_norm": 7.194660232162642e-12,
902
+ "learning_rate": 0.0029939787672318176,
903
+ "loss": -0.0,
904
+ "loss/policy_avg": -3.9639417082071304e-08,
905
+ "objective/entropy": 3.0249354949773988e-06,
906
+ "objective/kl": 0.0,
907
+ "objective/rlhf_reward": -1.0819144248962402,
908
+ "objective/scores": -1.082000732421875,
909
+ "policy/approxkl_avg": 0.0,
910
+ "policy/clipfrac_avg": 0.0,
911
+ "step": 38,
912
+ "timer/active_model_forward": 0.9271302223205566,
913
+ "timer/calc_advantages": 32.216827392578125,
914
+ "timer/calc_loss": 0.9388477802276611,
915
+ "timer/generate": 29.325376510620117,
916
+ "timer/get_reward": 0.6095032095909119,
917
+ "timer/model_forward": 0.9370960593223572,
918
+ "timer/ref_model_forward": 0.9444317817687988,
919
+ "timer/training_step": 36.04484558105469,
920
+ "val/num_eos_tokens": 0.0,
921
+ "val/ratio": 1.0
922
+ },
923
+ {
924
+ "epoch": 0.006179074327134455,
925
+ "grad_norm": 7.418058355079804e-12,
926
+ "learning_rate": 0.002993820313737918,
927
+ "loss": -0.0,
928
+ "loss/policy_avg": -4.330695446697064e-08,
929
+ "objective/entropy": 0.0,
930
+ "objective/kl": 0.0,
931
+ "objective/rlhf_reward": -1.1705617904663086,
932
+ "objective/scores": -1.170654296875,
933
+ "policy/approxkl_avg": 0.0,
934
+ "policy/clipfrac_avg": 0.0,
935
+ "step": 39,
936
+ "timer/active_model_forward": 0.7922002077102661,
937
+ "timer/calc_advantages": 31.311412811279297,
938
+ "timer/calc_loss": 0.8026509284973145,
939
+ "timer/generate": 28.80554962158203,
940
+ "timer/get_reward": 0.5210146903991699,
941
+ "timer/model_forward": 0.800923764705658,
942
+ "timer/ref_model_forward": 0.8090575933456421,
943
+ "timer/training_step": 34.634342193603516,
944
+ "val/num_eos_tokens": 0.0,
945
+ "val/ratio": 1.0
946
+ },
947
+ {
948
+ "epoch": 0.006337512130394312,
949
+ "grad_norm": 4.4267285150001445e-12,
950
+ "learning_rate": 0.0029936618602440183,
951
+ "loss": -0.0,
952
+ "loss/policy_avg": -3.7136487662792206e-08,
953
+ "objective/entropy": 0.0,
954
+ "objective/kl": 0.0,
955
+ "objective/rlhf_reward": -0.7537899017333984,
956
+ "objective/scores": -0.7537307739257812,
957
+ "policy/approxkl_avg": 0.0,
958
+ "policy/clipfrac_avg": 0.0,
959
+ "step": 40,
960
+ "timer/active_model_forward": 0.7792460322380066,
961
+ "timer/calc_advantages": 31.258018493652344,
962
+ "timer/calc_loss": 0.7891390323638916,
963
+ "timer/generate": 28.808034896850586,
964
+ "timer/get_reward": 0.51292884349823,
965
+ "timer/model_forward": 0.7873988151550293,
966
+ "timer/ref_model_forward": 0.795951247215271,
967
+ "timer/training_step": 34.5465087890625,
968
+ "val/num_eos_tokens": 0.0,
969
+ "val/ratio": 1.0
970
+ },
971
+ {
972
+ "epoch": 0.00649594993365417,
973
+ "grad_norm": 7.40804379645299e-12,
974
+ "learning_rate": 0.0029935034067501186,
975
+ "loss": -0.0,
976
+ "loss/policy_avg": -6.239861249923706e-08,
977
+ "objective/entropy": 0.0,
978
+ "objective/kl": 0.0,
979
+ "objective/rlhf_reward": -1.0780410766601562,
980
+ "objective/scores": -1.0780029296875,
981
+ "policy/approxkl_avg": 0.0,
982
+ "policy/clipfrac_avg": 0.0,
983
+ "step": 41,
984
+ "timer/active_model_forward": 0.9782811999320984,
985
+ "timer/calc_advantages": 32.72919845581055,
986
+ "timer/calc_loss": 0.9898173809051514,
987
+ "timer/generate": 29.66815185546875,
988
+ "timer/get_reward": 0.6456098556518555,
989
+ "timer/model_forward": 0.9880228042602539,
990
+ "timer/ref_model_forward": 0.9960311651229858,
991
+ "timer/training_step": 36.747779846191406,
992
+ "val/num_eos_tokens": 0.0,
993
+ "val/ratio": 1.0
994
+ },
995
+ {
996
+ "epoch": 0.0066543877369140275,
997
+ "grad_norm": 4.3237414516783534e-12,
998
+ "learning_rate": 0.0029933449532562194,
999
+ "loss": -0.0,
1000
+ "loss/policy_avg": -5.203764885663986e-08,
1001
+ "objective/entropy": 0.0,
1002
+ "objective/kl": 0.0,
1003
+ "objective/rlhf_reward": -1.0865364074707031,
1004
+ "objective/scores": -1.0865478515625,
1005
+ "policy/approxkl_avg": 0.0,
1006
+ "policy/clipfrac_avg": 0.0,
1007
+ "step": 42,
1008
+ "timer/active_model_forward": 0.9354751110076904,
1009
+ "timer/calc_advantages": 32.381378173828125,
1010
+ "timer/calc_loss": 0.9460563659667969,
1011
+ "timer/generate": 29.42566680908203,
1012
+ "timer/get_reward": 0.6166284680366516,
1013
+ "timer/model_forward": 0.9442933797836304,
1014
+ "timer/ref_model_forward": 0.9522690773010254,
1015
+ "timer/training_step": 36.222293853759766,
1016
+ "val/num_eos_tokens": 0.0,
1017
+ "val/ratio": 1.0
1018
+ },
1019
+ {
1020
+ "epoch": 0.0068128255401738854,
1021
+ "grad_norm": 3.2976431914993887e-12,
1022
+ "learning_rate": 0.0029931864997623197,
1023
+ "loss": -0.0,
1024
+ "loss/policy_avg": -4.0046870708465576e-08,
1025
+ "objective/entropy": 0.0,
1026
+ "objective/kl": 0.0,
1027
+ "objective/rlhf_reward": -0.9958648681640625,
1028
+ "objective/scores": -0.99603271484375,
1029
+ "policy/approxkl_avg": 0.0,
1030
+ "policy/clipfrac_avg": 0.0,
1031
+ "step": 43,
1032
+ "timer/active_model_forward": 0.7636290192604065,
1033
+ "timer/calc_advantages": 31.25667381286621,
1034
+ "timer/calc_loss": 0.7747111916542053,
1035
+ "timer/generate": 28.780200958251953,
1036
+ "timer/get_reward": 0.5010501146316528,
1037
+ "timer/model_forward": 0.7729100584983826,
1038
+ "timer/ref_model_forward": 0.7797021269798279,
1039
+ "timer/training_step": 34.46833038330078,
1040
+ "val/num_eos_tokens": 0.0,
1041
+ "val/ratio": 1.0
1042
+ },
1043
+ {
1044
+ "epoch": 0.006971263343433743,
1045
+ "grad_norm": 5.412355025963267e-12,
1046
+ "learning_rate": 0.00299302804626842,
1047
+ "loss": -0.0,
1048
+ "loss/policy_avg": -7.310882210731506e-08,
1049
+ "objective/entropy": 0.0,
1050
+ "objective/kl": 0.0,
1051
+ "objective/rlhf_reward": -1.306192398071289,
1052
+ "objective/scores": -1.30621337890625,
1053
+ "policy/approxkl_avg": 0.0,
1054
+ "policy/clipfrac_avg": 0.0,
1055
+ "step": 44,
1056
+ "timer/active_model_forward": 0.8105420470237732,
1057
+ "timer/calc_advantages": 31.56263542175293,
1058
+ "timer/calc_loss": 0.8194728493690491,
1059
+ "timer/generate": 28.94143295288086,
1060
+ "timer/get_reward": 0.5309967994689941,
1061
+ "timer/model_forward": 0.8177112340927124,
1062
+ "timer/ref_model_forward": 0.8259291052818298,
1063
+ "timer/training_step": 34.96255111694336,
1064
+ "val/num_eos_tokens": 0.0,
1065
+ "val/ratio": 1.0
1066
+ },
1067
+ {
1068
+ "epoch": 0.007129701146693601,
1069
+ "grad_norm": 3.038823749926256e-12,
1070
+ "learning_rate": 0.0029928695927745204,
1071
+ "loss": -0.0,
1072
+ "loss/policy_avg": -5.2619725465774536e-08,
1073
+ "objective/entropy": 0.0,
1074
+ "objective/kl": 0.0,
1075
+ "objective/rlhf_reward": -1.0578193664550781,
1076
+ "objective/scores": -1.05767822265625,
1077
+ "policy/approxkl_avg": 0.0,
1078
+ "policy/clipfrac_avg": 0.0,
1079
+ "step": 45,
1080
+ "timer/active_model_forward": 1.0397440195083618,
1081
+ "timer/calc_advantages": 33.09510803222656,
1082
+ "timer/calc_loss": 1.0498113632202148,
1083
+ "timer/generate": 29.858869552612305,
1084
+ "timer/get_reward": 0.6861503720283508,
1085
+ "timer/model_forward": 1.0480440855026245,
1086
+ "timer/ref_model_forward": 1.055608868598938,
1087
+ "timer/training_step": 37.31883239746094,
1088
+ "val/num_eos_tokens": 0.0,
1089
+ "val/ratio": 1.0
1090
+ },
1091
+ {
1092
+ "epoch": 0.007288138949953459,
1093
+ "grad_norm": 1.18787076529292e-12,
1094
+ "learning_rate": 0.0029927111392806212,
1095
+ "loss": -0.0,
1096
+ "loss/policy_avg": -2.1420419216156006e-08,
1097
+ "objective/entropy": 0.0,
1098
+ "objective/kl": 0.0,
1099
+ "objective/rlhf_reward": -0.8209381103515625,
1100
+ "objective/scores": -0.82086181640625,
1101
+ "policy/approxkl_avg": 0.0,
1102
+ "policy/clipfrac_avg": 0.0,
1103
+ "step": 46,
1104
+ "timer/active_model_forward": 1.0535329580307007,
1105
+ "timer/calc_advantages": 33.19418716430664,
1106
+ "timer/calc_loss": 1.0623672008514404,
1107
+ "timer/generate": 29.923227310180664,
1108
+ "timer/get_reward": 0.6941207647323608,
1109
+ "timer/model_forward": 1.0605970621109009,
1110
+ "timer/ref_model_forward": 1.070003628730774,
1111
+ "timer/training_step": 37.47746276855469,
1112
+ "val/num_eos_tokens": 0.0,
1113
+ "val/ratio": 1.0
1114
+ },
1115
+ {
1116
+ "epoch": 0.007446576753213317,
1117
+ "grad_norm": 2.7435091958655233e-12,
1118
+ "learning_rate": 0.0029925526857867216,
1119
+ "loss": -0.0,
1120
+ "loss/policy_avg": -5.3085386753082275e-08,
1121
+ "objective/entropy": 0.0,
1122
+ "objective/kl": 0.0,
1123
+ "objective/rlhf_reward": -0.9966888427734375,
1124
+ "objective/scores": -0.9967041015625,
1125
+ "policy/approxkl_avg": 0.0,
1126
+ "policy/clipfrac_avg": 0.0,
1127
+ "step": 47,
1128
+ "timer/active_model_forward": 0.7242672443389893,
1129
+ "timer/calc_advantages": 30.85483741760254,
1130
+ "timer/calc_loss": 0.7333927154541016,
1131
+ "timer/generate": 28.548473358154297,
1132
+ "timer/get_reward": 0.4773656129837036,
1133
+ "timer/model_forward": 0.7316561341285706,
1134
+ "timer/ref_model_forward": 0.7414883971214294,
1135
+ "timer/training_step": 33.913536071777344,
1136
+ "val/num_eos_tokens": 0.0,
1137
+ "val/ratio": 1.0
1138
+ },
1139
+ {
1140
+ "epoch": 0.007605014556473174,
1141
+ "grad_norm": 2.9446627428098004e-12,
1142
+ "learning_rate": 0.002992394232292822,
1143
+ "loss": -0.0,
1144
+ "loss/policy_avg": -5.8673322200775146e-08,
1145
+ "objective/entropy": 0.0,
1146
+ "objective/kl": 0.0,
1147
+ "objective/rlhf_reward": -1.2014045715332031,
1148
+ "objective/scores": -1.20123291015625,
1149
+ "policy/approxkl_avg": 0.0,
1150
+ "policy/clipfrac_avg": 0.0,
1151
+ "step": 48,
1152
+ "timer/active_model_forward": 0.7452950477600098,
1153
+ "timer/calc_advantages": 30.971996307373047,
1154
+ "timer/calc_loss": 0.7579642534255981,
1155
+ "timer/generate": 28.605379104614258,
1156
+ "timer/get_reward": 0.48896369338035583,
1157
+ "timer/model_forward": 0.7562232613563538,
1158
+ "timer/ref_model_forward": 0.7629098892211914,
1159
+ "timer/training_step": 34.12104797363281,
1160
+ "val/num_eos_tokens": 0.0,
1161
+ "val/ratio": 1.0
1162
+ }
1163
+ ],
1164
+ "logging_steps": 1,
1165
+ "max_steps": 18933,
1166
+ "num_input_tokens_seen": 0,
1167
+ "num_train_epochs": 3,
1168
+ "save_steps": 8,
1169
+ "total_flos": 2.519438650707149e+16,
1170
+ "train_batch_size": 4,
1171
+ "trial_name": null,
1172
+ "trial_params": null
1173
+ }
checkpoint-48/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6dee1690a11bb3358d66a6ba11dd88327a96654a51c9007c2a9830994ea29f7c
3
+ size 5240
runs/May03_18-30-59_2efa04e46d11/events.out.tfevents.1714761060.2efa04e46d11 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:90595cfdeb33d56599bf0ecc4c398b1bdad9ff1ce1180df6234674fa0fdb6d51
3
- size 65020
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a21b55444ab6929f286db7fb2951590364b981f5dd7e6595a51359d3a55abff3
3
+ size 66872