Hugging Face's logo

melihcatal
/
codedp-cpt-models-v2

PEFT
Safetensors
code-generation
differential-privacy
continued-pretraining
lora
Model card Files
xet
 Community
melihcatal commited on
Commit
076fd74
·
verified ·
1 Parent(s): 1a99fa7

Add files using upload-large-folder tool

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. llama3-8b/base/adapter/README.md +207 -0
  2. llama3-8b/base/adapter/adapter_config.json +46 -0
  3. llama3-8b/base/audit_results.json +137 -0
  4. llama3-8b/base/canary_meta.json +0 -0
  5. llama3-8b/base/metrics.jsonl +0 -0
  6. llama3-8b/base/resolved_config.yaml +102 -0
  7. llama3-8b/base/summary.json +71 -0
  8. llama3-8b/base/tokenizer/tokenizer_config.json +516 -0
  9. llama3-8b/base/train.log +274 -0
  10. llama3-8b/dp3/adapter/README.md +207 -0
  11. llama3-8b/dp3/adapter/adapter_config.json +46 -0
  12. llama3-8b/dp3/audit_results.json +137 -0
  13. llama3-8b/dp3/canary_meta.json +0 -0
  14. llama3-8b/dp3/metrics.jsonl +0 -0
  15. llama3-8b/dp3/resolved_config.yaml +102 -0
  16. llama3-8b/dp3/summary.json +72 -0
  17. llama3-8b/dp3/tokenizer/tokenizer_config.json +516 -0
  18. llama3-8b/dp3/train.log +276 -0
  19. llama3-8b/dp8/adapter/README.md +207 -0
  20. llama3-8b/dp8/adapter/adapter_config.json +46 -0
  21. llama3-8b/dp8/audit_results.json +137 -0
  22. llama3-8b/dp8/canary_meta.json +0 -0
  23. llama3-8b/dp8/metrics.jsonl +0 -0
  24. llama3-8b/dp8/resolved_config.yaml +102 -0
  25. llama3-8b/dp8/summary.json +72 -0
  26. llama3-8b/dp8/tokenizer/tokenizer_config.json +516 -0
  27. llama3-8b/dp8/train.log +276 -0
  28. llama3-8b/dp8_v2/adapter/README.md +207 -0
  29. llama3-8b/dp8_v2/adapter/adapter_config.json +46 -0
  30. llama3-8b/dp8_v2/audit_results.json +137 -0
  31. llama3-8b/dp8_v2/canary_meta.json +0 -0
  32. llama3-8b/dp8_v2/metrics.jsonl +0 -0
  33. llama3-8b/dp8_v2/resolved_config.yaml +102 -0
  34. llama3-8b/dp8_v2/summary.json +72 -0
  35. llama3-8b/dp8_v2/tokenizer/special_tokens_map.json +3519 -0
  36. llama3-8b/dp8_v2/tokenizer/tokenizer_config.json +0 -0
  37. llama3-8b/dp8_v2/train.log +441 -0
  38. qwen3-8b-base/base/adapter/README.md +207 -0
  39. qwen3-8b-base/base/adapter/adapter_config.json +46 -0
  40. qwen3-8b-base/base/tokenizer/chat_template.jinja +85 -0
  41. qwen3-8b-base/base/tokenizer/tokenizer_config.json +516 -0
  42. qwen3-8b-base/dp3/adapter/README.md +207 -0
  43. qwen3-8b-base/dp3/adapter/adapter_config.json +46 -0
  44. qwen3-8b-base/dp3/tokenizer/chat_template.jinja +85 -0
  45. qwen3-8b-base/dp3/tokenizer/tokenizer_config.json +516 -0
  46. qwen3-8b-base/dp3_v2/adapter/README.md +207 -0
  47. qwen3-8b-base/dp3_v2/adapter/adapter_config.json +46 -0
  48. qwen3-8b-base/dp3_v2/audit_results.json +137 -0
  49. qwen3-8b-base/dp3_v2/canary_meta.json +0 -0
  50. qwen3-8b-base/dp3_v2/metrics.jsonl +0 -0
llama3-8b/base/adapter/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: meta-llama/Meta-Llama-3-8B
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:meta-llama/Meta-Llama-3-8B
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.18.1
llama3-8b/base/adapter/adapter_config.json ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": true,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 32,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "megatron_config": null,
23
+ "megatron_core": "megatron.core",
24
+ "modules_to_save": [
25
+ "lm_head",
26
+ "embed_tokens"
27
+ ],
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.18.1",
30
+ "qalora_group_size": 16,
31
+ "r": 16,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "q_proj",
36
+ "v_proj",
37
+ "o_proj",
38
+ "k_proj"
39
+ ],
40
+ "target_parameters": null,
41
+ "task_type": "CAUSAL_LM",
42
+ "trainable_token_indices": null,
43
+ "use_dora": false,
44
+ "use_qalora": false,
45
+ "use_rslora": false
46
+ }
llama3-8b/base/audit_results.json ADDED
@@ -0,0 +1,137 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "delta": 1e-05,
3
+ "num_canaries": 500,
4
+ "num_members": 250,
5
+ "paper_guess_fraction": 0.2,
6
+ "paper_guess_steps": 20,
7
+ "loss": {
8
+ "auc": 1.0,
9
+ "empirical_epsilon": {
10
+ "0.05": 3.4791953936219215,
11
+ "0.01": 3.023197554051876
12
+ },
13
+ "empirical_epsilon_details": {
14
+ "0.05": {
15
+ "epsilon": 3.4791953936219215,
16
+ "num_guesses": 100,
17
+ "correct_guesses": 100,
18
+ "candidate_num_guesses": [
19
+ 5,
20
+ 10,
21
+ 15,
22
+ 20,
23
+ 25,
24
+ 30,
25
+ 35,
26
+ 40,
27
+ 45,
28
+ 50,
29
+ 55,
30
+ 60,
31
+ 65,
32
+ 70,
33
+ 75,
34
+ 80,
35
+ 85,
36
+ 90,
37
+ 95,
38
+ 100
39
+ ],
40
+ "direction": "lower"
41
+ },
42
+ "0.01": {
43
+ "epsilon": 3.023197554051876,
44
+ "num_guesses": 100,
45
+ "correct_guesses": 100,
46
+ "candidate_num_guesses": [
47
+ 5,
48
+ 10,
49
+ 15,
50
+ 20,
51
+ 25,
52
+ 30,
53
+ 35,
54
+ 40,
55
+ 45,
56
+ 50,
57
+ 55,
58
+ 60,
59
+ 65,
60
+ 70,
61
+ 75,
62
+ 80,
63
+ 85,
64
+ 90,
65
+ 95,
66
+ 100
67
+ ],
68
+ "direction": "lower"
69
+ }
70
+ }
71
+ },
72
+ "embedding": {
73
+ "auc": 0.558,
74
+ "empirical_epsilon": {
75
+ "0.05": 3.4791953936219215,
76
+ "0.01": 3.023197554051876
77
+ },
78
+ "empirical_epsilon_details": {
79
+ "0.05": {
80
+ "epsilon": 3.4791953936219215,
81
+ "num_guesses": 100,
82
+ "correct_guesses": 100,
83
+ "candidate_num_guesses": [
84
+ 5,
85
+ 10,
86
+ 15,
87
+ 20,
88
+ 25,
89
+ 30,
90
+ 35,
91
+ 40,
92
+ 45,
93
+ 50,
94
+ 55,
95
+ 60,
96
+ 65,
97
+ 70,
98
+ 75,
99
+ 80,
100
+ 85,
101
+ 90,
102
+ 95,
103
+ 100
104
+ ],
105
+ "direction": "lower"
106
+ },
107
+ "0.01": {
108
+ "epsilon": 3.023197554051876,
109
+ "num_guesses": 100,
110
+ "correct_guesses": 100,
111
+ "candidate_num_guesses": [
112
+ 5,
113
+ 10,
114
+ 15,
115
+ 20,
116
+ 25,
117
+ 30,
118
+ 35,
119
+ 40,
120
+ 45,
121
+ 50,
122
+ 55,
123
+ 60,
124
+ 65,
125
+ 70,
126
+ 75,
127
+ 80,
128
+ 85,
129
+ 90,
130
+ 95,
131
+ 100
132
+ ],
133
+ "direction": "lower"
134
+ }
135
+ }
136
+ }
137
+ }
llama3-8b/base/canary_meta.json ADDED
The diff for this file is too large to render. See raw diff
 
llama3-8b/base/metrics.jsonl ADDED
The diff for this file is too large to render. See raw diff
 
llama3-8b/base/resolved_config.yaml ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ model:
2
+ name: meta-llama/Meta-Llama-3-8B
3
+ tokenizer_name: meta-llama/Meta-Llama-3-8B
4
+ max_length: 1024
5
+ dtype: bfloat16
6
+ trust_remote_code: true
7
+ use_fast_tokenizer: true
8
+ cache_dir: null
9
+ local_files_only: false
10
+ low_cpu_mem_usage: true
11
+ tie_word_embeddings: true
12
+ gradient_checkpointing: false
13
+ use_chat_template: false
14
+ dataset:
15
+ name: melihcatal/codedp-cpt
16
+ split: train
17
+ mode: cpt
18
+ text_column: text
19
+ validation_ratio: 0.05
20
+ max_samples: -1
21
+ lora:
22
+ enabled: true
23
+ r: 16
24
+ alpha: 32
25
+ dropout: 0.05
26
+ target_modules:
27
+ - q_proj
28
+ - k_proj
29
+ - v_proj
30
+ - o_proj
31
+ modules_to_save:
32
+ - lm_head
33
+ bias: none
34
+ training:
35
+ seed: 42
36
+ epochs: 5
37
+ warmup_steps: null
38
+ warmup_ratio: 0.05
39
+ mixed_precision: false
40
+ mixed_precision_dtype: bfloat16
41
+ batch_size: 4
42
+ eval_batch_size: 8
43
+ eval_every_steps: 10
44
+ eval_every_epochs: 1
45
+ learning_rate: 0.0005
46
+ optimizer: adamw
47
+ lr_scheduler: cosine
48
+ adam_beta1: 0.9
49
+ adam_beta2: 0.999
50
+ adam_epsilon: 1.0e-08
51
+ sgd_momentum: 0.9
52
+ weight_decay: 0.01
53
+ max_grad_norm: 1.0
54
+ log_every: 5
55
+ gradient_accumulation_steps: 8
56
+ num_workers: 4
57
+ output_dir: /scratch/mcatal/runs/cpt/llama3-8b/base
58
+ min_lr_ratio: 0.15
59
+ distributed:
60
+ strategy: dpddp
61
+ backend: nccl
62
+ devices: null
63
+ dp:
64
+ module_validator: auto
65
+ target_delta: 1.0e-05
66
+ noise_multiplier: null
67
+ max_grad_norm: 1.0
68
+ grad_sample_mode: hooks
69
+ clipping: flat
70
+ secure_mode: false
71
+ enabled: false
72
+ target_epsilon: 8.0
73
+ audit:
74
+ enabled: true
75
+ run_every_epoch: true
76
+ epoch_device: cuda
77
+ q_canary: auto
78
+ num_canaries: 500
79
+ prefix_length: 49
80
+ num_digits: 12
81
+ batch_size: 32
82
+ delta: 1.0e-05
83
+ p_values:
84
+ - 0.05
85
+ - 0.01
86
+ paper_guess_fraction: 0.2
87
+ paper_guess_steps: 20
88
+ enable_holdout_empirical_epsilon: false
89
+ holdout_seed: 42
90
+ tie_seed: 42
91
+ tracking:
92
+ enabled: true
93
+ tensorboard: true
94
+ wandb: false
95
+ wandb_project: codedp-finetune-h200-audit
96
+ wandb_run_name: llama3-8b-cpt-base
97
+ wandb_mode: online
98
+ codecarbon: true
99
+ codecarbon_output_file: codecarbon.csv
100
+ codecarbon_measure_power_secs: 15
101
+ codecarbon_country_iso_code: null
102
+ codecarbon_project_name: codedp-llama3-8b-cpt-base
llama3-8b/base/summary.json ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "audit/delta": 1e-05,
3
+ "audit/embedding/auc": 0.558,
4
+ "audit/embedding/empirical_epsilon/0.01": 3.023197554051876,
5
+ "audit/embedding/empirical_epsilon/0.05": 3.4791953936219215,
6
+ "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 100.0,
7
+ "audit/embedding/empirical_epsilon_details/0.01/epsilon": 3.023197554051876,
8
+ "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 100.0,
9
+ "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 100.0,
10
+ "audit/embedding/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215,
11
+ "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 100.0,
12
+ "audit/loss/auc": 1.0,
13
+ "audit/loss/empirical_epsilon/0.01": 3.023197554051876,
14
+ "audit/loss/empirical_epsilon/0.05": 3.4791953936219215,
15
+ "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 100.0,
16
+ "audit/loss/empirical_epsilon_details/0.01/epsilon": 3.023197554051876,
17
+ "audit/loss/empirical_epsilon_details/0.01/num_guesses": 100.0,
18
+ "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 100.0,
19
+ "audit/loss/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215,
20
+ "audit/loss/empirical_epsilon_details/0.05/num_guesses": 100.0,
21
+ "audit/num_canaries": 500.0,
22
+ "audit/num_members": 250.0,
23
+ "audit/paper_guess_fraction": 0.2,
24
+ "audit/paper_guess_steps": 20.0,
25
+ "energy/codecarbon/cpu_count": 16.0,
26
+ "energy/codecarbon/cpu_energy": 0.1725324815965796,
27
+ "energy/codecarbon/cpu_power": 80.03188452660181,
28
+ "energy/codecarbon/cpu_utilization_percent": 2.938469209673398,
29
+ "energy/codecarbon/duration": 8064.424946603831,
30
+ "energy/codecarbon/emissions": 0.28934452755853146,
31
+ "energy/codecarbon/emissions_rate": 3.587912708895419e-05,
32
+ "energy/codecarbon/energy_consumed": 8.304236935927772,
33
+ "energy/codecarbon/gpu_count": 8.0,
34
+ "energy/codecarbon/gpu_energy": 8.049784840656088,
35
+ "energy/codecarbon/gpu_power": 3604.954522918001,
36
+ "energy/codecarbon/gpu_utilization_percent": 96.5048466525371,
37
+ "energy/codecarbon/latitude": 47.4843,
38
+ "energy/codecarbon/longitude": 8.212,
39
+ "energy/codecarbon/pue": 1.0,
40
+ "energy/codecarbon/ram_energy": 0.08191961367510629,
41
+ "energy/codecarbon/ram_power": 38.0,
42
+ "energy/codecarbon/ram_total_size": 128.0,
43
+ "energy/codecarbon/ram_used_gb": 508.35038226395585,
44
+ "energy/codecarbon/ram_utilization_percent": 25.6645599601097,
45
+ "energy/codecarbon/water_consumed": 0.0,
46
+ "energy/codecarbon/wue": 0.0,
47
+ "eval/duration_sec": 14.103881032206118,
48
+ "eval/loss": 0.8460299762395711,
49
+ "perf/audit_duration_sec": 9.559748985804617,
50
+ "perf/epoch_duration_sec": 1547.8333585211076,
51
+ "perf/epoch_samples": 47252.0,
52
+ "perf/epoch_samples_per_sec": 30.527834110738755,
53
+ "perf/epoch_tokens": 37200885.0,
54
+ "perf/epoch_tokens_per_sec": 24034.166724216324,
55
+ "perf/gradient_accumulation_steps": 8.0,
56
+ "perf/logical_batch_size": 32.0,
57
+ "perf/logical_token_count": 24452.0,
58
+ "perf/samples_per_sec": 4.475212891107686,
59
+ "perf/step_duration_sec": 7.150497815106064,
60
+ "perf/tokens_per_sec": 3419.6220504176604,
61
+ "system/cuda_epoch_peak_memory_gb": 73.35145711898804,
62
+ "system/cuda_max_memory_allocated_gb": 73.35145711898804,
63
+ "system/cuda_memory_allocated_gb": 23.090433597564697,
64
+ "train/epoch_canary_loss": 0.07487519531313218,
65
+ "train/epoch_loss": 0.39902570595419656,
66
+ "train/epoch_real_loss": 0.4018190494112548,
67
+ "train/lr": 7.5e-05,
68
+ "train/step_canary_loss": 2.384185791015625e-07,
69
+ "train/step_loss": 0.3472383450716734,
70
+ "train/step_real_loss": 0.3472383450716734
71
+ }
llama3-8b/base/tokenizer/tokenizer_config.json ADDED
@@ -0,0 +1,516 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "bos_token": "<|begin_of_text|>",
4
+ "clean_up_tokenization_spaces": true,
5
+ "eos_token": "<|end_of_text|>",
6
+ "extra_special_tokens": {
7
+ "extra_special_token_0": "865331112869",
8
+ "extra_special_token_1": "569765693871",
9
+ "extra_special_token_2": "485177821815",
10
+ "extra_special_token_3": "135441121756",
11
+ "extra_special_token_4": "367459894796",
12
+ "extra_special_token_5": "877482678543",
13
+ "extra_special_token_6": "457919547633",
14
+ "extra_special_token_7": "765474393376",
15
+ "extra_special_token_8": "114848338811",
16
+ "extra_special_token_9": "746285987371",
17
+ "extra_special_token_10": "649291669397",
18
+ "extra_special_token_11": "927914615679",
19
+ "extra_special_token_12": "445925149649",
20
+ "extra_special_token_13": "691587454538",
21
+ "extra_special_token_14": "143777992227",
22
+ "extra_special_token_15": "997981281989",
23
+ "extra_special_token_16": "425949483533",
24
+ "extra_special_token_17": "982993456429",
25
+ "extra_special_token_18": "718726519731",
26
+ "extra_special_token_19": "172599315861",
27
+ "extra_special_token_20": "643489267333",
28
+ "extra_special_token_21": "282322838685",
29
+ "extra_special_token_22": "781653545886",
30
+ "extra_special_token_23": "796415361892",
31
+ "extra_special_token_24": "841991688488",
32
+ "extra_special_token_25": "211411365397",
33
+ "extra_special_token_26": "698218415444",
34
+ "extra_special_token_27": "355977139358",
35
+ "extra_special_token_28": "682564697312",
36
+ "extra_special_token_29": "383837596997",
37
+ "extra_special_token_30": "689362171782",
38
+ "extra_special_token_31": "749966767285",
39
+ "extra_special_token_32": "753159165157",
40
+ "extra_special_token_33": "795693824762",
41
+ "extra_special_token_34": "669689115557",
42
+ "extra_special_token_35": "327491773134",
43
+ "extra_special_token_36": "983569279932",
44
+ "extra_special_token_37": "612128769512",
45
+ "extra_special_token_38": "374327157578",
46
+ "extra_special_token_39": "311632789559",
47
+ "extra_special_token_40": "523918658846",
48
+ "extra_special_token_41": "765981581453",
49
+ "extra_special_token_42": "794825141891",
50
+ "extra_special_token_43": "873898736873",
51
+ "extra_special_token_44": "447445629421",
52
+ "extra_special_token_45": "473822473819",
53
+ "extra_special_token_46": "181439694557",
54
+ "extra_special_token_47": "592538279337",
55
+ "extra_special_token_48": "668134915514",
56
+ "extra_special_token_49": "643692393748",
57
+ "extra_special_token_50": "696651276628",
58
+ "extra_special_token_51": "853859348234",
59
+ "extra_special_token_52": "778466723723",
60
+ "extra_special_token_53": "929826356991",
61
+ "extra_special_token_54": "272362973463",
62
+ "extra_special_token_55": "694235616268",
63
+ "extra_special_token_56": "281673864127",
64
+ "extra_special_token_57": "479676316326",
65
+ "extra_special_token_58": "646979124677",
66
+ "extra_special_token_59": "922327493433",
67
+ "extra_special_token_60": "883685933161",
68
+ "extra_special_token_61": "264259917554",
69
+ "extra_special_token_62": "836746273134",
70
+ "extra_special_token_63": "658481324922",
71
+ "extra_special_token_64": "481884157827",
72
+ "extra_special_token_65": "587787496812",
73
+ "extra_special_token_66": "579184949249",
74
+ "extra_special_token_67": "912193598348",
75
+ "extra_special_token_68": "529679678956",
76
+ "extra_special_token_69": "795838284624",
77
+ "extra_special_token_70": "159337222655",
78
+ "extra_special_token_71": "173781362446",
79
+ "extra_special_token_72": "773687856563",
80
+ "extra_special_token_73": "535787224917",
81
+ "extra_special_token_74": "351885857332",
82
+ "extra_special_token_75": "578827344666",
83
+ "extra_special_token_76": "198462689911",
84
+ "extra_special_token_77": "722618266242",
85
+ "extra_special_token_78": "952872416512",
86
+ "extra_special_token_79": "517778845323",
87
+ "extra_special_token_80": "749665846687",
88
+ "extra_special_token_81": "661436365453",
89
+ "extra_special_token_82": "259666844669",
90
+ "extra_special_token_83": "242851284913",
91
+ "extra_special_token_84": "514532995959",
92
+ "extra_special_token_85": "161588262349",
93
+ "extra_special_token_86": "742765629356",
94
+ "extra_special_token_87": "225164373623",
95
+ "extra_special_token_88": "676539973863",
96
+ "extra_special_token_89": "826214551218",
97
+ "extra_special_token_90": "182345464792",
98
+ "extra_special_token_91": "232776999554",
99
+ "extra_special_token_92": "337326533813",
100
+ "extra_special_token_93": "676676697292",
101
+ "extra_special_token_94": "929185622831",
102
+ "extra_special_token_95": "545512344383",
103
+ "extra_special_token_96": "499444466686",
104
+ "extra_special_token_97": "314697386682",
105
+ "extra_special_token_98": "517379856925",
106
+ "extra_special_token_99": "379557332953",
107
+ "extra_special_token_100": "614797267726",
108
+ "extra_special_token_101": "429781429464",
109
+ "extra_special_token_102": "922466849763",
110
+ "extra_special_token_103": "721737645236",
111
+ "extra_special_token_104": "479227349997",
112
+ "extra_special_token_105": "136931728327",
113
+ "extra_special_token_106": "259533577263",
114
+ "extra_special_token_107": "488538864842",
115
+ "extra_special_token_108": "937495658852",
116
+ "extra_special_token_109": "489991411364",
117
+ "extra_special_token_110": "499148455254",
118
+ "extra_special_token_111": "441373944925",
119
+ "extra_special_token_112": "899151413682",
120
+ "extra_special_token_113": "467893531755",
121
+ "extra_special_token_114": "527117488925",
122
+ "extra_special_token_115": "928335588653",
123
+ "extra_special_token_116": "374439448821",
124
+ "extra_special_token_117": "879425227932",
125
+ "extra_special_token_118": "867678158885",
126
+ "extra_special_token_119": "399749397872",
127
+ "extra_special_token_120": "129693547287",
128
+ "extra_special_token_121": "689285841825",
129
+ "extra_special_token_122": "771619544974",
130
+ "extra_special_token_123": "724883568652",
131
+ "extra_special_token_124": "516968424863",
132
+ "extra_special_token_125": "733737988257",
133
+ "extra_special_token_126": "852347289392",
134
+ "extra_special_token_127": "296953381169",
135
+ "extra_special_token_128": "377273562477",
136
+ "extra_special_token_129": "262296912232",
137
+ "extra_special_token_130": "547149832394",
138
+ "extra_special_token_131": "298464134954",
139
+ "extra_special_token_132": "216667245274",
140
+ "extra_special_token_133": "843998562287",
141
+ "extra_special_token_134": "572154333646",
142
+ "extra_special_token_135": "124589118494",
143
+ "extra_special_token_136": "841824384614",
144
+ "extra_special_token_137": "232896526252",
145
+ "extra_special_token_138": "295448593321",
146
+ "extra_special_token_139": "123741461297",
147
+ "extra_special_token_140": "653573457168",
148
+ "extra_special_token_141": "196735786156",
149
+ "extra_special_token_142": "377338713663",
150
+ "extra_special_token_143": "964342468552",
151
+ "extra_special_token_144": "586855179568",
152
+ "extra_special_token_145": "484773717614",
153
+ "extra_special_token_146": "894885246797",
154
+ "extra_special_token_147": "677896358599",
155
+ "extra_special_token_148": "848845611563",
156
+ "extra_special_token_149": "851852651677",
157
+ "extra_special_token_150": "398549545767",
158
+ "extra_special_token_151": "454244839926",
159
+ "extra_special_token_152": "799364566435",
160
+ "extra_special_token_153": "967114116556",
161
+ "extra_special_token_154": "817378986438",
162
+ "extra_special_token_155": "233795848681",
163
+ "extra_special_token_156": "824387273757",
164
+ "extra_special_token_157": "916198946615",
165
+ "extra_special_token_158": "563117729724",
166
+ "extra_special_token_159": "951794811935",
167
+ "extra_special_token_160": "374598961236",
168
+ "extra_special_token_161": "922867396683",
169
+ "extra_special_token_162": "765737843639",
170
+ "extra_special_token_163": "175469284871",
171
+ "extra_special_token_164": "231853711778",
172
+ "extra_special_token_165": "662426712668",
173
+ "extra_special_token_166": "711412347158",
174
+ "extra_special_token_167": "753466987363",
175
+ "extra_special_token_168": "513361312532",
176
+ "extra_special_token_169": "712992815957",
177
+ "extra_special_token_170": "971621888444",
178
+ "extra_special_token_171": "829235161526",
179
+ "extra_special_token_172": "585544633356",
180
+ "extra_special_token_173": "582471228164",
181
+ "extra_special_token_174": "678666359123",
182
+ "extra_special_token_175": "557533689478",
183
+ "extra_special_token_176": "632962475133",
184
+ "extra_special_token_177": "484489193824",
185
+ "extra_special_token_178": "489562189822",
186
+ "extra_special_token_179": "589547936288",
187
+ "extra_special_token_180": "363214487524",
188
+ "extra_special_token_181": "244885399387",
189
+ "extra_special_token_182": "431751228368",
190
+ "extra_special_token_183": "433581868192",
191
+ "extra_special_token_184": "486391569221",
192
+ "extra_special_token_185": "185438575221",
193
+ "extra_special_token_186": "126574388585",
194
+ "extra_special_token_187": "741757479784",
195
+ "extra_special_token_188": "529854679937",
196
+ "extra_special_token_189": "996116119839",
197
+ "extra_special_token_190": "616248973917",
198
+ "extra_special_token_191": "763531783491",
199
+ "extra_special_token_192": "955456118295",
200
+ "extra_special_token_193": "364196983365",
201
+ "extra_special_token_194": "195792996468",
202
+ "extra_special_token_195": "151859598873",
203
+ "extra_special_token_196": "399223169721",
204
+ "extra_special_token_197": "938488813964",
205
+ "extra_special_token_198": "961981959227",
206
+ "extra_special_token_199": "183368827562",
207
+ "extra_special_token_200": "533417736566",
208
+ "extra_special_token_201": "786391632558",
209
+ "extra_special_token_202": "665661658354",
210
+ "extra_special_token_203": "693281533643",
211
+ "extra_special_token_204": "475794684356",
212
+ "extra_special_token_205": "652154162978",
213
+ "extra_special_token_206": "753233719644",
214
+ "extra_special_token_207": "668514843129",
215
+ "extra_special_token_208": "819162623892",
216
+ "extra_special_token_209": "941169431859",
217
+ "extra_special_token_210": "877385381798",
218
+ "extra_special_token_211": "752644929761",
219
+ "extra_special_token_212": "881136466196",
220
+ "extra_special_token_213": "275597777299",
221
+ "extra_special_token_214": "731681792655",
222
+ "extra_special_token_215": "961133895172",
223
+ "extra_special_token_216": "864718285734",
224
+ "extra_special_token_217": "963852916563",
225
+ "extra_special_token_218": "319584985416",
226
+ "extra_special_token_219": "563365646341",
227
+ "extra_special_token_220": "811371928234",
228
+ "extra_special_token_221": "837131396371",
229
+ "extra_special_token_222": "267514771964",
230
+ "extra_special_token_223": "944513428457",
231
+ "extra_special_token_224": "117298239631",
232
+ "extra_special_token_225": "158142752582",
233
+ "extra_special_token_226": "252867443568",
234
+ "extra_special_token_227": "839269684865",
235
+ "extra_special_token_228": "612788593128",
236
+ "extra_special_token_229": "145669731981",
237
+ "extra_special_token_230": "121557291859",
238
+ "extra_special_token_231": "245416776926",
239
+ "extra_special_token_232": "799417897197",
240
+ "extra_special_token_233": "997958836435",
241
+ "extra_special_token_234": "892336777248",
242
+ "extra_special_token_235": "158929292238",
243
+ "extra_special_token_236": "581976444672",
244
+ "extra_special_token_237": "897784492783",
245
+ "extra_special_token_238": "492373714791",
246
+ "extra_special_token_239": "512659818733",
247
+ "extra_special_token_240": "881112998642",
248
+ "extra_special_token_241": "619454958782",
249
+ "extra_special_token_242": "431149748713",
250
+ "extra_special_token_243": "624221476921",
251
+ "extra_special_token_244": "125866399464",
252
+ "extra_special_token_245": "339882449689",
253
+ "extra_special_token_246": "186198784585",
254
+ "extra_special_token_247": "943193294691",
255
+ "extra_special_token_248": "955668961269",
256
+ "extra_special_token_249": "232787996724",
257
+ "extra_special_token_250": "215671314196",
258
+ "extra_special_token_251": "286173241916",
259
+ "extra_special_token_252": "745977673725",
260
+ "extra_special_token_253": "556976448182",
261
+ "extra_special_token_254": "599961512792",
262
+ "extra_special_token_255": "766294538337",
263
+ "extra_special_token_256": "934912591213",
264
+ "extra_special_token_257": "295118729589",
265
+ "extra_special_token_258": "529455466433",
266
+ "extra_special_token_259": "196119929397",
267
+ "extra_special_token_260": "379571934299",
268
+ "extra_special_token_261": "251789649997",
269
+ "extra_special_token_262": "564544131355",
270
+ "extra_special_token_263": "244371196654",
271
+ "extra_special_token_264": "384598329253",
272
+ "extra_special_token_265": "887753195844",
273
+ "extra_special_token_266": "364947325679",
274
+ "extra_special_token_267": "655517954651",
275
+ "extra_special_token_268": "673948786567",
276
+ "extra_special_token_269": "857231548835",
277
+ "extra_special_token_270": "816115936673",
278
+ "extra_special_token_271": "644234165531",
279
+ "extra_special_token_272": "182782912224",
280
+ "extra_special_token_273": "234316622259",
281
+ "extra_special_token_274": "421369185549",
282
+ "extra_special_token_275": "434632855397",
283
+ "extra_special_token_276": "921889371893",
284
+ "extra_special_token_277": "415956914763",
285
+ "extra_special_token_278": "598916996413",
286
+ "extra_special_token_279": "773671349113",
287
+ "extra_special_token_280": "952465217972",
288
+ "extra_special_token_281": "117657531962",
289
+ "extra_special_token_282": "729825168745",
290
+ "extra_special_token_283": "691315125346",
291
+ "extra_special_token_284": "768461952319",
292
+ "extra_special_token_285": "664847713559",
293
+ "extra_special_token_286": "953267689786",
294
+ "extra_special_token_287": "886464195129",
295
+ "extra_special_token_288": "824488329416",
296
+ "extra_special_token_289": "837873762491",
297
+ "extra_special_token_290": "532833541879",
298
+ "extra_special_token_291": "669183782449",
299
+ "extra_special_token_292": "941976537588",
300
+ "extra_special_token_293": "739394546916",
301
+ "extra_special_token_294": "267954879268",
302
+ "extra_special_token_295": "637551427887",
303
+ "extra_special_token_296": "217756494954",
304
+ "extra_special_token_297": "524444658383",
305
+ "extra_special_token_298": "117783274348",
306
+ "extra_special_token_299": "138218735276",
307
+ "extra_special_token_300": "814611949491",
308
+ "extra_special_token_301": "711641973413",
309
+ "extra_special_token_302": "499156317423",
310
+ "extra_special_token_303": "515856611931",
311
+ "extra_special_token_304": "454164859837",
312
+ "extra_special_token_305": "345271433112",
313
+ "extra_special_token_306": "462294118988",
314
+ "extra_special_token_307": "511785788222",
315
+ "extra_special_token_308": "497294727353",
316
+ "extra_special_token_309": "866519986723",
317
+ "extra_special_token_310": "334513529294",
318
+ "extra_special_token_311": "549946382131",
319
+ "extra_special_token_312": "284445431422",
320
+ "extra_special_token_313": "396521188476",
321
+ "extra_special_token_314": "421435255895",
322
+ "extra_special_token_315": "133373659361",
323
+ "extra_special_token_316": "322683334381",
324
+ "extra_special_token_317": "228358422847",
325
+ "extra_special_token_318": "291762694874",
326
+ "extra_special_token_319": "143182978129",
327
+ "extra_special_token_320": "511923256573",
328
+ "extra_special_token_321": "327158398268",
329
+ "extra_special_token_322": "879764613759",
330
+ "extra_special_token_323": "564395222747",
331
+ "extra_special_token_324": "451161679736",
332
+ "extra_special_token_325": "538631466654",
333
+ "extra_special_token_326": "221762325616",
334
+ "extra_special_token_327": "218391991184",
335
+ "extra_special_token_328": "322589379462",
336
+ "extra_special_token_329": "876537814263",
337
+ "extra_special_token_330": "152676556624",
338
+ "extra_special_token_331": "332522971941",
339
+ "extra_special_token_332": "884354318946",
340
+ "extra_special_token_333": "513349618943",
341
+ "extra_special_token_334": "116639746413",
342
+ "extra_special_token_335": "635185846287",
343
+ "extra_special_token_336": "993832498489",
344
+ "extra_special_token_337": "813981174797",
345
+ "extra_special_token_338": "438745114173",
346
+ "extra_special_token_339": "983493951323",
347
+ "extra_special_token_340": "724492262421",
348
+ "extra_special_token_341": "622553389126",
349
+ "extra_special_token_342": "889965243135",
350
+ "extra_special_token_343": "364492359246",
351
+ "extra_special_token_344": "154962668224",
352
+ "extra_special_token_345": "179564995814",
353
+ "extra_special_token_346": "418412875665",
354
+ "extra_special_token_347": "718951851413",
355
+ "extra_special_token_348": "699446724178",
356
+ "extra_special_token_349": "624266421831",
357
+ "extra_special_token_350": "815458725125",
358
+ "extra_special_token_351": "455423278865",
359
+ "extra_special_token_352": "393741199486",
360
+ "extra_special_token_353": "328552864359",
361
+ "extra_special_token_354": "211662639865",
362
+ "extra_special_token_355": "218784516525",
363
+ "extra_special_token_356": "762486672996",
364
+ "extra_special_token_357": "142799718159",
365
+ "extra_special_token_358": "858146415154",
366
+ "extra_special_token_359": "767858144912",
367
+ "extra_special_token_360": "571317457151",
368
+ "extra_special_token_361": "635127952696",
369
+ "extra_special_token_362": "116427191984",
370
+ "extra_special_token_363": "268921994538",
371
+ "extra_special_token_364": "523937669294",
372
+ "extra_special_token_365": "165429152138",
373
+ "extra_special_token_366": "739246183345",
374
+ "extra_special_token_367": "591464355756",
375
+ "extra_special_token_368": "212985874612",
376
+ "extra_special_token_369": "191887635211",
377
+ "extra_special_token_370": "967214577653",
378
+ "extra_special_token_371": "119342152414",
379
+ "extra_special_token_372": "946444632795",
380
+ "extra_special_token_373": "618423867817",
381
+ "extra_special_token_374": "228565148417",
382
+ "extra_special_token_375": "729116422489",
383
+ "extra_special_token_376": "527874729936",
384
+ "extra_special_token_377": "739784153482",
385
+ "extra_special_token_378": "387763951128",
386
+ "extra_special_token_379": "331369926711",
387
+ "extra_special_token_380": "562716493614",
388
+ "extra_special_token_381": "739667844957",
389
+ "extra_special_token_382": "562389434565",
390
+ "extra_special_token_383": "256497188281",
391
+ "extra_special_token_384": "859927364588",
392
+ "extra_special_token_385": "417668946583",
393
+ "extra_special_token_386": "357621613582",
394
+ "extra_special_token_387": "438435178228",
395
+ "extra_special_token_388": "485692541169",
396
+ "extra_special_token_389": "825815739116",
397
+ "extra_special_token_390": "342221452223",
398
+ "extra_special_token_391": "697747991249",
399
+ "extra_special_token_392": "716763689965",
400
+ "extra_special_token_393": "141499982867",
401
+ "extra_special_token_394": "818479319499",
402
+ "extra_special_token_395": "336813343298",
403
+ "extra_special_token_396": "594688742928",
404
+ "extra_special_token_397": "472129283475",
405
+ "extra_special_token_398": "514354144759",
406
+ "extra_special_token_399": "349249721685",
407
+ "extra_special_token_400": "546276298359",
408
+ "extra_special_token_401": "353755529131",
409
+ "extra_special_token_402": "315534574435",
410
+ "extra_special_token_403": "523723475786",
411
+ "extra_special_token_404": "215826764872",
412
+ "extra_special_token_405": "367968398551",
413
+ "extra_special_token_406": "569853653352",
414
+ "extra_special_token_407": "389715484387",
415
+ "extra_special_token_408": "293847485454",
416
+ "extra_special_token_409": "714738141818",
417
+ "extra_special_token_410": "178478368922",
418
+ "extra_special_token_411": "581493616981",
419
+ "extra_special_token_412": "589439538674",
420
+ "extra_special_token_413": "846657726193",
421
+ "extra_special_token_414": "722339992679",
422
+ "extra_special_token_415": "138154781148",
423
+ "extra_special_token_416": "757785319772",
424
+ "extra_special_token_417": "492516914298",
425
+ "extra_special_token_418": "919181521716",
426
+ "extra_special_token_419": "985781138935",
427
+ "extra_special_token_420": "476969195485",
428
+ "extra_special_token_421": "313145133463",
429
+ "extra_special_token_422": "758963111966",
430
+ "extra_special_token_423": "147541537162",
431
+ "extra_special_token_424": "557163366873",
432
+ "extra_special_token_425": "144373897488",
433
+ "extra_special_token_426": "522515164754",
434
+ "extra_special_token_427": "724964923582",
435
+ "extra_special_token_428": "284776712475",
436
+ "extra_special_token_429": "375429755114",
437
+ "extra_special_token_430": "181233596124",
438
+ "extra_special_token_431": "948585673431",
439
+ "extra_special_token_432": "243165586174",
440
+ "extra_special_token_433": "396847976144",
441
+ "extra_special_token_434": "997724962668",
442
+ "extra_special_token_435": "558837194455",
443
+ "extra_special_token_436": "163165456396",
444
+ "extra_special_token_437": "378749551722",
445
+ "extra_special_token_438": "161238482259",
446
+ "extra_special_token_439": "754978243758",
447
+ "extra_special_token_440": "195388849133",
448
+ "extra_special_token_441": "229775525672",
449
+ "extra_special_token_442": "262437452884",
450
+ "extra_special_token_443": "441377892146",
451
+ "extra_special_token_444": "451885565366",
452
+ "extra_special_token_445": "981277526855",
453
+ "extra_special_token_446": "762495822823",
454
+ "extra_special_token_447": "368763327262",
455
+ "extra_special_token_448": "757422791351",
456
+ "extra_special_token_449": "636324136426",
457
+ "extra_special_token_450": "214193645583",
458
+ "extra_special_token_451": "412843856172",
459
+ "extra_special_token_452": "179386156569",
460
+ "extra_special_token_453": "756916173536",
461
+ "extra_special_token_454": "892697125149",
462
+ "extra_special_token_455": "625334487352",
463
+ "extra_special_token_456": "941861857715",
464
+ "extra_special_token_457": "887417525236",
465
+ "extra_special_token_458": "649516938598",
466
+ "extra_special_token_459": "717628619782",
467
+ "extra_special_token_460": "438124184139",
468
+ "extra_special_token_461": "547563892268",
469
+ "extra_special_token_462": "856317483891",
470
+ "extra_special_token_463": "313313831273",
471
+ "extra_special_token_464": "371496153876",
472
+ "extra_special_token_465": "587541149322",
473
+ "extra_special_token_466": "265847332563",
474
+ "extra_special_token_467": "449549215429",
475
+ "extra_special_token_468": "163497196769",
476
+ "extra_special_token_469": "861342291298",
477
+ "extra_special_token_470": "268433315926",
478
+ "extra_special_token_471": "774679513717",
479
+ "extra_special_token_472": "851254219729",
480
+ "extra_special_token_473": "583527834464",
481
+ "extra_special_token_474": "488496781997",
482
+ "extra_special_token_475": "556814553861",
483
+ "extra_special_token_476": "482829231639",
484
+ "extra_special_token_477": "618878266619",
485
+ "extra_special_token_478": "147444452794",
486
+ "extra_special_token_479": "949235426629",
487
+ "extra_special_token_480": "357299947518",
488
+ "extra_special_token_481": "175528632226",
489
+ "extra_special_token_482": "645527857972",
490
+ "extra_special_token_483": "186872457894",
491
+ "extra_special_token_484": "552738847828",
492
+ "extra_special_token_485": "626748382482",
493
+ "extra_special_token_486": "921894985642",
494
+ "extra_special_token_487": "943878645871",
495
+ "extra_special_token_488": "859289776479",
496
+ "extra_special_token_489": "614583493135",
497
+ "extra_special_token_490": "933775286797",
498
+ "extra_special_token_491": "332234613346",
499
+ "extra_special_token_492": "325196781219",
500
+ "extra_special_token_493": "142526557681",
501
+ "extra_special_token_494": "356722692178",
502
+ "extra_special_token_495": "449318681694",
503
+ "extra_special_token_496": "687284547244",
504
+ "extra_special_token_497": "947262995132",
505
+ "extra_special_token_498": "893974619684",
506
+ "extra_special_token_499": "797238311233"
507
+ },
508
+ "is_local": false,
509
+ "model_input_names": [
510
+ "input_ids",
511
+ "attention_mask"
512
+ ],
513
+ "model_max_length": 1000000000000000019884624838656,
514
+ "pad_token": "<|end_of_text|>",
515
+ "tokenizer_class": "PreTrainedTokenizerFast"
516
+ }
llama3-8b/base/train.log ADDED
@@ -0,0 +1,274 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 2026-03-28 20:56:26,546 [INFO] new_opacus_codex.train_steps: epoch=1 step=5 loss=12.8613
2
+ 2026-03-28 20:57:01,255 [INFO] new_opacus_codex.train_steps: epoch=1 step=10 loss=12.2605
3
+ 2026-03-28 20:57:15,240 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=10 eval_loss=10.0588 duration_sec=13.97
4
+ 2026-03-28 20:57:50,264 [INFO] new_opacus_codex.train_steps: epoch=1 step=15 loss=10.1190
5
+ 2026-03-28 20:58:26,266 [INFO] new_opacus_codex.train_steps: epoch=1 step=20 loss=6.8648
6
+ 2026-03-28 20:58:40,318 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=20 eval_loss=3.7858 duration_sec=14.04
7
+ 2026-03-28 20:59:15,099 [INFO] new_opacus_codex.train_steps: epoch=1 step=25 loss=4.3052
8
+ 2026-03-28 20:59:49,805 [INFO] new_opacus_codex.train_steps: epoch=1 step=30 loss=2.8522
9
+ 2026-03-28 21:00:03,840 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=30 eval_loss=1.7647 duration_sec=14.02
10
+ 2026-03-28 21:00:39,662 [INFO] new_opacus_codex.train_steps: epoch=1 step=35 loss=2.0372
11
+ 2026-03-28 21:01:14,660 [INFO] new_opacus_codex.train_steps: epoch=1 step=40 loss=1.7104
12
+ 2026-03-28 21:01:28,749 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=40 eval_loss=1.3393 duration_sec=14.08
13
+ 2026-03-28 21:02:03,834 [INFO] new_opacus_codex.train_steps: epoch=1 step=45 loss=1.5088
14
+ 2026-03-28 21:02:38,485 [INFO] new_opacus_codex.train_steps: epoch=1 step=50 loss=1.3878
15
+ 2026-03-28 21:02:52,526 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=50 eval_loss=1.1982 duration_sec=14.03
16
+ 2026-03-28 21:03:27,540 [INFO] new_opacus_codex.train_steps: epoch=1 step=55 loss=1.2921
17
+ 2026-03-28 21:04:01,800 [INFO] new_opacus_codex.train_steps: epoch=1 step=60 loss=1.2207
18
+ 2026-03-28 21:04:15,862 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=60 eval_loss=1.1363 duration_sec=14.03
19
+ 2026-03-28 21:04:52,160 [INFO] new_opacus_codex.train_steps: epoch=1 step=65 loss=1.2457
20
+ 2026-03-28 21:05:26,828 [INFO] new_opacus_codex.train_steps: epoch=1 step=70 loss=1.1841
21
+ 2026-03-28 21:05:40,959 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=70 eval_loss=1.0855 duration_sec=14.10
22
+ 2026-03-28 21:06:17,277 [INFO] new_opacus_codex.train_steps: epoch=1 step=75 loss=1.2404
23
+ 2026-03-28 21:06:51,692 [INFO] new_opacus_codex.train_steps: epoch=1 step=80 loss=1.2855
24
+ 2026-03-28 21:07:05,807 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=80 eval_loss=1.0554 duration_sec=14.09
25
+ 2026-03-28 21:07:39,852 [INFO] new_opacus_codex.train_steps: epoch=1 step=85 loss=1.1834
26
+ 2026-03-28 21:08:14,926 [INFO] new_opacus_codex.train_steps: epoch=1 step=90 loss=1.2553
27
+ 2026-03-28 21:08:28,989 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=90 eval_loss=1.0419 duration_sec=14.05
28
+ 2026-03-28 21:09:04,763 [INFO] new_opacus_codex.train_steps: epoch=1 step=95 loss=1.2630
29
+ 2026-03-28 21:09:46,419 [INFO] new_opacus_codex.train_steps: epoch=1 step=100 loss=1.2037
30
+ 2026-03-28 21:10:00,429 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=100 eval_loss=1.0263 duration_sec=14.00
31
+ 2026-03-28 21:10:35,274 [INFO] new_opacus_codex.train_steps: epoch=1 step=105 loss=1.1383
32
+ 2026-03-28 21:11:11,048 [INFO] new_opacus_codex.train_steps: epoch=1 step=110 loss=1.0529
33
+ 2026-03-28 21:11:25,107 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=110 eval_loss=1.0032 duration_sec=14.05
34
+ 2026-03-28 21:11:59,487 [INFO] new_opacus_codex.train_steps: epoch=1 step=115 loss=1.0694
35
+ 2026-03-28 21:12:35,070 [INFO] new_opacus_codex.train_steps: epoch=1 step=120 loss=1.0754
36
+ 2026-03-28 21:12:49,131 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=120 eval_loss=0.9867 duration_sec=14.04
37
+ 2026-03-28 21:13:24,742 [INFO] new_opacus_codex.train_steps: epoch=1 step=125 loss=1.0723
38
+ 2026-03-28 21:13:59,516 [INFO] new_opacus_codex.train_steps: epoch=1 step=130 loss=1.1042
39
+ 2026-03-28 21:14:13,580 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=130 eval_loss=0.9733 duration_sec=14.05
40
+ 2026-03-28 21:14:48,067 [INFO] new_opacus_codex.train_steps: epoch=1 step=135 loss=1.1208
41
+ 2026-03-28 21:15:22,938 [INFO] new_opacus_codex.train_steps: epoch=1 step=140 loss=1.0620
42
+ 2026-03-28 21:15:36,976 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=140 eval_loss=0.9615 duration_sec=14.03
43
+ 2026-03-28 21:16:10,910 [INFO] new_opacus_codex.train_steps: epoch=1 step=145 loss=1.0273
44
+ 2026-03-28 21:16:45,756 [INFO] new_opacus_codex.train_steps: epoch=1 step=150 loss=1.0476
45
+ 2026-03-28 21:16:59,734 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=150 eval_loss=0.9529 duration_sec=13.96
46
+ 2026-03-28 21:17:34,620 [INFO] new_opacus_codex.train_steps: epoch=1 step=155 loss=1.0280
47
+ 2026-03-28 21:18:10,562 [INFO] new_opacus_codex.train_steps: epoch=1 step=160 loss=1.0403
48
+ 2026-03-28 21:18:24,652 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=160 eval_loss=0.9468 duration_sec=14.07
49
+ 2026-03-28 21:18:59,073 [INFO] new_opacus_codex.train_steps: epoch=1 step=165 loss=1.0142
50
+ 2026-03-28 21:19:33,965 [INFO] new_opacus_codex.train_steps: epoch=1 step=170 loss=0.9427
51
+ 2026-03-28 21:19:48,027 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=170 eval_loss=0.9378 duration_sec=14.04
52
+ 2026-03-28 21:20:23,506 [INFO] new_opacus_codex.train_steps: epoch=1 step=175 loss=0.9536
53
+ 2026-03-28 21:20:58,178 [INFO] new_opacus_codex.train_steps: epoch=1 step=180 loss=1.0034
54
+ 2026-03-28 21:21:12,260 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=180 eval_loss=0.9333 duration_sec=14.07
55
+ 2026-03-28 21:22:16,664 [INFO] new_opacus_codex.train_steps: epoch=2 step=185 loss=1.0552
56
+ 2026-03-28 21:22:51,313 [INFO] new_opacus_codex.train_steps: epoch=2 step=190 loss=0.9495
57
+ 2026-03-28 21:23:05,381 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=190 eval_loss=0.9330 duration_sec=14.06
58
+ 2026-03-28 21:23:40,172 [INFO] new_opacus_codex.train_steps: epoch=2 step=195 loss=0.8572
59
+ 2026-03-28 21:24:16,700 [INFO] new_opacus_codex.train_steps: epoch=2 step=200 loss=0.8417
60
+ 2026-03-28 21:24:30,721 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=200 eval_loss=0.9381 duration_sec=14.01
61
+ 2026-03-28 21:25:06,010 [INFO] new_opacus_codex.train_steps: epoch=2 step=205 loss=0.8215
62
+ 2026-03-28 21:25:40,901 [INFO] new_opacus_codex.train_steps: epoch=2 step=210 loss=0.8103
63
+ 2026-03-28 21:25:55,016 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=210 eval_loss=0.9246 duration_sec=14.10
64
+ 2026-03-28 21:26:32,398 [INFO] new_opacus_codex.train_steps: epoch=2 step=215 loss=0.8536
65
+ 2026-03-28 21:27:07,878 [INFO] new_opacus_codex.train_steps: epoch=2 step=220 loss=0.8250
66
+ 2026-03-28 21:27:22,006 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=220 eval_loss=0.9103 duration_sec=14.11
67
+ 2026-03-28 21:27:56,402 [INFO] new_opacus_codex.train_steps: epoch=2 step=225 loss=0.7749
68
+ 2026-03-28 21:28:33,950 [INFO] new_opacus_codex.train_steps: epoch=2 step=230 loss=0.7756
69
+ 2026-03-28 21:28:48,041 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=230 eval_loss=0.9046 duration_sec=14.08
70
+ 2026-03-28 21:29:23,106 [INFO] new_opacus_codex.train_steps: epoch=2 step=235 loss=0.7989
71
+ 2026-03-28 21:29:58,689 [INFO] new_opacus_codex.train_steps: epoch=2 step=240 loss=0.8179
72
+ 2026-03-28 21:30:12,826 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=240 eval_loss=0.9011 duration_sec=14.12
73
+ 2026-03-28 21:30:48,145 [INFO] new_opacus_codex.train_steps: epoch=2 step=245 loss=0.7930
74
+ 2026-03-28 21:31:22,306 [INFO] new_opacus_codex.train_steps: epoch=2 step=250 loss=0.7725
75
+ 2026-03-28 21:31:36,350 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=250 eval_loss=0.9041 duration_sec=14.04
76
+ 2026-03-28 21:32:11,848 [INFO] new_opacus_codex.train_steps: epoch=2 step=255 loss=0.7766
77
+ 2026-03-28 21:32:47,822 [INFO] new_opacus_codex.train_steps: epoch=2 step=260 loss=0.8018
78
+ 2026-03-28 21:33:02,024 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=260 eval_loss=0.9005 duration_sec=14.19
79
+ 2026-03-28 21:33:37,367 [INFO] new_opacus_codex.train_steps: epoch=2 step=265 loss=0.8110
80
+ 2026-03-28 21:34:11,806 [INFO] new_opacus_codex.train_steps: epoch=2 step=270 loss=0.7752
81
+ 2026-03-28 21:34:25,875 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=270 eval_loss=0.8990 duration_sec=14.06
82
+ 2026-03-28 21:35:01,247 [INFO] new_opacus_codex.train_steps: epoch=2 step=275 loss=0.7817
83
+ 2026-03-28 21:35:36,804 [INFO] new_opacus_codex.train_steps: epoch=2 step=280 loss=0.7905
84
+ 2026-03-28 21:35:51,001 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=280 eval_loss=0.8931 duration_sec=14.18
85
+ 2026-03-28 21:36:25,909 [INFO] new_opacus_codex.train_steps: epoch=2 step=285 loss=0.7414
86
+ 2026-03-28 21:37:01,168 [INFO] new_opacus_codex.train_steps: epoch=2 step=290 loss=0.7571
87
+ 2026-03-28 21:37:15,392 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=290 eval_loss=0.8868 duration_sec=14.21
88
+ 2026-03-28 21:37:51,250 [INFO] new_opacus_codex.train_steps: epoch=2 step=295 loss=0.8381
89
+ 2026-03-28 21:38:26,595 [INFO] new_opacus_codex.train_steps: epoch=2 step=300 loss=0.7943
90
+ 2026-03-28 21:38:40,659 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=300 eval_loss=0.8820 duration_sec=14.05
91
+ 2026-03-28 21:39:16,527 [INFO] new_opacus_codex.train_steps: epoch=2 step=305 loss=0.7557
92
+ 2026-03-28 21:39:52,431 [INFO] new_opacus_codex.train_steps: epoch=2 step=310 loss=0.7816
93
+ 2026-03-28 21:40:06,508 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=310 eval_loss=0.8758 duration_sec=14.06
94
+ 2026-03-28 21:40:41,127 [INFO] new_opacus_codex.train_steps: epoch=2 step=315 loss=0.7788
95
+ 2026-03-28 21:41:16,800 [INFO] new_opacus_codex.train_steps: epoch=2 step=320 loss=0.7685
96
+ 2026-03-28 21:41:30,886 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=320 eval_loss=0.8710 duration_sec=14.08
97
+ 2026-03-28 21:42:05,965 [INFO] new_opacus_codex.train_steps: epoch=2 step=325 loss=0.7570
98
+ 2026-03-28 21:42:39,891 [INFO] new_opacus_codex.train_steps: epoch=2 step=330 loss=0.7371
99
+ 2026-03-28 21:42:53,963 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=330 eval_loss=0.8704 duration_sec=14.06
100
+ 2026-03-28 21:43:28,777 [INFO] new_opacus_codex.train_steps: epoch=2 step=335 loss=0.7407
101
+ 2026-03-28 21:44:04,475 [INFO] new_opacus_codex.train_steps: epoch=2 step=340 loss=0.7488
102
+ 2026-03-28 21:44:18,602 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=340 eval_loss=0.8664 duration_sec=14.11
103
+ 2026-03-28 21:44:53,127 [INFO] new_opacus_codex.train_steps: epoch=2 step=345 loss=0.7434
104
+ 2026-03-28 21:45:28,928 [INFO] new_opacus_codex.train_steps: epoch=2 step=350 loss=0.7623
105
+ 2026-03-28 21:45:43,036 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=350 eval_loss=0.8677 duration_sec=14.09
106
+ 2026-03-28 21:46:18,906 [INFO] new_opacus_codex.train_steps: epoch=2 step=355 loss=0.7884
107
+ 2026-03-28 21:46:53,546 [INFO] new_opacus_codex.train_steps: epoch=2 step=360 loss=0.8095
108
+ 2026-03-28 21:47:07,606 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=360 eval_loss=0.8623 duration_sec=14.05
109
+ 2026-03-28 21:47:42,243 [INFO] new_opacus_codex.train_steps: epoch=2 step=365 loss=0.7600
110
+ 2026-03-28 21:48:49,752 [INFO] new_opacus_codex.train_steps: epoch=3 step=370 loss=0.6276
111
+ 2026-03-28 21:49:03,916 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=370 eval_loss=0.8717 duration_sec=14.15
112
+ 2026-03-28 21:49:39,132 [INFO] new_opacus_codex.train_steps: epoch=3 step=375 loss=0.6539
113
+ 2026-03-28 21:50:20,597 [INFO] new_opacus_codex.train_steps: epoch=3 step=380 loss=0.6426
114
+ 2026-03-28 21:50:34,723 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=380 eval_loss=0.8663 duration_sec=14.12
115
+ 2026-03-28 21:51:11,038 [INFO] new_opacus_codex.train_steps: epoch=3 step=385 loss=0.5852
116
+ 2026-03-28 21:51:47,216 [INFO] new_opacus_codex.train_steps: epoch=3 step=390 loss=0.5647
117
+ 2026-03-28 21:52:01,308 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=390 eval_loss=0.8669 duration_sec=14.07
118
+ 2026-03-28 21:52:37,801 [INFO] new_opacus_codex.train_steps: epoch=3 step=395 loss=0.5883
119
+ 2026-03-28 21:53:12,214 [INFO] new_opacus_codex.train_steps: epoch=3 step=400 loss=0.6391
120
+ 2026-03-28 21:53:26,309 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=400 eval_loss=0.8642 duration_sec=14.08
121
+ 2026-03-28 21:54:00,943 [INFO] new_opacus_codex.train_steps: epoch=3 step=405 loss=0.6465
122
+ 2026-03-28 21:54:36,833 [INFO] new_opacus_codex.train_steps: epoch=3 step=410 loss=0.6201
123
+ 2026-03-28 21:54:50,999 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=410 eval_loss=0.8614 duration_sec=14.16
124
+ 2026-03-28 21:55:25,647 [INFO] new_opacus_codex.train_steps: epoch=3 step=415 loss=0.5991
125
+ 2026-03-28 21:56:01,537 [INFO] new_opacus_codex.train_steps: epoch=3 step=420 loss=0.6230
126
+ 2026-03-28 21:56:15,595 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=420 eval_loss=0.8651 duration_sec=14.04
127
+ 2026-03-28 21:56:50,618 [INFO] new_opacus_codex.train_steps: epoch=3 step=425 loss=0.6310
128
+ 2026-03-28 21:57:26,719 [INFO] new_opacus_codex.train_steps: epoch=3 step=430 loss=0.5901
129
+ 2026-03-28 21:57:40,799 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=430 eval_loss=0.8557 duration_sec=14.07
130
+ 2026-03-28 21:58:15,681 [INFO] new_opacus_codex.train_steps: epoch=3 step=435 loss=0.6122
131
+ 2026-03-28 21:58:51,023 [INFO] new_opacus_codex.train_steps: epoch=3 step=440 loss=0.6217
132
+ 2026-03-28 21:59:05,147 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=440 eval_loss=0.8572 duration_sec=14.11
133
+ 2026-03-28 21:59:40,550 [INFO] new_opacus_codex.train_steps: epoch=3 step=445 loss=0.5839
134
+ 2026-03-28 22:00:16,035 [INFO] new_opacus_codex.train_steps: epoch=3 step=450 loss=0.5818
135
+ 2026-03-28 22:00:30,162 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=450 eval_loss=0.8531 duration_sec=14.11
136
+ 2026-03-28 22:01:06,246 [INFO] new_opacus_codex.train_steps: epoch=3 step=455 loss=0.6263
137
+ 2026-03-28 22:01:41,527 [INFO] new_opacus_codex.train_steps: epoch=3 step=460 loss=0.6332
138
+ 2026-03-28 22:01:55,653 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=460 eval_loss=0.8502 duration_sec=14.11
139
+ 2026-03-28 22:02:30,351 [INFO] new_opacus_codex.train_steps: epoch=3 step=465 loss=0.6334
140
+ 2026-03-28 22:03:05,370 [INFO] new_opacus_codex.train_steps: epoch=3 step=470 loss=0.6378
141
+ 2026-03-28 22:03:19,497 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=470 eval_loss=0.8455 duration_sec=14.11
142
+ 2026-03-28 22:03:54,911 [INFO] new_opacus_codex.train_steps: epoch=3 step=475 loss=0.6138
143
+ 2026-03-28 22:04:30,330 [INFO] new_opacus_codex.train_steps: epoch=3 step=480 loss=0.6024
144
+ 2026-03-28 22:04:44,413 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=480 eval_loss=0.8437 duration_sec=14.07
145
+ 2026-03-28 22:05:20,174 [INFO] new_opacus_codex.train_steps: epoch=3 step=485 loss=0.5991
146
+ 2026-03-28 22:05:55,136 [INFO] new_opacus_codex.train_steps: epoch=3 step=490 loss=0.5919
147
+ 2026-03-28 22:06:09,341 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=490 eval_loss=0.8431 duration_sec=14.19
148
+ 2026-03-28 22:06:43,832 [INFO] new_opacus_codex.train_steps: epoch=3 step=495 loss=0.5967
149
+ 2026-03-28 22:07:18,832 [INFO] new_opacus_codex.train_steps: epoch=3 step=500 loss=0.6140
150
+ 2026-03-28 22:07:33,134 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=500 eval_loss=0.8359 duration_sec=14.29
151
+ 2026-03-28 22:08:08,792 [INFO] new_opacus_codex.train_steps: epoch=3 step=505 loss=0.6154
152
+ 2026-03-28 22:08:43,971 [INFO] new_opacus_codex.train_steps: epoch=3 step=510 loss=0.6142
153
+ 2026-03-28 22:08:58,148 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=510 eval_loss=0.8381 duration_sec=14.17
154
+ 2026-03-28 22:09:34,057 [INFO] new_opacus_codex.train_steps: epoch=3 step=515 loss=0.6125
155
+ 2026-03-28 22:10:08,557 [INFO] new_opacus_codex.train_steps: epoch=3 step=520 loss=0.6209
156
+ 2026-03-28 22:10:22,677 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=520 eval_loss=0.8352 duration_sec=14.10
157
+ 2026-03-28 22:10:57,388 [INFO] new_opacus_codex.train_steps: epoch=3 step=525 loss=0.6048
158
+ 2026-03-28 22:11:33,442 [INFO] new_opacus_codex.train_steps: epoch=3 step=530 loss=0.5869
159
+ 2026-03-28 22:11:47,616 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=530 eval_loss=0.8335 duration_sec=14.16
160
+ 2026-03-28 22:12:23,493 [INFO] new_opacus_codex.train_steps: epoch=3 step=535 loss=0.5928
161
+ 2026-03-28 22:12:59,208 [INFO] new_opacus_codex.train_steps: epoch=3 step=540 loss=0.5727
162
+ 2026-03-28 22:13:13,294 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=540 eval_loss=0.8314 duration_sec=14.07
163
+ 2026-03-28 22:13:48,727 [INFO] new_opacus_codex.train_steps: epoch=3 step=545 loss=0.5787
164
+ 2026-03-28 22:14:54,716 [INFO] new_opacus_codex.train_steps: epoch=4 step=550 loss=0.4820
165
+ 2026-03-28 22:15:08,875 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=550 eval_loss=0.8311 duration_sec=14.15
166
+ 2026-03-28 22:15:45,103 [INFO] new_opacus_codex.train_steps: epoch=4 step=555 loss=0.4901
167
+ 2026-03-28 22:16:20,563 [INFO] new_opacus_codex.train_steps: epoch=4 step=560 loss=0.4995
168
+ 2026-03-28 22:16:34,661 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=560 eval_loss=0.8524 duration_sec=14.09
169
+ 2026-03-28 22:17:10,348 [INFO] new_opacus_codex.train_steps: epoch=4 step=565 loss=0.5060
170
+ 2026-03-28 22:17:45,358 [INFO] new_opacus_codex.train_steps: epoch=4 step=570 loss=0.4944
171
+ 2026-03-28 22:17:59,509 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=570 eval_loss=0.8441 duration_sec=14.14
172
+ 2026-03-28 22:18:35,454 [INFO] new_opacus_codex.train_steps: epoch=4 step=575 loss=0.4655
173
+ 2026-03-28 22:19:09,992 [INFO] new_opacus_codex.train_steps: epoch=4 step=580 loss=0.4715
174
+ 2026-03-28 22:19:24,098 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=580 eval_loss=0.8439 duration_sec=14.09
175
+ 2026-03-28 22:19:59,413 [INFO] new_opacus_codex.train_steps: epoch=4 step=585 loss=0.4903
176
+ 2026-03-28 22:20:35,807 [INFO] new_opacus_codex.train_steps: epoch=4 step=590 loss=0.4829
177
+ 2026-03-28 22:20:49,896 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=590 eval_loss=0.8448 duration_sec=14.07
178
+ 2026-03-28 22:21:25,790 [INFO] new_opacus_codex.train_steps: epoch=4 step=595 loss=0.5205
179
+ 2026-03-28 22:22:01,831 [INFO] new_opacus_codex.train_steps: epoch=4 step=600 loss=0.5171
180
+ 2026-03-28 22:22:15,930 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=600 eval_loss=0.8453 duration_sec=14.09
181
+ 2026-03-28 22:22:50,871 [INFO] new_opacus_codex.train_steps: epoch=4 step=605 loss=0.4960
182
+ 2026-03-28 22:23:27,408 [INFO] new_opacus_codex.train_steps: epoch=4 step=610 loss=0.4932
183
+ 2026-03-28 22:23:41,503 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=610 eval_loss=0.8427 duration_sec=14.08
184
+ 2026-03-28 22:24:15,919 [INFO] new_opacus_codex.train_steps: epoch=4 step=615 loss=0.4604
185
+ 2026-03-28 22:24:52,236 [INFO] new_opacus_codex.train_steps: epoch=4 step=620 loss=0.4674
186
+ 2026-03-28 22:25:06,359 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=620 eval_loss=0.8409 duration_sec=14.11
187
+ 2026-03-28 22:25:41,355 [INFO] new_opacus_codex.train_steps: epoch=4 step=625 loss=0.4671
188
+ 2026-03-28 22:26:17,695 [INFO] new_opacus_codex.train_steps: epoch=4 step=630 loss=0.4698
189
+ 2026-03-28 22:26:31,857 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=630 eval_loss=0.8427 duration_sec=14.14
190
+ 2026-03-28 22:27:08,191 [INFO] new_opacus_codex.train_steps: epoch=4 step=635 loss=0.5029
191
+ 2026-03-28 22:27:43,317 [INFO] new_opacus_codex.train_steps: epoch=4 step=640 loss=0.4860
192
+ 2026-03-28 22:27:57,353 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=640 eval_loss=0.8398 duration_sec=14.02
193
+ 2026-03-28 22:28:32,227 [INFO] new_opacus_codex.train_steps: epoch=4 step=645 loss=0.4673
194
+ 2026-03-28 22:29:07,352 [INFO] new_opacus_codex.train_steps: epoch=4 step=650 loss=0.4766
195
+ 2026-03-28 22:29:21,464 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=650 eval_loss=0.8393 duration_sec=14.10
196
+ 2026-03-28 22:29:57,071 [INFO] new_opacus_codex.train_steps: epoch=4 step=655 loss=0.4722
197
+ 2026-03-28 22:30:33,427 [INFO] new_opacus_codex.train_steps: epoch=4 step=660 loss=0.4940
198
+ 2026-03-28 22:30:47,480 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=660 eval_loss=0.8368 duration_sec=14.04
199
+ 2026-03-28 22:31:26,633 [INFO] new_opacus_codex.train_steps: epoch=4 step=665 loss=0.5114
200
+ 2026-03-28 22:32:02,005 [INFO] new_opacus_codex.train_steps: epoch=4 step=670 loss=0.4988
201
+ 2026-03-28 22:32:16,108 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=670 eval_loss=0.8360 duration_sec=14.09
202
+ 2026-03-28 22:32:51,451 [INFO] new_opacus_codex.train_steps: epoch=4 step=675 loss=0.4708
203
+ 2026-03-28 22:33:26,392 [INFO] new_opacus_codex.train_steps: epoch=4 step=680 loss=0.4640
204
+ 2026-03-28 22:33:40,456 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=680 eval_loss=0.8354 duration_sec=14.05
205
+ 2026-03-28 22:34:15,825 [INFO] new_opacus_codex.train_steps: epoch=4 step=685 loss=0.4745
206
+ 2026-03-28 22:34:51,948 [INFO] new_opacus_codex.train_steps: epoch=4 step=690 loss=0.4704
207
+ 2026-03-28 22:35:06,036 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=690 eval_loss=0.8319 duration_sec=14.08
208
+ 2026-03-28 22:35:41,417 [INFO] new_opacus_codex.train_steps: epoch=4 step=695 loss=0.4794
209
+ 2026-03-28 22:36:16,791 [INFO] new_opacus_codex.train_steps: epoch=4 step=700 loss=0.4845
210
+ 2026-03-28 22:36:30,915 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=700 eval_loss=0.8308 duration_sec=14.11
211
+ 2026-03-28 22:37:06,290 [INFO] new_opacus_codex.train_steps: epoch=4 step=705 loss=0.4946
212
+ 2026-03-28 22:37:41,692 [INFO] new_opacus_codex.train_steps: epoch=4 step=710 loss=0.5118
213
+ 2026-03-28 22:37:55,812 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=710 eval_loss=0.8308 duration_sec=14.11
214
+ 2026-03-28 22:38:31,559 [INFO] new_opacus_codex.train_steps: epoch=4 step=715 loss=0.4944
215
+ 2026-03-28 22:39:06,974 [INFO] new_opacus_codex.train_steps: epoch=4 step=720 loss=0.4851
216
+ 2026-03-28 22:39:21,095 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=720 eval_loss=0.8314 duration_sec=14.11
217
+ 2026-03-28 22:39:55,574 [INFO] new_opacus_codex.train_steps: epoch=4 step=725 loss=0.5051
218
+ 2026-03-28 22:40:29,779 [INFO] new_opacus_codex.train_steps: epoch=4 step=730 loss=0.5051
219
+ 2026-03-28 22:40:43,860 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=730 eval_loss=0.8270 duration_sec=14.07
220
+ 2026-03-28 22:41:50,785 [INFO] new_opacus_codex.train_steps: epoch=5 step=735 loss=0.3660
221
+ 2026-03-28 22:42:26,843 [INFO] new_opacus_codex.train_steps: epoch=5 step=740 loss=0.3864
222
+ 2026-03-28 22:42:40,958 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=740 eval_loss=0.8449 duration_sec=14.10
223
+ 2026-03-28 22:43:17,064 [INFO] new_opacus_codex.train_steps: epoch=5 step=745 loss=0.3984
224
+ 2026-03-28 22:43:51,986 [INFO] new_opacus_codex.train_steps: epoch=5 step=750 loss=0.3913
225
+ 2026-03-28 22:44:06,100 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=750 eval_loss=0.8497 duration_sec=14.10
226
+ 2026-03-28 22:44:41,293 [INFO] new_opacus_codex.train_steps: epoch=5 step=755 loss=0.3911
227
+ 2026-03-28 22:45:17,317 [INFO] new_opacus_codex.train_steps: epoch=5 step=760 loss=0.4024
228
+ 2026-03-28 22:45:31,458 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=760 eval_loss=0.8474 duration_sec=14.13
229
+ 2026-03-28 22:46:06,894 [INFO] new_opacus_codex.train_steps: epoch=5 step=765 loss=0.3897
230
+ 2026-03-28 22:46:43,519 [INFO] new_opacus_codex.train_steps: epoch=5 step=770 loss=0.3958
231
+ 2026-03-28 22:46:57,617 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=770 eval_loss=0.8484 duration_sec=14.08
232
+ 2026-03-28 22:47:32,720 [INFO] new_opacus_codex.train_steps: epoch=5 step=775 loss=0.4010
233
+ 2026-03-28 22:48:08,275 [INFO] new_opacus_codex.train_steps: epoch=5 step=780 loss=0.3965
234
+ 2026-03-28 22:48:22,341 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=780 eval_loss=0.8475 duration_sec=14.06
235
+ 2026-03-28 22:48:58,279 [INFO] new_opacus_codex.train_steps: epoch=5 step=785 loss=0.4111
236
+ 2026-03-28 22:49:33,507 [INFO] new_opacus_codex.train_steps: epoch=5 step=790 loss=0.4030
237
+ 2026-03-28 22:49:47,725 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=790 eval_loss=0.8489 duration_sec=14.20
238
+ 2026-03-28 22:50:22,634 [INFO] new_opacus_codex.train_steps: epoch=5 step=795 loss=0.4018
239
+ 2026-03-28 22:50:58,041 [INFO] new_opacus_codex.train_steps: epoch=5 step=800 loss=0.4189
240
+ 2026-03-28 22:51:12,231 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=800 eval_loss=0.8501 duration_sec=14.18
241
+ 2026-03-28 22:51:48,137 [INFO] new_opacus_codex.train_steps: epoch=5 step=805 loss=0.4070
242
+ 2026-03-28 22:52:23,827 [INFO] new_opacus_codex.train_steps: epoch=5 step=810 loss=0.3733
243
+ 2026-03-28 22:52:38,013 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=810 eval_loss=0.8499 duration_sec=14.17
244
+ 2026-03-28 22:53:12,475 [INFO] new_opacus_codex.train_steps: epoch=5 step=815 loss=0.3833
245
+ 2026-03-28 22:53:47,213 [INFO] new_opacus_codex.train_steps: epoch=5 step=820 loss=0.4063
246
+ 2026-03-28 22:54:01,453 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=820 eval_loss=0.8485 duration_sec=14.22
247
+ 2026-03-28 22:54:35,532 [INFO] new_opacus_codex.train_steps: epoch=5 step=825 loss=0.4107
248
+ 2026-03-28 22:55:10,253 [INFO] new_opacus_codex.train_steps: epoch=5 step=830 loss=0.4056
249
+ 2026-03-28 22:55:24,413 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=830 eval_loss=0.8477 duration_sec=14.14
250
+ 2026-03-28 22:56:00,538 [INFO] new_opacus_codex.train_steps: epoch=5 step=835 loss=0.4053
251
+ 2026-03-28 22:56:35,578 [INFO] new_opacus_codex.train_steps: epoch=5 step=840 loss=0.4152
252
+ 2026-03-28 22:56:49,721 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=840 eval_loss=0.8486 duration_sec=14.13
253
+ 2026-03-28 22:57:24,890 [INFO] new_opacus_codex.train_steps: epoch=5 step=845 loss=0.4165
254
+ 2026-03-28 22:57:59,854 [INFO] new_opacus_codex.train_steps: epoch=5 step=850 loss=0.4063
255
+ 2026-03-28 22:58:13,995 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=850 eval_loss=0.8471 duration_sec=14.13
256
+ 2026-03-28 22:58:48,649 [INFO] new_opacus_codex.train_steps: epoch=5 step=855 loss=0.4033
257
+ 2026-03-28 22:59:23,108 [INFO] new_opacus_codex.train_steps: epoch=5 step=860 loss=0.4217
258
+ 2026-03-28 22:59:37,262 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=860 eval_loss=0.8468 duration_sec=14.14
259
+ 2026-03-28 23:00:12,687 [INFO] new_opacus_codex.train_steps: epoch=5 step=865 loss=0.4179
260
+ 2026-03-28 23:00:48,109 [INFO] new_opacus_codex.train_steps: epoch=5 step=870 loss=0.3929
261
+ 2026-03-28 23:01:02,216 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=870 eval_loss=0.8467 duration_sec=14.09
262
+ 2026-03-28 23:01:38,095 [INFO] new_opacus_codex.train_steps: epoch=5 step=875 loss=0.3996
263
+ 2026-03-28 23:02:13,191 [INFO] new_opacus_codex.train_steps: epoch=5 step=880 loss=0.4206
264
+ 2026-03-28 23:02:27,388 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=880 eval_loss=0.8471 duration_sec=14.18
265
+ 2026-03-28 23:03:03,078 [INFO] new_opacus_codex.train_steps: epoch=5 step=885 loss=0.3948
266
+ 2026-03-28 23:03:38,787 [INFO] new_opacus_codex.train_steps: epoch=5 step=890 loss=0.3860
267
+ 2026-03-28 23:03:52,964 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=890 eval_loss=0.8470 duration_sec=14.16
268
+ 2026-03-28 23:04:27,877 [INFO] new_opacus_codex.train_steps: epoch=5 step=895 loss=0.4053
269
+ 2026-03-28 23:05:03,558 [INFO] new_opacus_codex.train_steps: epoch=5 step=900 loss=0.4076
270
+ 2026-03-28 23:05:17,646 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=900 eval_loss=0.8474 duration_sec=14.08
271
+ 2026-03-28 23:05:53,032 [INFO] new_opacus_codex.train_steps: epoch=5 step=905 loss=0.4012
272
+ 2026-03-28 23:06:27,722 [INFO] new_opacus_codex.train_steps: epoch=5 step=910 loss=0.4034
273
+ 2026-03-28 23:06:41,952 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=910 eval_loss=0.8459 duration_sec=14.22
274
+ 2026-03-28 23:07:16,666 [INFO] new_opacus_codex.train_steps: epoch=5 step=915 loss=0.4048
llama3-8b/dp3/adapter/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: meta-llama/Meta-Llama-3-8B
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:meta-llama/Meta-Llama-3-8B
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.18.1
llama3-8b/dp3/adapter/adapter_config.json ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": true,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 32,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "megatron_config": null,
23
+ "megatron_core": "megatron.core",
24
+ "modules_to_save": [
25
+ "lm_head",
26
+ "embed_tokens"
27
+ ],
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.18.1",
30
+ "qalora_group_size": 16,
31
+ "r": 16,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "o_proj",
36
+ "k_proj",
37
+ "v_proj",
38
+ "q_proj"
39
+ ],
40
+ "target_parameters": null,
41
+ "task_type": "CAUSAL_LM",
42
+ "trainable_token_indices": null,
43
+ "use_dora": false,
44
+ "use_qalora": false,
45
+ "use_rslora": false
46
+ }
llama3-8b/dp3/audit_results.json ADDED
@@ -0,0 +1,137 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "delta": 1e-05,
3
+ "num_canaries": 500,
4
+ "num_members": 250,
5
+ "paper_guess_fraction": 0.2,
6
+ "paper_guess_steps": 20,
7
+ "loss": {
8
+ "auc": 0.500744,
9
+ "empirical_epsilon": {
10
+ "0.05": 0.0,
11
+ "0.01": 0.0
12
+ },
13
+ "empirical_epsilon_details": {
14
+ "0.05": {
15
+ "epsilon": 0.0,
16
+ "num_guesses": 0,
17
+ "correct_guesses": 0,
18
+ "candidate_num_guesses": [
19
+ 5,
20
+ 10,
21
+ 15,
22
+ 20,
23
+ 25,
24
+ 30,
25
+ 35,
26
+ 40,
27
+ 45,
28
+ 50,
29
+ 55,
30
+ 60,
31
+ 65,
32
+ 70,
33
+ 75,
34
+ 80,
35
+ 85,
36
+ 90,
37
+ 95,
38
+ 100
39
+ ],
40
+ "direction": "lower"
41
+ },
42
+ "0.01": {
43
+ "epsilon": 0.0,
44
+ "num_guesses": 0,
45
+ "correct_guesses": 0,
46
+ "candidate_num_guesses": [
47
+ 5,
48
+ 10,
49
+ 15,
50
+ 20,
51
+ 25,
52
+ 30,
53
+ 35,
54
+ 40,
55
+ 45,
56
+ 50,
57
+ 55,
58
+ 60,
59
+ 65,
60
+ 70,
61
+ 75,
62
+ 80,
63
+ 85,
64
+ 90,
65
+ 95,
66
+ 100
67
+ ],
68
+ "direction": "lower"
69
+ }
70
+ }
71
+ },
72
+ "embedding": {
73
+ "auc": 0.503384,
74
+ "empirical_epsilon": {
75
+ "0.05": 0.0,
76
+ "0.01": 0.0
77
+ },
78
+ "empirical_epsilon_details": {
79
+ "0.05": {
80
+ "epsilon": 0.0,
81
+ "num_guesses": 0,
82
+ "correct_guesses": 0,
83
+ "candidate_num_guesses": [
84
+ 5,
85
+ 10,
86
+ 15,
87
+ 20,
88
+ 25,
89
+ 30,
90
+ 35,
91
+ 40,
92
+ 45,
93
+ 50,
94
+ 55,
95
+ 60,
96
+ 65,
97
+ 70,
98
+ 75,
99
+ 80,
100
+ 85,
101
+ 90,
102
+ 95,
103
+ 100
104
+ ],
105
+ "direction": "lower"
106
+ },
107
+ "0.01": {
108
+ "epsilon": 0.0,
109
+ "num_guesses": 0,
110
+ "correct_guesses": 0,
111
+ "candidate_num_guesses": [
112
+ 5,
113
+ 10,
114
+ 15,
115
+ 20,
116
+ 25,
117
+ 30,
118
+ 35,
119
+ 40,
120
+ 45,
121
+ 50,
122
+ 55,
123
+ 60,
124
+ 65,
125
+ 70,
126
+ 75,
127
+ 80,
128
+ 85,
129
+ 90,
130
+ 95,
131
+ 100
132
+ ],
133
+ "direction": "lower"
134
+ }
135
+ }
136
+ }
137
+ }
llama3-8b/dp3/canary_meta.json ADDED
The diff for this file is too large to render. See raw diff
 
llama3-8b/dp3/metrics.jsonl ADDED
The diff for this file is too large to render. See raw diff
 
llama3-8b/dp3/resolved_config.yaml ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ model:
2
+ name: meta-llama/Meta-Llama-3-8B
3
+ tokenizer_name: meta-llama/Meta-Llama-3-8B
4
+ max_length: 1024
5
+ dtype: bfloat16
6
+ trust_remote_code: true
7
+ use_fast_tokenizer: true
8
+ cache_dir: null
9
+ local_files_only: false
10
+ low_cpu_mem_usage: true
11
+ tie_word_embeddings: true
12
+ gradient_checkpointing: false
13
+ use_chat_template: false
14
+ dataset:
15
+ name: melihcatal/codedp-cpt
16
+ split: train
17
+ mode: cpt
18
+ text_column: text
19
+ validation_ratio: 0.05
20
+ max_samples: -1
21
+ lora:
22
+ enabled: true
23
+ r: 16
24
+ alpha: 32
25
+ dropout: 0.05
26
+ target_modules:
27
+ - q_proj
28
+ - k_proj
29
+ - v_proj
30
+ - o_proj
31
+ modules_to_save:
32
+ - lm_head
33
+ bias: none
34
+ training:
35
+ seed: 42
36
+ epochs: 5
37
+ warmup_steps: null
38
+ warmup_ratio: 0.05
39
+ mixed_precision: false
40
+ mixed_precision_dtype: bfloat16
41
+ batch_size: 4
42
+ eval_batch_size: 8
43
+ eval_every_steps: 10
44
+ eval_every_epochs: 1
45
+ learning_rate: 0.0005
46
+ optimizer: adamw
47
+ lr_scheduler: cosine
48
+ adam_beta1: 0.9
49
+ adam_beta2: 0.999
50
+ adam_epsilon: 1.0e-08
51
+ sgd_momentum: 0.9
52
+ weight_decay: 0.01
53
+ max_grad_norm: 1.0
54
+ log_every: 5
55
+ gradient_accumulation_steps: 8
56
+ num_workers: 4
57
+ output_dir: /scratch/mcatal/runs/cpt/llama3-8b/dp3
58
+ min_lr_ratio: 0.15
59
+ distributed:
60
+ strategy: dpddp
61
+ backend: nccl
62
+ devices: null
63
+ dp:
64
+ module_validator: auto
65
+ target_delta: 1.0e-05
66
+ noise_multiplier: null
67
+ max_grad_norm: 1.0
68
+ grad_sample_mode: hooks
69
+ clipping: per_layer
70
+ secure_mode: false
71
+ enabled: true
72
+ target_epsilon: 3.0
73
+ audit:
74
+ enabled: true
75
+ run_every_epoch: true
76
+ epoch_device: cuda
77
+ q_canary: auto
78
+ num_canaries: 500
79
+ prefix_length: 49
80
+ num_digits: 12
81
+ batch_size: 32
82
+ delta: 1.0e-05
83
+ p_values:
84
+ - 0.05
85
+ - 0.01
86
+ paper_guess_fraction: 0.2
87
+ paper_guess_steps: 20
88
+ enable_holdout_empirical_epsilon: false
89
+ holdout_seed: 42
90
+ tie_seed: 42
91
+ tracking:
92
+ enabled: true
93
+ tensorboard: true
94
+ wandb: false
95
+ wandb_project: codedp-finetune-h200-audit
96
+ wandb_run_name: llama3-8b-cpt-dp3
97
+ wandb_mode: online
98
+ codecarbon: true
99
+ codecarbon_output_file: codecarbon.csv
100
+ codecarbon_measure_power_secs: 15
101
+ codecarbon_country_iso_code: null
102
+ codecarbon_project_name: codedp-llama3-8b-cpt-dp3
llama3-8b/dp3/summary.json ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "audit/delta": 1e-05,
3
+ "audit/embedding/auc": 0.503384,
4
+ "audit/embedding/empirical_epsilon/0.01": 0.0,
5
+ "audit/embedding/empirical_epsilon/0.05": 0.0,
6
+ "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 0.0,
7
+ "audit/embedding/empirical_epsilon_details/0.01/epsilon": 0.0,
8
+ "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 0.0,
9
+ "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 0.0,
10
+ "audit/embedding/empirical_epsilon_details/0.05/epsilon": 0.0,
11
+ "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 0.0,
12
+ "audit/loss/auc": 0.500744,
13
+ "audit/loss/empirical_epsilon/0.01": 0.0,
14
+ "audit/loss/empirical_epsilon/0.05": 0.0,
15
+ "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 0.0,
16
+ "audit/loss/empirical_epsilon_details/0.01/epsilon": 0.0,
17
+ "audit/loss/empirical_epsilon_details/0.01/num_guesses": 0.0,
18
+ "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 0.0,
19
+ "audit/loss/empirical_epsilon_details/0.05/epsilon": 0.0,
20
+ "audit/loss/empirical_epsilon_details/0.05/num_guesses": 0.0,
21
+ "audit/num_canaries": 500.0,
22
+ "audit/num_members": 250.0,
23
+ "audit/paper_guess_fraction": 0.2,
24
+ "audit/paper_guess_steps": 20.0,
25
+ "energy/codecarbon/cpu_count": 16.0,
26
+ "energy/codecarbon/cpu_energy": 0.20031438086546308,
27
+ "energy/codecarbon/cpu_power": 80.0318118684876,
28
+ "energy/codecarbon/cpu_utilization_percent": 3.4759806034482756,
29
+ "energy/codecarbon/duration": 9359.37580287503,
30
+ "energy/codecarbon/emissions": 0.32605125814776115,
31
+ "energy/codecarbon/emissions_rate": 3.483685931786221e-05,
32
+ "energy/codecarbon/energy_consumed": 9.357726319426027,
33
+ "energy/codecarbon/gpu_count": 8.0,
34
+ "energy/codecarbon/gpu_energy": 9.062301173168478,
35
+ "energy/codecarbon/gpu_power": 3496.0195256746106,
36
+ "energy/codecarbon/gpu_utilization_percent": 95.7931707974138,
37
+ "energy/codecarbon/latitude": 47.4843,
38
+ "energy/codecarbon/longitude": 8.212,
39
+ "energy/codecarbon/pue": 1.0,
40
+ "energy/codecarbon/ram_energy": 0.09511076539208473,
41
+ "energy/codecarbon/ram_power": 38.0,
42
+ "energy/codecarbon/ram_total_size": 128.0,
43
+ "energy/codecarbon/ram_used_gb": 509.06644838966173,
44
+ "energy/codecarbon/ram_utilization_percent": 25.70332974137931,
45
+ "energy/codecarbon/water_consumed": 0.0,
46
+ "energy/codecarbon/wue": 0.0,
47
+ "eval/duration_sec": 13.853975180070847,
48
+ "eval/loss": 8.801205430275354,
49
+ "perf/audit_duration_sec": 7.164511439856142,
50
+ "perf/epoch_duration_sec": 1841.2133119190112,
51
+ "perf/epoch_samples": 51412.0,
52
+ "perf/epoch_samples_per_sec": 27.922891751426487,
53
+ "perf/epoch_tokens": 37419511.0,
54
+ "perf/epoch_tokens_per_sec": 20323.289408004213,
55
+ "perf/logical_batch_size": 3.0,
56
+ "perf/logical_token_count": 432.0,
57
+ "perf/physical_batches": 2.0,
58
+ "perf/samples_per_sec": 1.6842604666923884,
59
+ "perf/step_duration_sec": 1.781197183765471,
60
+ "perf/tokens_per_sec": 242.53350720370392,
61
+ "privacy/epsilon": 2.991647548922312,
62
+ "system/cuda_epoch_peak_memory_gb": 62.821791648864746,
63
+ "system/cuda_max_memory_allocated_gb": 62.821791648864746,
64
+ "system/cuda_memory_allocated_gb": 20.329766750335693,
65
+ "train/epoch_canary_loss": 10.677695805138764,
66
+ "train/epoch_loss": 9.224743401278454,
67
+ "train/epoch_real_loss": 8.985958187273985,
68
+ "train/lr": 7.5e-05,
69
+ "train/step_canary_loss": 9.791666666666666,
70
+ "train/step_loss": 9.522382020950317,
71
+ "train/step_real_loss": 8.714526176452637
72
+ }
llama3-8b/dp3/tokenizer/tokenizer_config.json ADDED
@@ -0,0 +1,516 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "bos_token": "<|begin_of_text|>",
4
+ "clean_up_tokenization_spaces": true,
5
+ "eos_token": "<|end_of_text|>",
6
+ "extra_special_tokens": {
7
+ "extra_special_token_0": "865331112869",
8
+ "extra_special_token_1": "569765693871",
9
+ "extra_special_token_2": "485177821815",
10
+ "extra_special_token_3": "135441121756",
11
+ "extra_special_token_4": "367459894796",
12
+ "extra_special_token_5": "877482678543",
13
+ "extra_special_token_6": "457919547633",
14
+ "extra_special_token_7": "765474393376",
15
+ "extra_special_token_8": "114848338811",
16
+ "extra_special_token_9": "746285987371",
17
+ "extra_special_token_10": "649291669397",
18
+ "extra_special_token_11": "927914615679",
19
+ "extra_special_token_12": "445925149649",
20
+ "extra_special_token_13": "691587454538",
21
+ "extra_special_token_14": "143777992227",
22
+ "extra_special_token_15": "997981281989",
23
+ "extra_special_token_16": "425949483533",
24
+ "extra_special_token_17": "982993456429",
25
+ "extra_special_token_18": "718726519731",
26
+ "extra_special_token_19": "172599315861",
27
+ "extra_special_token_20": "643489267333",
28
+ "extra_special_token_21": "282322838685",
29
+ "extra_special_token_22": "781653545886",
30
+ "extra_special_token_23": "796415361892",
31
+ "extra_special_token_24": "841991688488",
32
+ "extra_special_token_25": "211411365397",
33
+ "extra_special_token_26": "698218415444",
34
+ "extra_special_token_27": "355977139358",
35
+ "extra_special_token_28": "682564697312",
36
+ "extra_special_token_29": "383837596997",
37
+ "extra_special_token_30": "689362171782",
38
+ "extra_special_token_31": "749966767285",
39
+ "extra_special_token_32": "753159165157",
40
+ "extra_special_token_33": "795693824762",
41
+ "extra_special_token_34": "669689115557",
42
+ "extra_special_token_35": "327491773134",
43
+ "extra_special_token_36": "983569279932",
44
+ "extra_special_token_37": "612128769512",
45
+ "extra_special_token_38": "374327157578",
46
+ "extra_special_token_39": "311632789559",
47
+ "extra_special_token_40": "523918658846",
48
+ "extra_special_token_41": "765981581453",
49
+ "extra_special_token_42": "794825141891",
50
+ "extra_special_token_43": "873898736873",
51
+ "extra_special_token_44": "447445629421",
52
+ "extra_special_token_45": "473822473819",
53
+ "extra_special_token_46": "181439694557",
54
+ "extra_special_token_47": "592538279337",
55
+ "extra_special_token_48": "668134915514",
56
+ "extra_special_token_49": "643692393748",
57
+ "extra_special_token_50": "696651276628",
58
+ "extra_special_token_51": "853859348234",
59
+ "extra_special_token_52": "778466723723",
60
+ "extra_special_token_53": "929826356991",
61
+ "extra_special_token_54": "272362973463",
62
+ "extra_special_token_55": "694235616268",
63
+ "extra_special_token_56": "281673864127",
64
+ "extra_special_token_57": "479676316326",
65
+ "extra_special_token_58": "646979124677",
66
+ "extra_special_token_59": "922327493433",
67
+ "extra_special_token_60": "883685933161",
68
+ "extra_special_token_61": "264259917554",
69
+ "extra_special_token_62": "836746273134",
70
+ "extra_special_token_63": "658481324922",
71
+ "extra_special_token_64": "481884157827",
72
+ "extra_special_token_65": "587787496812",
73
+ "extra_special_token_66": "579184949249",
74
+ "extra_special_token_67": "912193598348",
75
+ "extra_special_token_68": "529679678956",
76
+ "extra_special_token_69": "795838284624",
77
+ "extra_special_token_70": "159337222655",
78
+ "extra_special_token_71": "173781362446",
79
+ "extra_special_token_72": "773687856563",
80
+ "extra_special_token_73": "535787224917",
81
+ "extra_special_token_74": "351885857332",
82
+ "extra_special_token_75": "578827344666",
83
+ "extra_special_token_76": "198462689911",
84
+ "extra_special_token_77": "722618266242",
85
+ "extra_special_token_78": "952872416512",
86
+ "extra_special_token_79": "517778845323",
87
+ "extra_special_token_80": "749665846687",
88
+ "extra_special_token_81": "661436365453",
89
+ "extra_special_token_82": "259666844669",
90
+ "extra_special_token_83": "242851284913",
91
+ "extra_special_token_84": "514532995959",
92
+ "extra_special_token_85": "161588262349",
93
+ "extra_special_token_86": "742765629356",
94
+ "extra_special_token_87": "225164373623",
95
+ "extra_special_token_88": "676539973863",
96
+ "extra_special_token_89": "826214551218",
97
+ "extra_special_token_90": "182345464792",
98
+ "extra_special_token_91": "232776999554",
99
+ "extra_special_token_92": "337326533813",
100
+ "extra_special_token_93": "676676697292",
101
+ "extra_special_token_94": "929185622831",
102
+ "extra_special_token_95": "545512344383",
103
+ "extra_special_token_96": "499444466686",
104
+ "extra_special_token_97": "314697386682",
105
+ "extra_special_token_98": "517379856925",
106
+ "extra_special_token_99": "379557332953",
107
+ "extra_special_token_100": "614797267726",
108
+ "extra_special_token_101": "429781429464",
109
+ "extra_special_token_102": "922466849763",
110
+ "extra_special_token_103": "721737645236",
111
+ "extra_special_token_104": "479227349997",
112
+ "extra_special_token_105": "136931728327",
113
+ "extra_special_token_106": "259533577263",
114
+ "extra_special_token_107": "488538864842",
115
+ "extra_special_token_108": "937495658852",
116
+ "extra_special_token_109": "489991411364",
117
+ "extra_special_token_110": "499148455254",
118
+ "extra_special_token_111": "441373944925",
119
+ "extra_special_token_112": "899151413682",
120
+ "extra_special_token_113": "467893531755",
121
+ "extra_special_token_114": "527117488925",
122
+ "extra_special_token_115": "928335588653",
123
+ "extra_special_token_116": "374439448821",
124
+ "extra_special_token_117": "879425227932",
125
+ "extra_special_token_118": "867678158885",
126
+ "extra_special_token_119": "399749397872",
127
+ "extra_special_token_120": "129693547287",
128
+ "extra_special_token_121": "689285841825",
129
+ "extra_special_token_122": "771619544974",
130
+ "extra_special_token_123": "724883568652",
131
+ "extra_special_token_124": "516968424863",
132
+ "extra_special_token_125": "733737988257",
133
+ "extra_special_token_126": "852347289392",
134
+ "extra_special_token_127": "296953381169",
135
+ "extra_special_token_128": "377273562477",
136
+ "extra_special_token_129": "262296912232",
137
+ "extra_special_token_130": "547149832394",
138
+ "extra_special_token_131": "298464134954",
139
+ "extra_special_token_132": "216667245274",
140
+ "extra_special_token_133": "843998562287",
141
+ "extra_special_token_134": "572154333646",
142
+ "extra_special_token_135": "124589118494",
143
+ "extra_special_token_136": "841824384614",
144
+ "extra_special_token_137": "232896526252",
145
+ "extra_special_token_138": "295448593321",
146
+ "extra_special_token_139": "123741461297",
147
+ "extra_special_token_140": "653573457168",
148
+ "extra_special_token_141": "196735786156",
149
+ "extra_special_token_142": "377338713663",
150
+ "extra_special_token_143": "964342468552",
151
+ "extra_special_token_144": "586855179568",
152
+ "extra_special_token_145": "484773717614",
153
+ "extra_special_token_146": "894885246797",
154
+ "extra_special_token_147": "677896358599",
155
+ "extra_special_token_148": "848845611563",
156
+ "extra_special_token_149": "851852651677",
157
+ "extra_special_token_150": "398549545767",
158
+ "extra_special_token_151": "454244839926",
159
+ "extra_special_token_152": "799364566435",
160
+ "extra_special_token_153": "967114116556",
161
+ "extra_special_token_154": "817378986438",
162
+ "extra_special_token_155": "233795848681",
163
+ "extra_special_token_156": "824387273757",
164
+ "extra_special_token_157": "916198946615",
165
+ "extra_special_token_158": "563117729724",
166
+ "extra_special_token_159": "951794811935",
167
+ "extra_special_token_160": "374598961236",
168
+ "extra_special_token_161": "922867396683",
169
+ "extra_special_token_162": "765737843639",
170
+ "extra_special_token_163": "175469284871",
171
+ "extra_special_token_164": "231853711778",
172
+ "extra_special_token_165": "662426712668",
173
+ "extra_special_token_166": "711412347158",
174
+ "extra_special_token_167": "753466987363",
175
+ "extra_special_token_168": "513361312532",
176
+ "extra_special_token_169": "712992815957",
177
+ "extra_special_token_170": "971621888444",
178
+ "extra_special_token_171": "829235161526",
179
+ "extra_special_token_172": "585544633356",
180
+ "extra_special_token_173": "582471228164",
181
+ "extra_special_token_174": "678666359123",
182
+ "extra_special_token_175": "557533689478",
183
+ "extra_special_token_176": "632962475133",
184
+ "extra_special_token_177": "484489193824",
185
+ "extra_special_token_178": "489562189822",
186
+ "extra_special_token_179": "589547936288",
187
+ "extra_special_token_180": "363214487524",
188
+ "extra_special_token_181": "244885399387",
189
+ "extra_special_token_182": "431751228368",
190
+ "extra_special_token_183": "433581868192",
191
+ "extra_special_token_184": "486391569221",
192
+ "extra_special_token_185": "185438575221",
193
+ "extra_special_token_186": "126574388585",
194
+ "extra_special_token_187": "741757479784",
195
+ "extra_special_token_188": "529854679937",
196
+ "extra_special_token_189": "996116119839",
197
+ "extra_special_token_190": "616248973917",
198
+ "extra_special_token_191": "763531783491",
199
+ "extra_special_token_192": "955456118295",
200
+ "extra_special_token_193": "364196983365",
201
+ "extra_special_token_194": "195792996468",
202
+ "extra_special_token_195": "151859598873",
203
+ "extra_special_token_196": "399223169721",
204
+ "extra_special_token_197": "938488813964",
205
+ "extra_special_token_198": "961981959227",
206
+ "extra_special_token_199": "183368827562",
207
+ "extra_special_token_200": "533417736566",
208
+ "extra_special_token_201": "786391632558",
209
+ "extra_special_token_202": "665661658354",
210
+ "extra_special_token_203": "693281533643",
211
+ "extra_special_token_204": "475794684356",
212
+ "extra_special_token_205": "652154162978",
213
+ "extra_special_token_206": "753233719644",
214
+ "extra_special_token_207": "668514843129",
215
+ "extra_special_token_208": "819162623892",
216
+ "extra_special_token_209": "941169431859",
217
+ "extra_special_token_210": "877385381798",
218
+ "extra_special_token_211": "752644929761",
219
+ "extra_special_token_212": "881136466196",
220
+ "extra_special_token_213": "275597777299",
221
+ "extra_special_token_214": "731681792655",
222
+ "extra_special_token_215": "961133895172",
223
+ "extra_special_token_216": "864718285734",
224
+ "extra_special_token_217": "963852916563",
225
+ "extra_special_token_218": "319584985416",
226
+ "extra_special_token_219": "563365646341",
227
+ "extra_special_token_220": "811371928234",
228
+ "extra_special_token_221": "837131396371",
229
+ "extra_special_token_222": "267514771964",
230
+ "extra_special_token_223": "944513428457",
231
+ "extra_special_token_224": "117298239631",
232
+ "extra_special_token_225": "158142752582",
233
+ "extra_special_token_226": "252867443568",
234
+ "extra_special_token_227": "839269684865",
235
+ "extra_special_token_228": "612788593128",
236
+ "extra_special_token_229": "145669731981",
237
+ "extra_special_token_230": "121557291859",
238
+ "extra_special_token_231": "245416776926",
239
+ "extra_special_token_232": "799417897197",
240
+ "extra_special_token_233": "997958836435",
241
+ "extra_special_token_234": "892336777248",
242
+ "extra_special_token_235": "158929292238",
243
+ "extra_special_token_236": "581976444672",
244
+ "extra_special_token_237": "897784492783",
245
+ "extra_special_token_238": "492373714791",
246
+ "extra_special_token_239": "512659818733",
247
+ "extra_special_token_240": "881112998642",
248
+ "extra_special_token_241": "619454958782",
249
+ "extra_special_token_242": "431149748713",
250
+ "extra_special_token_243": "624221476921",
251
+ "extra_special_token_244": "125866399464",
252
+ "extra_special_token_245": "339882449689",
253
+ "extra_special_token_246": "186198784585",
254
+ "extra_special_token_247": "943193294691",
255
+ "extra_special_token_248": "955668961269",
256
+ "extra_special_token_249": "232787996724",
257
+ "extra_special_token_250": "215671314196",
258
+ "extra_special_token_251": "286173241916",
259
+ "extra_special_token_252": "745977673725",
260
+ "extra_special_token_253": "556976448182",
261
+ "extra_special_token_254": "599961512792",
262
+ "extra_special_token_255": "766294538337",
263
+ "extra_special_token_256": "934912591213",
264
+ "extra_special_token_257": "295118729589",
265
+ "extra_special_token_258": "529455466433",
266
+ "extra_special_token_259": "196119929397",
267
+ "extra_special_token_260": "379571934299",
268
+ "extra_special_token_261": "251789649997",
269
+ "extra_special_token_262": "564544131355",
270
+ "extra_special_token_263": "244371196654",
271
+ "extra_special_token_264": "384598329253",
272
+ "extra_special_token_265": "887753195844",
273
+ "extra_special_token_266": "364947325679",
274
+ "extra_special_token_267": "655517954651",
275
+ "extra_special_token_268": "673948786567",
276
+ "extra_special_token_269": "857231548835",
277
+ "extra_special_token_270": "816115936673",
278
+ "extra_special_token_271": "644234165531",
279
+ "extra_special_token_272": "182782912224",
280
+ "extra_special_token_273": "234316622259",
281
+ "extra_special_token_274": "421369185549",
282
+ "extra_special_token_275": "434632855397",
283
+ "extra_special_token_276": "921889371893",
284
+ "extra_special_token_277": "415956914763",
285
+ "extra_special_token_278": "598916996413",
286
+ "extra_special_token_279": "773671349113",
287
+ "extra_special_token_280": "952465217972",
288
+ "extra_special_token_281": "117657531962",
289
+ "extra_special_token_282": "729825168745",
290
+ "extra_special_token_283": "691315125346",
291
+ "extra_special_token_284": "768461952319",
292
+ "extra_special_token_285": "664847713559",
293
+ "extra_special_token_286": "953267689786",
294
+ "extra_special_token_287": "886464195129",
295
+ "extra_special_token_288": "824488329416",
296
+ "extra_special_token_289": "837873762491",
297
+ "extra_special_token_290": "532833541879",
298
+ "extra_special_token_291": "669183782449",
299
+ "extra_special_token_292": "941976537588",
300
+ "extra_special_token_293": "739394546916",
301
+ "extra_special_token_294": "267954879268",
302
+ "extra_special_token_295": "637551427887",
303
+ "extra_special_token_296": "217756494954",
304
+ "extra_special_token_297": "524444658383",
305
+ "extra_special_token_298": "117783274348",
306
+ "extra_special_token_299": "138218735276",
307
+ "extra_special_token_300": "814611949491",
308
+ "extra_special_token_301": "711641973413",
309
+ "extra_special_token_302": "499156317423",
310
+ "extra_special_token_303": "515856611931",
311
+ "extra_special_token_304": "454164859837",
312
+ "extra_special_token_305": "345271433112",
313
+ "extra_special_token_306": "462294118988",
314
+ "extra_special_token_307": "511785788222",
315
+ "extra_special_token_308": "497294727353",
316
+ "extra_special_token_309": "866519986723",
317
+ "extra_special_token_310": "334513529294",
318
+ "extra_special_token_311": "549946382131",
319
+ "extra_special_token_312": "284445431422",
320
+ "extra_special_token_313": "396521188476",
321
+ "extra_special_token_314": "421435255895",
322
+ "extra_special_token_315": "133373659361",
323
+ "extra_special_token_316": "322683334381",
324
+ "extra_special_token_317": "228358422847",
325
+ "extra_special_token_318": "291762694874",
326
+ "extra_special_token_319": "143182978129",
327
+ "extra_special_token_320": "511923256573",
328
+ "extra_special_token_321": "327158398268",
329
+ "extra_special_token_322": "879764613759",
330
+ "extra_special_token_323": "564395222747",
331
+ "extra_special_token_324": "451161679736",
332
+ "extra_special_token_325": "538631466654",
333
+ "extra_special_token_326": "221762325616",
334
+ "extra_special_token_327": "218391991184",
335
+ "extra_special_token_328": "322589379462",
336
+ "extra_special_token_329": "876537814263",
337
+ "extra_special_token_330": "152676556624",
338
+ "extra_special_token_331": "332522971941",
339
+ "extra_special_token_332": "884354318946",
340
+ "extra_special_token_333": "513349618943",
341
+ "extra_special_token_334": "116639746413",
342
+ "extra_special_token_335": "635185846287",
343
+ "extra_special_token_336": "993832498489",
344
+ "extra_special_token_337": "813981174797",
345
+ "extra_special_token_338": "438745114173",
346
+ "extra_special_token_339": "983493951323",
347
+ "extra_special_token_340": "724492262421",
348
+ "extra_special_token_341": "622553389126",
349
+ "extra_special_token_342": "889965243135",
350
+ "extra_special_token_343": "364492359246",
351
+ "extra_special_token_344": "154962668224",
352
+ "extra_special_token_345": "179564995814",
353
+ "extra_special_token_346": "418412875665",
354
+ "extra_special_token_347": "718951851413",
355
+ "extra_special_token_348": "699446724178",
356
+ "extra_special_token_349": "624266421831",
357
+ "extra_special_token_350": "815458725125",
358
+ "extra_special_token_351": "455423278865",
359
+ "extra_special_token_352": "393741199486",
360
+ "extra_special_token_353": "328552864359",
361
+ "extra_special_token_354": "211662639865",
362
+ "extra_special_token_355": "218784516525",
363
+ "extra_special_token_356": "762486672996",
364
+ "extra_special_token_357": "142799718159",
365
+ "extra_special_token_358": "858146415154",
366
+ "extra_special_token_359": "767858144912",
367
+ "extra_special_token_360": "571317457151",
368
+ "extra_special_token_361": "635127952696",
369
+ "extra_special_token_362": "116427191984",
370
+ "extra_special_token_363": "268921994538",
371
+ "extra_special_token_364": "523937669294",
372
+ "extra_special_token_365": "165429152138",
373
+ "extra_special_token_366": "739246183345",
374
+ "extra_special_token_367": "591464355756",
375
+ "extra_special_token_368": "212985874612",
376
+ "extra_special_token_369": "191887635211",
377
+ "extra_special_token_370": "967214577653",
378
+ "extra_special_token_371": "119342152414",
379
+ "extra_special_token_372": "946444632795",
380
+ "extra_special_token_373": "618423867817",
381
+ "extra_special_token_374": "228565148417",
382
+ "extra_special_token_375": "729116422489",
383
+ "extra_special_token_376": "527874729936",
384
+ "extra_special_token_377": "739784153482",
385
+ "extra_special_token_378": "387763951128",
386
+ "extra_special_token_379": "331369926711",
387
+ "extra_special_token_380": "562716493614",
388
+ "extra_special_token_381": "739667844957",
389
+ "extra_special_token_382": "562389434565",
390
+ "extra_special_token_383": "256497188281",
391
+ "extra_special_token_384": "859927364588",
392
+ "extra_special_token_385": "417668946583",
393
+ "extra_special_token_386": "357621613582",
394
+ "extra_special_token_387": "438435178228",
395
+ "extra_special_token_388": "485692541169",
396
+ "extra_special_token_389": "825815739116",
397
+ "extra_special_token_390": "342221452223",
398
+ "extra_special_token_391": "697747991249",
399
+ "extra_special_token_392": "716763689965",
400
+ "extra_special_token_393": "141499982867",
401
+ "extra_special_token_394": "818479319499",
402
+ "extra_special_token_395": "336813343298",
403
+ "extra_special_token_396": "594688742928",
404
+ "extra_special_token_397": "472129283475",
405
+ "extra_special_token_398": "514354144759",
406
+ "extra_special_token_399": "349249721685",
407
+ "extra_special_token_400": "546276298359",
408
+ "extra_special_token_401": "353755529131",
409
+ "extra_special_token_402": "315534574435",
410
+ "extra_special_token_403": "523723475786",
411
+ "extra_special_token_404": "215826764872",
412
+ "extra_special_token_405": "367968398551",
413
+ "extra_special_token_406": "569853653352",
414
+ "extra_special_token_407": "389715484387",
415
+ "extra_special_token_408": "293847485454",
416
+ "extra_special_token_409": "714738141818",
417
+ "extra_special_token_410": "178478368922",
418
+ "extra_special_token_411": "581493616981",
419
+ "extra_special_token_412": "589439538674",
420
+ "extra_special_token_413": "846657726193",
421
+ "extra_special_token_414": "722339992679",
422
+ "extra_special_token_415": "138154781148",
423
+ "extra_special_token_416": "757785319772",
424
+ "extra_special_token_417": "492516914298",
425
+ "extra_special_token_418": "919181521716",
426
+ "extra_special_token_419": "985781138935",
427
+ "extra_special_token_420": "476969195485",
428
+ "extra_special_token_421": "313145133463",
429
+ "extra_special_token_422": "758963111966",
430
+ "extra_special_token_423": "147541537162",
431
+ "extra_special_token_424": "557163366873",
432
+ "extra_special_token_425": "144373897488",
433
+ "extra_special_token_426": "522515164754",
434
+ "extra_special_token_427": "724964923582",
435
+ "extra_special_token_428": "284776712475",
436
+ "extra_special_token_429": "375429755114",
437
+ "extra_special_token_430": "181233596124",
438
+ "extra_special_token_431": "948585673431",
439
+ "extra_special_token_432": "243165586174",
440
+ "extra_special_token_433": "396847976144",
441
+ "extra_special_token_434": "997724962668",
442
+ "extra_special_token_435": "558837194455",
443
+ "extra_special_token_436": "163165456396",
444
+ "extra_special_token_437": "378749551722",
445
+ "extra_special_token_438": "161238482259",
446
+ "extra_special_token_439": "754978243758",
447
+ "extra_special_token_440": "195388849133",
448
+ "extra_special_token_441": "229775525672",
449
+ "extra_special_token_442": "262437452884",
450
+ "extra_special_token_443": "441377892146",
451
+ "extra_special_token_444": "451885565366",
452
+ "extra_special_token_445": "981277526855",
453
+ "extra_special_token_446": "762495822823",
454
+ "extra_special_token_447": "368763327262",
455
+ "extra_special_token_448": "757422791351",
456
+ "extra_special_token_449": "636324136426",
457
+ "extra_special_token_450": "214193645583",
458
+ "extra_special_token_451": "412843856172",
459
+ "extra_special_token_452": "179386156569",
460
+ "extra_special_token_453": "756916173536",
461
+ "extra_special_token_454": "892697125149",
462
+ "extra_special_token_455": "625334487352",
463
+ "extra_special_token_456": "941861857715",
464
+ "extra_special_token_457": "887417525236",
465
+ "extra_special_token_458": "649516938598",
466
+ "extra_special_token_459": "717628619782",
467
+ "extra_special_token_460": "438124184139",
468
+ "extra_special_token_461": "547563892268",
469
+ "extra_special_token_462": "856317483891",
470
+ "extra_special_token_463": "313313831273",
471
+ "extra_special_token_464": "371496153876",
472
+ "extra_special_token_465": "587541149322",
473
+ "extra_special_token_466": "265847332563",
474
+ "extra_special_token_467": "449549215429",
475
+ "extra_special_token_468": "163497196769",
476
+ "extra_special_token_469": "861342291298",
477
+ "extra_special_token_470": "268433315926",
478
+ "extra_special_token_471": "774679513717",
479
+ "extra_special_token_472": "851254219729",
480
+ "extra_special_token_473": "583527834464",
481
+ "extra_special_token_474": "488496781997",
482
+ "extra_special_token_475": "556814553861",
483
+ "extra_special_token_476": "482829231639",
484
+ "extra_special_token_477": "618878266619",
485
+ "extra_special_token_478": "147444452794",
486
+ "extra_special_token_479": "949235426629",
487
+ "extra_special_token_480": "357299947518",
488
+ "extra_special_token_481": "175528632226",
489
+ "extra_special_token_482": "645527857972",
490
+ "extra_special_token_483": "186872457894",
491
+ "extra_special_token_484": "552738847828",
492
+ "extra_special_token_485": "626748382482",
493
+ "extra_special_token_486": "921894985642",
494
+ "extra_special_token_487": "943878645871",
495
+ "extra_special_token_488": "859289776479",
496
+ "extra_special_token_489": "614583493135",
497
+ "extra_special_token_490": "933775286797",
498
+ "extra_special_token_491": "332234613346",
499
+ "extra_special_token_492": "325196781219",
500
+ "extra_special_token_493": "142526557681",
501
+ "extra_special_token_494": "356722692178",
502
+ "extra_special_token_495": "449318681694",
503
+ "extra_special_token_496": "687284547244",
504
+ "extra_special_token_497": "947262995132",
505
+ "extra_special_token_498": "893974619684",
506
+ "extra_special_token_499": "797238311233"
507
+ },
508
+ "is_local": false,
509
+ "model_input_names": [
510
+ "input_ids",
511
+ "attention_mask"
512
+ ],
513
+ "model_max_length": 1000000000000000019884624838656,
514
+ "pad_token": "<|end_of_text|>",
515
+ "tokenizer_class": "PreTrainedTokenizerFast"
516
+ }
llama3-8b/dp3/train.log ADDED
@@ -0,0 +1,276 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 2026-03-28 23:11:01,165 [INFO] new_opacus_codex.train_steps: epoch=1 step=5 loss=12.6432
2
+ 2026-03-28 23:11:43,477 [INFO] new_opacus_codex.train_steps: epoch=1 step=10 loss=12.6414
3
+ 2026-03-28 23:11:57,325 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=10 eval_loss=12.9603 duration_sec=13.84
4
+ 2026-03-28 23:12:40,509 [INFO] new_opacus_codex.train_steps: epoch=1 step=15 loss=12.6380
5
+ 2026-03-28 23:13:23,312 [INFO] new_opacus_codex.train_steps: epoch=1 step=20 loss=12.5912
6
+ 2026-03-28 23:13:37,137 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=20 eval_loss=12.9465 duration_sec=13.81
7
+ 2026-03-28 23:14:19,557 [INFO] new_opacus_codex.train_steps: epoch=1 step=25 loss=12.6499
8
+ 2026-03-28 23:15:01,584 [INFO] new_opacus_codex.train_steps: epoch=1 step=30 loss=12.6518
9
+ 2026-03-28 23:15:15,421 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=30 eval_loss=12.9251 duration_sec=13.82
10
+ 2026-03-28 23:15:57,986 [INFO] new_opacus_codex.train_steps: epoch=1 step=35 loss=12.5443
11
+ 2026-03-28 23:16:39,903 [INFO] new_opacus_codex.train_steps: epoch=1 step=40 loss=12.6735
12
+ 2026-03-28 23:16:53,773 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=40 eval_loss=12.9026 duration_sec=13.86
13
+ 2026-03-28 23:17:36,151 [INFO] new_opacus_codex.train_steps: epoch=1 step=45 loss=12.7014
14
+ 2026-03-28 23:18:18,105 [INFO] new_opacus_codex.train_steps: epoch=1 step=50 loss=12.5941
15
+ 2026-03-28 23:18:31,999 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=50 eval_loss=12.8841 duration_sec=13.88
16
+ 2026-03-28 23:19:13,392 [INFO] new_opacus_codex.train_steps: epoch=1 step=55 loss=12.6160
17
+ 2026-03-28 23:19:55,620 [INFO] new_opacus_codex.train_steps: epoch=1 step=60 loss=12.6089
18
+ 2026-03-28 23:20:09,462 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=60 eval_loss=12.8592 duration_sec=13.83
19
+ 2026-03-28 23:20:51,573 [INFO] new_opacus_codex.train_steps: epoch=1 step=65 loss=12.6446
20
+ 2026-03-28 23:21:34,171 [INFO] new_opacus_codex.train_steps: epoch=1 step=70 loss=12.6905
21
+ 2026-03-28 23:21:47,989 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=70 eval_loss=12.8266 duration_sec=13.81
22
+ 2026-03-28 23:22:30,962 [INFO] new_opacus_codex.train_steps: epoch=1 step=75 loss=12.6714
23
+ 2026-03-28 23:23:15,250 [INFO] new_opacus_codex.train_steps: epoch=1 step=80 loss=12.5575
24
+ 2026-03-28 23:23:29,118 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=80 eval_loss=12.7882 duration_sec=13.85
25
+ 2026-03-28 23:24:11,826 [INFO] new_opacus_codex.train_steps: epoch=1 step=85 loss=12.4859
26
+ 2026-03-28 23:24:54,943 [INFO] new_opacus_codex.train_steps: epoch=1 step=90 loss=12.4751
27
+ 2026-03-28 23:25:08,792 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=90 eval_loss=12.7535 duration_sec=13.84
28
+ 2026-03-28 23:25:50,756 [INFO] new_opacus_codex.train_steps: epoch=1 step=95 loss=12.5629
29
+ 2026-03-28 23:26:32,824 [INFO] new_opacus_codex.train_steps: epoch=1 step=100 loss=12.6614
30
+ 2026-03-28 23:26:46,637 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=100 eval_loss=12.7082 duration_sec=13.80
31
+ 2026-03-28 23:27:29,979 [INFO] new_opacus_codex.train_steps: epoch=1 step=105 loss=12.5793
32
+ 2026-03-28 23:28:11,489 [INFO] new_opacus_codex.train_steps: epoch=1 step=110 loss=12.5831
33
+ 2026-03-28 23:28:25,421 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=110 eval_loss=12.6636 duration_sec=13.92
34
+ 2026-03-28 23:29:07,864 [INFO] new_opacus_codex.train_steps: epoch=1 step=115 loss=12.5200
35
+ 2026-03-28 23:29:49,800 [INFO] new_opacus_codex.train_steps: epoch=1 step=120 loss=12.4339
36
+ 2026-03-28 23:30:03,701 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=120 eval_loss=12.6109 duration_sec=13.88
37
+ 2026-03-28 23:30:45,512 [INFO] new_opacus_codex.train_steps: epoch=1 step=125 loss=12.5113
38
+ 2026-03-28 23:31:27,564 [INFO] new_opacus_codex.train_steps: epoch=1 step=130 loss=12.4431
39
+ 2026-03-28 23:31:41,472 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=130 eval_loss=12.5713 duration_sec=13.90
40
+ 2026-03-28 23:32:23,982 [INFO] new_opacus_codex.train_steps: epoch=1 step=135 loss=12.3718
41
+ 2026-03-28 23:33:05,352 [INFO] new_opacus_codex.train_steps: epoch=1 step=140 loss=12.4491
42
+ 2026-03-28 23:33:19,257 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=140 eval_loss=12.5249 duration_sec=13.89
43
+ 2026-03-28 23:34:01,593 [INFO] new_opacus_codex.train_steps: epoch=1 step=145 loss=12.4154
44
+ 2026-03-28 23:34:44,471 [INFO] new_opacus_codex.train_steps: epoch=1 step=150 loss=12.3379
45
+ 2026-03-28 23:34:58,341 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=150 eval_loss=12.4735 duration_sec=13.86
46
+ 2026-03-28 23:35:40,064 [INFO] new_opacus_codex.train_steps: epoch=1 step=155 loss=12.2029
47
+ 2026-03-28 23:36:20,719 [INFO] new_opacus_codex.train_steps: epoch=1 step=160 loss=12.2308
48
+ 2026-03-28 23:36:34,593 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=160 eval_loss=12.4154 duration_sec=13.86
49
+ 2026-03-28 23:37:17,271 [INFO] new_opacus_codex.train_steps: epoch=1 step=165 loss=12.3442
50
+ 2026-03-28 23:37:59,607 [INFO] new_opacus_codex.train_steps: epoch=1 step=170 loss=12.2505
51
+ 2026-03-28 23:38:13,514 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=170 eval_loss=12.3457 duration_sec=13.90
52
+ 2026-03-28 23:38:55,266 [INFO] new_opacus_codex.train_steps: epoch=1 step=175 loss=12.1596
53
+ 2026-03-28 23:39:37,975 [INFO] new_opacus_codex.train_steps: epoch=1 step=180 loss=12.1997
54
+ 2026-03-28 23:39:51,795 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=180 eval_loss=12.2751 duration_sec=13.81
55
+ 2026-03-28 23:40:53,138 [INFO] new_opacus_codex.train_steps: epoch=2 step=185 loss=12.4353
56
+ 2026-03-28 23:41:36,010 [INFO] new_opacus_codex.train_steps: epoch=2 step=190 loss=12.1768
57
+ 2026-03-28 23:41:49,970 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=190 eval_loss=12.2012 duration_sec=13.95
58
+ 2026-03-28 23:42:32,638 [INFO] new_opacus_codex.train_steps: epoch=2 step=195 loss=12.1148
59
+ 2026-03-28 23:43:15,056 [INFO] new_opacus_codex.train_steps: epoch=2 step=200 loss=12.0217
60
+ 2026-03-28 23:43:28,953 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=200 eval_loss=12.1232 duration_sec=13.89
61
+ 2026-03-28 23:44:11,394 [INFO] new_opacus_codex.train_steps: epoch=2 step=205 loss=11.9269
62
+ 2026-03-28 23:44:54,204 [INFO] new_opacus_codex.train_steps: epoch=2 step=210 loss=11.9084
63
+ 2026-03-28 23:45:08,052 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=210 eval_loss=12.0318 duration_sec=13.84
64
+ 2026-03-28 23:45:50,519 [INFO] new_opacus_codex.train_steps: epoch=2 step=215 loss=11.9058
65
+ 2026-03-28 23:46:33,374 [INFO] new_opacus_codex.train_steps: epoch=2 step=220 loss=11.9267
66
+ 2026-03-28 23:46:47,242 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=220 eval_loss=11.9401 duration_sec=13.86
67
+ 2026-03-28 23:47:29,864 [INFO] new_opacus_codex.train_steps: epoch=2 step=225 loss=11.8353
68
+ 2026-03-28 23:48:11,540 [INFO] new_opacus_codex.train_steps: epoch=2 step=230 loss=11.8237
69
+ 2026-03-28 23:48:25,456 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=230 eval_loss=11.8506 duration_sec=13.90
70
+ 2026-03-28 23:49:07,021 [INFO] new_opacus_codex.train_steps: epoch=2 step=235 loss=11.9273
71
+ 2026-03-28 23:49:50,030 [INFO] new_opacus_codex.train_steps: epoch=2 step=240 loss=11.8904
72
+ 2026-03-28 23:50:03,940 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=240 eval_loss=11.7533 duration_sec=13.90
73
+ 2026-03-28 23:50:46,082 [INFO] new_opacus_codex.train_steps: epoch=2 step=245 loss=11.6899
74
+ 2026-03-28 23:51:28,724 [INFO] new_opacus_codex.train_steps: epoch=2 step=250 loss=11.6593
75
+ 2026-03-28 23:51:42,609 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=250 eval_loss=11.6643 duration_sec=13.87
76
+ 2026-03-28 23:52:25,472 [INFO] new_opacus_codex.train_steps: epoch=2 step=255 loss=11.6412
77
+ 2026-03-28 23:53:06,845 [INFO] new_opacus_codex.train_steps: epoch=2 step=260 loss=11.5150
78
+ 2026-03-28 23:53:20,765 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=260 eval_loss=11.5679 duration_sec=13.91
79
+ 2026-03-28 23:54:02,397 [INFO] new_opacus_codex.train_steps: epoch=2 step=265 loss=11.4861
80
+ 2026-03-28 23:54:44,283 [INFO] new_opacus_codex.train_steps: epoch=2 step=270 loss=11.4992
81
+ 2026-03-28 23:54:58,187 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=270 eval_loss=11.4677 duration_sec=13.89
82
+ 2026-03-28 23:55:40,543 [INFO] new_opacus_codex.train_steps: epoch=2 step=275 loss=11.5161
83
+ 2026-03-28 23:56:24,051 [INFO] new_opacus_codex.train_steps: epoch=2 step=280 loss=11.4073
84
+ 2026-03-28 23:56:37,915 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=280 eval_loss=11.3674 duration_sec=13.85
85
+ 2026-03-28 23:57:20,070 [INFO] new_opacus_codex.train_steps: epoch=2 step=285 loss=11.3451
86
+ 2026-03-28 23:58:02,627 [INFO] new_opacus_codex.train_steps: epoch=2 step=290 loss=11.3256
87
+ 2026-03-28 23:58:16,559 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=290 eval_loss=11.2722 duration_sec=13.92
88
+ 2026-03-28 23:58:59,496 [INFO] new_opacus_codex.train_steps: epoch=2 step=295 loss=11.2640
89
+ 2026-03-28 23:59:41,663 [INFO] new_opacus_codex.train_steps: epoch=2 step=300 loss=11.2747
90
+ 2026-03-28 23:59:55,605 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=300 eval_loss=11.1742 duration_sec=13.93
91
+ 2026-03-29 00:00:38,906 [INFO] new_opacus_codex.train_steps: epoch=2 step=305 loss=11.2496
92
+ 2026-03-29 00:01:22,464 [INFO] new_opacus_codex.train_steps: epoch=2 step=310 loss=11.1926
93
+ 2026-03-29 00:01:36,318 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=310 eval_loss=11.0810 duration_sec=13.84
94
+ 2026-03-29 00:02:19,478 [INFO] new_opacus_codex.train_steps: epoch=2 step=315 loss=11.1307
95
+ 2026-03-29 00:03:02,826 [INFO] new_opacus_codex.train_steps: epoch=2 step=320 loss=11.1658
96
+ 2026-03-29 00:03:16,765 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=320 eval_loss=10.9768 duration_sec=13.92
97
+ 2026-03-29 00:03:59,581 [INFO] new_opacus_codex.train_steps: epoch=2 step=325 loss=11.1377
98
+ 2026-03-29 00:04:41,313 [INFO] new_opacus_codex.train_steps: epoch=2 step=330 loss=11.0942
99
+ 2026-03-29 00:04:55,214 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=330 eval_loss=10.8757 duration_sec=13.89
100
+ 2026-03-29 00:05:36,107 [INFO] new_opacus_codex.train_steps: epoch=2 step=335 loss=11.0275
101
+ 2026-03-29 00:06:18,469 [INFO] new_opacus_codex.train_steps: epoch=2 step=340 loss=10.8425
102
+ 2026-03-29 00:06:32,376 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=340 eval_loss=10.7737 duration_sec=13.89
103
+ 2026-03-29 00:07:14,222 [INFO] new_opacus_codex.train_steps: epoch=2 step=345 loss=10.7198
104
+ 2026-03-29 00:07:57,739 [INFO] new_opacus_codex.train_steps: epoch=2 step=350 loss=10.6885
105
+ 2026-03-29 00:08:11,674 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=350 eval_loss=10.6696 duration_sec=13.92
106
+ 2026-03-29 00:08:53,338 [INFO] new_opacus_codex.train_steps: epoch=2 step=355 loss=10.8078
107
+ 2026-03-29 00:09:35,471 [INFO] new_opacus_codex.train_steps: epoch=2 step=360 loss=10.8570
108
+ 2026-03-29 00:09:49,394 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=360 eval_loss=10.5701 duration_sec=13.91
109
+ 2026-03-29 00:10:31,542 [INFO] new_opacus_codex.train_steps: epoch=2 step=365 loss=10.7801
110
+ 2026-03-29 00:11:34,543 [INFO] new_opacus_codex.train_steps: epoch=3 step=370 loss=10.4806
111
+ 2026-03-29 00:11:48,418 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=370 eval_loss=10.4799 duration_sec=13.86
112
+ 2026-03-29 00:12:31,945 [INFO] new_opacus_codex.train_steps: epoch=3 step=375 loss=10.4048
113
+ 2026-03-29 00:13:15,313 [INFO] new_opacus_codex.train_steps: epoch=3 step=380 loss=10.4519
114
+ 2026-03-29 00:13:29,209 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=380 eval_loss=10.3853 duration_sec=13.88
115
+ 2026-03-29 00:14:11,479 [INFO] new_opacus_codex.train_steps: epoch=3 step=385 loss=10.5090
116
+ 2026-03-29 00:14:54,221 [INFO] new_opacus_codex.train_steps: epoch=3 step=390 loss=10.4564
117
+ 2026-03-29 00:15:08,156 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=390 eval_loss=10.2836 duration_sec=13.93
118
+ 2026-03-29 00:15:50,437 [INFO] new_opacus_codex.train_steps: epoch=3 step=395 loss=10.4278
119
+ 2026-03-29 00:16:33,647 [INFO] new_opacus_codex.train_steps: epoch=3 step=400 loss=10.3751
120
+ 2026-03-29 00:16:47,497 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=400 eval_loss=10.1865 duration_sec=13.83
121
+ 2026-03-29 00:17:29,937 [INFO] new_opacus_codex.train_steps: epoch=3 step=405 loss=10.2877
122
+ 2026-03-29 00:18:13,707 [INFO] new_opacus_codex.train_steps: epoch=3 step=410 loss=10.3461
123
+ 2026-03-29 00:18:27,688 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=410 eval_loss=10.0950 duration_sec=13.97
124
+ 2026-03-29 00:19:09,968 [INFO] new_opacus_codex.train_steps: epoch=3 step=415 loss=10.2461
125
+ 2026-03-29 00:19:53,060 [INFO] new_opacus_codex.train_steps: epoch=3 step=420 loss=10.1909
126
+ 2026-03-29 00:20:06,973 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=420 eval_loss=10.0074 duration_sec=13.89
127
+ 2026-03-29 00:20:51,039 [INFO] new_opacus_codex.train_steps: epoch=3 step=425 loss=10.1456
128
+ 2026-03-29 00:21:34,467 [INFO] new_opacus_codex.train_steps: epoch=3 step=430 loss=10.0728
129
+ 2026-03-29 00:21:48,360 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=430 eval_loss=9.9286 duration_sec=13.88
130
+ 2026-03-29 00:22:31,527 [INFO] new_opacus_codex.train_steps: epoch=3 step=435 loss=9.9865
131
+ 2026-03-29 00:23:12,581 [INFO] new_opacus_codex.train_steps: epoch=3 step=440 loss=9.8696
132
+ 2026-03-29 00:23:26,527 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=440 eval_loss=9.8568 duration_sec=13.93
133
+ 2026-03-29 00:24:09,768 [INFO] new_opacus_codex.train_steps: epoch=3 step=445 loss=9.9381
134
+ 2026-03-29 00:24:51,763 [INFO] new_opacus_codex.train_steps: epoch=3 step=450 loss=10.0390
135
+ 2026-03-29 00:25:05,754 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=450 eval_loss=9.7911 duration_sec=13.98
136
+ 2026-03-29 00:25:48,642 [INFO] new_opacus_codex.train_steps: epoch=3 step=455 loss=10.1205
137
+ 2026-03-29 00:26:30,523 [INFO] new_opacus_codex.train_steps: epoch=3 step=460 loss=10.0095
138
+ 2026-03-29 00:26:44,440 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=460 eval_loss=9.7353 duration_sec=13.90
139
+ 2026-03-29 00:27:26,955 [INFO] new_opacus_codex.train_steps: epoch=3 step=465 loss=9.9651
140
+ 2026-03-29 00:28:09,377 [INFO] new_opacus_codex.train_steps: epoch=3 step=470 loss=9.9953
141
+ 2026-03-29 00:28:23,451 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=470 eval_loss=9.6868 duration_sec=14.06
142
+ 2026-03-29 00:29:05,772 [INFO] new_opacus_codex.train_steps: epoch=3 step=475 loss=10.0260
143
+ 2026-03-29 00:29:48,191 [INFO] new_opacus_codex.train_steps: epoch=3 step=480 loss=9.9847
144
+ 2026-03-29 00:30:02,150 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=480 eval_loss=9.6375 duration_sec=13.94
145
+ 2026-03-29 00:30:44,755 [INFO] new_opacus_codex.train_steps: epoch=3 step=485 loss=9.8206
146
+ 2026-03-29 00:31:28,590 [INFO] new_opacus_codex.train_steps: epoch=3 step=490 loss=9.8692
147
+ 2026-03-29 00:31:42,514 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=490 eval_loss=9.5885 duration_sec=13.91
148
+ 2026-03-29 00:32:25,416 [INFO] new_opacus_codex.train_steps: epoch=3 step=495 loss=10.0079
149
+ 2026-03-29 00:33:08,860 [INFO] new_opacus_codex.train_steps: epoch=3 step=500 loss=9.9442
150
+ 2026-03-29 00:33:22,761 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=500 eval_loss=9.5385 duration_sec=13.89
151
+ 2026-03-29 00:34:05,128 [INFO] new_opacus_codex.train_steps: epoch=3 step=505 loss=9.7912
152
+ 2026-03-29 00:34:48,202 [INFO] new_opacus_codex.train_steps: epoch=3 step=510 loss=9.7889
153
+ 2026-03-29 00:35:02,131 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=510 eval_loss=9.4901 duration_sec=13.92
154
+ 2026-03-29 00:35:44,655 [INFO] new_opacus_codex.train_steps: epoch=3 step=515 loss=9.7205
155
+ 2026-03-29 00:36:26,936 [INFO] new_opacus_codex.train_steps: epoch=3 step=520 loss=9.7827
156
+ 2026-03-29 00:36:40,888 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=520 eval_loss=9.4519 duration_sec=13.93
157
+ 2026-03-29 00:37:22,035 [INFO] new_opacus_codex.train_steps: epoch=3 step=525 loss=9.7604
158
+ 2026-03-29 00:38:04,087 [INFO] new_opacus_codex.train_steps: epoch=3 step=530 loss=9.5351
159
+ 2026-03-29 00:38:18,013 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=530 eval_loss=9.4175 duration_sec=13.91
160
+ 2026-03-29 00:39:00,439 [INFO] new_opacus_codex.train_steps: epoch=3 step=535 loss=9.7260
161
+ 2026-03-29 00:39:43,484 [INFO] new_opacus_codex.train_steps: epoch=3 step=540 loss=9.6928
162
+ 2026-03-29 00:39:57,424 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=540 eval_loss=9.3856 duration_sec=13.93
163
+ 2026-03-29 00:40:39,734 [INFO] new_opacus_codex.train_steps: epoch=3 step=545 loss=9.5860
164
+ 2026-03-29 00:41:22,824 [INFO] new_opacus_codex.train_steps: epoch=3 step=550 loss=9.6520
165
+ 2026-03-29 00:41:36,699 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=550 eval_loss=9.3537 duration_sec=13.86
166
+ 2026-03-29 00:42:43,276 [INFO] new_opacus_codex.train_steps: epoch=4 step=555 loss=9.5472
167
+ 2026-03-29 00:43:28,307 [INFO] new_opacus_codex.train_steps: epoch=4 step=560 loss=9.4867
168
+ 2026-03-29 00:43:42,270 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=560 eval_loss=9.3217 duration_sec=13.95
169
+ 2026-03-29 00:44:24,504 [INFO] new_opacus_codex.train_steps: epoch=4 step=565 loss=9.5454
170
+ 2026-03-29 00:45:07,582 [INFO] new_opacus_codex.train_steps: epoch=4 step=570 loss=9.5886
171
+ 2026-03-29 00:45:21,556 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=570 eval_loss=9.2932 duration_sec=13.96
172
+ 2026-03-29 00:46:05,058 [INFO] new_opacus_codex.train_steps: epoch=4 step=575 loss=9.5040
173
+ 2026-03-29 00:46:47,536 [INFO] new_opacus_codex.train_steps: epoch=4 step=580 loss=9.4655
174
+ 2026-03-29 00:47:01,487 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=580 eval_loss=9.2688 duration_sec=13.94
175
+ 2026-03-29 00:47:45,567 [INFO] new_opacus_codex.train_steps: epoch=4 step=585 loss=9.5795
176
+ 2026-03-29 00:48:28,297 [INFO] new_opacus_codex.train_steps: epoch=4 step=590 loss=9.6444
177
+ 2026-03-29 00:48:42,238 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=590 eval_loss=9.2449 duration_sec=13.93
178
+ 2026-03-29 00:49:24,717 [INFO] new_opacus_codex.train_steps: epoch=4 step=595 loss=9.5073
179
+ 2026-03-29 00:50:09,332 [INFO] new_opacus_codex.train_steps: epoch=4 step=600 loss=9.3743
180
+ 2026-03-29 00:50:23,276 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=600 eval_loss=9.2133 duration_sec=13.93
181
+ 2026-03-29 00:51:05,172 [INFO] new_opacus_codex.train_steps: epoch=4 step=605 loss=9.4783
182
+ 2026-03-29 00:51:48,020 [INFO] new_opacus_codex.train_steps: epoch=4 step=610 loss=9.5370
183
+ 2026-03-29 00:52:01,990 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=610 eval_loss=9.1861 duration_sec=13.95
184
+ 2026-03-29 00:52:44,965 [INFO] new_opacus_codex.train_steps: epoch=4 step=615 loss=9.5148
185
+ 2026-03-29 00:53:27,645 [INFO] new_opacus_codex.train_steps: epoch=4 step=620 loss=9.6177
186
+ 2026-03-29 00:53:41,641 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=620 eval_loss=9.1642 duration_sec=13.98
187
+ 2026-03-29 00:54:24,911 [INFO] new_opacus_codex.train_steps: epoch=4 step=625 loss=9.5634
188
+ 2026-03-29 00:55:08,041 [INFO] new_opacus_codex.train_steps: epoch=4 step=630 loss=9.5399
189
+ 2026-03-29 00:55:21,995 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=630 eval_loss=9.1399 duration_sec=13.94
190
+ 2026-03-29 00:56:05,806 [INFO] new_opacus_codex.train_steps: epoch=4 step=635 loss=9.6083
191
+ 2026-03-29 00:56:49,072 [INFO] new_opacus_codex.train_steps: epoch=4 step=640 loss=9.5350
192
+ 2026-03-29 00:57:03,060 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=640 eval_loss=9.1155 duration_sec=13.97
193
+ 2026-03-29 00:57:46,556 [INFO] new_opacus_codex.train_steps: epoch=4 step=645 loss=9.4773
194
+ 2026-03-29 00:58:28,665 [INFO] new_opacus_codex.train_steps: epoch=4 step=650 loss=9.4427
195
+ 2026-03-29 00:58:42,624 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=650 eval_loss=9.0919 duration_sec=13.95
196
+ 2026-03-29 00:59:25,691 [INFO] new_opacus_codex.train_steps: epoch=4 step=655 loss=9.4582
197
+ 2026-03-29 01:00:08,453 [INFO] new_opacus_codex.train_steps: epoch=4 step=660 loss=9.3193
198
+ 2026-03-29 01:00:22,401 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=660 eval_loss=9.0724 duration_sec=13.94
199
+ 2026-03-29 01:01:05,356 [INFO] new_opacus_codex.train_steps: epoch=4 step=665 loss=9.2979
200
+ 2026-03-29 01:01:48,127 [INFO] new_opacus_codex.train_steps: epoch=4 step=670 loss=9.4743
201
+ 2026-03-29 01:02:02,078 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=670 eval_loss=9.0592 duration_sec=13.94
202
+ 2026-03-29 01:02:44,645 [INFO] new_opacus_codex.train_steps: epoch=4 step=675 loss=9.4385
203
+ 2026-03-29 01:03:27,308 [INFO] new_opacus_codex.train_steps: epoch=4 step=680 loss=9.3588
204
+ 2026-03-29 01:03:41,278 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=680 eval_loss=9.0397 duration_sec=13.95
205
+ 2026-03-29 01:04:23,668 [INFO] new_opacus_codex.train_steps: epoch=4 step=685 loss=9.2823
206
+ 2026-03-29 01:05:07,013 [INFO] new_opacus_codex.train_steps: epoch=4 step=690 loss=9.3099
207
+ 2026-03-29 01:05:20,950 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=690 eval_loss=9.0243 duration_sec=13.92
208
+ 2026-03-29 01:06:03,624 [INFO] new_opacus_codex.train_steps: epoch=4 step=695 loss=9.4757
209
+ 2026-03-29 01:06:46,858 [INFO] new_opacus_codex.train_steps: epoch=4 step=700 loss=9.4604
210
+ 2026-03-29 01:07:00,801 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=700 eval_loss=9.0088 duration_sec=13.93
211
+ 2026-03-29 01:07:42,644 [INFO] new_opacus_codex.train_steps: epoch=4 step=705 loss=9.3163
212
+ 2026-03-29 01:08:25,830 [INFO] new_opacus_codex.train_steps: epoch=4 step=710 loss=9.3739
213
+ 2026-03-29 01:08:39,805 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=710 eval_loss=8.9954 duration_sec=13.96
214
+ 2026-03-29 01:09:22,085 [INFO] new_opacus_codex.train_steps: epoch=4 step=715 loss=9.3267
215
+ 2026-03-29 01:10:06,017 [INFO] new_opacus_codex.train_steps: epoch=4 step=720 loss=9.3430
216
+ 2026-03-29 01:10:19,983 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=720 eval_loss=8.9829 duration_sec=13.95
217
+ 2026-03-29 01:11:03,147 [INFO] new_opacus_codex.train_steps: epoch=4 step=725 loss=9.3345
218
+ 2026-03-29 01:11:46,333 [INFO] new_opacus_codex.train_steps: epoch=4 step=730 loss=9.2972
219
+ 2026-03-29 01:12:00,293 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=730 eval_loss=8.9715 duration_sec=13.94
220
+ 2026-03-29 01:12:43,471 [INFO] new_opacus_codex.train_steps: epoch=4 step=735 loss=9.3228
221
+ 2026-03-29 01:13:49,500 [INFO] new_opacus_codex.train_steps: epoch=5 step=740 loss=9.4831
222
+ 2026-03-29 01:14:03,480 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=740 eval_loss=8.9593 duration_sec=13.97
223
+ 2026-03-29 01:14:46,457 [INFO] new_opacus_codex.train_steps: epoch=5 step=745 loss=9.5675
224
+ 2026-03-29 01:15:29,460 [INFO] new_opacus_codex.train_steps: epoch=5 step=750 loss=9.3214
225
+ 2026-03-29 01:15:43,450 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=750 eval_loss=8.9491 duration_sec=13.98
226
+ 2026-03-29 01:16:27,541 [INFO] new_opacus_codex.train_steps: epoch=5 step=755 loss=9.1354
227
+ 2026-03-29 01:17:09,559 [INFO] new_opacus_codex.train_steps: epoch=5 step=760 loss=9.3490
228
+ 2026-03-29 01:17:23,589 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=760 eval_loss=8.9367 duration_sec=14.02
229
+ 2026-03-29 01:18:07,476 [INFO] new_opacus_codex.train_steps: epoch=5 step=765 loss=9.3444
230
+ 2026-03-29 01:18:49,238 [INFO] new_opacus_codex.train_steps: epoch=5 step=770 loss=9.2800
231
+ 2026-03-29 01:19:03,207 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=770 eval_loss=8.9254 duration_sec=13.96
232
+ 2026-03-29 01:19:45,442 [INFO] new_opacus_codex.train_steps: epoch=5 step=775 loss=9.3355
233
+ 2026-03-29 01:20:28,338 [INFO] new_opacus_codex.train_steps: epoch=5 step=780 loss=9.4885
234
+ 2026-03-29 01:20:42,288 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=780 eval_loss=8.9148 duration_sec=13.94
235
+ 2026-03-29 01:21:25,138 [INFO] new_opacus_codex.train_steps: epoch=5 step=785 loss=9.4515
236
+ 2026-03-29 01:22:07,400 [INFO] new_opacus_codex.train_steps: epoch=5 step=790 loss=9.3068
237
+ 2026-03-29 01:22:21,340 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=790 eval_loss=8.9047 duration_sec=13.93
238
+ 2026-03-29 01:23:04,731 [INFO] new_opacus_codex.train_steps: epoch=5 step=795 loss=9.3529
239
+ 2026-03-29 01:23:46,750 [INFO] new_opacus_codex.train_steps: epoch=5 step=800 loss=9.4846
240
+ 2026-03-29 01:24:00,732 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=800 eval_loss=8.8960 duration_sec=13.97
241
+ 2026-03-29 01:24:43,922 [INFO] new_opacus_codex.train_steps: epoch=5 step=805 loss=9.5191
242
+ 2026-03-29 01:25:26,689 [INFO] new_opacus_codex.train_steps: epoch=5 step=810 loss=9.4019
243
+ 2026-03-29 01:25:40,661 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=810 eval_loss=8.8863 duration_sec=13.96
244
+ 2026-03-29 01:26:23,489 [INFO] new_opacus_codex.train_steps: epoch=5 step=815 loss=9.3000
245
+ 2026-03-29 01:27:06,991 [INFO] new_opacus_codex.train_steps: epoch=5 step=820 loss=9.0916
246
+ 2026-03-29 01:27:20,986 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=820 eval_loss=8.8748 duration_sec=13.98
247
+ 2026-03-29 01:28:02,861 [INFO] new_opacus_codex.train_steps: epoch=5 step=825 loss=9.0650
248
+ 2026-03-29 01:28:45,797 [INFO] new_opacus_codex.train_steps: epoch=5 step=830 loss=9.1268
249
+ 2026-03-29 01:28:59,660 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=830 eval_loss=8.8657 duration_sec=13.85
250
+ 2026-03-29 01:29:45,179 [INFO] new_opacus_codex.train_steps: epoch=5 step=835 loss=8.9559
251
+ 2026-03-29 01:30:27,551 [INFO] new_opacus_codex.train_steps: epoch=5 step=840 loss=9.0989
252
+ 2026-03-29 01:30:41,451 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=840 eval_loss=8.8569 duration_sec=13.88
253
+ 2026-03-29 01:31:23,881 [INFO] new_opacus_codex.train_steps: epoch=5 step=845 loss=9.1965
254
+ 2026-03-29 01:32:05,668 [INFO] new_opacus_codex.train_steps: epoch=5 step=850 loss=9.1001
255
+ 2026-03-29 01:32:19,642 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=850 eval_loss=8.8499 duration_sec=13.97
256
+ 2026-03-29 01:33:02,561 [INFO] new_opacus_codex.train_steps: epoch=5 step=855 loss=9.1822
257
+ 2026-03-29 01:33:45,029 [INFO] new_opacus_codex.train_steps: epoch=5 step=860 loss=9.0988
258
+ 2026-03-29 01:33:59,008 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=860 eval_loss=8.8429 duration_sec=13.97
259
+ 2026-03-29 01:34:41,412 [INFO] new_opacus_codex.train_steps: epoch=5 step=865 loss=9.0254
260
+ 2026-03-29 01:35:24,659 [INFO] new_opacus_codex.train_steps: epoch=5 step=870 loss=9.1267
261
+ 2026-03-29 01:35:38,664 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=870 eval_loss=8.8359 duration_sec=13.99
262
+ 2026-03-29 01:36:22,774 [INFO] new_opacus_codex.train_steps: epoch=5 step=875 loss=9.1624
263
+ 2026-03-29 01:37:06,699 [INFO] new_opacus_codex.train_steps: epoch=5 step=880 loss=9.1647
264
+ 2026-03-29 01:37:20,631 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=880 eval_loss=8.8293 duration_sec=13.92
265
+ 2026-03-29 01:38:04,734 [INFO] new_opacus_codex.train_steps: epoch=5 step=885 loss=9.2199
266
+ 2026-03-29 01:38:47,791 [INFO] new_opacus_codex.train_steps: epoch=5 step=890 loss=9.1405
267
+ 2026-03-29 01:39:01,779 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=890 eval_loss=8.8222 duration_sec=13.97
268
+ 2026-03-29 01:39:45,899 [INFO] new_opacus_codex.train_steps: epoch=5 step=895 loss=9.0507
269
+ 2026-03-29 01:40:28,852 [INFO] new_opacus_codex.train_steps: epoch=5 step=900 loss=9.1232
270
+ 2026-03-29 01:40:42,814 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=900 eval_loss=8.8154 duration_sec=13.95
271
+ 2026-03-29 01:41:25,440 [INFO] new_opacus_codex.train_steps: epoch=5 step=905 loss=9.3886
272
+ 2026-03-29 01:42:08,847 [INFO] new_opacus_codex.train_steps: epoch=5 step=910 loss=9.2949
273
+ 2026-03-29 01:42:22,744 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=910 eval_loss=8.8077 duration_sec=13.88
274
+ 2026-03-29 01:43:06,196 [INFO] new_opacus_codex.train_steps: epoch=5 step=915 loss=9.1431
275
+ 2026-03-29 01:43:41,660 [INFO] new_opacus_codex.train_steps: epoch=5 step=920 loss=9.3398
276
+ 2026-03-29 01:43:55,525 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=920 eval_loss=8.8012 duration_sec=13.85
llama3-8b/dp8/adapter/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: meta-llama/Meta-Llama-3-8B
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:meta-llama/Meta-Llama-3-8B
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.18.1
llama3-8b/dp8/adapter/adapter_config.json ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": true,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 32,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "megatron_config": null,
23
+ "megatron_core": "megatron.core",
24
+ "modules_to_save": [
25
+ "lm_head",
26
+ "embed_tokens"
27
+ ],
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.18.1",
30
+ "qalora_group_size": 16,
31
+ "r": 16,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "q_proj",
36
+ "k_proj",
37
+ "v_proj",
38
+ "o_proj"
39
+ ],
40
+ "target_parameters": null,
41
+ "task_type": "CAUSAL_LM",
42
+ "trainable_token_indices": null,
43
+ "use_dora": false,
44
+ "use_qalora": false,
45
+ "use_rslora": false
46
+ }
llama3-8b/dp8/audit_results.json ADDED
@@ -0,0 +1,137 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "delta": 1e-05,
3
+ "num_canaries": 500,
4
+ "num_members": 250,
5
+ "paper_guess_fraction": 0.2,
6
+ "paper_guess_steps": 20,
7
+ "loss": {
8
+ "auc": 0.533056,
9
+ "empirical_epsilon": {
10
+ "0.05": 0.0,
11
+ "0.01": 0.0
12
+ },
13
+ "empirical_epsilon_details": {
14
+ "0.05": {
15
+ "epsilon": 0.0,
16
+ "num_guesses": 0,
17
+ "correct_guesses": 0,
18
+ "candidate_num_guesses": [
19
+ 5,
20
+ 10,
21
+ 15,
22
+ 20,
23
+ 25,
24
+ 30,
25
+ 35,
26
+ 40,
27
+ 45,
28
+ 50,
29
+ 55,
30
+ 60,
31
+ 65,
32
+ 70,
33
+ 75,
34
+ 80,
35
+ 85,
36
+ 90,
37
+ 95,
38
+ 100
39
+ ],
40
+ "direction": "lower"
41
+ },
42
+ "0.01": {
43
+ "epsilon": 0.0,
44
+ "num_guesses": 0,
45
+ "correct_guesses": 0,
46
+ "candidate_num_guesses": [
47
+ 5,
48
+ 10,
49
+ 15,
50
+ 20,
51
+ 25,
52
+ 30,
53
+ 35,
54
+ 40,
55
+ 45,
56
+ 50,
57
+ 55,
58
+ 60,
59
+ 65,
60
+ 70,
61
+ 75,
62
+ 80,
63
+ 85,
64
+ 90,
65
+ 95,
66
+ 100
67
+ ],
68
+ "direction": "lower"
69
+ }
70
+ }
71
+ },
72
+ "embedding": {
73
+ "auc": 0.500312,
74
+ "empirical_epsilon": {
75
+ "0.05": 0.0,
76
+ "0.01": 0.0
77
+ },
78
+ "empirical_epsilon_details": {
79
+ "0.05": {
80
+ "epsilon": 0.0,
81
+ "num_guesses": 0,
82
+ "correct_guesses": 0,
83
+ "candidate_num_guesses": [
84
+ 5,
85
+ 10,
86
+ 15,
87
+ 20,
88
+ 25,
89
+ 30,
90
+ 35,
91
+ 40,
92
+ 45,
93
+ 50,
94
+ 55,
95
+ 60,
96
+ 65,
97
+ 70,
98
+ 75,
99
+ 80,
100
+ 85,
101
+ 90,
102
+ 95,
103
+ 100
104
+ ],
105
+ "direction": "lower"
106
+ },
107
+ "0.01": {
108
+ "epsilon": 0.0,
109
+ "num_guesses": 0,
110
+ "correct_guesses": 0,
111
+ "candidate_num_guesses": [
112
+ 5,
113
+ 10,
114
+ 15,
115
+ 20,
116
+ 25,
117
+ 30,
118
+ 35,
119
+ 40,
120
+ 45,
121
+ 50,
122
+ 55,
123
+ 60,
124
+ 65,
125
+ 70,
126
+ 75,
127
+ 80,
128
+ 85,
129
+ 90,
130
+ 95,
131
+ 100
132
+ ],
133
+ "direction": "lower"
134
+ }
135
+ }
136
+ }
137
+ }
llama3-8b/dp8/canary_meta.json ADDED
The diff for this file is too large to render. See raw diff
 
llama3-8b/dp8/metrics.jsonl ADDED
The diff for this file is too large to render. See raw diff
 
llama3-8b/dp8/resolved_config.yaml ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ model:
2
+ name: meta-llama/Meta-Llama-3-8B
3
+ tokenizer_name: meta-llama/Meta-Llama-3-8B
4
+ max_length: 1024
5
+ dtype: bfloat16
6
+ trust_remote_code: true
7
+ use_fast_tokenizer: true
8
+ cache_dir: null
9
+ local_files_only: false
10
+ low_cpu_mem_usage: true
11
+ tie_word_embeddings: true
12
+ gradient_checkpointing: false
13
+ use_chat_template: false
14
+ dataset:
15
+ name: melihcatal/codedp-cpt
16
+ split: train
17
+ mode: cpt
18
+ text_column: text
19
+ validation_ratio: 0.05
20
+ max_samples: -1
21
+ lora:
22
+ enabled: true
23
+ r: 16
24
+ alpha: 32
25
+ dropout: 0.05
26
+ target_modules:
27
+ - q_proj
28
+ - k_proj
29
+ - v_proj
30
+ - o_proj
31
+ modules_to_save:
32
+ - lm_head
33
+ bias: none
34
+ training:
35
+ seed: 42
36
+ epochs: 5
37
+ warmup_steps: null
38
+ warmup_ratio: 0.05
39
+ mixed_precision: false
40
+ mixed_precision_dtype: bfloat16
41
+ batch_size: 4
42
+ eval_batch_size: 8
43
+ eval_every_steps: 10
44
+ eval_every_epochs: 1
45
+ learning_rate: 0.0005
46
+ optimizer: adamw
47
+ lr_scheduler: cosine
48
+ adam_beta1: 0.9
49
+ adam_beta2: 0.999
50
+ adam_epsilon: 1.0e-08
51
+ sgd_momentum: 0.9
52
+ weight_decay: 0.01
53
+ max_grad_norm: 1.0
54
+ log_every: 5
55
+ gradient_accumulation_steps: 8
56
+ num_workers: 4
57
+ output_dir: /scratch/mcatal/runs/cpt/llama3-8b/dp8
58
+ min_lr_ratio: 0.15
59
+ distributed:
60
+ strategy: dpddp
61
+ backend: nccl
62
+ devices: null
63
+ dp:
64
+ module_validator: auto
65
+ target_delta: 1.0e-05
66
+ noise_multiplier: null
67
+ max_grad_norm: 1.0
68
+ grad_sample_mode: hooks
69
+ clipping: per_layer
70
+ secure_mode: false
71
+ enabled: true
72
+ target_epsilon: 8.0
73
+ audit:
74
+ enabled: true
75
+ run_every_epoch: true
76
+ epoch_device: cuda
77
+ q_canary: auto
78
+ num_canaries: 500
79
+ prefix_length: 49
80
+ num_digits: 12
81
+ batch_size: 32
82
+ delta: 1.0e-05
83
+ p_values:
84
+ - 0.05
85
+ - 0.01
86
+ paper_guess_fraction: 0.2
87
+ paper_guess_steps: 20
88
+ enable_holdout_empirical_epsilon: false
89
+ holdout_seed: 42
90
+ tie_seed: 42
91
+ tracking:
92
+ enabled: true
93
+ tensorboard: true
94
+ wandb: false
95
+ wandb_project: codedp-finetune-h200-audit
96
+ wandb_run_name: llama3-8b-cpt-dp8
97
+ wandb_mode: online
98
+ codecarbon: true
99
+ codecarbon_output_file: codecarbon.csv
100
+ codecarbon_measure_power_secs: 15
101
+ codecarbon_country_iso_code: null
102
+ codecarbon_project_name: codedp-llama3-8b-cpt-dp8
llama3-8b/dp8/summary.json ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "audit/delta": 1e-05,
3
+ "audit/embedding/auc": 0.500312,
4
+ "audit/embedding/empirical_epsilon/0.01": 0.0,
5
+ "audit/embedding/empirical_epsilon/0.05": 0.0,
6
+ "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 0.0,
7
+ "audit/embedding/empirical_epsilon_details/0.01/epsilon": 0.0,
8
+ "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 0.0,
9
+ "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 0.0,
10
+ "audit/embedding/empirical_epsilon_details/0.05/epsilon": 0.0,
11
+ "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 0.0,
12
+ "audit/loss/auc": 0.533056,
13
+ "audit/loss/empirical_epsilon/0.01": 0.0,
14
+ "audit/loss/empirical_epsilon/0.05": 0.0,
15
+ "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 0.0,
16
+ "audit/loss/empirical_epsilon_details/0.01/epsilon": 0.0,
17
+ "audit/loss/empirical_epsilon_details/0.01/num_guesses": 0.0,
18
+ "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 0.0,
19
+ "audit/loss/empirical_epsilon_details/0.05/epsilon": 0.0,
20
+ "audit/loss/empirical_epsilon_details/0.05/num_guesses": 0.0,
21
+ "audit/num_canaries": 500.0,
22
+ "audit/num_members": 250.0,
23
+ "audit/paper_guess_fraction": 0.2,
24
+ "audit/paper_guess_steps": 20.0,
25
+ "energy/codecarbon/cpu_count": 16.0,
26
+ "energy/codecarbon/cpu_energy": 0.202142001044712,
27
+ "energy/codecarbon/cpu_power": 80.03183964072312,
28
+ "energy/codecarbon/cpu_utilization_percent": 3.536022654413336,
29
+ "energy/codecarbon/duration": 9446.989458259195,
30
+ "energy/codecarbon/emissions": 0.32692423999452075,
31
+ "energy/codecarbon/emissions_rate": 3.460618236518741e-05,
32
+ "energy/codecarbon/energy_consumed": 9.38278104625092,
33
+ "energy/codecarbon/gpu_count": 8.0,
34
+ "energy/codecarbon/gpu_energy": 9.084660437167017,
35
+ "energy/codecarbon/gpu_power": 3472.1677074789927,
36
+ "energy/codecarbon/gpu_utilization_percent": 95.0607635178457,
37
+ "energy/codecarbon/latitude": 47.4843,
38
+ "energy/codecarbon/longitude": 8.212,
39
+ "energy/codecarbon/pue": 1.0,
40
+ "energy/codecarbon/ram_energy": 0.09597860803917964,
41
+ "energy/codecarbon/ram_power": 38.0,
42
+ "energy/codecarbon/ram_total_size": 128.0,
43
+ "energy/codecarbon/ram_used_gb": 508.7419379540656,
44
+ "energy/codecarbon/ram_utilization_percent": 25.690094037187432,
45
+ "energy/codecarbon/water_consumed": 0.0,
46
+ "energy/codecarbon/wue": 0.0,
47
+ "eval/duration_sec": 13.979894496966153,
48
+ "eval/loss": 7.628438680599897,
49
+ "perf/audit_duration_sec": 7.4041956178843975,
50
+ "perf/epoch_duration_sec": 1848.8750976803713,
51
+ "perf/epoch_samples": 51412.0,
52
+ "perf/epoch_samples_per_sec": 27.80717857279939,
53
+ "perf/epoch_tokens": 37419511.0,
54
+ "perf/epoch_tokens_per_sec": 20239.069176142362,
55
+ "perf/logical_batch_size": 3.0,
56
+ "perf/logical_token_count": 432.0,
57
+ "perf/physical_batches": 2.0,
58
+ "perf/samples_per_sec": 1.6753487106915488,
59
+ "perf/step_duration_sec": 1.7906719841994345,
60
+ "perf/tokens_per_sec": 241.250214339583,
61
+ "privacy/epsilon": 7.992470385789166,
62
+ "system/cuda_epoch_peak_memory_gb": 62.821791648864746,
63
+ "system/cuda_max_memory_allocated_gb": 62.821791648864746,
64
+ "system/cuda_memory_allocated_gb": 20.329766750335693,
65
+ "train/epoch_canary_loss": 10.158678088822576,
66
+ "train/epoch_loss": 8.089672184988586,
67
+ "train/epoch_real_loss": 7.779524202661022,
68
+ "train/lr": 7.5e-05,
69
+ "train/step_canary_loss": 9.229166666666666,
70
+ "train/step_loss": 8.765607833862305,
71
+ "train/step_real_loss": 7.374928951263428
72
+ }
llama3-8b/dp8/tokenizer/tokenizer_config.json ADDED
@@ -0,0 +1,516 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "bos_token": "<|begin_of_text|>",
4
+ "clean_up_tokenization_spaces": true,
5
+ "eos_token": "<|end_of_text|>",
6
+ "extra_special_tokens": {
7
+ "extra_special_token_0": "865331112869",
8
+ "extra_special_token_1": "569765693871",
9
+ "extra_special_token_2": "485177821815",
10
+ "extra_special_token_3": "135441121756",
11
+ "extra_special_token_4": "367459894796",
12
+ "extra_special_token_5": "877482678543",
13
+ "extra_special_token_6": "457919547633",
14
+ "extra_special_token_7": "765474393376",
15
+ "extra_special_token_8": "114848338811",
16
+ "extra_special_token_9": "746285987371",
17
+ "extra_special_token_10": "649291669397",
18
+ "extra_special_token_11": "927914615679",
19
+ "extra_special_token_12": "445925149649",
20
+ "extra_special_token_13": "691587454538",
21
+ "extra_special_token_14": "143777992227",
22
+ "extra_special_token_15": "997981281989",
23
+ "extra_special_token_16": "425949483533",
24
+ "extra_special_token_17": "982993456429",
25
+ "extra_special_token_18": "718726519731",
26
+ "extra_special_token_19": "172599315861",
27
+ "extra_special_token_20": "643489267333",
28
+ "extra_special_token_21": "282322838685",
29
+ "extra_special_token_22": "781653545886",
30
+ "extra_special_token_23": "796415361892",
31
+ "extra_special_token_24": "841991688488",
32
+ "extra_special_token_25": "211411365397",
33
+ "extra_special_token_26": "698218415444",
34
+ "extra_special_token_27": "355977139358",
35
+ "extra_special_token_28": "682564697312",
36
+ "extra_special_token_29": "383837596997",
37
+ "extra_special_token_30": "689362171782",
38
+ "extra_special_token_31": "749966767285",
39
+ "extra_special_token_32": "753159165157",
40
+ "extra_special_token_33": "795693824762",
41
+ "extra_special_token_34": "669689115557",
42
+ "extra_special_token_35": "327491773134",
43
+ "extra_special_token_36": "983569279932",
44
+ "extra_special_token_37": "612128769512",
45
+ "extra_special_token_38": "374327157578",
46
+ "extra_special_token_39": "311632789559",
47
+ "extra_special_token_40": "523918658846",
48
+ "extra_special_token_41": "765981581453",
49
+ "extra_special_token_42": "794825141891",
50
+ "extra_special_token_43": "873898736873",
51
+ "extra_special_token_44": "447445629421",
52
+ "extra_special_token_45": "473822473819",
53
+ "extra_special_token_46": "181439694557",
54
+ "extra_special_token_47": "592538279337",
55
+ "extra_special_token_48": "668134915514",
56
+ "extra_special_token_49": "643692393748",
57
+ "extra_special_token_50": "696651276628",
58
+ "extra_special_token_51": "853859348234",
59
+ "extra_special_token_52": "778466723723",
60
+ "extra_special_token_53": "929826356991",
61
+ "extra_special_token_54": "272362973463",
62
+ "extra_special_token_55": "694235616268",
63
+ "extra_special_token_56": "281673864127",
64
+ "extra_special_token_57": "479676316326",
65
+ "extra_special_token_58": "646979124677",
66
+ "extra_special_token_59": "922327493433",
67
+ "extra_special_token_60": "883685933161",
68
+ "extra_special_token_61": "264259917554",
69
+ "extra_special_token_62": "836746273134",
70
+ "extra_special_token_63": "658481324922",
71
+ "extra_special_token_64": "481884157827",
72
+ "extra_special_token_65": "587787496812",
73
+ "extra_special_token_66": "579184949249",
74
+ "extra_special_token_67": "912193598348",
75
+ "extra_special_token_68": "529679678956",
76
+ "extra_special_token_69": "795838284624",
77
+ "extra_special_token_70": "159337222655",
78
+ "extra_special_token_71": "173781362446",
79
+ "extra_special_token_72": "773687856563",
80
+ "extra_special_token_73": "535787224917",
81
+ "extra_special_token_74": "351885857332",
82
+ "extra_special_token_75": "578827344666",
83
+ "extra_special_token_76": "198462689911",
84
+ "extra_special_token_77": "722618266242",
85
+ "extra_special_token_78": "952872416512",
86
+ "extra_special_token_79": "517778845323",
87
+ "extra_special_token_80": "749665846687",
88
+ "extra_special_token_81": "661436365453",
89
+ "extra_special_token_82": "259666844669",
90
+ "extra_special_token_83": "242851284913",
91
+ "extra_special_token_84": "514532995959",
92
+ "extra_special_token_85": "161588262349",
93
+ "extra_special_token_86": "742765629356",
94
+ "extra_special_token_87": "225164373623",
95
+ "extra_special_token_88": "676539973863",
96
+ "extra_special_token_89": "826214551218",
97
+ "extra_special_token_90": "182345464792",
98
+ "extra_special_token_91": "232776999554",
99
+ "extra_special_token_92": "337326533813",
100
+ "extra_special_token_93": "676676697292",
101
+ "extra_special_token_94": "929185622831",
102
+ "extra_special_token_95": "545512344383",
103
+ "extra_special_token_96": "499444466686",
104
+ "extra_special_token_97": "314697386682",
105
+ "extra_special_token_98": "517379856925",
106
+ "extra_special_token_99": "379557332953",
107
+ "extra_special_token_100": "614797267726",
108
+ "extra_special_token_101": "429781429464",
109
+ "extra_special_token_102": "922466849763",
110
+ "extra_special_token_103": "721737645236",
111
+ "extra_special_token_104": "479227349997",
112
+ "extra_special_token_105": "136931728327",
113
+ "extra_special_token_106": "259533577263",
114
+ "extra_special_token_107": "488538864842",
115
+ "extra_special_token_108": "937495658852",
116
+ "extra_special_token_109": "489991411364",
117
+ "extra_special_token_110": "499148455254",
118
+ "extra_special_token_111": "441373944925",
119
+ "extra_special_token_112": "899151413682",
120
+ "extra_special_token_113": "467893531755",
121
+ "extra_special_token_114": "527117488925",
122
+ "extra_special_token_115": "928335588653",
123
+ "extra_special_token_116": "374439448821",
124
+ "extra_special_token_117": "879425227932",
125
+ "extra_special_token_118": "867678158885",
126
+ "extra_special_token_119": "399749397872",
127
+ "extra_special_token_120": "129693547287",
128
+ "extra_special_token_121": "689285841825",
129
+ "extra_special_token_122": "771619544974",
130
+ "extra_special_token_123": "724883568652",
131
+ "extra_special_token_124": "516968424863",
132
+ "extra_special_token_125": "733737988257",
133
+ "extra_special_token_126": "852347289392",
134
+ "extra_special_token_127": "296953381169",
135
+ "extra_special_token_128": "377273562477",
136
+ "extra_special_token_129": "262296912232",
137
+ "extra_special_token_130": "547149832394",
138
+ "extra_special_token_131": "298464134954",
139
+ "extra_special_token_132": "216667245274",
140
+ "extra_special_token_133": "843998562287",
141
+ "extra_special_token_134": "572154333646",
142
+ "extra_special_token_135": "124589118494",
143
+ "extra_special_token_136": "841824384614",
144
+ "extra_special_token_137": "232896526252",
145
+ "extra_special_token_138": "295448593321",
146
+ "extra_special_token_139": "123741461297",
147
+ "extra_special_token_140": "653573457168",
148
+ "extra_special_token_141": "196735786156",
149
+ "extra_special_token_142": "377338713663",
150
+ "extra_special_token_143": "964342468552",
151
+ "extra_special_token_144": "586855179568",
152
+ "extra_special_token_145": "484773717614",
153
+ "extra_special_token_146": "894885246797",
154
+ "extra_special_token_147": "677896358599",
155
+ "extra_special_token_148": "848845611563",
156
+ "extra_special_token_149": "851852651677",
157
+ "extra_special_token_150": "398549545767",
158
+ "extra_special_token_151": "454244839926",
159
+ "extra_special_token_152": "799364566435",
160
+ "extra_special_token_153": "967114116556",
161
+ "extra_special_token_154": "817378986438",
162
+ "extra_special_token_155": "233795848681",
163
+ "extra_special_token_156": "824387273757",
164
+ "extra_special_token_157": "916198946615",
165
+ "extra_special_token_158": "563117729724",
166
+ "extra_special_token_159": "951794811935",
167
+ "extra_special_token_160": "374598961236",
168
+ "extra_special_token_161": "922867396683",
169
+ "extra_special_token_162": "765737843639",
170
+ "extra_special_token_163": "175469284871",
171
+ "extra_special_token_164": "231853711778",
172
+ "extra_special_token_165": "662426712668",
173
+ "extra_special_token_166": "711412347158",
174
+ "extra_special_token_167": "753466987363",
175
+ "extra_special_token_168": "513361312532",
176
+ "extra_special_token_169": "712992815957",
177
+ "extra_special_token_170": "971621888444",
178
+ "extra_special_token_171": "829235161526",
179
+ "extra_special_token_172": "585544633356",
180
+ "extra_special_token_173": "582471228164",
181
+ "extra_special_token_174": "678666359123",
182
+ "extra_special_token_175": "557533689478",
183
+ "extra_special_token_176": "632962475133",
184
+ "extra_special_token_177": "484489193824",
185
+ "extra_special_token_178": "489562189822",
186
+ "extra_special_token_179": "589547936288",
187
+ "extra_special_token_180": "363214487524",
188
+ "extra_special_token_181": "244885399387",
189
+ "extra_special_token_182": "431751228368",
190
+ "extra_special_token_183": "433581868192",
191
+ "extra_special_token_184": "486391569221",
192
+ "extra_special_token_185": "185438575221",
193
+ "extra_special_token_186": "126574388585",
194
+ "extra_special_token_187": "741757479784",
195
+ "extra_special_token_188": "529854679937",
196
+ "extra_special_token_189": "996116119839",
197
+ "extra_special_token_190": "616248973917",
198
+ "extra_special_token_191": "763531783491",
199
+ "extra_special_token_192": "955456118295",
200
+ "extra_special_token_193": "364196983365",
201
+ "extra_special_token_194": "195792996468",
202
+ "extra_special_token_195": "151859598873",
203
+ "extra_special_token_196": "399223169721",
204
+ "extra_special_token_197": "938488813964",
205
+ "extra_special_token_198": "961981959227",
206
+ "extra_special_token_199": "183368827562",
207
+ "extra_special_token_200": "533417736566",
208
+ "extra_special_token_201": "786391632558",
209
+ "extra_special_token_202": "665661658354",
210
+ "extra_special_token_203": "693281533643",
211
+ "extra_special_token_204": "475794684356",
212
+ "extra_special_token_205": "652154162978",
213
+ "extra_special_token_206": "753233719644",
214
+ "extra_special_token_207": "668514843129",
215
+ "extra_special_token_208": "819162623892",
216
+ "extra_special_token_209": "941169431859",
217
+ "extra_special_token_210": "877385381798",
218
+ "extra_special_token_211": "752644929761",
219
+ "extra_special_token_212": "881136466196",
220
+ "extra_special_token_213": "275597777299",
221
+ "extra_special_token_214": "731681792655",
222
+ "extra_special_token_215": "961133895172",
223
+ "extra_special_token_216": "864718285734",
224
+ "extra_special_token_217": "963852916563",
225
+ "extra_special_token_218": "319584985416",
226
+ "extra_special_token_219": "563365646341",
227
+ "extra_special_token_220": "811371928234",
228
+ "extra_special_token_221": "837131396371",
229
+ "extra_special_token_222": "267514771964",
230
+ "extra_special_token_223": "944513428457",
231
+ "extra_special_token_224": "117298239631",
232
+ "extra_special_token_225": "158142752582",
233
+ "extra_special_token_226": "252867443568",
234
+ "extra_special_token_227": "839269684865",
235
+ "extra_special_token_228": "612788593128",
236
+ "extra_special_token_229": "145669731981",
237
+ "extra_special_token_230": "121557291859",
238
+ "extra_special_token_231": "245416776926",
239
+ "extra_special_token_232": "799417897197",
240
+ "extra_special_token_233": "997958836435",
241
+ "extra_special_token_234": "892336777248",
242
+ "extra_special_token_235": "158929292238",
243
+ "extra_special_token_236": "581976444672",
244
+ "extra_special_token_237": "897784492783",
245
+ "extra_special_token_238": "492373714791",
246
+ "extra_special_token_239": "512659818733",
247
+ "extra_special_token_240": "881112998642",
248
+ "extra_special_token_241": "619454958782",
249
+ "extra_special_token_242": "431149748713",
250
+ "extra_special_token_243": "624221476921",
251
+ "extra_special_token_244": "125866399464",
252
+ "extra_special_token_245": "339882449689",
253
+ "extra_special_token_246": "186198784585",
254
+ "extra_special_token_247": "943193294691",
255
+ "extra_special_token_248": "955668961269",
256
+ "extra_special_token_249": "232787996724",
257
+ "extra_special_token_250": "215671314196",
258
+ "extra_special_token_251": "286173241916",
259
+ "extra_special_token_252": "745977673725",
260
+ "extra_special_token_253": "556976448182",
261
+ "extra_special_token_254": "599961512792",
262
+ "extra_special_token_255": "766294538337",
263
+ "extra_special_token_256": "934912591213",
264
+ "extra_special_token_257": "295118729589",
265
+ "extra_special_token_258": "529455466433",
266
+ "extra_special_token_259": "196119929397",
267
+ "extra_special_token_260": "379571934299",
268
+ "extra_special_token_261": "251789649997",
269
+ "extra_special_token_262": "564544131355",
270
+ "extra_special_token_263": "244371196654",
271
+ "extra_special_token_264": "384598329253",
272
+ "extra_special_token_265": "887753195844",
273
+ "extra_special_token_266": "364947325679",
274
+ "extra_special_token_267": "655517954651",
275
+ "extra_special_token_268": "673948786567",
276
+ "extra_special_token_269": "857231548835",
277
+ "extra_special_token_270": "816115936673",
278
+ "extra_special_token_271": "644234165531",
279
+ "extra_special_token_272": "182782912224",
280
+ "extra_special_token_273": "234316622259",
281
+ "extra_special_token_274": "421369185549",
282
+ "extra_special_token_275": "434632855397",
283
+ "extra_special_token_276": "921889371893",
284
+ "extra_special_token_277": "415956914763",
285
+ "extra_special_token_278": "598916996413",
286
+ "extra_special_token_279": "773671349113",
287
+ "extra_special_token_280": "952465217972",
288
+ "extra_special_token_281": "117657531962",
289
+ "extra_special_token_282": "729825168745",
290
+ "extra_special_token_283": "691315125346",
291
+ "extra_special_token_284": "768461952319",
292
+ "extra_special_token_285": "664847713559",
293
+ "extra_special_token_286": "953267689786",
294
+ "extra_special_token_287": "886464195129",
295
+ "extra_special_token_288": "824488329416",
296
+ "extra_special_token_289": "837873762491",
297
+ "extra_special_token_290": "532833541879",
298
+ "extra_special_token_291": "669183782449",
299
+ "extra_special_token_292": "941976537588",
300
+ "extra_special_token_293": "739394546916",
301
+ "extra_special_token_294": "267954879268",
302
+ "extra_special_token_295": "637551427887",
303
+ "extra_special_token_296": "217756494954",
304
+ "extra_special_token_297": "524444658383",
305
+ "extra_special_token_298": "117783274348",
306
+ "extra_special_token_299": "138218735276",
307
+ "extra_special_token_300": "814611949491",
308
+ "extra_special_token_301": "711641973413",
309
+ "extra_special_token_302": "499156317423",
310
+ "extra_special_token_303": "515856611931",
311
+ "extra_special_token_304": "454164859837",
312
+ "extra_special_token_305": "345271433112",
313
+ "extra_special_token_306": "462294118988",
314
+ "extra_special_token_307": "511785788222",
315
+ "extra_special_token_308": "497294727353",
316
+ "extra_special_token_309": "866519986723",
317
+ "extra_special_token_310": "334513529294",
318
+ "extra_special_token_311": "549946382131",
319
+ "extra_special_token_312": "284445431422",
320
+ "extra_special_token_313": "396521188476",
321
+ "extra_special_token_314": "421435255895",
322
+ "extra_special_token_315": "133373659361",
323
+ "extra_special_token_316": "322683334381",
324
+ "extra_special_token_317": "228358422847",
325
+ "extra_special_token_318": "291762694874",
326
+ "extra_special_token_319": "143182978129",
327
+ "extra_special_token_320": "511923256573",
328
+ "extra_special_token_321": "327158398268",
329
+ "extra_special_token_322": "879764613759",
330
+ "extra_special_token_323": "564395222747",
331
+ "extra_special_token_324": "451161679736",
332
+ "extra_special_token_325": "538631466654",
333
+ "extra_special_token_326": "221762325616",
334
+ "extra_special_token_327": "218391991184",
335
+ "extra_special_token_328": "322589379462",
336
+ "extra_special_token_329": "876537814263",
337
+ "extra_special_token_330": "152676556624",
338
+ "extra_special_token_331": "332522971941",
339
+ "extra_special_token_332": "884354318946",
340
+ "extra_special_token_333": "513349618943",
341
+ "extra_special_token_334": "116639746413",
342
+ "extra_special_token_335": "635185846287",
343
+ "extra_special_token_336": "993832498489",
344
+ "extra_special_token_337": "813981174797",
345
+ "extra_special_token_338": "438745114173",
346
+ "extra_special_token_339": "983493951323",
347
+ "extra_special_token_340": "724492262421",
348
+ "extra_special_token_341": "622553389126",
349
+ "extra_special_token_342": "889965243135",
350
+ "extra_special_token_343": "364492359246",
351
+ "extra_special_token_344": "154962668224",
352
+ "extra_special_token_345": "179564995814",
353
+ "extra_special_token_346": "418412875665",
354
+ "extra_special_token_347": "718951851413",
355
+ "extra_special_token_348": "699446724178",
356
+ "extra_special_token_349": "624266421831",
357
+ "extra_special_token_350": "815458725125",
358
+ "extra_special_token_351": "455423278865",
359
+ "extra_special_token_352": "393741199486",
360
+ "extra_special_token_353": "328552864359",
361
+ "extra_special_token_354": "211662639865",
362
+ "extra_special_token_355": "218784516525",
363
+ "extra_special_token_356": "762486672996",
364
+ "extra_special_token_357": "142799718159",
365
+ "extra_special_token_358": "858146415154",
366
+ "extra_special_token_359": "767858144912",
367
+ "extra_special_token_360": "571317457151",
368
+ "extra_special_token_361": "635127952696",
369
+ "extra_special_token_362": "116427191984",
370
+ "extra_special_token_363": "268921994538",
371
+ "extra_special_token_364": "523937669294",
372
+ "extra_special_token_365": "165429152138",
373
+ "extra_special_token_366": "739246183345",
374
+ "extra_special_token_367": "591464355756",
375
+ "extra_special_token_368": "212985874612",
376
+ "extra_special_token_369": "191887635211",
377
+ "extra_special_token_370": "967214577653",
378
+ "extra_special_token_371": "119342152414",
379
+ "extra_special_token_372": "946444632795",
380
+ "extra_special_token_373": "618423867817",
381
+ "extra_special_token_374": "228565148417",
382
+ "extra_special_token_375": "729116422489",
383
+ "extra_special_token_376": "527874729936",
384
+ "extra_special_token_377": "739784153482",
385
+ "extra_special_token_378": "387763951128",
386
+ "extra_special_token_379": "331369926711",
387
+ "extra_special_token_380": "562716493614",
388
+ "extra_special_token_381": "739667844957",
389
+ "extra_special_token_382": "562389434565",
390
+ "extra_special_token_383": "256497188281",
391
+ "extra_special_token_384": "859927364588",
392
+ "extra_special_token_385": "417668946583",
393
+ "extra_special_token_386": "357621613582",
394
+ "extra_special_token_387": "438435178228",
395
+ "extra_special_token_388": "485692541169",
396
+ "extra_special_token_389": "825815739116",
397
+ "extra_special_token_390": "342221452223",
398
+ "extra_special_token_391": "697747991249",
399
+ "extra_special_token_392": "716763689965",
400
+ "extra_special_token_393": "141499982867",
401
+ "extra_special_token_394": "818479319499",
402
+ "extra_special_token_395": "336813343298",
403
+ "extra_special_token_396": "594688742928",
404
+ "extra_special_token_397": "472129283475",
405
+ "extra_special_token_398": "514354144759",
406
+ "extra_special_token_399": "349249721685",
407
+ "extra_special_token_400": "546276298359",
408
+ "extra_special_token_401": "353755529131",
409
+ "extra_special_token_402": "315534574435",
410
+ "extra_special_token_403": "523723475786",
411
+ "extra_special_token_404": "215826764872",
412
+ "extra_special_token_405": "367968398551",
413
+ "extra_special_token_406": "569853653352",
414
+ "extra_special_token_407": "389715484387",
415
+ "extra_special_token_408": "293847485454",
416
+ "extra_special_token_409": "714738141818",
417
+ "extra_special_token_410": "178478368922",
418
+ "extra_special_token_411": "581493616981",
419
+ "extra_special_token_412": "589439538674",
420
+ "extra_special_token_413": "846657726193",
421
+ "extra_special_token_414": "722339992679",
422
+ "extra_special_token_415": "138154781148",
423
+ "extra_special_token_416": "757785319772",
424
+ "extra_special_token_417": "492516914298",
425
+ "extra_special_token_418": "919181521716",
426
+ "extra_special_token_419": "985781138935",
427
+ "extra_special_token_420": "476969195485",
428
+ "extra_special_token_421": "313145133463",
429
+ "extra_special_token_422": "758963111966",
430
+ "extra_special_token_423": "147541537162",
431
+ "extra_special_token_424": "557163366873",
432
+ "extra_special_token_425": "144373897488",
433
+ "extra_special_token_426": "522515164754",
434
+ "extra_special_token_427": "724964923582",
435
+ "extra_special_token_428": "284776712475",
436
+ "extra_special_token_429": "375429755114",
437
+ "extra_special_token_430": "181233596124",
438
+ "extra_special_token_431": "948585673431",
439
+ "extra_special_token_432": "243165586174",
440
+ "extra_special_token_433": "396847976144",
441
+ "extra_special_token_434": "997724962668",
442
+ "extra_special_token_435": "558837194455",
443
+ "extra_special_token_436": "163165456396",
444
+ "extra_special_token_437": "378749551722",
445
+ "extra_special_token_438": "161238482259",
446
+ "extra_special_token_439": "754978243758",
447
+ "extra_special_token_440": "195388849133",
448
+ "extra_special_token_441": "229775525672",
449
+ "extra_special_token_442": "262437452884",
450
+ "extra_special_token_443": "441377892146",
451
+ "extra_special_token_444": "451885565366",
452
+ "extra_special_token_445": "981277526855",
453
+ "extra_special_token_446": "762495822823",
454
+ "extra_special_token_447": "368763327262",
455
+ "extra_special_token_448": "757422791351",
456
+ "extra_special_token_449": "636324136426",
457
+ "extra_special_token_450": "214193645583",
458
+ "extra_special_token_451": "412843856172",
459
+ "extra_special_token_452": "179386156569",
460
+ "extra_special_token_453": "756916173536",
461
+ "extra_special_token_454": "892697125149",
462
+ "extra_special_token_455": "625334487352",
463
+ "extra_special_token_456": "941861857715",
464
+ "extra_special_token_457": "887417525236",
465
+ "extra_special_token_458": "649516938598",
466
+ "extra_special_token_459": "717628619782",
467
+ "extra_special_token_460": "438124184139",
468
+ "extra_special_token_461": "547563892268",
469
+ "extra_special_token_462": "856317483891",
470
+ "extra_special_token_463": "313313831273",
471
+ "extra_special_token_464": "371496153876",
472
+ "extra_special_token_465": "587541149322",
473
+ "extra_special_token_466": "265847332563",
474
+ "extra_special_token_467": "449549215429",
475
+ "extra_special_token_468": "163497196769",
476
+ "extra_special_token_469": "861342291298",
477
+ "extra_special_token_470": "268433315926",
478
+ "extra_special_token_471": "774679513717",
479
+ "extra_special_token_472": "851254219729",
480
+ "extra_special_token_473": "583527834464",
481
+ "extra_special_token_474": "488496781997",
482
+ "extra_special_token_475": "556814553861",
483
+ "extra_special_token_476": "482829231639",
484
+ "extra_special_token_477": "618878266619",
485
+ "extra_special_token_478": "147444452794",
486
+ "extra_special_token_479": "949235426629",
487
+ "extra_special_token_480": "357299947518",
488
+ "extra_special_token_481": "175528632226",
489
+ "extra_special_token_482": "645527857972",
490
+ "extra_special_token_483": "186872457894",
491
+ "extra_special_token_484": "552738847828",
492
+ "extra_special_token_485": "626748382482",
493
+ "extra_special_token_486": "921894985642",
494
+ "extra_special_token_487": "943878645871",
495
+ "extra_special_token_488": "859289776479",
496
+ "extra_special_token_489": "614583493135",
497
+ "extra_special_token_490": "933775286797",
498
+ "extra_special_token_491": "332234613346",
499
+ "extra_special_token_492": "325196781219",
500
+ "extra_special_token_493": "142526557681",
501
+ "extra_special_token_494": "356722692178",
502
+ "extra_special_token_495": "449318681694",
503
+ "extra_special_token_496": "687284547244",
504
+ "extra_special_token_497": "947262995132",
505
+ "extra_special_token_498": "893974619684",
506
+ "extra_special_token_499": "797238311233"
507
+ },
508
+ "is_local": false,
509
+ "model_input_names": [
510
+ "input_ids",
511
+ "attention_mask"
512
+ ],
513
+ "model_max_length": 1000000000000000019884624838656,
514
+ "pad_token": "<|end_of_text|>",
515
+ "tokenizer_class": "PreTrainedTokenizerFast"
516
+ }
llama3-8b/dp8/train.log ADDED
@@ -0,0 +1,276 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 2026-03-29 01:48:17,842 [INFO] new_opacus_codex.train_steps: epoch=1 step=5 loss=12.6436
2
+ 2026-03-29 01:49:00,550 [INFO] new_opacus_codex.train_steps: epoch=1 step=10 loss=12.6403
3
+ 2026-03-29 01:49:14,476 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=10 eval_loss=12.9540 duration_sec=13.92
4
+ 2026-03-29 01:49:57,849 [INFO] new_opacus_codex.train_steps: epoch=1 step=15 loss=12.6327
5
+ 2026-03-29 01:50:40,962 [INFO] new_opacus_codex.train_steps: epoch=1 step=20 loss=12.5780
6
+ 2026-03-29 01:50:54,793 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=20 eval_loss=12.9172 duration_sec=13.82
7
+ 2026-03-29 01:51:37,462 [INFO] new_opacus_codex.train_steps: epoch=1 step=25 loss=12.6256
8
+ 2026-03-29 01:52:19,666 [INFO] new_opacus_codex.train_steps: epoch=1 step=30 loss=12.6127
9
+ 2026-03-29 01:52:33,519 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=30 eval_loss=12.8571 duration_sec=13.85
10
+ 2026-03-29 01:53:16,437 [INFO] new_opacus_codex.train_steps: epoch=1 step=35 loss=12.4864
11
+ 2026-03-29 01:53:58,787 [INFO] new_opacus_codex.train_steps: epoch=1 step=40 loss=12.5944
12
+ 2026-03-29 01:54:12,661 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=40 eval_loss=12.7783 duration_sec=13.86
13
+ 2026-03-29 01:54:55,274 [INFO] new_opacus_codex.train_steps: epoch=1 step=45 loss=12.5971
14
+ 2026-03-29 01:55:37,472 [INFO] new_opacus_codex.train_steps: epoch=1 step=50 loss=12.4586
15
+ 2026-03-29 01:55:51,328 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=50 eval_loss=12.6873 duration_sec=13.84
16
+ 2026-03-29 01:56:32,782 [INFO] new_opacus_codex.train_steps: epoch=1 step=55 loss=12.4464
17
+ 2026-03-29 01:57:15,166 [INFO] new_opacus_codex.train_steps: epoch=1 step=60 loss=12.4083
18
+ 2026-03-29 01:57:29,046 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=60 eval_loss=12.5836 duration_sec=13.87
19
+ 2026-03-29 01:58:11,568 [INFO] new_opacus_codex.train_steps: epoch=1 step=65 loss=12.4112
20
+ 2026-03-29 01:58:54,114 [INFO] new_opacus_codex.train_steps: epoch=1 step=70 loss=12.4203
21
+ 2026-03-29 01:59:08,035 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=70 eval_loss=12.4688 duration_sec=13.91
22
+ 2026-03-29 01:59:50,754 [INFO] new_opacus_codex.train_steps: epoch=1 step=75 loss=12.3712
23
+ 2026-03-29 03:00:34,796 [INFO] new_opacus_codex.train_steps: epoch=1 step=80 loss=12.2271
24
+ 2026-03-29 03:00:48,664 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=80 eval_loss=12.3409 duration_sec=13.86
25
+ 2026-03-29 03:01:31,672 [INFO] new_opacus_codex.train_steps: epoch=1 step=85 loss=12.1119
26
+ 2026-03-29 03:02:14,870 [INFO] new_opacus_codex.train_steps: epoch=1 step=90 loss=12.0607
27
+ 2026-03-29 03:02:28,762 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=90 eval_loss=12.2062 duration_sec=13.88
28
+ 2026-03-29 03:03:10,876 [INFO] new_opacus_codex.train_steps: epoch=1 step=95 loss=12.1098
29
+ 2026-03-29 03:03:52,994 [INFO] new_opacus_codex.train_steps: epoch=1 step=100 loss=12.1589
30
+ 2026-03-29 03:04:06,833 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=100 eval_loss=12.0450 duration_sec=13.83
31
+ 2026-03-29 03:04:50,523 [INFO] new_opacus_codex.train_steps: epoch=1 step=105 loss=12.0365
32
+ 2026-03-29 03:05:32,261 [INFO] new_opacus_codex.train_steps: epoch=1 step=110 loss=11.9842
33
+ 2026-03-29 03:05:46,132 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=110 eval_loss=11.8622 duration_sec=13.86
34
+ 2026-03-29 03:06:28,790 [INFO] new_opacus_codex.train_steps: epoch=1 step=115 loss=11.8362
35
+ 2026-03-29 03:07:10,748 [INFO] new_opacus_codex.train_steps: epoch=1 step=120 loss=11.6789
36
+ 2026-03-29 03:07:24,755 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=120 eval_loss=11.6674 duration_sec=13.99
37
+ 2026-03-29 03:08:06,382 [INFO] new_opacus_codex.train_steps: epoch=1 step=125 loss=11.6873
38
+ 2026-03-29 03:08:48,677 [INFO] new_opacus_codex.train_steps: epoch=1 step=130 loss=11.5385
39
+ 2026-03-29 03:09:02,528 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=130 eval_loss=11.4582 duration_sec=13.84
40
+ 2026-03-29 03:09:45,129 [INFO] new_opacus_codex.train_steps: epoch=1 step=135 loss=11.4124
41
+ 2026-03-29 03:10:26,493 [INFO] new_opacus_codex.train_steps: epoch=1 step=140 loss=11.4211
42
+ 2026-03-29 03:10:40,384 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=140 eval_loss=11.2309 duration_sec=13.88
43
+ 2026-03-29 03:11:22,692 [INFO] new_opacus_codex.train_steps: epoch=1 step=145 loss=11.2841
44
+ 2026-03-29 03:12:05,729 [INFO] new_opacus_codex.train_steps: epoch=1 step=150 loss=11.1467
45
+ 2026-03-29 03:12:19,648 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=150 eval_loss=10.9978 duration_sec=13.88
46
+ 2026-03-29 03:13:01,456 [INFO] new_opacus_codex.train_steps: epoch=1 step=155 loss=10.9408
47
+ 2026-03-29 03:13:42,209 [INFO] new_opacus_codex.train_steps: epoch=1 step=160 loss=10.8491
48
+ 2026-03-29 03:13:56,100 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=160 eval_loss=10.7718 duration_sec=13.88
49
+ 2026-03-29 03:14:39,221 [INFO] new_opacus_codex.train_steps: epoch=1 step=165 loss=10.8826
50
+ 2026-03-29 03:15:21,847 [INFO] new_opacus_codex.train_steps: epoch=1 step=170 loss=10.7471
51
+ 2026-03-29 03:15:35,748 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=170 eval_loss=10.5485 duration_sec=13.89
52
+ 2026-03-29 03:16:17,979 [INFO] new_opacus_codex.train_steps: epoch=1 step=175 loss=10.5876
53
+ 2026-03-29 03:17:00,724 [INFO] new_opacus_codex.train_steps: epoch=1 step=180 loss=10.5963
54
+ 2026-03-29 03:17:14,713 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=180 eval_loss=10.3261 duration_sec=13.98
55
+ 2026-03-29 03:18:19,545 [INFO] new_opacus_codex.train_steps: epoch=2 step=185 loss=10.7562
56
+ 2026-03-29 03:19:02,390 [INFO] new_opacus_codex.train_steps: epoch=2 step=190 loss=10.4240
57
+ 2026-03-29 03:19:16,287 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=190 eval_loss=10.1163 duration_sec=13.88
58
+ 2026-03-29 03:19:59,249 [INFO] new_opacus_codex.train_steps: epoch=2 step=195 loss=10.3020
59
+ 2026-03-29 03:20:41,674 [INFO] new_opacus_codex.train_steps: epoch=2 step=200 loss=10.1524
60
+ 2026-03-29 03:20:55,577 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=200 eval_loss=9.9156 duration_sec=13.88
61
+ 2026-03-29 03:21:38,138 [INFO] new_opacus_codex.train_steps: epoch=2 step=205 loss=10.0216
62
+ 2026-03-29 03:22:21,300 [INFO] new_opacus_codex.train_steps: epoch=2 step=210 loss=9.9256
63
+ 2026-03-29 03:22:35,214 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=210 eval_loss=9.7202 duration_sec=13.90
64
+ 2026-03-29 03:23:18,159 [INFO] new_opacus_codex.train_steps: epoch=2 step=215 loss=9.8738
65
+ 2026-03-29 03:24:00,837 [INFO] new_opacus_codex.train_steps: epoch=2 step=220 loss=9.8461
66
+ 2026-03-29 03:24:14,740 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=220 eval_loss=9.5444 duration_sec=13.89
67
+ 2026-03-29 03:24:58,432 [INFO] new_opacus_codex.train_steps: epoch=2 step=225 loss=9.7056
68
+ 2026-03-29 03:25:40,224 [INFO] new_opacus_codex.train_steps: epoch=2 step=230 loss=9.6949
69
+ 2026-03-29 03:25:54,091 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=230 eval_loss=9.3817 duration_sec=13.85
70
+ 2026-03-29 03:26:36,008 [INFO] new_opacus_codex.train_steps: epoch=2 step=235 loss=9.7830
71
+ 2026-03-29 03:27:19,228 [INFO] new_opacus_codex.train_steps: epoch=2 step=240 loss=9.7200
72
+ 2026-03-29 03:27:33,111 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=240 eval_loss=9.2225 duration_sec=13.88
73
+ 2026-03-29 03:28:15,576 [INFO] new_opacus_codex.train_steps: epoch=2 step=245 loss=9.4512
74
+ 2026-03-29 03:28:58,421 [INFO] new_opacus_codex.train_steps: epoch=2 step=250 loss=9.3841
75
+ 2026-03-29 03:29:12,319 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=250 eval_loss=9.0764 duration_sec=13.89
76
+ 2026-03-29 03:29:54,965 [INFO] new_opacus_codex.train_steps: epoch=2 step=255 loss=9.3551
77
+ 2026-03-29 03:30:37,214 [INFO] new_opacus_codex.train_steps: epoch=2 step=260 loss=9.1777
78
+ 2026-03-29 03:30:51,128 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=260 eval_loss=8.9443 duration_sec=13.90
79
+ 2026-03-29 03:31:32,719 [INFO] new_opacus_codex.train_steps: epoch=2 step=265 loss=9.1285
80
+ 2026-03-29 03:32:14,730 [INFO] new_opacus_codex.train_steps: epoch=2 step=270 loss=9.1577
81
+ 2026-03-29 03:32:28,620 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=270 eval_loss=8.8264 duration_sec=13.88
82
+ 2026-03-29 03:33:11,504 [INFO] new_opacus_codex.train_steps: epoch=2 step=275 loss=9.1826
83
+ 2026-03-29 03:33:55,311 [INFO] new_opacus_codex.train_steps: epoch=2 step=280 loss=9.0614
84
+ 2026-03-29 03:34:09,191 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=280 eval_loss=8.7266 duration_sec=13.86
85
+ 2026-03-29 03:34:51,649 [INFO] new_opacus_codex.train_steps: epoch=2 step=285 loss=8.9843
86
+ 2026-03-29 03:35:34,301 [INFO] new_opacus_codex.train_steps: epoch=2 step=290 loss=8.9571
87
+ 2026-03-29 03:35:48,228 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=290 eval_loss=8.6414 duration_sec=13.92
88
+ 2026-03-29 03:36:31,168 [INFO] new_opacus_codex.train_steps: epoch=2 step=295 loss=8.9224
89
+ 2026-03-29 03:37:13,601 [INFO] new_opacus_codex.train_steps: epoch=2 step=300 loss=8.9701
90
+ 2026-03-29 03:37:27,490 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=300 eval_loss=8.5703 duration_sec=13.88
91
+ 2026-03-29 03:38:10,814 [INFO] new_opacus_codex.train_steps: epoch=2 step=305 loss=8.9565
92
+ 2026-03-29 03:38:54,935 [INFO] new_opacus_codex.train_steps: epoch=2 step=310 loss=8.9234
93
+ 2026-03-29 03:39:08,795 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=310 eval_loss=8.5074 duration_sec=13.85
94
+ 2026-03-29 03:39:52,436 [INFO] new_opacus_codex.train_steps: epoch=2 step=315 loss=8.8654
95
+ 2026-03-29 03:40:35,927 [INFO] new_opacus_codex.train_steps: epoch=2 step=320 loss=8.9284
96
+ 2026-03-29 03:40:49,952 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=320 eval_loss=8.4488 duration_sec=14.01
97
+ 2026-03-29 03:41:33,051 [INFO] new_opacus_codex.train_steps: epoch=2 step=325 loss=8.9184
98
+ 2026-03-29 03:42:14,507 [INFO] new_opacus_codex.train_steps: epoch=2 step=330 loss=8.8525
99
+ 2026-03-29 03:42:28,476 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=330 eval_loss=8.3984 duration_sec=13.95
100
+ 2026-03-29 03:43:10,099 [INFO] new_opacus_codex.train_steps: epoch=2 step=335 loss=8.7556
101
+ 2026-03-29 03:43:52,701 [INFO] new_opacus_codex.train_steps: epoch=2 step=340 loss=8.5616
102
+ 2026-03-29 03:44:06,605 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=340 eval_loss=8.3567 duration_sec=13.89
103
+ 2026-03-29 03:44:48,598 [INFO] new_opacus_codex.train_steps: epoch=2 step=345 loss=8.4590
104
+ 2026-03-29 03:45:32,101 [INFO] new_opacus_codex.train_steps: epoch=2 step=350 loss=8.4827
105
+ 2026-03-29 03:45:46,006 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=350 eval_loss=8.3249 duration_sec=13.89
106
+ 2026-03-29 03:46:27,751 [INFO] new_opacus_codex.train_steps: epoch=2 step=355 loss=8.7077
107
+ 2026-03-29 03:47:10,022 [INFO] new_opacus_codex.train_steps: epoch=2 step=360 loss=8.8328
108
+ 2026-03-29 03:47:23,959 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=360 eval_loss=8.2958 duration_sec=13.92
109
+ 2026-03-29 03:48:06,427 [INFO] new_opacus_codex.train_steps: epoch=2 step=365 loss=8.7603
110
+ 2026-03-29 03:49:13,838 [INFO] new_opacus_codex.train_steps: epoch=3 step=370 loss=8.5907
111
+ 2026-03-29 03:49:27,747 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=370 eval_loss=8.2695 duration_sec=13.90
112
+ 2026-03-29 03:50:10,388 [INFO] new_opacus_codex.train_steps: epoch=3 step=375 loss=8.4038
113
+ 2026-03-29 03:50:53,487 [INFO] new_opacus_codex.train_steps: epoch=3 step=380 loss=8.4507
114
+ 2026-03-29 03:51:07,362 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=380 eval_loss=8.2469 duration_sec=13.86
115
+ 2026-03-29 03:51:49,509 [INFO] new_opacus_codex.train_steps: epoch=3 step=385 loss=8.6010
116
+ 2026-03-29 03:52:32,543 [INFO] new_opacus_codex.train_steps: epoch=3 step=390 loss=8.5997
117
+ 2026-03-29 03:52:46,419 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=390 eval_loss=8.2199 duration_sec=13.86
118
+ 2026-03-29 03:53:28,789 [INFO] new_opacus_codex.train_steps: epoch=3 step=395 loss=8.5738
119
+ 2026-03-29 03:54:11,623 [INFO] new_opacus_codex.train_steps: epoch=3 step=400 loss=8.5178
120
+ 2026-03-29 03:54:25,570 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=400 eval_loss=8.1958 duration_sec=13.94
121
+ 2026-03-29 03:55:08,025 [INFO] new_opacus_codex.train_steps: epoch=3 step=405 loss=8.4449
122
+ 2026-03-29 03:55:51,712 [INFO] new_opacus_codex.train_steps: epoch=3 step=410 loss=8.5738
123
+ 2026-03-29 03:56:05,624 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=410 eval_loss=8.1682 duration_sec=13.90
124
+ 2026-03-29 03:56:48,179 [INFO] new_opacus_codex.train_steps: epoch=3 step=415 loss=8.5169
125
+ 2026-03-29 03:57:31,318 [INFO] new_opacus_codex.train_steps: epoch=3 step=420 loss=8.4918
126
+ 2026-03-29 03:57:45,219 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=420 eval_loss=8.1408 duration_sec=13.89
127
+ 2026-03-29 03:58:29,258 [INFO] new_opacus_codex.train_steps: epoch=3 step=425 loss=8.4555
128
+ 2026-03-29 03:59:11,720 [INFO] new_opacus_codex.train_steps: epoch=3 step=430 loss=8.3939
129
+ 2026-03-29 03:59:25,643 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=430 eval_loss=8.1182 duration_sec=13.91
130
+ 2026-03-29 04:00:09,201 [INFO] new_opacus_codex.train_steps: epoch=3 step=435 loss=8.3582
131
+ 2026-03-29 04:00:50,735 [INFO] new_opacus_codex.train_steps: epoch=3 step=440 loss=8.2233
132
+ 2026-03-29 04:01:04,653 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=440 eval_loss=8.0980 duration_sec=13.91
133
+ 2026-03-29 04:01:48,782 [INFO] new_opacus_codex.train_steps: epoch=3 step=445 loss=8.3102
134
+ 2026-03-29 04:02:30,689 [INFO] new_opacus_codex.train_steps: epoch=3 step=450 loss=8.4583
135
+ 2026-03-29 04:02:44,582 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=450 eval_loss=8.0771 duration_sec=13.88
136
+ 2026-03-29 04:03:27,701 [INFO] new_opacus_codex.train_steps: epoch=3 step=455 loss=8.5624
137
+ 2026-03-29 04:04:10,032 [INFO] new_opacus_codex.train_steps: epoch=3 step=460 loss=8.4621
138
+ 2026-03-29 04:04:24,049 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=460 eval_loss=8.0561 duration_sec=14.01
139
+ 2026-03-29 04:05:06,003 [INFO] new_opacus_codex.train_steps: epoch=3 step=465 loss=8.4351
140
+ 2026-03-29 04:05:48,361 [INFO] new_opacus_codex.train_steps: epoch=3 step=470 loss=8.4753
141
+ 2026-03-29 04:06:02,270 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=470 eval_loss=8.0388 duration_sec=13.90
142
+ 2026-03-29 04:06:45,281 [INFO] new_opacus_codex.train_steps: epoch=3 step=475 loss=8.4969
143
+ 2026-03-29 04:07:27,774 [INFO] new_opacus_codex.train_steps: epoch=3 step=480 loss=8.4943
144
+ 2026-03-29 04:07:41,697 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=480 eval_loss=8.0227 duration_sec=13.91
145
+ 2026-03-29 04:08:24,024 [INFO] new_opacus_codex.train_steps: epoch=3 step=485 loss=8.3448
146
+ 2026-03-29 04:09:07,681 [INFO] new_opacus_codex.train_steps: epoch=3 step=490 loss=8.3846
147
+ 2026-03-29 04:09:21,595 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=490 eval_loss=8.0054 duration_sec=13.90
148
+ 2026-03-29 04:10:04,490 [INFO] new_opacus_codex.train_steps: epoch=3 step=495 loss=8.5365
149
+ 2026-03-29 04:10:47,863 [INFO] new_opacus_codex.train_steps: epoch=3 step=500 loss=8.4996
150
+ 2026-03-29 04:11:01,798 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=500 eval_loss=7.9882 duration_sec=13.92
151
+ 2026-03-29 04:11:44,463 [INFO] new_opacus_codex.train_steps: epoch=3 step=505 loss=8.3776
152
+ 2026-03-29 04:12:27,278 [INFO] new_opacus_codex.train_steps: epoch=3 step=510 loss=8.3997
153
+ 2026-03-29 04:12:41,217 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=510 eval_loss=7.9706 duration_sec=13.93
154
+ 2026-03-29 04:13:24,134 [INFO] new_opacus_codex.train_steps: epoch=3 step=515 loss=8.3370
155
+ 2026-03-29 04:14:06,196 [INFO] new_opacus_codex.train_steps: epoch=3 step=520 loss=8.4006
156
+ 2026-03-29 04:14:20,121 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=520 eval_loss=7.9543 duration_sec=13.91
157
+ 2026-03-29 04:15:01,209 [INFO] new_opacus_codex.train_steps: epoch=3 step=525 loss=8.3610
158
+ 2026-03-29 04:15:43,376 [INFO] new_opacus_codex.train_steps: epoch=3 step=530 loss=8.1149
159
+ 2026-03-29 04:15:57,289 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=530 eval_loss=7.9399 duration_sec=13.90
160
+ 2026-03-29 04:16:39,590 [INFO] new_opacus_codex.train_steps: epoch=3 step=535 loss=8.3729
161
+ 2026-03-29 04:17:23,131 [INFO] new_opacus_codex.train_steps: epoch=3 step=540 loss=8.3566
162
+ 2026-03-29 04:17:37,063 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=540 eval_loss=7.9260 duration_sec=13.92
163
+ 2026-03-29 04:18:19,552 [INFO] new_opacus_codex.train_steps: epoch=3 step=545 loss=8.2228
164
+ 2026-03-29 04:19:03,051 [INFO] new_opacus_codex.train_steps: epoch=3 step=550 loss=8.2941
165
+ 2026-03-29 04:19:16,961 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=550 eval_loss=7.9110 duration_sec=13.89
166
+ 2026-03-29 04:20:21,857 [INFO] new_opacus_codex.train_steps: epoch=4 step=555 loss=8.1343
167
+ 2026-03-29 04:21:06,411 [INFO] new_opacus_codex.train_steps: epoch=4 step=560 loss=8.1155
168
+ 2026-03-29 04:21:20,283 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=560 eval_loss=7.8967 duration_sec=13.86
169
+ 2026-03-29 04:22:02,666 [INFO] new_opacus_codex.train_steps: epoch=4 step=565 loss=8.2031
170
+ 2026-03-29 04:22:46,440 [INFO] new_opacus_codex.train_steps: epoch=4 step=570 loss=8.2715
171
+ 2026-03-29 04:23:00,332 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=570 eval_loss=7.8837 duration_sec=13.88
172
+ 2026-03-29 04:23:43,580 [INFO] new_opacus_codex.train_steps: epoch=4 step=575 loss=8.1927
173
+ 2026-03-29 04:24:27,210 [INFO] new_opacus_codex.train_steps: epoch=4 step=580 loss=8.1510
174
+ 2026-03-29 04:24:41,100 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=580 eval_loss=7.8707 duration_sec=13.88
175
+ 2026-03-29 04:25:25,118 [INFO] new_opacus_codex.train_steps: epoch=4 step=585 loss=8.2746
176
+ 2026-03-29 04:26:07,970 [INFO] new_opacus_codex.train_steps: epoch=4 step=590 loss=8.3548
177
+ 2026-03-29 04:26:21,834 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=590 eval_loss=7.8582 duration_sec=13.85
178
+ 2026-03-29 04:27:04,118 [INFO] new_opacus_codex.train_steps: epoch=4 step=595 loss=8.2218
179
+ 2026-03-29 04:27:48,164 [INFO] new_opacus_codex.train_steps: epoch=4 step=600 loss=8.0846
180
+ 2026-03-29 04:28:02,084 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=600 eval_loss=7.8456 duration_sec=13.91
181
+ 2026-03-29 04:28:43,980 [INFO] new_opacus_codex.train_steps: epoch=4 step=605 loss=8.2157
182
+ 2026-03-29 04:29:27,227 [INFO] new_opacus_codex.train_steps: epoch=4 step=610 loss=8.2702
183
+ 2026-03-29 04:29:41,148 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=610 eval_loss=7.8329 duration_sec=13.91
184
+ 2026-03-29 04:30:24,835 [INFO] new_opacus_codex.train_steps: epoch=4 step=615 loss=8.2566
185
+ 2026-03-29 04:31:07,498 [INFO] new_opacus_codex.train_steps: epoch=4 step=620 loss=8.3520
186
+ 2026-03-29 04:31:21,352 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=620 eval_loss=7.8224 duration_sec=13.84
187
+ 2026-03-29 04:32:05,122 [INFO] new_opacus_codex.train_steps: epoch=4 step=625 loss=8.2735
188
+ 2026-03-29 04:32:48,687 [INFO] new_opacus_codex.train_steps: epoch=4 step=630 loss=8.2926
189
+ 2026-03-29 04:33:02,610 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=630 eval_loss=7.8112 duration_sec=13.90
190
+ 2026-03-29 04:33:46,402 [INFO] new_opacus_codex.train_steps: epoch=4 step=635 loss=8.3737
191
+ 2026-03-29 04:34:29,487 [INFO] new_opacus_codex.train_steps: epoch=4 step=640 loss=8.2948
192
+ 2026-03-29 04:34:43,448 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=640 eval_loss=7.7998 duration_sec=13.95
193
+ 2026-03-29 04:35:26,885 [INFO] new_opacus_codex.train_steps: epoch=4 step=645 loss=8.2378
194
+ 2026-03-29 04:36:09,756 [INFO] new_opacus_codex.train_steps: epoch=4 step=650 loss=8.2147
195
+ 2026-03-29 04:36:23,638 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=650 eval_loss=7.7902 duration_sec=13.87
196
+ 2026-03-29 04:37:06,425 [INFO] new_opacus_codex.train_steps: epoch=4 step=655 loss=8.2516
197
+ 2026-03-29 04:37:49,046 [INFO] new_opacus_codex.train_steps: epoch=4 step=660 loss=8.0961
198
+ 2026-03-29 04:38:02,979 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=660 eval_loss=7.7821 duration_sec=13.92
199
+ 2026-03-29 04:38:45,855 [INFO] new_opacus_codex.train_steps: epoch=4 step=665 loss=8.0751
200
+ 2026-03-29 04:39:28,104 [INFO] new_opacus_codex.train_steps: epoch=4 step=670 loss=8.2636
201
+ 2026-03-29 04:39:42,004 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=670 eval_loss=7.7745 duration_sec=13.88
202
+ 2026-03-29 04:40:25,062 [INFO] new_opacus_codex.train_steps: epoch=4 step=675 loss=8.2509
203
+ 2026-03-29 04:41:07,613 [INFO] new_opacus_codex.train_steps: epoch=4 step=680 loss=8.1759
204
+ 2026-03-29 04:41:21,544 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=680 eval_loss=7.7649 duration_sec=13.92
205
+ 2026-03-29 04:42:03,782 [INFO] new_opacus_codex.train_steps: epoch=4 step=685 loss=8.0666
206
+ 2026-03-29 04:42:47,266 [INFO] new_opacus_codex.train_steps: epoch=4 step=690 loss=8.1033
207
+ 2026-03-29 04:43:01,174 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=690 eval_loss=7.7554 duration_sec=13.90
208
+ 2026-03-29 04:43:43,702 [INFO] new_opacus_codex.train_steps: epoch=4 step=695 loss=8.2774
209
+ 2026-03-29 04:44:26,942 [INFO] new_opacus_codex.train_steps: epoch=4 step=700 loss=8.2652
210
+ 2026-03-29 04:44:40,960 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=700 eval_loss=7.7458 duration_sec=14.01
211
+ 2026-03-29 04:45:22,848 [INFO] new_opacus_codex.train_steps: epoch=4 step=705 loss=8.1177
212
+ 2026-03-29 04:46:06,188 [INFO] new_opacus_codex.train_steps: epoch=4 step=710 loss=8.1770
213
+ 2026-03-29 04:46:20,125 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=710 eval_loss=7.7370 duration_sec=13.93
214
+ 2026-03-29 04:47:02,448 [INFO] new_opacus_codex.train_steps: epoch=4 step=715 loss=8.1182
215
+ 2026-03-29 04:47:46,247 [INFO] new_opacus_codex.train_steps: epoch=4 step=720 loss=8.1244
216
+ 2026-03-29 04:48:00,234 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=720 eval_loss=7.7299 duration_sec=13.98
217
+ 2026-03-29 04:48:43,498 [INFO] new_opacus_codex.train_steps: epoch=4 step=725 loss=8.1529
218
+ 2026-03-29 04:49:27,722 [INFO] new_opacus_codex.train_steps: epoch=4 step=730 loss=8.1000
219
+ 2026-03-29 04:49:41,661 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=730 eval_loss=7.7235 duration_sec=13.93
220
+ 2026-03-29 04:50:25,428 [INFO] new_opacus_codex.train_steps: epoch=4 step=735 loss=8.1157
221
+ 2026-03-29 04:51:30,165 [INFO] new_opacus_codex.train_steps: epoch=5 step=740 loss=8.2690
222
+ 2026-03-29 04:51:44,147 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=740 eval_loss=7.7169 duration_sec=13.97
223
+ 2026-03-29 04:52:27,017 [INFO] new_opacus_codex.train_steps: epoch=5 step=745 loss=8.3354
224
+ 2026-03-29 04:53:09,855 [INFO] new_opacus_codex.train_steps: epoch=5 step=750 loss=8.1002
225
+ 2026-03-29 04:53:23,788 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=750 eval_loss=7.7109 duration_sec=13.92
226
+ 2026-03-29 04:54:07,599 [INFO] new_opacus_codex.train_steps: epoch=5 step=755 loss=7.9482
227
+ 2026-03-29 04:54:49,559 [INFO] new_opacus_codex.train_steps: epoch=5 step=760 loss=8.1674
228
+ 2026-03-29 04:55:03,458 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=760 eval_loss=7.7040 duration_sec=13.89
229
+ 2026-03-29 04:55:47,050 [INFO] new_opacus_codex.train_steps: epoch=5 step=765 loss=8.1573
230
+ 2026-03-29 04:56:28,659 [INFO] new_opacus_codex.train_steps: epoch=5 step=770 loss=8.0822
231
+ 2026-03-29 04:56:42,591 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=770 eval_loss=7.6979 duration_sec=13.92
232
+ 2026-03-29 04:57:25,217 [INFO] new_opacus_codex.train_steps: epoch=5 step=775 loss=8.1537
233
+ 2026-03-29 04:58:08,111 [INFO] new_opacus_codex.train_steps: epoch=5 step=780 loss=8.2979
234
+ 2026-03-29 04:58:21,989 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=780 eval_loss=7.6921 duration_sec=13.86
235
+ 2026-03-29 04:59:04,749 [INFO] new_opacus_codex.train_steps: epoch=5 step=785 loss=8.2653
236
+ 2026-03-29 04:59:47,501 [INFO] new_opacus_codex.train_steps: epoch=5 step=790 loss=8.1694
237
+ 2026-03-29 05:00:01,368 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=790 eval_loss=7.6868 duration_sec=13.86
238
+ 2026-03-29 05:00:44,857 [INFO] new_opacus_codex.train_steps: epoch=5 step=795 loss=8.2119
239
+ 2026-03-29 05:01:27,220 [INFO] new_opacus_codex.train_steps: epoch=5 step=800 loss=8.3282
240
+ 2026-03-29 05:01:41,146 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=800 eval_loss=7.6810 duration_sec=13.91
241
+ 2026-03-29 05:02:24,303 [INFO] new_opacus_codex.train_steps: epoch=5 step=805 loss=8.3409
242
+ 2026-03-29 05:03:07,079 [INFO] new_opacus_codex.train_steps: epoch=5 step=810 loss=8.2251
243
+ 2026-03-29 05:03:21,010 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=810 eval_loss=7.6753 duration_sec=13.92
244
+ 2026-03-29 05:04:04,033 [INFO] new_opacus_codex.train_steps: epoch=5 step=815 loss=8.1414
245
+ 2026-03-29 05:04:47,670 [INFO] new_opacus_codex.train_steps: epoch=5 step=820 loss=7.9533
246
+ 2026-03-29 05:05:01,655 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=820 eval_loss=7.6704 duration_sec=13.97
247
+ 2026-03-29 05:05:43,608 [INFO] new_opacus_codex.train_steps: epoch=5 step=825 loss=7.9214
248
+ 2026-03-29 05:06:27,358 [INFO] new_opacus_codex.train_steps: epoch=5 step=830 loss=7.9782
249
+ 2026-03-29 05:06:41,231 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=830 eval_loss=7.6658 duration_sec=13.86
250
+ 2026-03-29 05:07:26,877 [INFO] new_opacus_codex.train_steps: epoch=5 step=835 loss=7.8309
251
+ 2026-03-29 05:08:09,511 [INFO] new_opacus_codex.train_steps: epoch=5 step=840 loss=7.9757
252
+ 2026-03-29 05:08:23,445 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=840 eval_loss=7.6605 duration_sec=13.92
253
+ 2026-03-29 05:09:06,513 [INFO] new_opacus_codex.train_steps: epoch=5 step=845 loss=8.0575
254
+ 2026-03-29 05:09:48,679 [INFO] new_opacus_codex.train_steps: epoch=5 step=850 loss=7.9575
255
+ 2026-03-29 05:10:02,594 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=850 eval_loss=7.6557 duration_sec=13.91
256
+ 2026-03-29 05:10:46,225 [INFO] new_opacus_codex.train_steps: epoch=5 step=855 loss=8.0380
257
+ 2026-03-29 05:11:29,039 [INFO] new_opacus_codex.train_steps: epoch=5 step=860 loss=7.9293
258
+ 2026-03-29 05:11:43,008 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=860 eval_loss=7.6514 duration_sec=13.96
259
+ 2026-03-29 05:12:25,957 [INFO] new_opacus_codex.train_steps: epoch=5 step=865 loss=7.8763
260
+ 2026-03-29 05:13:09,310 [INFO] new_opacus_codex.train_steps: epoch=5 step=870 loss=7.9866
261
+ 2026-03-29 05:13:23,193 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=870 eval_loss=7.6477 duration_sec=13.87
262
+ 2026-03-29 05:14:07,547 [INFO] new_opacus_codex.train_steps: epoch=5 step=875 loss=8.0219
263
+ 2026-03-29 05:14:51,763 [INFO] new_opacus_codex.train_steps: epoch=5 step=880 loss=8.0358
264
+ 2026-03-29 05:15:05,669 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=880 eval_loss=7.6443 duration_sec=13.89
265
+ 2026-03-29 05:15:49,867 [INFO] new_opacus_codex.train_steps: epoch=5 step=885 loss=8.0623
266
+ 2026-03-29 05:16:32,994 [INFO] new_opacus_codex.train_steps: epoch=5 step=890 loss=7.9799
267
+ 2026-03-29 05:16:46,869 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=890 eval_loss=7.6403 duration_sec=13.86
268
+ 2026-03-29 05:17:31,768 [INFO] new_opacus_codex.train_steps: epoch=5 step=895 loss=7.9192
269
+ 2026-03-29 05:18:15,007 [INFO] new_opacus_codex.train_steps: epoch=5 step=900 loss=7.9882
270
+ 2026-03-29 05:18:29,016 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=900 eval_loss=7.6359 duration_sec=14.00
271
+ 2026-03-29 05:19:12,657 [INFO] new_opacus_codex.train_steps: epoch=5 step=905 loss=8.2397
272
+ 2026-03-29 05:19:56,268 [INFO] new_opacus_codex.train_steps: epoch=5 step=910 loss=8.1681
273
+ 2026-03-29 05:20:10,199 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=910 eval_loss=7.6323 duration_sec=13.92
274
+ 2026-03-29 05:20:53,881 [INFO] new_opacus_codex.train_steps: epoch=5 step=915 loss=8.0207
275
+ 2026-03-29 05:21:29,340 [INFO] new_opacus_codex.train_steps: epoch=5 step=920 loss=8.2316
276
+ 2026-03-29 05:21:43,333 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=920 eval_loss=7.6284 duration_sec=13.98
llama3-8b/dp8_v2/adapter/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: meta-llama/Meta-Llama-3-8B
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:meta-llama/Meta-Llama-3-8B
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.18.1
llama3-8b/dp8_v2/adapter/adapter_config.json ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": true,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 32,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "megatron_config": null,
23
+ "megatron_core": "megatron.core",
24
+ "modules_to_save": [
25
+ "lm_head",
26
+ "embed_tokens"
27
+ ],
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.18.1",
30
+ "qalora_group_size": 16,
31
+ "r": 16,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "k_proj",
36
+ "q_proj",
37
+ "o_proj",
38
+ "v_proj"
39
+ ],
40
+ "target_parameters": null,
41
+ "task_type": "CAUSAL_LM",
42
+ "trainable_token_indices": null,
43
+ "use_dora": false,
44
+ "use_qalora": false,
45
+ "use_rslora": false
46
+ }
llama3-8b/dp8_v2/audit_results.json ADDED
@@ -0,0 +1,137 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "delta": 1e-05,
3
+ "num_canaries": 500,
4
+ "num_members": 250,
5
+ "paper_guess_fraction": 0.2,
6
+ "paper_guess_steps": 20,
7
+ "loss": {
8
+ "auc": 0.540616,
9
+ "empirical_epsilon": {
10
+ "0.05": 0.0,
11
+ "0.01": 0.0
12
+ },
13
+ "empirical_epsilon_details": {
14
+ "0.05": {
15
+ "epsilon": 0.0,
16
+ "num_guesses": 0,
17
+ "correct_guesses": 0,
18
+ "candidate_num_guesses": [
19
+ 5,
20
+ 10,
21
+ 15,
22
+ 20,
23
+ 25,
24
+ 30,
25
+ 35,
26
+ 40,
27
+ 45,
28
+ 50,
29
+ 55,
30
+ 60,
31
+ 65,
32
+ 70,
33
+ 75,
34
+ 80,
35
+ 85,
36
+ 90,
37
+ 95,
38
+ 100
39
+ ],
40
+ "direction": "lower"
41
+ },
42
+ "0.01": {
43
+ "epsilon": 0.0,
44
+ "num_guesses": 0,
45
+ "correct_guesses": 0,
46
+ "candidate_num_guesses": [
47
+ 5,
48
+ 10,
49
+ 15,
50
+ 20,
51
+ 25,
52
+ 30,
53
+ 35,
54
+ 40,
55
+ 45,
56
+ 50,
57
+ 55,
58
+ 60,
59
+ 65,
60
+ 70,
61
+ 75,
62
+ 80,
63
+ 85,
64
+ 90,
65
+ 95,
66
+ 100
67
+ ],
68
+ "direction": "lower"
69
+ }
70
+ }
71
+ },
72
+ "embedding": {
73
+ "auc": 0.506472,
74
+ "empirical_epsilon": {
75
+ "0.05": 0.0,
76
+ "0.01": 0.0
77
+ },
78
+ "empirical_epsilon_details": {
79
+ "0.05": {
80
+ "epsilon": 0.0,
81
+ "num_guesses": 0,
82
+ "correct_guesses": 0,
83
+ "candidate_num_guesses": [
84
+ 5,
85
+ 10,
86
+ 15,
87
+ 20,
88
+ 25,
89
+ 30,
90
+ 35,
91
+ 40,
92
+ 45,
93
+ 50,
94
+ 55,
95
+ 60,
96
+ 65,
97
+ 70,
98
+ 75,
99
+ 80,
100
+ 85,
101
+ 90,
102
+ 95,
103
+ 100
104
+ ],
105
+ "direction": "lower"
106
+ },
107
+ "0.01": {
108
+ "epsilon": 0.0,
109
+ "num_guesses": 0,
110
+ "correct_guesses": 0,
111
+ "candidate_num_guesses": [
112
+ 5,
113
+ 10,
114
+ 15,
115
+ 20,
116
+ 25,
117
+ 30,
118
+ 35,
119
+ 40,
120
+ 45,
121
+ 50,
122
+ 55,
123
+ 60,
124
+ 65,
125
+ 70,
126
+ 75,
127
+ 80,
128
+ 85,
129
+ 90,
130
+ 95,
131
+ 100
132
+ ],
133
+ "direction": "lower"
134
+ }
135
+ }
136
+ }
137
+ }
llama3-8b/dp8_v2/canary_meta.json ADDED
The diff for this file is too large to render. See raw diff
 
llama3-8b/dp8_v2/metrics.jsonl ADDED
The diff for this file is too large to render. See raw diff
 
llama3-8b/dp8_v2/resolved_config.yaml ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ model:
2
+ name: meta-llama/Meta-Llama-3-8B
3
+ tokenizer_name: meta-llama/Meta-Llama-3-8B
4
+ max_length: 1024
5
+ dtype: bfloat16
6
+ trust_remote_code: true
7
+ use_fast_tokenizer: true
8
+ cache_dir: null
9
+ local_files_only: false
10
+ low_cpu_mem_usage: true
11
+ tie_word_embeddings: true
12
+ gradient_checkpointing: false
13
+ use_chat_template: false
14
+ dataset:
15
+ name: melihcatal/codedp-cpt
16
+ split: train
17
+ mode: cpt
18
+ text_column: text
19
+ validation_ratio: 0.05
20
+ max_samples: -1
21
+ lora:
22
+ enabled: true
23
+ r: 16
24
+ alpha: 32
25
+ dropout: 0.05
26
+ target_modules:
27
+ - q_proj
28
+ - k_proj
29
+ - v_proj
30
+ - o_proj
31
+ modules_to_save:
32
+ - lm_head
33
+ bias: none
34
+ training:
35
+ seed: 42
36
+ epochs: 8
37
+ warmup_steps: null
38
+ warmup_ratio: 0.05
39
+ mixed_precision: false
40
+ mixed_precision_dtype: bfloat16
41
+ batch_size: 4
42
+ eval_batch_size: 8
43
+ eval_every_steps: 10
44
+ eval_every_epochs: 1
45
+ learning_rate: 0.0005
46
+ optimizer: adamw
47
+ lr_scheduler: cosine
48
+ adam_beta1: 0.9
49
+ adam_beta2: 0.999
50
+ adam_epsilon: 1.0e-08
51
+ sgd_momentum: 0.9
52
+ weight_decay: 0.01
53
+ max_grad_norm: 1.0
54
+ log_every: 5
55
+ gradient_accumulation_steps: 8
56
+ num_workers: 4
57
+ output_dir: /scratch/mcatal/runs/cpt/llama3-8b/dp8_v2
58
+ min_lr_ratio: 0.15
59
+ distributed:
60
+ strategy: dpddp
61
+ backend: nccl
62
+ devices: null
63
+ dp:
64
+ module_validator: auto
65
+ target_delta: 1.0e-05
66
+ noise_multiplier: null
67
+ max_grad_norm: 1.0
68
+ grad_sample_mode: hooks
69
+ clipping: per_layer
70
+ secure_mode: false
71
+ enabled: true
72
+ target_epsilon: 8.0
73
+ audit:
74
+ enabled: true
75
+ run_every_epoch: true
76
+ epoch_device: cuda
77
+ q_canary: auto
78
+ num_canaries: 500
79
+ prefix_length: 49
80
+ num_digits: 12
81
+ batch_size: 32
82
+ delta: 1.0e-05
83
+ p_values:
84
+ - 0.05
85
+ - 0.01
86
+ paper_guess_fraction: 0.2
87
+ paper_guess_steps: 20
88
+ enable_holdout_empirical_epsilon: false
89
+ holdout_seed: 42
90
+ tie_seed: 42
91
+ tracking:
92
+ enabled: true
93
+ tensorboard: true
94
+ wandb: false
95
+ wandb_project: codedp-finetune-h200-audit
96
+ wandb_run_name: llama3-8b-cpt-dp8
97
+ wandb_mode: online
98
+ codecarbon: true
99
+ codecarbon_output_file: codecarbon.csv
100
+ codecarbon_measure_power_secs: 15
101
+ codecarbon_country_iso_code: null
102
+ codecarbon_project_name: codedp-llama3-8b-cpt-dp8
llama3-8b/dp8_v2/summary.json ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "audit/delta": 1e-05,
3
+ "audit/embedding/auc": 0.506472,
4
+ "audit/embedding/empirical_epsilon/0.01": 0.0,
5
+ "audit/embedding/empirical_epsilon/0.05": 0.0,
6
+ "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 0.0,
7
+ "audit/embedding/empirical_epsilon_details/0.01/epsilon": 0.0,
8
+ "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 0.0,
9
+ "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 0.0,
10
+ "audit/embedding/empirical_epsilon_details/0.05/epsilon": 0.0,
11
+ "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 0.0,
12
+ "audit/loss/auc": 0.540616,
13
+ "audit/loss/empirical_epsilon/0.01": 0.0,
14
+ "audit/loss/empirical_epsilon/0.05": 0.0,
15
+ "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 0.0,
16
+ "audit/loss/empirical_epsilon_details/0.01/epsilon": 0.0,
17
+ "audit/loss/empirical_epsilon_details/0.01/num_guesses": 0.0,
18
+ "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 0.0,
19
+ "audit/loss/empirical_epsilon_details/0.05/epsilon": 0.0,
20
+ "audit/loss/empirical_epsilon_details/0.05/num_guesses": 0.0,
21
+ "audit/num_canaries": 500.0,
22
+ "audit/num_members": 250.0,
23
+ "audit/paper_guess_fraction": 0.2,
24
+ "audit/paper_guess_steps": 20.0,
25
+ "energy/codecarbon/cpu_count": 32.0,
26
+ "energy/codecarbon/cpu_energy": 0.32995656977925963,
27
+ "energy/codecarbon/cpu_power": 81.77024728452554,
28
+ "energy/codecarbon/cpu_utilization_percent": 6.016326258176479,
29
+ "energy/codecarbon/duration": 15065.575935233996,
30
+ "energy/codecarbon/emissions": 0.5238648891587854,
31
+ "energy/codecarbon/emissions_rate": 3.477231082375138e-05,
32
+ "energy/codecarbon/energy_consumed": 15.035011025422182,
33
+ "energy/codecarbon/gpu_count": 8.0,
34
+ "energy/codecarbon/gpu_energy": 14.55171868358786,
35
+ "energy/codecarbon/gpu_power": 3485.627546226233,
36
+ "energy/codecarbon/gpu_utilization_percent": 96.56516152716593,
37
+ "energy/codecarbon/latitude": 47.4843,
38
+ "energy/codecarbon/longitude": 8.212,
39
+ "energy/codecarbon/pue": 1.0,
40
+ "energy/codecarbon/ram_energy": 0.15333577205503765,
41
+ "energy/codecarbon/ram_power": 38.0,
42
+ "energy/codecarbon/ram_total_size": 256.0,
43
+ "energy/codecarbon/ram_used_gb": 46.15334321494224,
44
+ "energy/codecarbon/ram_utilization_percent": 2.7207115204912564,
45
+ "energy/codecarbon/water_consumed": 0.0,
46
+ "energy/codecarbon/wue": 0.0,
47
+ "eval/duration_sec": 14.466551557008643,
48
+ "eval/loss": 7.411571566875165,
49
+ "perf/audit_duration_sec": 5.254391202994157,
50
+ "perf/epoch_duration_sec": 1851.9040329359996,
51
+ "perf/epoch_samples": 51455.0,
52
+ "perf/epoch_samples_per_sec": 27.784917082567986,
53
+ "perf/epoch_tokens": 37421704.0,
54
+ "perf/epoch_tokens_per_sec": 20207.15076724133,
55
+ "perf/logical_batch_size": 34.0,
56
+ "perf/logical_token_count": 18921.0,
57
+ "perf/physical_batches": 10.0,
58
+ "perf/samples_per_sec": 3.7932708590165265,
59
+ "perf/step_duration_sec": 8.963240765995579,
60
+ "perf/tokens_per_sec": 2110.9552330426973,
61
+ "privacy/epsilon": 7.994810949157526,
62
+ "system/cuda_epoch_peak_memory_gb": 62.82171964645386,
63
+ "system/cuda_max_memory_allocated_gb": 62.82171964645386,
64
+ "system/cuda_memory_allocated_gb": 20.584996223449707,
65
+ "train/epoch_canary_loss": 10.26028393006185,
66
+ "train/epoch_loss": 7.892161646281565,
67
+ "train/epoch_real_loss": 7.555891874813191,
68
+ "train/lr": 7.500214621685675e-05,
69
+ "train/step_canary_loss": 8.75,
70
+ "train/step_loss": 7.9287278311593195,
71
+ "train/step_real_loss": 7.851733565330505
72
+ }
llama3-8b/dp8_v2/tokenizer/special_tokens_map.json ADDED
@@ -0,0 +1,3519 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ {
4
+ "content": "865331112869",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ {
11
+ "content": "569765693871",
12
+ "lstrip": false,
13
+ "normalized": false,
14
+ "rstrip": false,
15
+ "single_word": false
16
+ },
17
+ {
18
+ "content": "485177821815",
19
+ "lstrip": false,
20
+ "normalized": false,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ },
24
+ {
25
+ "content": "135441121756",
26
+ "lstrip": false,
27
+ "normalized": false,
28
+ "rstrip": false,
29
+ "single_word": false
30
+ },
31
+ {
32
+ "content": "367459894796",
33
+ "lstrip": false,
34
+ "normalized": false,
35
+ "rstrip": false,
36
+ "single_word": false
37
+ },
38
+ {
39
+ "content": "877482678543",
40
+ "lstrip": false,
41
+ "normalized": false,
42
+ "rstrip": false,
43
+ "single_word": false
44
+ },
45
+ {
46
+ "content": "457919547633",
47
+ "lstrip": false,
48
+ "normalized": false,
49
+ "rstrip": false,
50
+ "single_word": false
51
+ },
52
+ {
53
+ "content": "765474393376",
54
+ "lstrip": false,
55
+ "normalized": false,
56
+ "rstrip": false,
57
+ "single_word": false
58
+ },
59
+ {
60
+ "content": "114848338811",
61
+ "lstrip": false,
62
+ "normalized": false,
63
+ "rstrip": false,
64
+ "single_word": false
65
+ },
66
+ {
67
+ "content": "746285987371",
68
+ "lstrip": false,
69
+ "normalized": false,
70
+ "rstrip": false,
71
+ "single_word": false
72
+ },
73
+ {
74
+ "content": "649291669397",
75
+ "lstrip": false,
76
+ "normalized": false,
77
+ "rstrip": false,
78
+ "single_word": false
79
+ },
80
+ {
81
+ "content": "927914615679",
82
+ "lstrip": false,
83
+ "normalized": false,
84
+ "rstrip": false,
85
+ "single_word": false
86
+ },
87
+ {
88
+ "content": "445925149649",
89
+ "lstrip": false,
90
+ "normalized": false,
91
+ "rstrip": false,
92
+ "single_word": false
93
+ },
94
+ {
95
+ "content": "691587454538",
96
+ "lstrip": false,
97
+ "normalized": false,
98
+ "rstrip": false,
99
+ "single_word": false
100
+ },
101
+ {
102
+ "content": "143777992227",
103
+ "lstrip": false,
104
+ "normalized": false,
105
+ "rstrip": false,
106
+ "single_word": false
107
+ },
108
+ {
109
+ "content": "997981281989",
110
+ "lstrip": false,
111
+ "normalized": false,
112
+ "rstrip": false,
113
+ "single_word": false
114
+ },
115
+ {
116
+ "content": "425949483533",
117
+ "lstrip": false,
118
+ "normalized": false,
119
+ "rstrip": false,
120
+ "single_word": false
121
+ },
122
+ {
123
+ "content": "982993456429",
124
+ "lstrip": false,
125
+ "normalized": false,
126
+ "rstrip": false,
127
+ "single_word": false
128
+ },
129
+ {
130
+ "content": "718726519731",
131
+ "lstrip": false,
132
+ "normalized": false,
133
+ "rstrip": false,
134
+ "single_word": false
135
+ },
136
+ {
137
+ "content": "172599315861",
138
+ "lstrip": false,
139
+ "normalized": false,
140
+ "rstrip": false,
141
+ "single_word": false
142
+ },
143
+ {
144
+ "content": "643489267333",
145
+ "lstrip": false,
146
+ "normalized": false,
147
+ "rstrip": false,
148
+ "single_word": false
149
+ },
150
+ {
151
+ "content": "282322838685",
152
+ "lstrip": false,
153
+ "normalized": false,
154
+ "rstrip": false,
155
+ "single_word": false
156
+ },
157
+ {
158
+ "content": "781653545886",
159
+ "lstrip": false,
160
+ "normalized": false,
161
+ "rstrip": false,
162
+ "single_word": false
163
+ },
164
+ {
165
+ "content": "796415361892",
166
+ "lstrip": false,
167
+ "normalized": false,
168
+ "rstrip": false,
169
+ "single_word": false
170
+ },
171
+ {
172
+ "content": "841991688488",
173
+ "lstrip": false,
174
+ "normalized": false,
175
+ "rstrip": false,
176
+ "single_word": false
177
+ },
178
+ {
179
+ "content": "211411365397",
180
+ "lstrip": false,
181
+ "normalized": false,
182
+ "rstrip": false,
183
+ "single_word": false
184
+ },
185
+ {
186
+ "content": "698218415444",
187
+ "lstrip": false,
188
+ "normalized": false,
189
+ "rstrip": false,
190
+ "single_word": false
191
+ },
192
+ {
193
+ "content": "355977139358",
194
+ "lstrip": false,
195
+ "normalized": false,
196
+ "rstrip": false,
197
+ "single_word": false
198
+ },
199
+ {
200
+ "content": "682564697312",
201
+ "lstrip": false,
202
+ "normalized": false,
203
+ "rstrip": false,
204
+ "single_word": false
205
+ },
206
+ {
207
+ "content": "383837596997",
208
+ "lstrip": false,
209
+ "normalized": false,
210
+ "rstrip": false,
211
+ "single_word": false
212
+ },
213
+ {
214
+ "content": "689362171782",
215
+ "lstrip": false,
216
+ "normalized": false,
217
+ "rstrip": false,
218
+ "single_word": false
219
+ },
220
+ {
221
+ "content": "749966767285",
222
+ "lstrip": false,
223
+ "normalized": false,
224
+ "rstrip": false,
225
+ "single_word": false
226
+ },
227
+ {
228
+ "content": "753159165157",
229
+ "lstrip": false,
230
+ "normalized": false,
231
+ "rstrip": false,
232
+ "single_word": false
233
+ },
234
+ {
235
+ "content": "795693824762",
236
+ "lstrip": false,
237
+ "normalized": false,
238
+ "rstrip": false,
239
+ "single_word": false
240
+ },
241
+ {
242
+ "content": "669689115557",
243
+ "lstrip": false,
244
+ "normalized": false,
245
+ "rstrip": false,
246
+ "single_word": false
247
+ },
248
+ {
249
+ "content": "327491773134",
250
+ "lstrip": false,
251
+ "normalized": false,
252
+ "rstrip": false,
253
+ "single_word": false
254
+ },
255
+ {
256
+ "content": "983569279932",
257
+ "lstrip": false,
258
+ "normalized": false,
259
+ "rstrip": false,
260
+ "single_word": false
261
+ },
262
+ {
263
+ "content": "612128769512",
264
+ "lstrip": false,
265
+ "normalized": false,
266
+ "rstrip": false,
267
+ "single_word": false
268
+ },
269
+ {
270
+ "content": "374327157578",
271
+ "lstrip": false,
272
+ "normalized": false,
273
+ "rstrip": false,
274
+ "single_word": false
275
+ },
276
+ {
277
+ "content": "311632789559",
278
+ "lstrip": false,
279
+ "normalized": false,
280
+ "rstrip": false,
281
+ "single_word": false
282
+ },
283
+ {
284
+ "content": "523918658846",
285
+ "lstrip": false,
286
+ "normalized": false,
287
+ "rstrip": false,
288
+ "single_word": false
289
+ },
290
+ {
291
+ "content": "765981581453",
292
+ "lstrip": false,
293
+ "normalized": false,
294
+ "rstrip": false,
295
+ "single_word": false
296
+ },
297
+ {
298
+ "content": "794825141891",
299
+ "lstrip": false,
300
+ "normalized": false,
301
+ "rstrip": false,
302
+ "single_word": false
303
+ },
304
+ {
305
+ "content": "873898736873",
306
+ "lstrip": false,
307
+ "normalized": false,
308
+ "rstrip": false,
309
+ "single_word": false
310
+ },
311
+ {
312
+ "content": "447445629421",
313
+ "lstrip": false,
314
+ "normalized": false,
315
+ "rstrip": false,
316
+ "single_word": false
317
+ },
318
+ {
319
+ "content": "473822473819",
320
+ "lstrip": false,
321
+ "normalized": false,
322
+ "rstrip": false,
323
+ "single_word": false
324
+ },
325
+ {
326
+ "content": "181439694557",
327
+ "lstrip": false,
328
+ "normalized": false,
329
+ "rstrip": false,
330
+ "single_word": false
331
+ },
332
+ {
333
+ "content": "592538279337",
334
+ "lstrip": false,
335
+ "normalized": false,
336
+ "rstrip": false,
337
+ "single_word": false
338
+ },
339
+ {
340
+ "content": "668134915514",
341
+ "lstrip": false,
342
+ "normalized": false,
343
+ "rstrip": false,
344
+ "single_word": false
345
+ },
346
+ {
347
+ "content": "643692393748",
348
+ "lstrip": false,
349
+ "normalized": false,
350
+ "rstrip": false,
351
+ "single_word": false
352
+ },
353
+ {
354
+ "content": "696651276628",
355
+ "lstrip": false,
356
+ "normalized": false,
357
+ "rstrip": false,
358
+ "single_word": false
359
+ },
360
+ {
361
+ "content": "853859348234",
362
+ "lstrip": false,
363
+ "normalized": false,
364
+ "rstrip": false,
365
+ "single_word": false
366
+ },
367
+ {
368
+ "content": "778466723723",
369
+ "lstrip": false,
370
+ "normalized": false,
371
+ "rstrip": false,
372
+ "single_word": false
373
+ },
374
+ {
375
+ "content": "929826356991",
376
+ "lstrip": false,
377
+ "normalized": false,
378
+ "rstrip": false,
379
+ "single_word": false
380
+ },
381
+ {
382
+ "content": "272362973463",
383
+ "lstrip": false,
384
+ "normalized": false,
385
+ "rstrip": false,
386
+ "single_word": false
387
+ },
388
+ {
389
+ "content": "694235616268",
390
+ "lstrip": false,
391
+ "normalized": false,
392
+ "rstrip": false,
393
+ "single_word": false
394
+ },
395
+ {
396
+ "content": "281673864127",
397
+ "lstrip": false,
398
+ "normalized": false,
399
+ "rstrip": false,
400
+ "single_word": false
401
+ },
402
+ {
403
+ "content": "479676316326",
404
+ "lstrip": false,
405
+ "normalized": false,
406
+ "rstrip": false,
407
+ "single_word": false
408
+ },
409
+ {
410
+ "content": "646979124677",
411
+ "lstrip": false,
412
+ "normalized": false,
413
+ "rstrip": false,
414
+ "single_word": false
415
+ },
416
+ {
417
+ "content": "922327493433",
418
+ "lstrip": false,
419
+ "normalized": false,
420
+ "rstrip": false,
421
+ "single_word": false
422
+ },
423
+ {
424
+ "content": "883685933161",
425
+ "lstrip": false,
426
+ "normalized": false,
427
+ "rstrip": false,
428
+ "single_word": false
429
+ },
430
+ {
431
+ "content": "264259917554",
432
+ "lstrip": false,
433
+ "normalized": false,
434
+ "rstrip": false,
435
+ "single_word": false
436
+ },
437
+ {
438
+ "content": "836746273134",
439
+ "lstrip": false,
440
+ "normalized": false,
441
+ "rstrip": false,
442
+ "single_word": false
443
+ },
444
+ {
445
+ "content": "658481324922",
446
+ "lstrip": false,
447
+ "normalized": false,
448
+ "rstrip": false,
449
+ "single_word": false
450
+ },
451
+ {
452
+ "content": "481884157827",
453
+ "lstrip": false,
454
+ "normalized": false,
455
+ "rstrip": false,
456
+ "single_word": false
457
+ },
458
+ {
459
+ "content": "587787496812",
460
+ "lstrip": false,
461
+ "normalized": false,
462
+ "rstrip": false,
463
+ "single_word": false
464
+ },
465
+ {
466
+ "content": "579184949249",
467
+ "lstrip": false,
468
+ "normalized": false,
469
+ "rstrip": false,
470
+ "single_word": false
471
+ },
472
+ {
473
+ "content": "912193598348",
474
+ "lstrip": false,
475
+ "normalized": false,
476
+ "rstrip": false,
477
+ "single_word": false
478
+ },
479
+ {
480
+ "content": "529679678956",
481
+ "lstrip": false,
482
+ "normalized": false,
483
+ "rstrip": false,
484
+ "single_word": false
485
+ },
486
+ {
487
+ "content": "795838284624",
488
+ "lstrip": false,
489
+ "normalized": false,
490
+ "rstrip": false,
491
+ "single_word": false
492
+ },
493
+ {
494
+ "content": "159337222655",
495
+ "lstrip": false,
496
+ "normalized": false,
497
+ "rstrip": false,
498
+ "single_word": false
499
+ },
500
+ {
501
+ "content": "173781362446",
502
+ "lstrip": false,
503
+ "normalized": false,
504
+ "rstrip": false,
505
+ "single_word": false
506
+ },
507
+ {
508
+ "content": "773687856563",
509
+ "lstrip": false,
510
+ "normalized": false,
511
+ "rstrip": false,
512
+ "single_word": false
513
+ },
514
+ {
515
+ "content": "535787224917",
516
+ "lstrip": false,
517
+ "normalized": false,
518
+ "rstrip": false,
519
+ "single_word": false
520
+ },
521
+ {
522
+ "content": "351885857332",
523
+ "lstrip": false,
524
+ "normalized": false,
525
+ "rstrip": false,
526
+ "single_word": false
527
+ },
528
+ {
529
+ "content": "578827344666",
530
+ "lstrip": false,
531
+ "normalized": false,
532
+ "rstrip": false,
533
+ "single_word": false
534
+ },
535
+ {
536
+ "content": "198462689911",
537
+ "lstrip": false,
538
+ "normalized": false,
539
+ "rstrip": false,
540
+ "single_word": false
541
+ },
542
+ {
543
+ "content": "722618266242",
544
+ "lstrip": false,
545
+ "normalized": false,
546
+ "rstrip": false,
547
+ "single_word": false
548
+ },
549
+ {
550
+ "content": "952872416512",
551
+ "lstrip": false,
552
+ "normalized": false,
553
+ "rstrip": false,
554
+ "single_word": false
555
+ },
556
+ {
557
+ "content": "517778845323",
558
+ "lstrip": false,
559
+ "normalized": false,
560
+ "rstrip": false,
561
+ "single_word": false
562
+ },
563
+ {
564
+ "content": "749665846687",
565
+ "lstrip": false,
566
+ "normalized": false,
567
+ "rstrip": false,
568
+ "single_word": false
569
+ },
570
+ {
571
+ "content": "661436365453",
572
+ "lstrip": false,
573
+ "normalized": false,
574
+ "rstrip": false,
575
+ "single_word": false
576
+ },
577
+ {
578
+ "content": "259666844669",
579
+ "lstrip": false,
580
+ "normalized": false,
581
+ "rstrip": false,
582
+ "single_word": false
583
+ },
584
+ {
585
+ "content": "242851284913",
586
+ "lstrip": false,
587
+ "normalized": false,
588
+ "rstrip": false,
589
+ "single_word": false
590
+ },
591
+ {
592
+ "content": "514532995959",
593
+ "lstrip": false,
594
+ "normalized": false,
595
+ "rstrip": false,
596
+ "single_word": false
597
+ },
598
+ {
599
+ "content": "161588262349",
600
+ "lstrip": false,
601
+ "normalized": false,
602
+ "rstrip": false,
603
+ "single_word": false
604
+ },
605
+ {
606
+ "content": "742765629356",
607
+ "lstrip": false,
608
+ "normalized": false,
609
+ "rstrip": false,
610
+ "single_word": false
611
+ },
612
+ {
613
+ "content": "225164373623",
614
+ "lstrip": false,
615
+ "normalized": false,
616
+ "rstrip": false,
617
+ "single_word": false
618
+ },
619
+ {
620
+ "content": "676539973863",
621
+ "lstrip": false,
622
+ "normalized": false,
623
+ "rstrip": false,
624
+ "single_word": false
625
+ },
626
+ {
627
+ "content": "826214551218",
628
+ "lstrip": false,
629
+ "normalized": false,
630
+ "rstrip": false,
631
+ "single_word": false
632
+ },
633
+ {
634
+ "content": "182345464792",
635
+ "lstrip": false,
636
+ "normalized": false,
637
+ "rstrip": false,
638
+ "single_word": false
639
+ },
640
+ {
641
+ "content": "232776999554",
642
+ "lstrip": false,
643
+ "normalized": false,
644
+ "rstrip": false,
645
+ "single_word": false
646
+ },
647
+ {
648
+ "content": "337326533813",
649
+ "lstrip": false,
650
+ "normalized": false,
651
+ "rstrip": false,
652
+ "single_word": false
653
+ },
654
+ {
655
+ "content": "676676697292",
656
+ "lstrip": false,
657
+ "normalized": false,
658
+ "rstrip": false,
659
+ "single_word": false
660
+ },
661
+ {
662
+ "content": "929185622831",
663
+ "lstrip": false,
664
+ "normalized": false,
665
+ "rstrip": false,
666
+ "single_word": false
667
+ },
668
+ {
669
+ "content": "545512344383",
670
+ "lstrip": false,
671
+ "normalized": false,
672
+ "rstrip": false,
673
+ "single_word": false
674
+ },
675
+ {
676
+ "content": "499444466686",
677
+ "lstrip": false,
678
+ "normalized": false,
679
+ "rstrip": false,
680
+ "single_word": false
681
+ },
682
+ {
683
+ "content": "314697386682",
684
+ "lstrip": false,
685
+ "normalized": false,
686
+ "rstrip": false,
687
+ "single_word": false
688
+ },
689
+ {
690
+ "content": "517379856925",
691
+ "lstrip": false,
692
+ "normalized": false,
693
+ "rstrip": false,
694
+ "single_word": false
695
+ },
696
+ {
697
+ "content": "379557332953",
698
+ "lstrip": false,
699
+ "normalized": false,
700
+ "rstrip": false,
701
+ "single_word": false
702
+ },
703
+ {
704
+ "content": "614797267726",
705
+ "lstrip": false,
706
+ "normalized": false,
707
+ "rstrip": false,
708
+ "single_word": false
709
+ },
710
+ {
711
+ "content": "429781429464",
712
+ "lstrip": false,
713
+ "normalized": false,
714
+ "rstrip": false,
715
+ "single_word": false
716
+ },
717
+ {
718
+ "content": "922466849763",
719
+ "lstrip": false,
720
+ "normalized": false,
721
+ "rstrip": false,
722
+ "single_word": false
723
+ },
724
+ {
725
+ "content": "721737645236",
726
+ "lstrip": false,
727
+ "normalized": false,
728
+ "rstrip": false,
729
+ "single_word": false
730
+ },
731
+ {
732
+ "content": "479227349997",
733
+ "lstrip": false,
734
+ "normalized": false,
735
+ "rstrip": false,
736
+ "single_word": false
737
+ },
738
+ {
739
+ "content": "136931728327",
740
+ "lstrip": false,
741
+ "normalized": false,
742
+ "rstrip": false,
743
+ "single_word": false
744
+ },
745
+ {
746
+ "content": "259533577263",
747
+ "lstrip": false,
748
+ "normalized": false,
749
+ "rstrip": false,
750
+ "single_word": false
751
+ },
752
+ {
753
+ "content": "488538864842",
754
+ "lstrip": false,
755
+ "normalized": false,
756
+ "rstrip": false,
757
+ "single_word": false
758
+ },
759
+ {
760
+ "content": "937495658852",
761
+ "lstrip": false,
762
+ "normalized": false,
763
+ "rstrip": false,
764
+ "single_word": false
765
+ },
766
+ {
767
+ "content": "489991411364",
768
+ "lstrip": false,
769
+ "normalized": false,
770
+ "rstrip": false,
771
+ "single_word": false
772
+ },
773
+ {
774
+ "content": "499148455254",
775
+ "lstrip": false,
776
+ "normalized": false,
777
+ "rstrip": false,
778
+ "single_word": false
779
+ },
780
+ {
781
+ "content": "441373944925",
782
+ "lstrip": false,
783
+ "normalized": false,
784
+ "rstrip": false,
785
+ "single_word": false
786
+ },
787
+ {
788
+ "content": "899151413682",
789
+ "lstrip": false,
790
+ "normalized": false,
791
+ "rstrip": false,
792
+ "single_word": false
793
+ },
794
+ {
795
+ "content": "467893531755",
796
+ "lstrip": false,
797
+ "normalized": false,
798
+ "rstrip": false,
799
+ "single_word": false
800
+ },
801
+ {
802
+ "content": "527117488925",
803
+ "lstrip": false,
804
+ "normalized": false,
805
+ "rstrip": false,
806
+ "single_word": false
807
+ },
808
+ {
809
+ "content": "928335588653",
810
+ "lstrip": false,
811
+ "normalized": false,
812
+ "rstrip": false,
813
+ "single_word": false
814
+ },
815
+ {
816
+ "content": "374439448821",
817
+ "lstrip": false,
818
+ "normalized": false,
819
+ "rstrip": false,
820
+ "single_word": false
821
+ },
822
+ {
823
+ "content": "879425227932",
824
+ "lstrip": false,
825
+ "normalized": false,
826
+ "rstrip": false,
827
+ "single_word": false
828
+ },
829
+ {
830
+ "content": "867678158885",
831
+ "lstrip": false,
832
+ "normalized": false,
833
+ "rstrip": false,
834
+ "single_word": false
835
+ },
836
+ {
837
+ "content": "399749397872",
838
+ "lstrip": false,
839
+ "normalized": false,
840
+ "rstrip": false,
841
+ "single_word": false
842
+ },
843
+ {
844
+ "content": "129693547287",
845
+ "lstrip": false,
846
+ "normalized": false,
847
+ "rstrip": false,
848
+ "single_word": false
849
+ },
850
+ {
851
+ "content": "689285841825",
852
+ "lstrip": false,
853
+ "normalized": false,
854
+ "rstrip": false,
855
+ "single_word": false
856
+ },
857
+ {
858
+ "content": "771619544974",
859
+ "lstrip": false,
860
+ "normalized": false,
861
+ "rstrip": false,
862
+ "single_word": false
863
+ },
864
+ {
865
+ "content": "724883568652",
866
+ "lstrip": false,
867
+ "normalized": false,
868
+ "rstrip": false,
869
+ "single_word": false
870
+ },
871
+ {
872
+ "content": "516968424863",
873
+ "lstrip": false,
874
+ "normalized": false,
875
+ "rstrip": false,
876
+ "single_word": false
877
+ },
878
+ {
879
+ "content": "733737988257",
880
+ "lstrip": false,
881
+ "normalized": false,
882
+ "rstrip": false,
883
+ "single_word": false
884
+ },
885
+ {
886
+ "content": "852347289392",
887
+ "lstrip": false,
888
+ "normalized": false,
889
+ "rstrip": false,
890
+ "single_word": false
891
+ },
892
+ {
893
+ "content": "296953381169",
894
+ "lstrip": false,
895
+ "normalized": false,
896
+ "rstrip": false,
897
+ "single_word": false
898
+ },
899
+ {
900
+ "content": "377273562477",
901
+ "lstrip": false,
902
+ "normalized": false,
903
+ "rstrip": false,
904
+ "single_word": false
905
+ },
906
+ {
907
+ "content": "262296912232",
908
+ "lstrip": false,
909
+ "normalized": false,
910
+ "rstrip": false,
911
+ "single_word": false
912
+ },
913
+ {
914
+ "content": "547149832394",
915
+ "lstrip": false,
916
+ "normalized": false,
917
+ "rstrip": false,
918
+ "single_word": false
919
+ },
920
+ {
921
+ "content": "298464134954",
922
+ "lstrip": false,
923
+ "normalized": false,
924
+ "rstrip": false,
925
+ "single_word": false
926
+ },
927
+ {
928
+ "content": "216667245274",
929
+ "lstrip": false,
930
+ "normalized": false,
931
+ "rstrip": false,
932
+ "single_word": false
933
+ },
934
+ {
935
+ "content": "843998562287",
936
+ "lstrip": false,
937
+ "normalized": false,
938
+ "rstrip": false,
939
+ "single_word": false
940
+ },
941
+ {
942
+ "content": "572154333646",
943
+ "lstrip": false,
944
+ "normalized": false,
945
+ "rstrip": false,
946
+ "single_word": false
947
+ },
948
+ {
949
+ "content": "124589118494",
950
+ "lstrip": false,
951
+ "normalized": false,
952
+ "rstrip": false,
953
+ "single_word": false
954
+ },
955
+ {
956
+ "content": "841824384614",
957
+ "lstrip": false,
958
+ "normalized": false,
959
+ "rstrip": false,
960
+ "single_word": false
961
+ },
962
+ {
963
+ "content": "232896526252",
964
+ "lstrip": false,
965
+ "normalized": false,
966
+ "rstrip": false,
967
+ "single_word": false
968
+ },
969
+ {
970
+ "content": "295448593321",
971
+ "lstrip": false,
972
+ "normalized": false,
973
+ "rstrip": false,
974
+ "single_word": false
975
+ },
976
+ {
977
+ "content": "123741461297",
978
+ "lstrip": false,
979
+ "normalized": false,
980
+ "rstrip": false,
981
+ "single_word": false
982
+ },
983
+ {
984
+ "content": "653573457168",
985
+ "lstrip": false,
986
+ "normalized": false,
987
+ "rstrip": false,
988
+ "single_word": false
989
+ },
990
+ {
991
+ "content": "196735786156",
992
+ "lstrip": false,
993
+ "normalized": false,
994
+ "rstrip": false,
995
+ "single_word": false
996
+ },
997
+ {
998
+ "content": "377338713663",
999
+ "lstrip": false,
1000
+ "normalized": false,
1001
+ "rstrip": false,
1002
+ "single_word": false
1003
+ },
1004
+ {
1005
+ "content": "964342468552",
1006
+ "lstrip": false,
1007
+ "normalized": false,
1008
+ "rstrip": false,
1009
+ "single_word": false
1010
+ },
1011
+ {
1012
+ "content": "586855179568",
1013
+ "lstrip": false,
1014
+ "normalized": false,
1015
+ "rstrip": false,
1016
+ "single_word": false
1017
+ },
1018
+ {
1019
+ "content": "484773717614",
1020
+ "lstrip": false,
1021
+ "normalized": false,
1022
+ "rstrip": false,
1023
+ "single_word": false
1024
+ },
1025
+ {
1026
+ "content": "894885246797",
1027
+ "lstrip": false,
1028
+ "normalized": false,
1029
+ "rstrip": false,
1030
+ "single_word": false
1031
+ },
1032
+ {
1033
+ "content": "677896358599",
1034
+ "lstrip": false,
1035
+ "normalized": false,
1036
+ "rstrip": false,
1037
+ "single_word": false
1038
+ },
1039
+ {
1040
+ "content": "848845611563",
1041
+ "lstrip": false,
1042
+ "normalized": false,
1043
+ "rstrip": false,
1044
+ "single_word": false
1045
+ },
1046
+ {
1047
+ "content": "851852651677",
1048
+ "lstrip": false,
1049
+ "normalized": false,
1050
+ "rstrip": false,
1051
+ "single_word": false
1052
+ },
1053
+ {
1054
+ "content": "398549545767",
1055
+ "lstrip": false,
1056
+ "normalized": false,
1057
+ "rstrip": false,
1058
+ "single_word": false
1059
+ },
1060
+ {
1061
+ "content": "454244839926",
1062
+ "lstrip": false,
1063
+ "normalized": false,
1064
+ "rstrip": false,
1065
+ "single_word": false
1066
+ },
1067
+ {
1068
+ "content": "799364566435",
1069
+ "lstrip": false,
1070
+ "normalized": false,
1071
+ "rstrip": false,
1072
+ "single_word": false
1073
+ },
1074
+ {
1075
+ "content": "967114116556",
1076
+ "lstrip": false,
1077
+ "normalized": false,
1078
+ "rstrip": false,
1079
+ "single_word": false
1080
+ },
1081
+ {
1082
+ "content": "817378986438",
1083
+ "lstrip": false,
1084
+ "normalized": false,
1085
+ "rstrip": false,
1086
+ "single_word": false
1087
+ },
1088
+ {
1089
+ "content": "233795848681",
1090
+ "lstrip": false,
1091
+ "normalized": false,
1092
+ "rstrip": false,
1093
+ "single_word": false
1094
+ },
1095
+ {
1096
+ "content": "824387273757",
1097
+ "lstrip": false,
1098
+ "normalized": false,
1099
+ "rstrip": false,
1100
+ "single_word": false
1101
+ },
1102
+ {
1103
+ "content": "916198946615",
1104
+ "lstrip": false,
1105
+ "normalized": false,
1106
+ "rstrip": false,
1107
+ "single_word": false
1108
+ },
1109
+ {
1110
+ "content": "563117729724",
1111
+ "lstrip": false,
1112
+ "normalized": false,
1113
+ "rstrip": false,
1114
+ "single_word": false
1115
+ },
1116
+ {
1117
+ "content": "951794811935",
1118
+ "lstrip": false,
1119
+ "normalized": false,
1120
+ "rstrip": false,
1121
+ "single_word": false
1122
+ },
1123
+ {
1124
+ "content": "374598961236",
1125
+ "lstrip": false,
1126
+ "normalized": false,
1127
+ "rstrip": false,
1128
+ "single_word": false
1129
+ },
1130
+ {
1131
+ "content": "922867396683",
1132
+ "lstrip": false,
1133
+ "normalized": false,
1134
+ "rstrip": false,
1135
+ "single_word": false
1136
+ },
1137
+ {
1138
+ "content": "765737843639",
1139
+ "lstrip": false,
1140
+ "normalized": false,
1141
+ "rstrip": false,
1142
+ "single_word": false
1143
+ },
1144
+ {
1145
+ "content": "175469284871",
1146
+ "lstrip": false,
1147
+ "normalized": false,
1148
+ "rstrip": false,
1149
+ "single_word": false
1150
+ },
1151
+ {
1152
+ "content": "231853711778",
1153
+ "lstrip": false,
1154
+ "normalized": false,
1155
+ "rstrip": false,
1156
+ "single_word": false
1157
+ },
1158
+ {
1159
+ "content": "662426712668",
1160
+ "lstrip": false,
1161
+ "normalized": false,
1162
+ "rstrip": false,
1163
+ "single_word": false
1164
+ },
1165
+ {
1166
+ "content": "711412347158",
1167
+ "lstrip": false,
1168
+ "normalized": false,
1169
+ "rstrip": false,
1170
+ "single_word": false
1171
+ },
1172
+ {
1173
+ "content": "753466987363",
1174
+ "lstrip": false,
1175
+ "normalized": false,
1176
+ "rstrip": false,
1177
+ "single_word": false
1178
+ },
1179
+ {
1180
+ "content": "513361312532",
1181
+ "lstrip": false,
1182
+ "normalized": false,
1183
+ "rstrip": false,
1184
+ "single_word": false
1185
+ },
1186
+ {
1187
+ "content": "712992815957",
1188
+ "lstrip": false,
1189
+ "normalized": false,
1190
+ "rstrip": false,
1191
+ "single_word": false
1192
+ },
1193
+ {
1194
+ "content": "971621888444",
1195
+ "lstrip": false,
1196
+ "normalized": false,
1197
+ "rstrip": false,
1198
+ "single_word": false
1199
+ },
1200
+ {
1201
+ "content": "829235161526",
1202
+ "lstrip": false,
1203
+ "normalized": false,
1204
+ "rstrip": false,
1205
+ "single_word": false
1206
+ },
1207
+ {
1208
+ "content": "585544633356",
1209
+ "lstrip": false,
1210
+ "normalized": false,
1211
+ "rstrip": false,
1212
+ "single_word": false
1213
+ },
1214
+ {
1215
+ "content": "582471228164",
1216
+ "lstrip": false,
1217
+ "normalized": false,
1218
+ "rstrip": false,
1219
+ "single_word": false
1220
+ },
1221
+ {
1222
+ "content": "678666359123",
1223
+ "lstrip": false,
1224
+ "normalized": false,
1225
+ "rstrip": false,
1226
+ "single_word": false
1227
+ },
1228
+ {
1229
+ "content": "557533689478",
1230
+ "lstrip": false,
1231
+ "normalized": false,
1232
+ "rstrip": false,
1233
+ "single_word": false
1234
+ },
1235
+ {
1236
+ "content": "632962475133",
1237
+ "lstrip": false,
1238
+ "normalized": false,
1239
+ "rstrip": false,
1240
+ "single_word": false
1241
+ },
1242
+ {
1243
+ "content": "484489193824",
1244
+ "lstrip": false,
1245
+ "normalized": false,
1246
+ "rstrip": false,
1247
+ "single_word": false
1248
+ },
1249
+ {
1250
+ "content": "489562189822",
1251
+ "lstrip": false,
1252
+ "normalized": false,
1253
+ "rstrip": false,
1254
+ "single_word": false
1255
+ },
1256
+ {
1257
+ "content": "589547936288",
1258
+ "lstrip": false,
1259
+ "normalized": false,
1260
+ "rstrip": false,
1261
+ "single_word": false
1262
+ },
1263
+ {
1264
+ "content": "363214487524",
1265
+ "lstrip": false,
1266
+ "normalized": false,
1267
+ "rstrip": false,
1268
+ "single_word": false
1269
+ },
1270
+ {
1271
+ "content": "244885399387",
1272
+ "lstrip": false,
1273
+ "normalized": false,
1274
+ "rstrip": false,
1275
+ "single_word": false
1276
+ },
1277
+ {
1278
+ "content": "431751228368",
1279
+ "lstrip": false,
1280
+ "normalized": false,
1281
+ "rstrip": false,
1282
+ "single_word": false
1283
+ },
1284
+ {
1285
+ "content": "433581868192",
1286
+ "lstrip": false,
1287
+ "normalized": false,
1288
+ "rstrip": false,
1289
+ "single_word": false
1290
+ },
1291
+ {
1292
+ "content": "486391569221",
1293
+ "lstrip": false,
1294
+ "normalized": false,
1295
+ "rstrip": false,
1296
+ "single_word": false
1297
+ },
1298
+ {
1299
+ "content": "185438575221",
1300
+ "lstrip": false,
1301
+ "normalized": false,
1302
+ "rstrip": false,
1303
+ "single_word": false
1304
+ },
1305
+ {
1306
+ "content": "126574388585",
1307
+ "lstrip": false,
1308
+ "normalized": false,
1309
+ "rstrip": false,
1310
+ "single_word": false
1311
+ },
1312
+ {
1313
+ "content": "741757479784",
1314
+ "lstrip": false,
1315
+ "normalized": false,
1316
+ "rstrip": false,
1317
+ "single_word": false
1318
+ },
1319
+ {
1320
+ "content": "529854679937",
1321
+ "lstrip": false,
1322
+ "normalized": false,
1323
+ "rstrip": false,
1324
+ "single_word": false
1325
+ },
1326
+ {
1327
+ "content": "996116119839",
1328
+ "lstrip": false,
1329
+ "normalized": false,
1330
+ "rstrip": false,
1331
+ "single_word": false
1332
+ },
1333
+ {
1334
+ "content": "616248973917",
1335
+ "lstrip": false,
1336
+ "normalized": false,
1337
+ "rstrip": false,
1338
+ "single_word": false
1339
+ },
1340
+ {
1341
+ "content": "763531783491",
1342
+ "lstrip": false,
1343
+ "normalized": false,
1344
+ "rstrip": false,
1345
+ "single_word": false
1346
+ },
1347
+ {
1348
+ "content": "955456118295",
1349
+ "lstrip": false,
1350
+ "normalized": false,
1351
+ "rstrip": false,
1352
+ "single_word": false
1353
+ },
1354
+ {
1355
+ "content": "364196983365",
1356
+ "lstrip": false,
1357
+ "normalized": false,
1358
+ "rstrip": false,
1359
+ "single_word": false
1360
+ },
1361
+ {
1362
+ "content": "195792996468",
1363
+ "lstrip": false,
1364
+ "normalized": false,
1365
+ "rstrip": false,
1366
+ "single_word": false
1367
+ },
1368
+ {
1369
+ "content": "151859598873",
1370
+ "lstrip": false,
1371
+ "normalized": false,
1372
+ "rstrip": false,
1373
+ "single_word": false
1374
+ },
1375
+ {
1376
+ "content": "399223169721",
1377
+ "lstrip": false,
1378
+ "normalized": false,
1379
+ "rstrip": false,
1380
+ "single_word": false
1381
+ },
1382
+ {
1383
+ "content": "938488813964",
1384
+ "lstrip": false,
1385
+ "normalized": false,
1386
+ "rstrip": false,
1387
+ "single_word": false
1388
+ },
1389
+ {
1390
+ "content": "961981959227",
1391
+ "lstrip": false,
1392
+ "normalized": false,
1393
+ "rstrip": false,
1394
+ "single_word": false
1395
+ },
1396
+ {
1397
+ "content": "183368827562",
1398
+ "lstrip": false,
1399
+ "normalized": false,
1400
+ "rstrip": false,
1401
+ "single_word": false
1402
+ },
1403
+ {
1404
+ "content": "533417736566",
1405
+ "lstrip": false,
1406
+ "normalized": false,
1407
+ "rstrip": false,
1408
+ "single_word": false
1409
+ },
1410
+ {
1411
+ "content": "786391632558",
1412
+ "lstrip": false,
1413
+ "normalized": false,
1414
+ "rstrip": false,
1415
+ "single_word": false
1416
+ },
1417
+ {
1418
+ "content": "665661658354",
1419
+ "lstrip": false,
1420
+ "normalized": false,
1421
+ "rstrip": false,
1422
+ "single_word": false
1423
+ },
1424
+ {
1425
+ "content": "693281533643",
1426
+ "lstrip": false,
1427
+ "normalized": false,
1428
+ "rstrip": false,
1429
+ "single_word": false
1430
+ },
1431
+ {
1432
+ "content": "475794684356",
1433
+ "lstrip": false,
1434
+ "normalized": false,
1435
+ "rstrip": false,
1436
+ "single_word": false
1437
+ },
1438
+ {
1439
+ "content": "652154162978",
1440
+ "lstrip": false,
1441
+ "normalized": false,
1442
+ "rstrip": false,
1443
+ "single_word": false
1444
+ },
1445
+ {
1446
+ "content": "753233719644",
1447
+ "lstrip": false,
1448
+ "normalized": false,
1449
+ "rstrip": false,
1450
+ "single_word": false
1451
+ },
1452
+ {
1453
+ "content": "668514843129",
1454
+ "lstrip": false,
1455
+ "normalized": false,
1456
+ "rstrip": false,
1457
+ "single_word": false
1458
+ },
1459
+ {
1460
+ "content": "819162623892",
1461
+ "lstrip": false,
1462
+ "normalized": false,
1463
+ "rstrip": false,
1464
+ "single_word": false
1465
+ },
1466
+ {
1467
+ "content": "941169431859",
1468
+ "lstrip": false,
1469
+ "normalized": false,
1470
+ "rstrip": false,
1471
+ "single_word": false
1472
+ },
1473
+ {
1474
+ "content": "877385381798",
1475
+ "lstrip": false,
1476
+ "normalized": false,
1477
+ "rstrip": false,
1478
+ "single_word": false
1479
+ },
1480
+ {
1481
+ "content": "752644929761",
1482
+ "lstrip": false,
1483
+ "normalized": false,
1484
+ "rstrip": false,
1485
+ "single_word": false
1486
+ },
1487
+ {
1488
+ "content": "881136466196",
1489
+ "lstrip": false,
1490
+ "normalized": false,
1491
+ "rstrip": false,
1492
+ "single_word": false
1493
+ },
1494
+ {
1495
+ "content": "275597777299",
1496
+ "lstrip": false,
1497
+ "normalized": false,
1498
+ "rstrip": false,
1499
+ "single_word": false
1500
+ },
1501
+ {
1502
+ "content": "731681792655",
1503
+ "lstrip": false,
1504
+ "normalized": false,
1505
+ "rstrip": false,
1506
+ "single_word": false
1507
+ },
1508
+ {
1509
+ "content": "961133895172",
1510
+ "lstrip": false,
1511
+ "normalized": false,
1512
+ "rstrip": false,
1513
+ "single_word": false
1514
+ },
1515
+ {
1516
+ "content": "864718285734",
1517
+ "lstrip": false,
1518
+ "normalized": false,
1519
+ "rstrip": false,
1520
+ "single_word": false
1521
+ },
1522
+ {
1523
+ "content": "963852916563",
1524
+ "lstrip": false,
1525
+ "normalized": false,
1526
+ "rstrip": false,
1527
+ "single_word": false
1528
+ },
1529
+ {
1530
+ "content": "319584985416",
1531
+ "lstrip": false,
1532
+ "normalized": false,
1533
+ "rstrip": false,
1534
+ "single_word": false
1535
+ },
1536
+ {
1537
+ "content": "563365646341",
1538
+ "lstrip": false,
1539
+ "normalized": false,
1540
+ "rstrip": false,
1541
+ "single_word": false
1542
+ },
1543
+ {
1544
+ "content": "811371928234",
1545
+ "lstrip": false,
1546
+ "normalized": false,
1547
+ "rstrip": false,
1548
+ "single_word": false
1549
+ },
1550
+ {
1551
+ "content": "837131396371",
1552
+ "lstrip": false,
1553
+ "normalized": false,
1554
+ "rstrip": false,
1555
+ "single_word": false
1556
+ },
1557
+ {
1558
+ "content": "267514771964",
1559
+ "lstrip": false,
1560
+ "normalized": false,
1561
+ "rstrip": false,
1562
+ "single_word": false
1563
+ },
1564
+ {
1565
+ "content": "944513428457",
1566
+ "lstrip": false,
1567
+ "normalized": false,
1568
+ "rstrip": false,
1569
+ "single_word": false
1570
+ },
1571
+ {
1572
+ "content": "117298239631",
1573
+ "lstrip": false,
1574
+ "normalized": false,
1575
+ "rstrip": false,
1576
+ "single_word": false
1577
+ },
1578
+ {
1579
+ "content": "158142752582",
1580
+ "lstrip": false,
1581
+ "normalized": false,
1582
+ "rstrip": false,
1583
+ "single_word": false
1584
+ },
1585
+ {
1586
+ "content": "252867443568",
1587
+ "lstrip": false,
1588
+ "normalized": false,
1589
+ "rstrip": false,
1590
+ "single_word": false
1591
+ },
1592
+ {
1593
+ "content": "839269684865",
1594
+ "lstrip": false,
1595
+ "normalized": false,
1596
+ "rstrip": false,
1597
+ "single_word": false
1598
+ },
1599
+ {
1600
+ "content": "612788593128",
1601
+ "lstrip": false,
1602
+ "normalized": false,
1603
+ "rstrip": false,
1604
+ "single_word": false
1605
+ },
1606
+ {
1607
+ "content": "145669731981",
1608
+ "lstrip": false,
1609
+ "normalized": false,
1610
+ "rstrip": false,
1611
+ "single_word": false
1612
+ },
1613
+ {
1614
+ "content": "121557291859",
1615
+ "lstrip": false,
1616
+ "normalized": false,
1617
+ "rstrip": false,
1618
+ "single_word": false
1619
+ },
1620
+ {
1621
+ "content": "245416776926",
1622
+ "lstrip": false,
1623
+ "normalized": false,
1624
+ "rstrip": false,
1625
+ "single_word": false
1626
+ },
1627
+ {
1628
+ "content": "799417897197",
1629
+ "lstrip": false,
1630
+ "normalized": false,
1631
+ "rstrip": false,
1632
+ "single_word": false
1633
+ },
1634
+ {
1635
+ "content": "997958836435",
1636
+ "lstrip": false,
1637
+ "normalized": false,
1638
+ "rstrip": false,
1639
+ "single_word": false
1640
+ },
1641
+ {
1642
+ "content": "892336777248",
1643
+ "lstrip": false,
1644
+ "normalized": false,
1645
+ "rstrip": false,
1646
+ "single_word": false
1647
+ },
1648
+ {
1649
+ "content": "158929292238",
1650
+ "lstrip": false,
1651
+ "normalized": false,
1652
+ "rstrip": false,
1653
+ "single_word": false
1654
+ },
1655
+ {
1656
+ "content": "581976444672",
1657
+ "lstrip": false,
1658
+ "normalized": false,
1659
+ "rstrip": false,
1660
+ "single_word": false
1661
+ },
1662
+ {
1663
+ "content": "897784492783",
1664
+ "lstrip": false,
1665
+ "normalized": false,
1666
+ "rstrip": false,
1667
+ "single_word": false
1668
+ },
1669
+ {
1670
+ "content": "492373714791",
1671
+ "lstrip": false,
1672
+ "normalized": false,
1673
+ "rstrip": false,
1674
+ "single_word": false
1675
+ },
1676
+ {
1677
+ "content": "512659818733",
1678
+ "lstrip": false,
1679
+ "normalized": false,
1680
+ "rstrip": false,
1681
+ "single_word": false
1682
+ },
1683
+ {
1684
+ "content": "881112998642",
1685
+ "lstrip": false,
1686
+ "normalized": false,
1687
+ "rstrip": false,
1688
+ "single_word": false
1689
+ },
1690
+ {
1691
+ "content": "619454958782",
1692
+ "lstrip": false,
1693
+ "normalized": false,
1694
+ "rstrip": false,
1695
+ "single_word": false
1696
+ },
1697
+ {
1698
+ "content": "431149748713",
1699
+ "lstrip": false,
1700
+ "normalized": false,
1701
+ "rstrip": false,
1702
+ "single_word": false
1703
+ },
1704
+ {
1705
+ "content": "624221476921",
1706
+ "lstrip": false,
1707
+ "normalized": false,
1708
+ "rstrip": false,
1709
+ "single_word": false
1710
+ },
1711
+ {
1712
+ "content": "125866399464",
1713
+ "lstrip": false,
1714
+ "normalized": false,
1715
+ "rstrip": false,
1716
+ "single_word": false
1717
+ },
1718
+ {
1719
+ "content": "339882449689",
1720
+ "lstrip": false,
1721
+ "normalized": false,
1722
+ "rstrip": false,
1723
+ "single_word": false
1724
+ },
1725
+ {
1726
+ "content": "186198784585",
1727
+ "lstrip": false,
1728
+ "normalized": false,
1729
+ "rstrip": false,
1730
+ "single_word": false
1731
+ },
1732
+ {
1733
+ "content": "943193294691",
1734
+ "lstrip": false,
1735
+ "normalized": false,
1736
+ "rstrip": false,
1737
+ "single_word": false
1738
+ },
1739
+ {
1740
+ "content": "955668961269",
1741
+ "lstrip": false,
1742
+ "normalized": false,
1743
+ "rstrip": false,
1744
+ "single_word": false
1745
+ },
1746
+ {
1747
+ "content": "232787996724",
1748
+ "lstrip": false,
1749
+ "normalized": false,
1750
+ "rstrip": false,
1751
+ "single_word": false
1752
+ },
1753
+ {
1754
+ "content": "215671314196",
1755
+ "lstrip": false,
1756
+ "normalized": false,
1757
+ "rstrip": false,
1758
+ "single_word": false
1759
+ },
1760
+ {
1761
+ "content": "286173241916",
1762
+ "lstrip": false,
1763
+ "normalized": false,
1764
+ "rstrip": false,
1765
+ "single_word": false
1766
+ },
1767
+ {
1768
+ "content": "745977673725",
1769
+ "lstrip": false,
1770
+ "normalized": false,
1771
+ "rstrip": false,
1772
+ "single_word": false
1773
+ },
1774
+ {
1775
+ "content": "556976448182",
1776
+ "lstrip": false,
1777
+ "normalized": false,
1778
+ "rstrip": false,
1779
+ "single_word": false
1780
+ },
1781
+ {
1782
+ "content": "599961512792",
1783
+ "lstrip": false,
1784
+ "normalized": false,
1785
+ "rstrip": false,
1786
+ "single_word": false
1787
+ },
1788
+ {
1789
+ "content": "766294538337",
1790
+ "lstrip": false,
1791
+ "normalized": false,
1792
+ "rstrip": false,
1793
+ "single_word": false
1794
+ },
1795
+ {
1796
+ "content": "934912591213",
1797
+ "lstrip": false,
1798
+ "normalized": false,
1799
+ "rstrip": false,
1800
+ "single_word": false
1801
+ },
1802
+ {
1803
+ "content": "295118729589",
1804
+ "lstrip": false,
1805
+ "normalized": false,
1806
+ "rstrip": false,
1807
+ "single_word": false
1808
+ },
1809
+ {
1810
+ "content": "529455466433",
1811
+ "lstrip": false,
1812
+ "normalized": false,
1813
+ "rstrip": false,
1814
+ "single_word": false
1815
+ },
1816
+ {
1817
+ "content": "196119929397",
1818
+ "lstrip": false,
1819
+ "normalized": false,
1820
+ "rstrip": false,
1821
+ "single_word": false
1822
+ },
1823
+ {
1824
+ "content": "379571934299",
1825
+ "lstrip": false,
1826
+ "normalized": false,
1827
+ "rstrip": false,
1828
+ "single_word": false
1829
+ },
1830
+ {
1831
+ "content": "251789649997",
1832
+ "lstrip": false,
1833
+ "normalized": false,
1834
+ "rstrip": false,
1835
+ "single_word": false
1836
+ },
1837
+ {
1838
+ "content": "564544131355",
1839
+ "lstrip": false,
1840
+ "normalized": false,
1841
+ "rstrip": false,
1842
+ "single_word": false
1843
+ },
1844
+ {
1845
+ "content": "244371196654",
1846
+ "lstrip": false,
1847
+ "normalized": false,
1848
+ "rstrip": false,
1849
+ "single_word": false
1850
+ },
1851
+ {
1852
+ "content": "384598329253",
1853
+ "lstrip": false,
1854
+ "normalized": false,
1855
+ "rstrip": false,
1856
+ "single_word": false
1857
+ },
1858
+ {
1859
+ "content": "887753195844",
1860
+ "lstrip": false,
1861
+ "normalized": false,
1862
+ "rstrip": false,
1863
+ "single_word": false
1864
+ },
1865
+ {
1866
+ "content": "364947325679",
1867
+ "lstrip": false,
1868
+ "normalized": false,
1869
+ "rstrip": false,
1870
+ "single_word": false
1871
+ },
1872
+ {
1873
+ "content": "655517954651",
1874
+ "lstrip": false,
1875
+ "normalized": false,
1876
+ "rstrip": false,
1877
+ "single_word": false
1878
+ },
1879
+ {
1880
+ "content": "673948786567",
1881
+ "lstrip": false,
1882
+ "normalized": false,
1883
+ "rstrip": false,
1884
+ "single_word": false
1885
+ },
1886
+ {
1887
+ "content": "857231548835",
1888
+ "lstrip": false,
1889
+ "normalized": false,
1890
+ "rstrip": false,
1891
+ "single_word": false
1892
+ },
1893
+ {
1894
+ "content": "816115936673",
1895
+ "lstrip": false,
1896
+ "normalized": false,
1897
+ "rstrip": false,
1898
+ "single_word": false
1899
+ },
1900
+ {
1901
+ "content": "644234165531",
1902
+ "lstrip": false,
1903
+ "normalized": false,
1904
+ "rstrip": false,
1905
+ "single_word": false
1906
+ },
1907
+ {
1908
+ "content": "182782912224",
1909
+ "lstrip": false,
1910
+ "normalized": false,
1911
+ "rstrip": false,
1912
+ "single_word": false
1913
+ },
1914
+ {
1915
+ "content": "234316622259",
1916
+ "lstrip": false,
1917
+ "normalized": false,
1918
+ "rstrip": false,
1919
+ "single_word": false
1920
+ },
1921
+ {
1922
+ "content": "421369185549",
1923
+ "lstrip": false,
1924
+ "normalized": false,
1925
+ "rstrip": false,
1926
+ "single_word": false
1927
+ },
1928
+ {
1929
+ "content": "434632855397",
1930
+ "lstrip": false,
1931
+ "normalized": false,
1932
+ "rstrip": false,
1933
+ "single_word": false
1934
+ },
1935
+ {
1936
+ "content": "921889371893",
1937
+ "lstrip": false,
1938
+ "normalized": false,
1939
+ "rstrip": false,
1940
+ "single_word": false
1941
+ },
1942
+ {
1943
+ "content": "415956914763",
1944
+ "lstrip": false,
1945
+ "normalized": false,
1946
+ "rstrip": false,
1947
+ "single_word": false
1948
+ },
1949
+ {
1950
+ "content": "598916996413",
1951
+ "lstrip": false,
1952
+ "normalized": false,
1953
+ "rstrip": false,
1954
+ "single_word": false
1955
+ },
1956
+ {
1957
+ "content": "773671349113",
1958
+ "lstrip": false,
1959
+ "normalized": false,
1960
+ "rstrip": false,
1961
+ "single_word": false
1962
+ },
1963
+ {
1964
+ "content": "952465217972",
1965
+ "lstrip": false,
1966
+ "normalized": false,
1967
+ "rstrip": false,
1968
+ "single_word": false
1969
+ },
1970
+ {
1971
+ "content": "117657531962",
1972
+ "lstrip": false,
1973
+ "normalized": false,
1974
+ "rstrip": false,
1975
+ "single_word": false
1976
+ },
1977
+ {
1978
+ "content": "729825168745",
1979
+ "lstrip": false,
1980
+ "normalized": false,
1981
+ "rstrip": false,
1982
+ "single_word": false
1983
+ },
1984
+ {
1985
+ "content": "691315125346",
1986
+ "lstrip": false,
1987
+ "normalized": false,
1988
+ "rstrip": false,
1989
+ "single_word": false
1990
+ },
1991
+ {
1992
+ "content": "768461952319",
1993
+ "lstrip": false,
1994
+ "normalized": false,
1995
+ "rstrip": false,
1996
+ "single_word": false
1997
+ },
1998
+ {
1999
+ "content": "664847713559",
2000
+ "lstrip": false,
2001
+ "normalized": false,
2002
+ "rstrip": false,
2003
+ "single_word": false
2004
+ },
2005
+ {
2006
+ "content": "953267689786",
2007
+ "lstrip": false,
2008
+ "normalized": false,
2009
+ "rstrip": false,
2010
+ "single_word": false
2011
+ },
2012
+ {
2013
+ "content": "886464195129",
2014
+ "lstrip": false,
2015
+ "normalized": false,
2016
+ "rstrip": false,
2017
+ "single_word": false
2018
+ },
2019
+ {
2020
+ "content": "824488329416",
2021
+ "lstrip": false,
2022
+ "normalized": false,
2023
+ "rstrip": false,
2024
+ "single_word": false
2025
+ },
2026
+ {
2027
+ "content": "837873762491",
2028
+ "lstrip": false,
2029
+ "normalized": false,
2030
+ "rstrip": false,
2031
+ "single_word": false
2032
+ },
2033
+ {
2034
+ "content": "532833541879",
2035
+ "lstrip": false,
2036
+ "normalized": false,
2037
+ "rstrip": false,
2038
+ "single_word": false
2039
+ },
2040
+ {
2041
+ "content": "669183782449",
2042
+ "lstrip": false,
2043
+ "normalized": false,
2044
+ "rstrip": false,
2045
+ "single_word": false
2046
+ },
2047
+ {
2048
+ "content": "941976537588",
2049
+ "lstrip": false,
2050
+ "normalized": false,
2051
+ "rstrip": false,
2052
+ "single_word": false
2053
+ },
2054
+ {
2055
+ "content": "739394546916",
2056
+ "lstrip": false,
2057
+ "normalized": false,
2058
+ "rstrip": false,
2059
+ "single_word": false
2060
+ },
2061
+ {
2062
+ "content": "267954879268",
2063
+ "lstrip": false,
2064
+ "normalized": false,
2065
+ "rstrip": false,
2066
+ "single_word": false
2067
+ },
2068
+ {
2069
+ "content": "637551427887",
2070
+ "lstrip": false,
2071
+ "normalized": false,
2072
+ "rstrip": false,
2073
+ "single_word": false
2074
+ },
2075
+ {
2076
+ "content": "217756494954",
2077
+ "lstrip": false,
2078
+ "normalized": false,
2079
+ "rstrip": false,
2080
+ "single_word": false
2081
+ },
2082
+ {
2083
+ "content": "524444658383",
2084
+ "lstrip": false,
2085
+ "normalized": false,
2086
+ "rstrip": false,
2087
+ "single_word": false
2088
+ },
2089
+ {
2090
+ "content": "117783274348",
2091
+ "lstrip": false,
2092
+ "normalized": false,
2093
+ "rstrip": false,
2094
+ "single_word": false
2095
+ },
2096
+ {
2097
+ "content": "138218735276",
2098
+ "lstrip": false,
2099
+ "normalized": false,
2100
+ "rstrip": false,
2101
+ "single_word": false
2102
+ },
2103
+ {
2104
+ "content": "814611949491",
2105
+ "lstrip": false,
2106
+ "normalized": false,
2107
+ "rstrip": false,
2108
+ "single_word": false
2109
+ },
2110
+ {
2111
+ "content": "711641973413",
2112
+ "lstrip": false,
2113
+ "normalized": false,
2114
+ "rstrip": false,
2115
+ "single_word": false
2116
+ },
2117
+ {
2118
+ "content": "499156317423",
2119
+ "lstrip": false,
2120
+ "normalized": false,
2121
+ "rstrip": false,
2122
+ "single_word": false
2123
+ },
2124
+ {
2125
+ "content": "515856611931",
2126
+ "lstrip": false,
2127
+ "normalized": false,
2128
+ "rstrip": false,
2129
+ "single_word": false
2130
+ },
2131
+ {
2132
+ "content": "454164859837",
2133
+ "lstrip": false,
2134
+ "normalized": false,
2135
+ "rstrip": false,
2136
+ "single_word": false
2137
+ },
2138
+ {
2139
+ "content": "345271433112",
2140
+ "lstrip": false,
2141
+ "normalized": false,
2142
+ "rstrip": false,
2143
+ "single_word": false
2144
+ },
2145
+ {
2146
+ "content": "462294118988",
2147
+ "lstrip": false,
2148
+ "normalized": false,
2149
+ "rstrip": false,
2150
+ "single_word": false
2151
+ },
2152
+ {
2153
+ "content": "511785788222",
2154
+ "lstrip": false,
2155
+ "normalized": false,
2156
+ "rstrip": false,
2157
+ "single_word": false
2158
+ },
2159
+ {
2160
+ "content": "497294727353",
2161
+ "lstrip": false,
2162
+ "normalized": false,
2163
+ "rstrip": false,
2164
+ "single_word": false
2165
+ },
2166
+ {
2167
+ "content": "866519986723",
2168
+ "lstrip": false,
2169
+ "normalized": false,
2170
+ "rstrip": false,
2171
+ "single_word": false
2172
+ },
2173
+ {
2174
+ "content": "334513529294",
2175
+ "lstrip": false,
2176
+ "normalized": false,
2177
+ "rstrip": false,
2178
+ "single_word": false
2179
+ },
2180
+ {
2181
+ "content": "549946382131",
2182
+ "lstrip": false,
2183
+ "normalized": false,
2184
+ "rstrip": false,
2185
+ "single_word": false
2186
+ },
2187
+ {
2188
+ "content": "284445431422",
2189
+ "lstrip": false,
2190
+ "normalized": false,
2191
+ "rstrip": false,
2192
+ "single_word": false
2193
+ },
2194
+ {
2195
+ "content": "396521188476",
2196
+ "lstrip": false,
2197
+ "normalized": false,
2198
+ "rstrip": false,
2199
+ "single_word": false
2200
+ },
2201
+ {
2202
+ "content": "421435255895",
2203
+ "lstrip": false,
2204
+ "normalized": false,
2205
+ "rstrip": false,
2206
+ "single_word": false
2207
+ },
2208
+ {
2209
+ "content": "133373659361",
2210
+ "lstrip": false,
2211
+ "normalized": false,
2212
+ "rstrip": false,
2213
+ "single_word": false
2214
+ },
2215
+ {
2216
+ "content": "322683334381",
2217
+ "lstrip": false,
2218
+ "normalized": false,
2219
+ "rstrip": false,
2220
+ "single_word": false
2221
+ },
2222
+ {
2223
+ "content": "228358422847",
2224
+ "lstrip": false,
2225
+ "normalized": false,
2226
+ "rstrip": false,
2227
+ "single_word": false
2228
+ },
2229
+ {
2230
+ "content": "291762694874",
2231
+ "lstrip": false,
2232
+ "normalized": false,
2233
+ "rstrip": false,
2234
+ "single_word": false
2235
+ },
2236
+ {
2237
+ "content": "143182978129",
2238
+ "lstrip": false,
2239
+ "normalized": false,
2240
+ "rstrip": false,
2241
+ "single_word": false
2242
+ },
2243
+ {
2244
+ "content": "511923256573",
2245
+ "lstrip": false,
2246
+ "normalized": false,
2247
+ "rstrip": false,
2248
+ "single_word": false
2249
+ },
2250
+ {
2251
+ "content": "327158398268",
2252
+ "lstrip": false,
2253
+ "normalized": false,
2254
+ "rstrip": false,
2255
+ "single_word": false
2256
+ },
2257
+ {
2258
+ "content": "879764613759",
2259
+ "lstrip": false,
2260
+ "normalized": false,
2261
+ "rstrip": false,
2262
+ "single_word": false
2263
+ },
2264
+ {
2265
+ "content": "564395222747",
2266
+ "lstrip": false,
2267
+ "normalized": false,
2268
+ "rstrip": false,
2269
+ "single_word": false
2270
+ },
2271
+ {
2272
+ "content": "451161679736",
2273
+ "lstrip": false,
2274
+ "normalized": false,
2275
+ "rstrip": false,
2276
+ "single_word": false
2277
+ },
2278
+ {
2279
+ "content": "538631466654",
2280
+ "lstrip": false,
2281
+ "normalized": false,
2282
+ "rstrip": false,
2283
+ "single_word": false
2284
+ },
2285
+ {
2286
+ "content": "221762325616",
2287
+ "lstrip": false,
2288
+ "normalized": false,
2289
+ "rstrip": false,
2290
+ "single_word": false
2291
+ },
2292
+ {
2293
+ "content": "218391991184",
2294
+ "lstrip": false,
2295
+ "normalized": false,
2296
+ "rstrip": false,
2297
+ "single_word": false
2298
+ },
2299
+ {
2300
+ "content": "322589379462",
2301
+ "lstrip": false,
2302
+ "normalized": false,
2303
+ "rstrip": false,
2304
+ "single_word": false
2305
+ },
2306
+ {
2307
+ "content": "876537814263",
2308
+ "lstrip": false,
2309
+ "normalized": false,
2310
+ "rstrip": false,
2311
+ "single_word": false
2312
+ },
2313
+ {
2314
+ "content": "152676556624",
2315
+ "lstrip": false,
2316
+ "normalized": false,
2317
+ "rstrip": false,
2318
+ "single_word": false
2319
+ },
2320
+ {
2321
+ "content": "332522971941",
2322
+ "lstrip": false,
2323
+ "normalized": false,
2324
+ "rstrip": false,
2325
+ "single_word": false
2326
+ },
2327
+ {
2328
+ "content": "884354318946",
2329
+ "lstrip": false,
2330
+ "normalized": false,
2331
+ "rstrip": false,
2332
+ "single_word": false
2333
+ },
2334
+ {
2335
+ "content": "513349618943",
2336
+ "lstrip": false,
2337
+ "normalized": false,
2338
+ "rstrip": false,
2339
+ "single_word": false
2340
+ },
2341
+ {
2342
+ "content": "116639746413",
2343
+ "lstrip": false,
2344
+ "normalized": false,
2345
+ "rstrip": false,
2346
+ "single_word": false
2347
+ },
2348
+ {
2349
+ "content": "635185846287",
2350
+ "lstrip": false,
2351
+ "normalized": false,
2352
+ "rstrip": false,
2353
+ "single_word": false
2354
+ },
2355
+ {
2356
+ "content": "993832498489",
2357
+ "lstrip": false,
2358
+ "normalized": false,
2359
+ "rstrip": false,
2360
+ "single_word": false
2361
+ },
2362
+ {
2363
+ "content": "813981174797",
2364
+ "lstrip": false,
2365
+ "normalized": false,
2366
+ "rstrip": false,
2367
+ "single_word": false
2368
+ },
2369
+ {
2370
+ "content": "438745114173",
2371
+ "lstrip": false,
2372
+ "normalized": false,
2373
+ "rstrip": false,
2374
+ "single_word": false
2375
+ },
2376
+ {
2377
+ "content": "983493951323",
2378
+ "lstrip": false,
2379
+ "normalized": false,
2380
+ "rstrip": false,
2381
+ "single_word": false
2382
+ },
2383
+ {
2384
+ "content": "724492262421",
2385
+ "lstrip": false,
2386
+ "normalized": false,
2387
+ "rstrip": false,
2388
+ "single_word": false
2389
+ },
2390
+ {
2391
+ "content": "622553389126",
2392
+ "lstrip": false,
2393
+ "normalized": false,
2394
+ "rstrip": false,
2395
+ "single_word": false
2396
+ },
2397
+ {
2398
+ "content": "889965243135",
2399
+ "lstrip": false,
2400
+ "normalized": false,
2401
+ "rstrip": false,
2402
+ "single_word": false
2403
+ },
2404
+ {
2405
+ "content": "364492359246",
2406
+ "lstrip": false,
2407
+ "normalized": false,
2408
+ "rstrip": false,
2409
+ "single_word": false
2410
+ },
2411
+ {
2412
+ "content": "154962668224",
2413
+ "lstrip": false,
2414
+ "normalized": false,
2415
+ "rstrip": false,
2416
+ "single_word": false
2417
+ },
2418
+ {
2419
+ "content": "179564995814",
2420
+ "lstrip": false,
2421
+ "normalized": false,
2422
+ "rstrip": false,
2423
+ "single_word": false
2424
+ },
2425
+ {
2426
+ "content": "418412875665",
2427
+ "lstrip": false,
2428
+ "normalized": false,
2429
+ "rstrip": false,
2430
+ "single_word": false
2431
+ },
2432
+ {
2433
+ "content": "718951851413",
2434
+ "lstrip": false,
2435
+ "normalized": false,
2436
+ "rstrip": false,
2437
+ "single_word": false
2438
+ },
2439
+ {
2440
+ "content": "699446724178",
2441
+ "lstrip": false,
2442
+ "normalized": false,
2443
+ "rstrip": false,
2444
+ "single_word": false
2445
+ },
2446
+ {
2447
+ "content": "624266421831",
2448
+ "lstrip": false,
2449
+ "normalized": false,
2450
+ "rstrip": false,
2451
+ "single_word": false
2452
+ },
2453
+ {
2454
+ "content": "815458725125",
2455
+ "lstrip": false,
2456
+ "normalized": false,
2457
+ "rstrip": false,
2458
+ "single_word": false
2459
+ },
2460
+ {
2461
+ "content": "455423278865",
2462
+ "lstrip": false,
2463
+ "normalized": false,
2464
+ "rstrip": false,
2465
+ "single_word": false
2466
+ },
2467
+ {
2468
+ "content": "393741199486",
2469
+ "lstrip": false,
2470
+ "normalized": false,
2471
+ "rstrip": false,
2472
+ "single_word": false
2473
+ },
2474
+ {
2475
+ "content": "328552864359",
2476
+ "lstrip": false,
2477
+ "normalized": false,
2478
+ "rstrip": false,
2479
+ "single_word": false
2480
+ },
2481
+ {
2482
+ "content": "211662639865",
2483
+ "lstrip": false,
2484
+ "normalized": false,
2485
+ "rstrip": false,
2486
+ "single_word": false
2487
+ },
2488
+ {
2489
+ "content": "218784516525",
2490
+ "lstrip": false,
2491
+ "normalized": false,
2492
+ "rstrip": false,
2493
+ "single_word": false
2494
+ },
2495
+ {
2496
+ "content": "762486672996",
2497
+ "lstrip": false,
2498
+ "normalized": false,
2499
+ "rstrip": false,
2500
+ "single_word": false
2501
+ },
2502
+ {
2503
+ "content": "142799718159",
2504
+ "lstrip": false,
2505
+ "normalized": false,
2506
+ "rstrip": false,
2507
+ "single_word": false
2508
+ },
2509
+ {
2510
+ "content": "858146415154",
2511
+ "lstrip": false,
2512
+ "normalized": false,
2513
+ "rstrip": false,
2514
+ "single_word": false
2515
+ },
2516
+ {
2517
+ "content": "767858144912",
2518
+ "lstrip": false,
2519
+ "normalized": false,
2520
+ "rstrip": false,
2521
+ "single_word": false
2522
+ },
2523
+ {
2524
+ "content": "571317457151",
2525
+ "lstrip": false,
2526
+ "normalized": false,
2527
+ "rstrip": false,
2528
+ "single_word": false
2529
+ },
2530
+ {
2531
+ "content": "635127952696",
2532
+ "lstrip": false,
2533
+ "normalized": false,
2534
+ "rstrip": false,
2535
+ "single_word": false
2536
+ },
2537
+ {
2538
+ "content": "116427191984",
2539
+ "lstrip": false,
2540
+ "normalized": false,
2541
+ "rstrip": false,
2542
+ "single_word": false
2543
+ },
2544
+ {
2545
+ "content": "268921994538",
2546
+ "lstrip": false,
2547
+ "normalized": false,
2548
+ "rstrip": false,
2549
+ "single_word": false
2550
+ },
2551
+ {
2552
+ "content": "523937669294",
2553
+ "lstrip": false,
2554
+ "normalized": false,
2555
+ "rstrip": false,
2556
+ "single_word": false
2557
+ },
2558
+ {
2559
+ "content": "165429152138",
2560
+ "lstrip": false,
2561
+ "normalized": false,
2562
+ "rstrip": false,
2563
+ "single_word": false
2564
+ },
2565
+ {
2566
+ "content": "739246183345",
2567
+ "lstrip": false,
2568
+ "normalized": false,
2569
+ "rstrip": false,
2570
+ "single_word": false
2571
+ },
2572
+ {
2573
+ "content": "591464355756",
2574
+ "lstrip": false,
2575
+ "normalized": false,
2576
+ "rstrip": false,
2577
+ "single_word": false
2578
+ },
2579
+ {
2580
+ "content": "212985874612",
2581
+ "lstrip": false,
2582
+ "normalized": false,
2583
+ "rstrip": false,
2584
+ "single_word": false
2585
+ },
2586
+ {
2587
+ "content": "191887635211",
2588
+ "lstrip": false,
2589
+ "normalized": false,
2590
+ "rstrip": false,
2591
+ "single_word": false
2592
+ },
2593
+ {
2594
+ "content": "967214577653",
2595
+ "lstrip": false,
2596
+ "normalized": false,
2597
+ "rstrip": false,
2598
+ "single_word": false
2599
+ },
2600
+ {
2601
+ "content": "119342152414",
2602
+ "lstrip": false,
2603
+ "normalized": false,
2604
+ "rstrip": false,
2605
+ "single_word": false
2606
+ },
2607
+ {
2608
+ "content": "946444632795",
2609
+ "lstrip": false,
2610
+ "normalized": false,
2611
+ "rstrip": false,
2612
+ "single_word": false
2613
+ },
2614
+ {
2615
+ "content": "618423867817",
2616
+ "lstrip": false,
2617
+ "normalized": false,
2618
+ "rstrip": false,
2619
+ "single_word": false
2620
+ },
2621
+ {
2622
+ "content": "228565148417",
2623
+ "lstrip": false,
2624
+ "normalized": false,
2625
+ "rstrip": false,
2626
+ "single_word": false
2627
+ },
2628
+ {
2629
+ "content": "729116422489",
2630
+ "lstrip": false,
2631
+ "normalized": false,
2632
+ "rstrip": false,
2633
+ "single_word": false
2634
+ },
2635
+ {
2636
+ "content": "527874729936",
2637
+ "lstrip": false,
2638
+ "normalized": false,
2639
+ "rstrip": false,
2640
+ "single_word": false
2641
+ },
2642
+ {
2643
+ "content": "739784153482",
2644
+ "lstrip": false,
2645
+ "normalized": false,
2646
+ "rstrip": false,
2647
+ "single_word": false
2648
+ },
2649
+ {
2650
+ "content": "387763951128",
2651
+ "lstrip": false,
2652
+ "normalized": false,
2653
+ "rstrip": false,
2654
+ "single_word": false
2655
+ },
2656
+ {
2657
+ "content": "331369926711",
2658
+ "lstrip": false,
2659
+ "normalized": false,
2660
+ "rstrip": false,
2661
+ "single_word": false
2662
+ },
2663
+ {
2664
+ "content": "562716493614",
2665
+ "lstrip": false,
2666
+ "normalized": false,
2667
+ "rstrip": false,
2668
+ "single_word": false
2669
+ },
2670
+ {
2671
+ "content": "739667844957",
2672
+ "lstrip": false,
2673
+ "normalized": false,
2674
+ "rstrip": false,
2675
+ "single_word": false
2676
+ },
2677
+ {
2678
+ "content": "562389434565",
2679
+ "lstrip": false,
2680
+ "normalized": false,
2681
+ "rstrip": false,
2682
+ "single_word": false
2683
+ },
2684
+ {
2685
+ "content": "256497188281",
2686
+ "lstrip": false,
2687
+ "normalized": false,
2688
+ "rstrip": false,
2689
+ "single_word": false
2690
+ },
2691
+ {
2692
+ "content": "859927364588",
2693
+ "lstrip": false,
2694
+ "normalized": false,
2695
+ "rstrip": false,
2696
+ "single_word": false
2697
+ },
2698
+ {
2699
+ "content": "417668946583",
2700
+ "lstrip": false,
2701
+ "normalized": false,
2702
+ "rstrip": false,
2703
+ "single_word": false
2704
+ },
2705
+ {
2706
+ "content": "357621613582",
2707
+ "lstrip": false,
2708
+ "normalized": false,
2709
+ "rstrip": false,
2710
+ "single_word": false
2711
+ },
2712
+ {
2713
+ "content": "438435178228",
2714
+ "lstrip": false,
2715
+ "normalized": false,
2716
+ "rstrip": false,
2717
+ "single_word": false
2718
+ },
2719
+ {
2720
+ "content": "485692541169",
2721
+ "lstrip": false,
2722
+ "normalized": false,
2723
+ "rstrip": false,
2724
+ "single_word": false
2725
+ },
2726
+ {
2727
+ "content": "825815739116",
2728
+ "lstrip": false,
2729
+ "normalized": false,
2730
+ "rstrip": false,
2731
+ "single_word": false
2732
+ },
2733
+ {
2734
+ "content": "342221452223",
2735
+ "lstrip": false,
2736
+ "normalized": false,
2737
+ "rstrip": false,
2738
+ "single_word": false
2739
+ },
2740
+ {
2741
+ "content": "697747991249",
2742
+ "lstrip": false,
2743
+ "normalized": false,
2744
+ "rstrip": false,
2745
+ "single_word": false
2746
+ },
2747
+ {
2748
+ "content": "716763689965",
2749
+ "lstrip": false,
2750
+ "normalized": false,
2751
+ "rstrip": false,
2752
+ "single_word": false
2753
+ },
2754
+ {
2755
+ "content": "141499982867",
2756
+ "lstrip": false,
2757
+ "normalized": false,
2758
+ "rstrip": false,
2759
+ "single_word": false
2760
+ },
2761
+ {
2762
+ "content": "818479319499",
2763
+ "lstrip": false,
2764
+ "normalized": false,
2765
+ "rstrip": false,
2766
+ "single_word": false
2767
+ },
2768
+ {
2769
+ "content": "336813343298",
2770
+ "lstrip": false,
2771
+ "normalized": false,
2772
+ "rstrip": false,
2773
+ "single_word": false
2774
+ },
2775
+ {
2776
+ "content": "594688742928",
2777
+ "lstrip": false,
2778
+ "normalized": false,
2779
+ "rstrip": false,
2780
+ "single_word": false
2781
+ },
2782
+ {
2783
+ "content": "472129283475",
2784
+ "lstrip": false,
2785
+ "normalized": false,
2786
+ "rstrip": false,
2787
+ "single_word": false
2788
+ },
2789
+ {
2790
+ "content": "514354144759",
2791
+ "lstrip": false,
2792
+ "normalized": false,
2793
+ "rstrip": false,
2794
+ "single_word": false
2795
+ },
2796
+ {
2797
+ "content": "349249721685",
2798
+ "lstrip": false,
2799
+ "normalized": false,
2800
+ "rstrip": false,
2801
+ "single_word": false
2802
+ },
2803
+ {
2804
+ "content": "546276298359",
2805
+ "lstrip": false,
2806
+ "normalized": false,
2807
+ "rstrip": false,
2808
+ "single_word": false
2809
+ },
2810
+ {
2811
+ "content": "353755529131",
2812
+ "lstrip": false,
2813
+ "normalized": false,
2814
+ "rstrip": false,
2815
+ "single_word": false
2816
+ },
2817
+ {
2818
+ "content": "315534574435",
2819
+ "lstrip": false,
2820
+ "normalized": false,
2821
+ "rstrip": false,
2822
+ "single_word": false
2823
+ },
2824
+ {
2825
+ "content": "523723475786",
2826
+ "lstrip": false,
2827
+ "normalized": false,
2828
+ "rstrip": false,
2829
+ "single_word": false
2830
+ },
2831
+ {
2832
+ "content": "215826764872",
2833
+ "lstrip": false,
2834
+ "normalized": false,
2835
+ "rstrip": false,
2836
+ "single_word": false
2837
+ },
2838
+ {
2839
+ "content": "367968398551",
2840
+ "lstrip": false,
2841
+ "normalized": false,
2842
+ "rstrip": false,
2843
+ "single_word": false
2844
+ },
2845
+ {
2846
+ "content": "569853653352",
2847
+ "lstrip": false,
2848
+ "normalized": false,
2849
+ "rstrip": false,
2850
+ "single_word": false
2851
+ },
2852
+ {
2853
+ "content": "389715484387",
2854
+ "lstrip": false,
2855
+ "normalized": false,
2856
+ "rstrip": false,
2857
+ "single_word": false
2858
+ },
2859
+ {
2860
+ "content": "293847485454",
2861
+ "lstrip": false,
2862
+ "normalized": false,
2863
+ "rstrip": false,
2864
+ "single_word": false
2865
+ },
2866
+ {
2867
+ "content": "714738141818",
2868
+ "lstrip": false,
2869
+ "normalized": false,
2870
+ "rstrip": false,
2871
+ "single_word": false
2872
+ },
2873
+ {
2874
+ "content": "178478368922",
2875
+ "lstrip": false,
2876
+ "normalized": false,
2877
+ "rstrip": false,
2878
+ "single_word": false
2879
+ },
2880
+ {
2881
+ "content": "581493616981",
2882
+ "lstrip": false,
2883
+ "normalized": false,
2884
+ "rstrip": false,
2885
+ "single_word": false
2886
+ },
2887
+ {
2888
+ "content": "589439538674",
2889
+ "lstrip": false,
2890
+ "normalized": false,
2891
+ "rstrip": false,
2892
+ "single_word": false
2893
+ },
2894
+ {
2895
+ "content": "846657726193",
2896
+ "lstrip": false,
2897
+ "normalized": false,
2898
+ "rstrip": false,
2899
+ "single_word": false
2900
+ },
2901
+ {
2902
+ "content": "722339992679",
2903
+ "lstrip": false,
2904
+ "normalized": false,
2905
+ "rstrip": false,
2906
+ "single_word": false
2907
+ },
2908
+ {
2909
+ "content": "138154781148",
2910
+ "lstrip": false,
2911
+ "normalized": false,
2912
+ "rstrip": false,
2913
+ "single_word": false
2914
+ },
2915
+ {
2916
+ "content": "757785319772",
2917
+ "lstrip": false,
2918
+ "normalized": false,
2919
+ "rstrip": false,
2920
+ "single_word": false
2921
+ },
2922
+ {
2923
+ "content": "492516914298",
2924
+ "lstrip": false,
2925
+ "normalized": false,
2926
+ "rstrip": false,
2927
+ "single_word": false
2928
+ },
2929
+ {
2930
+ "content": "919181521716",
2931
+ "lstrip": false,
2932
+ "normalized": false,
2933
+ "rstrip": false,
2934
+ "single_word": false
2935
+ },
2936
+ {
2937
+ "content": "985781138935",
2938
+ "lstrip": false,
2939
+ "normalized": false,
2940
+ "rstrip": false,
2941
+ "single_word": false
2942
+ },
2943
+ {
2944
+ "content": "476969195485",
2945
+ "lstrip": false,
2946
+ "normalized": false,
2947
+ "rstrip": false,
2948
+ "single_word": false
2949
+ },
2950
+ {
2951
+ "content": "313145133463",
2952
+ "lstrip": false,
2953
+ "normalized": false,
2954
+ "rstrip": false,
2955
+ "single_word": false
2956
+ },
2957
+ {
2958
+ "content": "758963111966",
2959
+ "lstrip": false,
2960
+ "normalized": false,
2961
+ "rstrip": false,
2962
+ "single_word": false
2963
+ },
2964
+ {
2965
+ "content": "147541537162",
2966
+ "lstrip": false,
2967
+ "normalized": false,
2968
+ "rstrip": false,
2969
+ "single_word": false
2970
+ },
2971
+ {
2972
+ "content": "557163366873",
2973
+ "lstrip": false,
2974
+ "normalized": false,
2975
+ "rstrip": false,
2976
+ "single_word": false
2977
+ },
2978
+ {
2979
+ "content": "144373897488",
2980
+ "lstrip": false,
2981
+ "normalized": false,
2982
+ "rstrip": false,
2983
+ "single_word": false
2984
+ },
2985
+ {
2986
+ "content": "522515164754",
2987
+ "lstrip": false,
2988
+ "normalized": false,
2989
+ "rstrip": false,
2990
+ "single_word": false
2991
+ },
2992
+ {
2993
+ "content": "724964923582",
2994
+ "lstrip": false,
2995
+ "normalized": false,
2996
+ "rstrip": false,
2997
+ "single_word": false
2998
+ },
2999
+ {
3000
+ "content": "284776712475",
3001
+ "lstrip": false,
3002
+ "normalized": false,
3003
+ "rstrip": false,
3004
+ "single_word": false
3005
+ },
3006
+ {
3007
+ "content": "375429755114",
3008
+ "lstrip": false,
3009
+ "normalized": false,
3010
+ "rstrip": false,
3011
+ "single_word": false
3012
+ },
3013
+ {
3014
+ "content": "181233596124",
3015
+ "lstrip": false,
3016
+ "normalized": false,
3017
+ "rstrip": false,
3018
+ "single_word": false
3019
+ },
3020
+ {
3021
+ "content": "948585673431",
3022
+ "lstrip": false,
3023
+ "normalized": false,
3024
+ "rstrip": false,
3025
+ "single_word": false
3026
+ },
3027
+ {
3028
+ "content": "243165586174",
3029
+ "lstrip": false,
3030
+ "normalized": false,
3031
+ "rstrip": false,
3032
+ "single_word": false
3033
+ },
3034
+ {
3035
+ "content": "396847976144",
3036
+ "lstrip": false,
3037
+ "normalized": false,
3038
+ "rstrip": false,
3039
+ "single_word": false
3040
+ },
3041
+ {
3042
+ "content": "997724962668",
3043
+ "lstrip": false,
3044
+ "normalized": false,
3045
+ "rstrip": false,
3046
+ "single_word": false
3047
+ },
3048
+ {
3049
+ "content": "558837194455",
3050
+ "lstrip": false,
3051
+ "normalized": false,
3052
+ "rstrip": false,
3053
+ "single_word": false
3054
+ },
3055
+ {
3056
+ "content": "163165456396",
3057
+ "lstrip": false,
3058
+ "normalized": false,
3059
+ "rstrip": false,
3060
+ "single_word": false
3061
+ },
3062
+ {
3063
+ "content": "378749551722",
3064
+ "lstrip": false,
3065
+ "normalized": false,
3066
+ "rstrip": false,
3067
+ "single_word": false
3068
+ },
3069
+ {
3070
+ "content": "161238482259",
3071
+ "lstrip": false,
3072
+ "normalized": false,
3073
+ "rstrip": false,
3074
+ "single_word": false
3075
+ },
3076
+ {
3077
+ "content": "754978243758",
3078
+ "lstrip": false,
3079
+ "normalized": false,
3080
+ "rstrip": false,
3081
+ "single_word": false
3082
+ },
3083
+ {
3084
+ "content": "195388849133",
3085
+ "lstrip": false,
3086
+ "normalized": false,
3087
+ "rstrip": false,
3088
+ "single_word": false
3089
+ },
3090
+ {
3091
+ "content": "229775525672",
3092
+ "lstrip": false,
3093
+ "normalized": false,
3094
+ "rstrip": false,
3095
+ "single_word": false
3096
+ },
3097
+ {
3098
+ "content": "262437452884",
3099
+ "lstrip": false,
3100
+ "normalized": false,
3101
+ "rstrip": false,
3102
+ "single_word": false
3103
+ },
3104
+ {
3105
+ "content": "441377892146",
3106
+ "lstrip": false,
3107
+ "normalized": false,
3108
+ "rstrip": false,
3109
+ "single_word": false
3110
+ },
3111
+ {
3112
+ "content": "451885565366",
3113
+ "lstrip": false,
3114
+ "normalized": false,
3115
+ "rstrip": false,
3116
+ "single_word": false
3117
+ },
3118
+ {
3119
+ "content": "981277526855",
3120
+ "lstrip": false,
3121
+ "normalized": false,
3122
+ "rstrip": false,
3123
+ "single_word": false
3124
+ },
3125
+ {
3126
+ "content": "762495822823",
3127
+ "lstrip": false,
3128
+ "normalized": false,
3129
+ "rstrip": false,
3130
+ "single_word": false
3131
+ },
3132
+ {
3133
+ "content": "368763327262",
3134
+ "lstrip": false,
3135
+ "normalized": false,
3136
+ "rstrip": false,
3137
+ "single_word": false
3138
+ },
3139
+ {
3140
+ "content": "757422791351",
3141
+ "lstrip": false,
3142
+ "normalized": false,
3143
+ "rstrip": false,
3144
+ "single_word": false
3145
+ },
3146
+ {
3147
+ "content": "636324136426",
3148
+ "lstrip": false,
3149
+ "normalized": false,
3150
+ "rstrip": false,
3151
+ "single_word": false
3152
+ },
3153
+ {
3154
+ "content": "214193645583",
3155
+ "lstrip": false,
3156
+ "normalized": false,
3157
+ "rstrip": false,
3158
+ "single_word": false
3159
+ },
3160
+ {
3161
+ "content": "412843856172",
3162
+ "lstrip": false,
3163
+ "normalized": false,
3164
+ "rstrip": false,
3165
+ "single_word": false
3166
+ },
3167
+ {
3168
+ "content": "179386156569",
3169
+ "lstrip": false,
3170
+ "normalized": false,
3171
+ "rstrip": false,
3172
+ "single_word": false
3173
+ },
3174
+ {
3175
+ "content": "756916173536",
3176
+ "lstrip": false,
3177
+ "normalized": false,
3178
+ "rstrip": false,
3179
+ "single_word": false
3180
+ },
3181
+ {
3182
+ "content": "892697125149",
3183
+ "lstrip": false,
3184
+ "normalized": false,
3185
+ "rstrip": false,
3186
+ "single_word": false
3187
+ },
3188
+ {
3189
+ "content": "625334487352",
3190
+ "lstrip": false,
3191
+ "normalized": false,
3192
+ "rstrip": false,
3193
+ "single_word": false
3194
+ },
3195
+ {
3196
+ "content": "941861857715",
3197
+ "lstrip": false,
3198
+ "normalized": false,
3199
+ "rstrip": false,
3200
+ "single_word": false
3201
+ },
3202
+ {
3203
+ "content": "887417525236",
3204
+ "lstrip": false,
3205
+ "normalized": false,
3206
+ "rstrip": false,
3207
+ "single_word": false
3208
+ },
3209
+ {
3210
+ "content": "649516938598",
3211
+ "lstrip": false,
3212
+ "normalized": false,
3213
+ "rstrip": false,
3214
+ "single_word": false
3215
+ },
3216
+ {
3217
+ "content": "717628619782",
3218
+ "lstrip": false,
3219
+ "normalized": false,
3220
+ "rstrip": false,
3221
+ "single_word": false
3222
+ },
3223
+ {
3224
+ "content": "438124184139",
3225
+ "lstrip": false,
3226
+ "normalized": false,
3227
+ "rstrip": false,
3228
+ "single_word": false
3229
+ },
3230
+ {
3231
+ "content": "547563892268",
3232
+ "lstrip": false,
3233
+ "normalized": false,
3234
+ "rstrip": false,
3235
+ "single_word": false
3236
+ },
3237
+ {
3238
+ "content": "856317483891",
3239
+ "lstrip": false,
3240
+ "normalized": false,
3241
+ "rstrip": false,
3242
+ "single_word": false
3243
+ },
3244
+ {
3245
+ "content": "313313831273",
3246
+ "lstrip": false,
3247
+ "normalized": false,
3248
+ "rstrip": false,
3249
+ "single_word": false
3250
+ },
3251
+ {
3252
+ "content": "371496153876",
3253
+ "lstrip": false,
3254
+ "normalized": false,
3255
+ "rstrip": false,
3256
+ "single_word": false
3257
+ },
3258
+ {
3259
+ "content": "587541149322",
3260
+ "lstrip": false,
3261
+ "normalized": false,
3262
+ "rstrip": false,
3263
+ "single_word": false
3264
+ },
3265
+ {
3266
+ "content": "265847332563",
3267
+ "lstrip": false,
3268
+ "normalized": false,
3269
+ "rstrip": false,
3270
+ "single_word": false
3271
+ },
3272
+ {
3273
+ "content": "449549215429",
3274
+ "lstrip": false,
3275
+ "normalized": false,
3276
+ "rstrip": false,
3277
+ "single_word": false
3278
+ },
3279
+ {
3280
+ "content": "163497196769",
3281
+ "lstrip": false,
3282
+ "normalized": false,
3283
+ "rstrip": false,
3284
+ "single_word": false
3285
+ },
3286
+ {
3287
+ "content": "861342291298",
3288
+ "lstrip": false,
3289
+ "normalized": false,
3290
+ "rstrip": false,
3291
+ "single_word": false
3292
+ },
3293
+ {
3294
+ "content": "268433315926",
3295
+ "lstrip": false,
3296
+ "normalized": false,
3297
+ "rstrip": false,
3298
+ "single_word": false
3299
+ },
3300
+ {
3301
+ "content": "774679513717",
3302
+ "lstrip": false,
3303
+ "normalized": false,
3304
+ "rstrip": false,
3305
+ "single_word": false
3306
+ },
3307
+ {
3308
+ "content": "851254219729",
3309
+ "lstrip": false,
3310
+ "normalized": false,
3311
+ "rstrip": false,
3312
+ "single_word": false
3313
+ },
3314
+ {
3315
+ "content": "583527834464",
3316
+ "lstrip": false,
3317
+ "normalized": false,
3318
+ "rstrip": false,
3319
+ "single_word": false
3320
+ },
3321
+ {
3322
+ "content": "488496781997",
3323
+ "lstrip": false,
3324
+ "normalized": false,
3325
+ "rstrip": false,
3326
+ "single_word": false
3327
+ },
3328
+ {
3329
+ "content": "556814553861",
3330
+ "lstrip": false,
3331
+ "normalized": false,
3332
+ "rstrip": false,
3333
+ "single_word": false
3334
+ },
3335
+ {
3336
+ "content": "482829231639",
3337
+ "lstrip": false,
3338
+ "normalized": false,
3339
+ "rstrip": false,
3340
+ "single_word": false
3341
+ },
3342
+ {
3343
+ "content": "618878266619",
3344
+ "lstrip": false,
3345
+ "normalized": false,
3346
+ "rstrip": false,
3347
+ "single_word": false
3348
+ },
3349
+ {
3350
+ "content": "147444452794",
3351
+ "lstrip": false,
3352
+ "normalized": false,
3353
+ "rstrip": false,
3354
+ "single_word": false
3355
+ },
3356
+ {
3357
+ "content": "949235426629",
3358
+ "lstrip": false,
3359
+ "normalized": false,
3360
+ "rstrip": false,
3361
+ "single_word": false
3362
+ },
3363
+ {
3364
+ "content": "357299947518",
3365
+ "lstrip": false,
3366
+ "normalized": false,
3367
+ "rstrip": false,
3368
+ "single_word": false
3369
+ },
3370
+ {
3371
+ "content": "175528632226",
3372
+ "lstrip": false,
3373
+ "normalized": false,
3374
+ "rstrip": false,
3375
+ "single_word": false
3376
+ },
3377
+ {
3378
+ "content": "645527857972",
3379
+ "lstrip": false,
3380
+ "normalized": false,
3381
+ "rstrip": false,
3382
+ "single_word": false
3383
+ },
3384
+ {
3385
+ "content": "186872457894",
3386
+ "lstrip": false,
3387
+ "normalized": false,
3388
+ "rstrip": false,
3389
+ "single_word": false
3390
+ },
3391
+ {
3392
+ "content": "552738847828",
3393
+ "lstrip": false,
3394
+ "normalized": false,
3395
+ "rstrip": false,
3396
+ "single_word": false
3397
+ },
3398
+ {
3399
+ "content": "626748382482",
3400
+ "lstrip": false,
3401
+ "normalized": false,
3402
+ "rstrip": false,
3403
+ "single_word": false
3404
+ },
3405
+ {
3406
+ "content": "921894985642",
3407
+ "lstrip": false,
3408
+ "normalized": false,
3409
+ "rstrip": false,
3410
+ "single_word": false
3411
+ },
3412
+ {
3413
+ "content": "943878645871",
3414
+ "lstrip": false,
3415
+ "normalized": false,
3416
+ "rstrip": false,
3417
+ "single_word": false
3418
+ },
3419
+ {
3420
+ "content": "859289776479",
3421
+ "lstrip": false,
3422
+ "normalized": false,
3423
+ "rstrip": false,
3424
+ "single_word": false
3425
+ },
3426
+ {
3427
+ "content": "614583493135",
3428
+ "lstrip": false,
3429
+ "normalized": false,
3430
+ "rstrip": false,
3431
+ "single_word": false
3432
+ },
3433
+ {
3434
+ "content": "933775286797",
3435
+ "lstrip": false,
3436
+ "normalized": false,
3437
+ "rstrip": false,
3438
+ "single_word": false
3439
+ },
3440
+ {
3441
+ "content": "332234613346",
3442
+ "lstrip": false,
3443
+ "normalized": false,
3444
+ "rstrip": false,
3445
+ "single_word": false
3446
+ },
3447
+ {
3448
+ "content": "325196781219",
3449
+ "lstrip": false,
3450
+ "normalized": false,
3451
+ "rstrip": false,
3452
+ "single_word": false
3453
+ },
3454
+ {
3455
+ "content": "142526557681",
3456
+ "lstrip": false,
3457
+ "normalized": false,
3458
+ "rstrip": false,
3459
+ "single_word": false
3460
+ },
3461
+ {
3462
+ "content": "356722692178",
3463
+ "lstrip": false,
3464
+ "normalized": false,
3465
+ "rstrip": false,
3466
+ "single_word": false
3467
+ },
3468
+ {
3469
+ "content": "449318681694",
3470
+ "lstrip": false,
3471
+ "normalized": false,
3472
+ "rstrip": false,
3473
+ "single_word": false
3474
+ },
3475
+ {
3476
+ "content": "687284547244",
3477
+ "lstrip": false,
3478
+ "normalized": false,
3479
+ "rstrip": false,
3480
+ "single_word": false
3481
+ },
3482
+ {
3483
+ "content": "947262995132",
3484
+ "lstrip": false,
3485
+ "normalized": false,
3486
+ "rstrip": false,
3487
+ "single_word": false
3488
+ },
3489
+ {
3490
+ "content": "893974619684",
3491
+ "lstrip": false,
3492
+ "normalized": false,
3493
+ "rstrip": false,
3494
+ "single_word": false
3495
+ },
3496
+ {
3497
+ "content": "797238311233",
3498
+ "lstrip": false,
3499
+ "normalized": false,
3500
+ "rstrip": false,
3501
+ "single_word": false
3502
+ }
3503
+ ],
3504
+ "bos_token": {
3505
+ "content": "<|begin_of_text|>",
3506
+ "lstrip": false,
3507
+ "normalized": false,
3508
+ "rstrip": false,
3509
+ "single_word": false
3510
+ },
3511
+ "eos_token": {
3512
+ "content": "<|end_of_text|>",
3513
+ "lstrip": false,
3514
+ "normalized": false,
3515
+ "rstrip": false,
3516
+ "single_word": false
3517
+ },
3518
+ "pad_token": "<|end_of_text|>"
3519
+ }
llama3-8b/dp8_v2/tokenizer/tokenizer_config.json ADDED
The diff for this file is too large to render. See raw diff
 
llama3-8b/dp8_v2/train.log ADDED
@@ -0,0 +1,441 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 2026-04-04 00:18:33,732 [INFO] new_opacus_codex.train_steps: epoch=1 step=5 loss=12.6437
2
+ 2026-04-04 00:19:17,246 [INFO] new_opacus_codex.train_steps: epoch=1 step=10 loss=12.6414
3
+ 2026-04-04 00:19:31,628 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=10 eval_loss=12.9598 duration_sec=14.35
4
+ 2026-04-04 00:20:14,632 [INFO] new_opacus_codex.train_steps: epoch=1 step=15 loss=12.6369
5
+ 2026-04-04 00:20:57,340 [INFO] new_opacus_codex.train_steps: epoch=1 step=20 loss=12.5891
6
+ 2026-04-04 00:21:11,157 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=20 eval_loss=12.9415 duration_sec=13.81
7
+ 2026-04-04 00:21:53,744 [INFO] new_opacus_codex.train_steps: epoch=1 step=25 loss=12.6459
8
+ 2026-04-04 00:22:36,086 [INFO] new_opacus_codex.train_steps: epoch=1 step=30 loss=12.6439
9
+ 2026-04-04 00:22:49,944 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=30 eval_loss=12.9105 duration_sec=13.85
10
+ 2026-04-04 00:23:33,602 [INFO] new_opacus_codex.train_steps: epoch=1 step=35 loss=12.5316
11
+ 2026-04-04 00:24:15,891 [INFO] new_opacus_codex.train_steps: epoch=1 step=40 loss=12.6559
12
+ 2026-04-04 00:24:29,767 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=40 eval_loss=12.8708 duration_sec=13.87
13
+ 2026-04-04 00:25:13,706 [INFO] new_opacus_codex.train_steps: epoch=1 step=45 loss=12.6748
14
+ 2026-04-04 00:25:56,079 [INFO] new_opacus_codex.train_steps: epoch=1 step=50 loss=12.5540
15
+ 2026-04-04 00:26:09,921 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=50 eval_loss=12.8248 duration_sec=13.83
16
+ 2026-04-04 00:26:52,143 [INFO] new_opacus_codex.train_steps: epoch=1 step=55 loss=12.5633
17
+ 2026-04-04 00:27:34,622 [INFO] new_opacus_codex.train_steps: epoch=1 step=60 loss=12.5407
18
+ 2026-04-04 00:27:48,425 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=60 eval_loss=12.7645 duration_sec=13.78
19
+ 2026-04-04 00:28:31,586 [INFO] new_opacus_codex.train_steps: epoch=1 step=65 loss=12.5575
20
+ 2026-04-04 00:29:14,360 [INFO] new_opacus_codex.train_steps: epoch=1 step=70 loss=12.5858
21
+ 2026-04-04 00:29:28,105 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=70 eval_loss=12.6885 duration_sec=13.73
22
+ 2026-04-04 00:30:11,519 [INFO] new_opacus_codex.train_steps: epoch=1 step=75 loss=12.5502
23
+ 2026-04-04 00:30:54,825 [INFO] new_opacus_codex.train_steps: epoch=1 step=80 loss=12.4159
24
+ 2026-04-04 00:31:08,681 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=80 eval_loss=12.5984 duration_sec=13.84
25
+ 2026-04-04 00:31:51,675 [INFO] new_opacus_codex.train_steps: epoch=1 step=85 loss=12.3167
26
+ 2026-04-04 00:32:36,008 [INFO] new_opacus_codex.train_steps: epoch=1 step=90 loss=12.2845
27
+ 2026-04-04 00:32:49,812 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=90 eval_loss=12.5105 duration_sec=13.79
28
+ 2026-04-04 00:33:32,248 [INFO] new_opacus_codex.train_steps: epoch=1 step=95 loss=12.3508
29
+ 2026-04-04 00:34:15,338 [INFO] new_opacus_codex.train_steps: epoch=1 step=100 loss=12.4227
30
+ 2026-04-04 00:34:29,148 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=100 eval_loss=12.4091 duration_sec=13.78
31
+ 2026-04-04 00:35:13,463 [INFO] new_opacus_codex.train_steps: epoch=1 step=105 loss=12.3208
32
+ 2026-04-04 00:35:56,100 [INFO] new_opacus_codex.train_steps: epoch=1 step=110 loss=12.3008
33
+ 2026-04-04 00:36:09,970 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=110 eval_loss=12.3035 duration_sec=13.85
34
+ 2026-04-04 00:36:52,459 [INFO] new_opacus_codex.train_steps: epoch=1 step=115 loss=12.2038
35
+ 2026-04-04 00:37:35,636 [INFO] new_opacus_codex.train_steps: epoch=1 step=120 loss=12.0833
36
+ 2026-04-04 00:37:49,439 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=120 eval_loss=12.1823 duration_sec=13.79
37
+ 2026-04-04 00:38:31,184 [INFO] new_opacus_codex.train_steps: epoch=1 step=125 loss=12.1275
38
+ 2026-04-04 00:39:14,332 [INFO] new_opacus_codex.train_steps: epoch=1 step=130 loss=12.0210
39
+ 2026-04-04 00:39:28,143 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=130 eval_loss=12.0569 duration_sec=13.79
40
+ 2026-04-04 00:40:11,438 [INFO] new_opacus_codex.train_steps: epoch=1 step=135 loss=11.9207
41
+ 2026-04-04 00:40:54,075 [INFO] new_opacus_codex.train_steps: epoch=1 step=140 loss=11.9587
42
+ 2026-04-04 00:41:07,916 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=140 eval_loss=11.9201 duration_sec=13.83
43
+ 2026-04-04 00:41:50,210 [INFO] new_opacus_codex.train_steps: epoch=1 step=145 loss=11.8719
44
+ 2026-04-04 00:42:33,803 [INFO] new_opacus_codex.train_steps: epoch=1 step=150 loss=11.7624
45
+ 2026-04-04 00:42:47,636 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=150 eval_loss=11.7611 duration_sec=13.83
46
+ 2026-04-04 00:43:30,810 [INFO] new_opacus_codex.train_steps: epoch=1 step=155 loss=11.5857
47
+ 2026-04-04 00:44:12,512 [INFO] new_opacus_codex.train_steps: epoch=1 step=160 loss=11.5446
48
+ 2026-04-04 00:44:26,398 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=160 eval_loss=11.5874 duration_sec=13.87
49
+ 2026-04-04 00:45:09,399 [INFO] new_opacus_codex.train_steps: epoch=1 step=165 loss=11.6043
50
+ 2026-04-04 00:45:52,780 [INFO] new_opacus_codex.train_steps: epoch=1 step=170 loss=11.4718
51
+ 2026-04-04 00:46:06,608 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=170 eval_loss=11.3996 duration_sec=13.81
52
+ 2026-04-04 00:46:49,721 [INFO] new_opacus_codex.train_steps: epoch=1 step=175 loss=11.3325
53
+ 2026-04-04 00:47:33,563 [INFO] new_opacus_codex.train_steps: epoch=1 step=180 loss=11.3421
54
+ 2026-04-04 00:47:47,465 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=180 eval_loss=11.2089 duration_sec=13.89
55
+ 2026-04-04 00:48:48,867 [INFO] new_opacus_codex.train_steps: epoch=2 step=185 loss=11.4799
56
+ 2026-04-04 00:49:32,821 [INFO] new_opacus_codex.train_steps: epoch=2 step=190 loss=11.1957
57
+ 2026-04-04 00:49:46,627 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=190 eval_loss=11.0243 duration_sec=13.80
58
+ 2026-04-04 00:50:30,498 [INFO] new_opacus_codex.train_steps: epoch=2 step=195 loss=11.0877
59
+ 2026-04-04 00:51:13,820 [INFO] new_opacus_codex.train_steps: epoch=2 step=200 loss=10.9481
60
+ 2026-04-04 00:51:27,610 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=200 eval_loss=10.8387 duration_sec=13.77
61
+ 2026-04-04 00:52:10,687 [INFO] new_opacus_codex.train_steps: epoch=2 step=205 loss=10.8243
62
+ 2026-04-04 00:52:54,419 [INFO] new_opacus_codex.train_steps: epoch=2 step=210 loss=10.7420
63
+ 2026-04-04 00:53:08,252 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=210 eval_loss=10.6455 duration_sec=13.82
64
+ 2026-04-04 00:53:51,578 [INFO] new_opacus_codex.train_steps: epoch=2 step=215 loss=10.6979
65
+ 2026-04-04 00:54:34,317 [INFO] new_opacus_codex.train_steps: epoch=2 step=220 loss=10.6712
66
+ 2026-04-04 00:54:48,213 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=220 eval_loss=10.4531 duration_sec=13.88
67
+ 2026-04-04 00:55:32,238 [INFO] new_opacus_codex.train_steps: epoch=2 step=225 loss=10.5195
68
+ 2026-04-04 00:56:15,009 [INFO] new_opacus_codex.train_steps: epoch=2 step=230 loss=10.4842
69
+ 2026-04-04 00:56:29,149 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=230 eval_loss=10.2655 duration_sec=14.11
70
+ 2026-04-04 00:57:11,572 [INFO] new_opacus_codex.train_steps: epoch=2 step=235 loss=10.5610
71
+ 2026-04-04 00:57:54,752 [INFO] new_opacus_codex.train_steps: epoch=2 step=240 loss=10.4847
72
+ 2026-04-04 00:58:08,604 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=240 eval_loss=10.0795 duration_sec=13.83
73
+ 2026-04-04 00:58:51,989 [INFO] new_opacus_codex.train_steps: epoch=2 step=245 loss=10.2159
74
+ 2026-04-04 00:59:35,102 [INFO] new_opacus_codex.train_steps: epoch=2 step=250 loss=10.1418
75
+ 2026-04-04 00:59:48,934 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=250 eval_loss=9.9085 duration_sec=13.81
76
+ 2026-04-04 01:00:32,161 [INFO] new_opacus_codex.train_steps: epoch=2 step=255 loss=10.0980
77
+ 2026-04-04 01:01:14,102 [INFO] new_opacus_codex.train_steps: epoch=2 step=260 loss=9.9173
78
+ 2026-04-04 01:01:27,927 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=260 eval_loss=9.7428 duration_sec=13.81
79
+ 2026-04-04 01:02:10,481 [INFO] new_opacus_codex.train_steps: epoch=2 step=265 loss=9.8524
80
+ 2026-04-04 01:02:53,046 [INFO] new_opacus_codex.train_steps: epoch=2 step=270 loss=9.8633
81
+ 2026-04-04 01:03:06,889 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=270 eval_loss=9.5842 duration_sec=13.82
82
+ 2026-04-04 01:03:49,525 [INFO] new_opacus_codex.train_steps: epoch=2 step=275 loss=9.8625
83
+ 2026-04-04 01:04:33,646 [INFO] new_opacus_codex.train_steps: epoch=2 step=280 loss=9.7221
84
+ 2026-04-04 01:04:47,522 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=280 eval_loss=9.4427 duration_sec=13.87
85
+ 2026-04-04 01:05:30,732 [INFO] new_opacus_codex.train_steps: epoch=2 step=285 loss=9.6413
86
+ 2026-04-04 01:06:14,353 [INFO] new_opacus_codex.train_steps: epoch=2 step=290 loss=9.5965
87
+ 2026-04-04 01:06:28,236 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=290 eval_loss=9.3080 duration_sec=13.87
88
+ 2026-04-04 01:07:11,802 [INFO] new_opacus_codex.train_steps: epoch=2 step=295 loss=9.5249
89
+ 2026-04-04 01:07:54,787 [INFO] new_opacus_codex.train_steps: epoch=2 step=300 loss=9.5400
90
+ 2026-04-04 01:08:08,630 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=300 eval_loss=9.1786 duration_sec=13.83
91
+ 2026-04-04 01:08:52,226 [INFO] new_opacus_codex.train_steps: epoch=2 step=305 loss=9.5074
92
+ 2026-04-04 01:09:36,534 [INFO] new_opacus_codex.train_steps: epoch=2 step=310 loss=9.4456
93
+ 2026-04-04 01:09:50,391 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=310 eval_loss=9.0669 duration_sec=13.85
94
+ 2026-04-04 01:10:34,319 [INFO] new_opacus_codex.train_steps: epoch=2 step=315 loss=9.3628
95
+ 2026-04-04 01:11:17,731 [INFO] new_opacus_codex.train_steps: epoch=2 step=320 loss=9.3919
96
+ 2026-04-04 01:11:31,562 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=320 eval_loss=8.9646 duration_sec=13.81
97
+ 2026-04-04 01:12:15,815 [INFO] new_opacus_codex.train_steps: epoch=2 step=325 loss=9.3651
98
+ 2026-04-04 01:12:58,495 [INFO] new_opacus_codex.train_steps: epoch=2 step=330 loss=9.3098
99
+ 2026-04-04 01:13:12,551 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=330 eval_loss=8.8728 duration_sec=14.04
100
+ 2026-04-04 01:13:54,548 [INFO] new_opacus_codex.train_steps: epoch=2 step=335 loss=9.2054
101
+ 2026-04-04 01:14:37,681 [INFO] new_opacus_codex.train_steps: epoch=2 step=340 loss=8.9976
102
+ 2026-04-04 01:14:51,545 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=340 eval_loss=8.7960 duration_sec=13.84
103
+ 2026-04-04 01:15:33,850 [INFO] new_opacus_codex.train_steps: epoch=2 step=345 loss=8.8728
104
+ 2026-04-04 01:16:17,481 [INFO] new_opacus_codex.train_steps: epoch=2 step=350 loss=8.8782
105
+ 2026-04-04 01:16:31,327 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=350 eval_loss=8.7341 duration_sec=13.83
106
+ 2026-04-04 01:17:14,618 [INFO] new_opacus_codex.train_steps: epoch=2 step=355 loss=9.0876
107
+ 2026-04-04 01:17:58,159 [INFO] new_opacus_codex.train_steps: epoch=2 step=360 loss=9.1970
108
+ 2026-04-04 01:18:11,950 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=360 eval_loss=8.6868 duration_sec=13.77
109
+ 2026-04-04 01:18:55,294 [INFO] new_opacus_codex.train_steps: epoch=2 step=365 loss=9.1232
110
+ 2026-04-04 01:19:58,807 [INFO] new_opacus_codex.train_steps: epoch=3 step=370 loss=8.9323
111
+ 2026-04-04 01:20:12,617 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=370 eval_loss=8.6505 duration_sec=13.79
112
+ 2026-04-04 01:20:56,402 [INFO] new_opacus_codex.train_steps: epoch=3 step=375 loss=8.7468
113
+ 2026-04-04 01:21:39,662 [INFO] new_opacus_codex.train_steps: epoch=3 step=380 loss=8.7873
114
+ 2026-04-04 01:21:53,466 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=380 eval_loss=8.6128 duration_sec=13.79
115
+ 2026-04-04 01:22:36,320 [INFO] new_opacus_codex.train_steps: epoch=3 step=385 loss=8.9283
116
+ 2026-04-04 01:23:19,253 [INFO] new_opacus_codex.train_steps: epoch=3 step=390 loss=8.9223
117
+ 2026-04-04 01:23:33,138 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=390 eval_loss=8.5732 duration_sec=13.87
118
+ 2026-04-04 01:24:15,804 [INFO] new_opacus_codex.train_steps: epoch=3 step=395 loss=8.8993
119
+ 2026-04-04 01:24:58,846 [INFO] new_opacus_codex.train_steps: epoch=3 step=400 loss=8.8413
120
+ 2026-04-04 01:25:13,228 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=400 eval_loss=8.5340 duration_sec=14.37
121
+ 2026-04-04 01:25:55,801 [INFO] new_opacus_codex.train_steps: epoch=3 step=405 loss=8.7646
122
+ 2026-04-04 01:26:39,604 [INFO] new_opacus_codex.train_steps: epoch=3 step=410 loss=8.8812
123
+ 2026-04-04 01:26:53,424 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=410 eval_loss=8.4936 duration_sec=13.80
124
+ 2026-04-04 01:27:36,770 [INFO] new_opacus_codex.train_steps: epoch=3 step=415 loss=8.8209
125
+ 2026-04-04 01:28:20,109 [INFO] new_opacus_codex.train_steps: epoch=3 step=420 loss=8.7990
126
+ 2026-04-04 01:28:33,911 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=420 eval_loss=8.4546 duration_sec=13.78
127
+ 2026-04-04 01:29:17,885 [INFO] new_opacus_codex.train_steps: epoch=3 step=425 loss=8.7555
128
+ 2026-04-04 01:30:01,122 [INFO] new_opacus_codex.train_steps: epoch=3 step=430 loss=8.6954
129
+ 2026-04-04 01:30:14,976 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=430 eval_loss=8.4279 duration_sec=13.85
130
+ 2026-04-04 01:30:58,776 [INFO] new_opacus_codex.train_steps: epoch=3 step=435 loss=8.6517
131
+ 2026-04-04 01:31:40,436 [INFO] new_opacus_codex.train_steps: epoch=3 step=440 loss=8.5113
132
+ 2026-04-04 01:31:54,261 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=440 eval_loss=8.4102 duration_sec=13.80
133
+ 2026-04-04 01:32:37,763 [INFO] new_opacus_codex.train_steps: epoch=3 step=445 loss=8.5980
134
+ 2026-04-04 01:33:20,666 [INFO] new_opacus_codex.train_steps: epoch=3 step=450 loss=8.7562
135
+ 2026-04-04 01:33:35,039 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=450 eval_loss=8.3842 duration_sec=14.36
136
+ 2026-04-04 01:34:17,500 [INFO] new_opacus_codex.train_steps: epoch=3 step=455 loss=8.8614
137
+ 2026-04-04 01:35:00,971 [INFO] new_opacus_codex.train_steps: epoch=3 step=460 loss=8.7587
138
+ 2026-04-04 01:35:14,828 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=460 eval_loss=8.3524 duration_sec=13.85
139
+ 2026-04-04 01:35:57,782 [INFO] new_opacus_codex.train_steps: epoch=3 step=465 loss=8.7260
140
+ 2026-04-04 01:36:40,965 [INFO] new_opacus_codex.train_steps: epoch=3 step=470 loss=8.7542
141
+ 2026-04-04 01:36:54,799 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=470 eval_loss=8.3323 duration_sec=13.82
142
+ 2026-04-04 01:37:37,163 [INFO] new_opacus_codex.train_steps: epoch=3 step=475 loss=8.7748
143
+ 2026-04-04 01:38:20,448 [INFO] new_opacus_codex.train_steps: epoch=3 step=480 loss=8.7551
144
+ 2026-04-04 01:38:34,298 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=480 eval_loss=8.3151 duration_sec=13.84
145
+ 2026-04-04 01:39:17,777 [INFO] new_opacus_codex.train_steps: epoch=3 step=485 loss=8.6135
146
+ 2026-04-04 01:40:01,146 [INFO] new_opacus_codex.train_steps: epoch=3 step=490 loss=8.6653
147
+ 2026-04-04 01:40:14,947 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=490 eval_loss=8.2928 duration_sec=13.78
148
+ 2026-04-04 01:40:57,340 [INFO] new_opacus_codex.train_steps: epoch=3 step=495 loss=8.8211
149
+ 2026-04-04 01:41:40,384 [INFO] new_opacus_codex.train_steps: epoch=3 step=500 loss=8.7912
150
+ 2026-04-04 01:41:54,194 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=500 eval_loss=8.2759 duration_sec=13.80
151
+ 2026-04-04 01:42:38,115 [INFO] new_opacus_codex.train_steps: epoch=3 step=505 loss=8.6512
152
+ 2026-04-04 01:43:21,428 [INFO] new_opacus_codex.train_steps: epoch=3 step=510 loss=8.6774
153
+ 2026-04-04 01:43:35,259 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=510 eval_loss=8.2577 duration_sec=13.80
154
+ 2026-04-04 01:44:19,000 [INFO] new_opacus_codex.train_steps: epoch=3 step=515 loss=8.6249
155
+ 2026-04-04 01:45:02,406 [INFO] new_opacus_codex.train_steps: epoch=3 step=520 loss=8.7024
156
+ 2026-04-04 01:45:16,230 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=520 eval_loss=8.2370 duration_sec=13.81
157
+ 2026-04-04 01:45:58,105 [INFO] new_opacus_codex.train_steps: epoch=3 step=525 loss=8.6607
158
+ 2026-04-04 01:46:41,561 [INFO] new_opacus_codex.train_steps: epoch=3 step=530 loss=8.3961
159
+ 2026-04-04 01:46:55,357 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=530 eval_loss=8.2238 duration_sec=13.78
160
+ 2026-04-04 01:47:38,420 [INFO] new_opacus_codex.train_steps: epoch=3 step=535 loss=8.6423
161
+ 2026-04-04 01:48:21,525 [INFO] new_opacus_codex.train_steps: epoch=3 step=540 loss=8.6160
162
+ 2026-04-04 01:48:35,343 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=540 eval_loss=8.2091 duration_sec=13.80
163
+ 2026-04-04 01:49:18,109 [INFO] new_opacus_codex.train_steps: epoch=3 step=545 loss=8.4943
164
+ 2026-04-04 01:50:01,290 [INFO] new_opacus_codex.train_steps: epoch=3 step=550 loss=8.5813
165
+ 2026-04-04 01:50:15,097 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=3 step=550 eval_loss=8.1917 duration_sec=13.75
166
+ 2026-04-04 01:51:16,213 [INFO] new_opacus_codex.train_steps: epoch=4 step=555 loss=8.4671
167
+ 2026-04-04 01:52:00,217 [INFO] new_opacus_codex.train_steps: epoch=4 step=560 loss=8.4131
168
+ 2026-04-04 01:52:14,025 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=560 eval_loss=8.1692 duration_sec=13.79
169
+ 2026-04-04 01:52:57,127 [INFO] new_opacus_codex.train_steps: epoch=4 step=565 loss=8.4649
170
+ 2026-04-04 01:53:39,777 [INFO] new_opacus_codex.train_steps: epoch=4 step=570 loss=8.5219
171
+ 2026-04-04 01:53:54,218 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=570 eval_loss=8.1523 duration_sec=14.43
172
+ 2026-04-04 01:54:37,333 [INFO] new_opacus_codex.train_steps: epoch=4 step=575 loss=8.4504
173
+ 2026-04-04 01:55:20,380 [INFO] new_opacus_codex.train_steps: epoch=4 step=580 loss=8.4120
174
+ 2026-04-04 01:55:34,171 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=580 eval_loss=8.1378 duration_sec=13.78
175
+ 2026-04-04 01:56:18,582 [INFO] new_opacus_codex.train_steps: epoch=4 step=585 loss=8.5372
176
+ 2026-04-04 01:57:01,697 [INFO] new_opacus_codex.train_steps: epoch=4 step=590 loss=8.6137
177
+ 2026-04-04 01:57:15,488 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=590 eval_loss=8.1206 duration_sec=13.76
178
+ 2026-04-04 01:57:58,310 [INFO] new_opacus_codex.train_steps: epoch=4 step=595 loss=8.4798
179
+ 2026-04-04 01:58:42,795 [INFO] new_opacus_codex.train_steps: epoch=4 step=600 loss=8.3445
180
+ 2026-04-04 01:58:56,595 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=600 eval_loss=8.1009 duration_sec=13.78
181
+ 2026-04-04 01:59:38,655 [INFO] new_opacus_codex.train_steps: epoch=4 step=605 loss=8.4822
182
+ 2026-04-04 02:00:22,159 [INFO] new_opacus_codex.train_steps: epoch=4 step=610 loss=8.5309
183
+ 2026-04-04 02:00:35,968 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=610 eval_loss=8.0791 duration_sec=13.79
184
+ 2026-04-04 02:01:18,946 [INFO] new_opacus_codex.train_steps: epoch=4 step=615 loss=8.5083
185
+ 2026-04-04 02:02:01,906 [INFO] new_opacus_codex.train_steps: epoch=4 step=620 loss=8.6011
186
+ 2026-04-04 02:02:15,672 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=620 eval_loss=8.0637 duration_sec=13.76
187
+ 2026-04-04 02:02:59,316 [INFO] new_opacus_codex.train_steps: epoch=4 step=625 loss=8.5162
188
+ 2026-04-04 02:03:43,021 [INFO] new_opacus_codex.train_steps: epoch=4 step=630 loss=8.5410
189
+ 2026-04-04 02:03:56,802 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=630 eval_loss=8.0428 duration_sec=13.77
190
+ 2026-04-04 02:04:40,515 [INFO] new_opacus_codex.train_steps: epoch=4 step=635 loss=8.6322
191
+ 2026-04-04 02:05:23,923 [INFO] new_opacus_codex.train_steps: epoch=4 step=640 loss=8.5511
192
+ 2026-04-04 02:05:37,781 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=640 eval_loss=8.0233 duration_sec=13.84
193
+ 2026-04-04 02:06:21,500 [INFO] new_opacus_codex.train_steps: epoch=4 step=645 loss=8.4754
194
+ 2026-04-04 02:07:04,462 [INFO] new_opacus_codex.train_steps: epoch=4 step=650 loss=8.4420
195
+ 2026-04-04 02:07:18,300 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=650 eval_loss=8.0109 duration_sec=13.82
196
+ 2026-04-04 02:08:01,247 [INFO] new_opacus_codex.train_steps: epoch=4 step=655 loss=8.4984
197
+ 2026-04-04 02:08:44,522 [INFO] new_opacus_codex.train_steps: epoch=4 step=660 loss=8.3464
198
+ 2026-04-04 02:08:58,313 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=660 eval_loss=7.9993 duration_sec=13.78
199
+ 2026-04-04 02:09:41,310 [INFO] new_opacus_codex.train_steps: epoch=4 step=665 loss=8.3040
200
+ 2026-04-04 02:10:22,823 [INFO] new_opacus_codex.train_steps: epoch=4 step=670 loss=8.4839
201
+ 2026-04-04 02:10:37,241 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=670 eval_loss=7.9897 duration_sec=14.41
202
+ 2026-04-04 02:11:20,437 [INFO] new_opacus_codex.train_steps: epoch=4 step=675 loss=8.4712
203
+ 2026-04-04 02:12:03,542 [INFO] new_opacus_codex.train_steps: epoch=4 step=680 loss=8.3963
204
+ 2026-04-04 02:12:17,785 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=680 eval_loss=7.9754 duration_sec=14.24
205
+ 2026-04-04 02:12:59,983 [INFO] new_opacus_codex.train_steps: epoch=4 step=685 loss=8.2955
206
+ 2026-04-04 02:13:43,191 [INFO] new_opacus_codex.train_steps: epoch=4 step=690 loss=8.3325
207
+ 2026-04-04 02:13:56,976 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=690 eval_loss=7.9610 duration_sec=13.77
208
+ 2026-04-04 02:14:40,305 [INFO] new_opacus_codex.train_steps: epoch=4 step=695 loss=8.4914
209
+ 2026-04-04 02:15:22,502 [INFO] new_opacus_codex.train_steps: epoch=4 step=700 loss=8.4728
210
+ 2026-04-04 02:15:36,290 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=700 eval_loss=7.9464 duration_sec=13.77
211
+ 2026-04-04 02:16:18,313 [INFO] new_opacus_codex.train_steps: epoch=4 step=705 loss=8.3303
212
+ 2026-04-04 02:17:01,366 [INFO] new_opacus_codex.train_steps: epoch=4 step=710 loss=8.3862
213
+ 2026-04-04 02:17:15,147 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=710 eval_loss=7.9326 duration_sec=13.75
214
+ 2026-04-04 02:17:58,426 [INFO] new_opacus_codex.train_steps: epoch=4 step=715 loss=8.3278
215
+ 2026-04-04 02:18:41,851 [INFO] new_opacus_codex.train_steps: epoch=4 step=720 loss=8.3386
216
+ 2026-04-04 02:18:55,645 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=720 eval_loss=7.9223 duration_sec=13.78
217
+ 2026-04-04 02:19:39,096 [INFO] new_opacus_codex.train_steps: epoch=4 step=725 loss=8.3392
218
+ 2026-04-04 02:20:22,851 [INFO] new_opacus_codex.train_steps: epoch=4 step=730 loss=8.2772
219
+ 2026-04-04 02:20:36,679 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=4 step=730 eval_loss=7.9113 duration_sec=13.82
220
+ 2026-04-04 02:21:20,307 [INFO] new_opacus_codex.train_steps: epoch=4 step=735 loss=8.3190
221
+ 2026-04-04 02:22:22,434 [INFO] new_opacus_codex.train_steps: epoch=5 step=740 loss=8.4833
222
+ 2026-04-04 02:22:36,286 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=740 eval_loss=7.9005 duration_sec=13.82
223
+ 2026-04-04 02:23:18,428 [INFO] new_opacus_codex.train_steps: epoch=5 step=745 loss=8.5397
224
+ 2026-04-04 02:24:00,998 [INFO] new_opacus_codex.train_steps: epoch=5 step=750 loss=8.2904
225
+ 2026-04-04 02:24:14,813 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=750 eval_loss=7.8903 duration_sec=13.80
226
+ 2026-04-04 02:24:58,245 [INFO] new_opacus_codex.train_steps: epoch=5 step=755 loss=8.1349
227
+ 2026-04-04 02:25:41,058 [INFO] new_opacus_codex.train_steps: epoch=5 step=760 loss=8.3589
228
+ 2026-04-04 02:25:54,900 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=760 eval_loss=7.8781 duration_sec=13.82
229
+ 2026-04-04 02:26:38,332 [INFO] new_opacus_codex.train_steps: epoch=5 step=765 loss=8.3706
230
+ 2026-04-04 02:27:20,376 [INFO] new_opacus_codex.train_steps: epoch=5 step=770 loss=8.2894
231
+ 2026-04-04 02:27:34,145 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=770 eval_loss=7.8679 duration_sec=13.76
232
+ 2026-04-04 02:28:17,207 [INFO] new_opacus_codex.train_steps: epoch=5 step=775 loss=8.3295
233
+ 2026-04-04 02:29:00,208 [INFO] new_opacus_codex.train_steps: epoch=5 step=780 loss=8.4758
234
+ 2026-04-04 02:29:14,483 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=780 eval_loss=7.8575 duration_sec=14.26
235
+ 2026-04-04 02:29:57,416 [INFO] new_opacus_codex.train_steps: epoch=5 step=785 loss=8.4437
236
+ 2026-04-04 02:30:40,304 [INFO] new_opacus_codex.train_steps: epoch=5 step=790 loss=8.3511
237
+ 2026-04-04 02:30:54,095 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=790 eval_loss=7.8431 duration_sec=13.78
238
+ 2026-04-04 02:31:37,938 [INFO] new_opacus_codex.train_steps: epoch=5 step=795 loss=8.3924
239
+ 2026-04-04 02:32:20,762 [INFO] new_opacus_codex.train_steps: epoch=5 step=800 loss=8.5000
240
+ 2026-04-04 02:32:35,170 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=800 eval_loss=7.8293 duration_sec=14.39
241
+ 2026-04-04 02:33:17,398 [INFO] new_opacus_codex.train_steps: epoch=5 step=805 loss=8.4957
242
+ 2026-04-04 02:34:00,725 [INFO] new_opacus_codex.train_steps: epoch=5 step=810 loss=8.3498
243
+ 2026-04-04 02:34:14,531 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=810 eval_loss=7.8180 duration_sec=13.79
244
+ 2026-04-04 02:34:56,771 [INFO] new_opacus_codex.train_steps: epoch=5 step=815 loss=8.2656
245
+ 2026-04-04 02:35:39,948 [INFO] new_opacus_codex.train_steps: epoch=5 step=820 loss=8.0829
246
+ 2026-04-04 02:35:53,893 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=820 eval_loss=7.8090 duration_sec=13.92
247
+ 2026-04-04 02:36:36,282 [INFO] new_opacus_codex.train_steps: epoch=5 step=825 loss=8.0488
248
+ 2026-04-04 02:37:19,465 [INFO] new_opacus_codex.train_steps: epoch=5 step=830 loss=8.1209
249
+ 2026-04-04 02:37:33,284 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=830 eval_loss=7.7979 duration_sec=13.80
250
+ 2026-04-04 02:38:18,004 [INFO] new_opacus_codex.train_steps: epoch=5 step=835 loss=7.9800
251
+ 2026-04-04 02:39:00,928 [INFO] new_opacus_codex.train_steps: epoch=5 step=840 loss=8.1144
252
+ 2026-04-04 02:39:14,752 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=840 eval_loss=7.7864 duration_sec=13.81
253
+ 2026-04-04 02:39:57,111 [INFO] new_opacus_codex.train_steps: epoch=5 step=845 loss=8.1965
254
+ 2026-04-04 02:40:39,546 [INFO] new_opacus_codex.train_steps: epoch=5 step=850 loss=8.1048
255
+ 2026-04-04 02:40:53,708 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=850 eval_loss=7.7754 duration_sec=14.16
256
+ 2026-04-04 02:41:37,328 [INFO] new_opacus_codex.train_steps: epoch=5 step=855 loss=8.1781
257
+ 2026-04-04 02:42:19,987 [INFO] new_opacus_codex.train_steps: epoch=5 step=860 loss=8.0520
258
+ 2026-04-04 02:42:33,797 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=860 eval_loss=7.7649 duration_sec=13.80
259
+ 2026-04-04 02:43:17,214 [INFO] new_opacus_codex.train_steps: epoch=5 step=865 loss=7.9792
260
+ 2026-04-04 02:44:00,140 [INFO] new_opacus_codex.train_steps: epoch=5 step=870 loss=8.1016
261
+ 2026-04-04 02:44:13,948 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=870 eval_loss=7.7557 duration_sec=13.78
262
+ 2026-04-04 02:44:57,688 [INFO] new_opacus_codex.train_steps: epoch=5 step=875 loss=8.1375
263
+ 2026-04-04 02:45:41,818 [INFO] new_opacus_codex.train_steps: epoch=5 step=880 loss=8.1276
264
+ 2026-04-04 02:45:55,622 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=880 eval_loss=7.7472 duration_sec=13.79
265
+ 2026-04-04 02:46:39,695 [INFO] new_opacus_codex.train_steps: epoch=5 step=885 loss=8.1559
266
+ 2026-04-04 02:47:22,918 [INFO] new_opacus_codex.train_steps: epoch=5 step=890 loss=8.0823
267
+ 2026-04-04 02:47:36,851 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=890 eval_loss=7.7365 duration_sec=13.92
268
+ 2026-04-04 02:48:21,723 [INFO] new_opacus_codex.train_steps: epoch=5 step=895 loss=8.0371
269
+ 2026-04-04 02:49:04,909 [INFO] new_opacus_codex.train_steps: epoch=5 step=900 loss=8.1086
270
+ 2026-04-04 02:49:19,177 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=900 eval_loss=7.7266 duration_sec=14.25
271
+ 2026-04-04 02:50:02,605 [INFO] new_opacus_codex.train_steps: epoch=5 step=905 loss=8.3246
272
+ 2026-04-04 02:50:47,607 [INFO] new_opacus_codex.train_steps: epoch=5 step=910 loss=8.2274
273
+ 2026-04-04 02:51:01,603 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=910 eval_loss=7.7166 duration_sec=13.98
274
+ 2026-04-04 02:51:45,538 [INFO] new_opacus_codex.train_steps: epoch=5 step=915 loss=8.0968
275
+ 2026-04-04 02:52:21,737 [INFO] new_opacus_codex.train_steps: epoch=5 step=920 loss=8.2959
276
+ 2026-04-04 02:52:36,444 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=5 step=920 eval_loss=7.7061 duration_sec=14.69
277
+ 2026-04-04 02:53:34,218 [INFO] new_opacus_codex.train_steps: epoch=6 step=925 loss=7.9518
278
+ 2026-04-04 02:54:18,316 [INFO] new_opacus_codex.train_steps: epoch=6 step=930 loss=8.0396
279
+ 2026-04-04 02:54:32,261 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=930 eval_loss=7.6963 duration_sec=13.93
280
+ 2026-04-04 02:55:16,634 [INFO] new_opacus_codex.train_steps: epoch=6 step=935 loss=8.0651
281
+ 2026-04-04 02:56:01,273 [INFO] new_opacus_codex.train_steps: epoch=6 step=940 loss=8.1868
282
+ 2026-04-04 02:56:15,297 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=940 eval_loss=7.6861 duration_sec=14.00
283
+ 2026-04-04 02:56:59,405 [INFO] new_opacus_codex.train_steps: epoch=6 step=945 loss=8.1514
284
+ 2026-04-04 02:57:42,657 [INFO] new_opacus_codex.train_steps: epoch=6 step=950 loss=8.0463
285
+ 2026-04-04 02:57:56,531 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=950 eval_loss=7.6773 duration_sec=13.86
286
+ 2026-04-04 02:58:39,439 [INFO] new_opacus_codex.train_steps: epoch=6 step=955 loss=8.0333
287
+ 2026-04-04 02:59:25,053 [INFO] new_opacus_codex.train_steps: epoch=6 step=960 loss=8.0159
288
+ 2026-04-04 02:59:39,007 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=960 eval_loss=7.6684 duration_sec=13.94
289
+ 2026-04-04 03:00:24,043 [INFO] new_opacus_codex.train_steps: epoch=6 step=965 loss=8.1489
290
+ 2026-04-04 03:01:08,260 [INFO] new_opacus_codex.train_steps: epoch=6 step=970 loss=8.1264
291
+ 2026-04-04 03:01:22,232 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=970 eval_loss=7.6584 duration_sec=13.96
292
+ 2026-04-04 03:02:06,718 [INFO] new_opacus_codex.train_steps: epoch=6 step=975 loss=8.1535
293
+ 2026-04-04 03:02:50,024 [INFO] new_opacus_codex.train_steps: epoch=6 step=980 loss=8.0353
294
+ 2026-04-04 03:03:04,096 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=980 eval_loss=7.6495 duration_sec=14.06
295
+ 2026-04-04 03:03:48,737 [INFO] new_opacus_codex.train_steps: epoch=6 step=985 loss=7.7855
296
+ 2026-04-04 03:04:33,050 [INFO] new_opacus_codex.train_steps: epoch=6 step=990 loss=8.0162
297
+ 2026-04-04 03:04:47,158 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=990 eval_loss=7.6408 duration_sec=14.10
298
+ 2026-04-04 03:05:30,555 [INFO] new_opacus_codex.train_steps: epoch=6 step=995 loss=8.2446
299
+ 2026-04-04 03:06:14,112 [INFO] new_opacus_codex.train_steps: epoch=6 step=1000 loss=8.1190
300
+ 2026-04-04 03:06:28,103 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=1000 eval_loss=7.6325 duration_sec=13.97
301
+ 2026-04-04 03:07:11,977 [INFO] new_opacus_codex.train_steps: epoch=6 step=1005 loss=8.0308
302
+ 2026-04-04 03:07:56,486 [INFO] new_opacus_codex.train_steps: epoch=6 step=1010 loss=8.1305
303
+ 2026-04-04 03:08:10,513 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=1010 eval_loss=7.6263 duration_sec=14.01
304
+ 2026-04-04 03:08:55,768 [INFO] new_opacus_codex.train_steps: epoch=6 step=1015 loss=8.2296
305
+ 2026-04-04 03:09:39,029 [INFO] new_opacus_codex.train_steps: epoch=6 step=1020 loss=8.1635
306
+ 2026-04-04 03:09:53,014 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=1020 eval_loss=7.6193 duration_sec=13.97
307
+ 2026-04-04 03:10:37,340 [INFO] new_opacus_codex.train_steps: epoch=6 step=1025 loss=8.0311
308
+ 2026-04-04 03:11:21,859 [INFO] new_opacus_codex.train_steps: epoch=6 step=1030 loss=7.8382
309
+ 2026-04-04 03:11:36,203 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=1030 eval_loss=7.6122 duration_sec=14.33
310
+ 2026-04-04 03:12:20,880 [INFO] new_opacus_codex.train_steps: epoch=6 step=1035 loss=8.1024
311
+ 2026-04-04 03:13:06,050 [INFO] new_opacus_codex.train_steps: epoch=6 step=1040 loss=8.2337
312
+ 2026-04-04 03:13:20,015 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=1040 eval_loss=7.6052 duration_sec=13.95
313
+ 2026-04-04 03:14:04,740 [INFO] new_opacus_codex.train_steps: epoch=6 step=1045 loss=8.0441
314
+ 2026-04-04 03:14:49,316 [INFO] new_opacus_codex.train_steps: epoch=6 step=1050 loss=8.0393
315
+ 2026-04-04 03:15:03,261 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=1050 eval_loss=7.5991 duration_sec=13.92
316
+ 2026-04-04 03:15:44,487 [INFO] new_opacus_codex.train_steps: epoch=6 step=1055 loss=8.0681
317
+ 2026-04-04 03:16:29,018 [INFO] new_opacus_codex.train_steps: epoch=6 step=1060 loss=8.1620
318
+ 2026-04-04 03:16:42,991 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=1060 eval_loss=7.5938 duration_sec=13.96
319
+ 2026-04-04 03:17:26,688 [INFO] new_opacus_codex.train_steps: epoch=6 step=1065 loss=8.1887
320
+ 2026-04-04 03:18:10,862 [INFO] new_opacus_codex.train_steps: epoch=6 step=1070 loss=8.1010
321
+ 2026-04-04 03:18:24,876 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=1070 eval_loss=7.5871 duration_sec=14.00
322
+ 2026-04-04 03:19:08,479 [INFO] new_opacus_codex.train_steps: epoch=6 step=1075 loss=8.0485
323
+ 2026-04-04 03:19:52,719 [INFO] new_opacus_codex.train_steps: epoch=6 step=1080 loss=7.9842
324
+ 2026-04-04 03:20:06,731 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=1080 eval_loss=7.5816 duration_sec=14.00
325
+ 2026-04-04 03:20:49,993 [INFO] new_opacus_codex.train_steps: epoch=6 step=1085 loss=7.8979
326
+ 2026-04-04 03:21:33,945 [INFO] new_opacus_codex.train_steps: epoch=6 step=1090 loss=7.9975
327
+ 2026-04-04 03:21:47,918 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=1090 eval_loss=7.5764 duration_sec=13.96
328
+ 2026-04-04 03:22:31,671 [INFO] new_opacus_codex.train_steps: epoch=6 step=1095 loss=8.0491
329
+ 2026-04-04 03:23:17,102 [INFO] new_opacus_codex.train_steps: epoch=6 step=1100 loss=8.0233
330
+ 2026-04-04 03:23:31,427 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=6 step=1100 eval_loss=7.5709 duration_sec=14.31
331
+ 2026-04-04 03:24:33,669 [INFO] new_opacus_codex.train_steps: epoch=7 step=1105 loss=7.6577
332
+ 2026-04-04 03:25:20,263 [INFO] new_opacus_codex.train_steps: epoch=7 step=1110 loss=7.9462
333
+ 2026-04-04 03:25:34,131 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1110 eval_loss=7.5656 duration_sec=13.86
334
+ 2026-04-04 03:26:17,589 [INFO] new_opacus_codex.train_steps: epoch=7 step=1115 loss=7.9321
335
+ 2026-04-04 03:27:01,291 [INFO] new_opacus_codex.train_steps: epoch=7 step=1120 loss=7.9827
336
+ 2026-04-04 03:27:15,203 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1120 eval_loss=7.5606 duration_sec=13.90
337
+ 2026-04-04 03:27:58,629 [INFO] new_opacus_codex.train_steps: epoch=7 step=1125 loss=8.0301
338
+ 2026-04-04 03:28:41,518 [INFO] new_opacus_codex.train_steps: epoch=7 step=1130 loss=7.8712
339
+ 2026-04-04 03:28:55,423 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1130 eval_loss=7.5550 duration_sec=13.89
340
+ 2026-04-04 03:29:38,358 [INFO] new_opacus_codex.train_steps: epoch=7 step=1135 loss=7.8581
341
+ 2026-04-04 03:30:22,427 [INFO] new_opacus_codex.train_steps: epoch=7 step=1140 loss=7.9740
342
+ 2026-04-04 03:30:36,274 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1140 eval_loss=7.5497 duration_sec=13.83
343
+ 2026-04-04 03:31:18,590 [INFO] new_opacus_codex.train_steps: epoch=7 step=1145 loss=7.9829
344
+ 2026-04-04 03:32:01,788 [INFO] new_opacus_codex.train_steps: epoch=7 step=1150 loss=7.8215
345
+ 2026-04-04 03:32:15,635 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1150 eval_loss=7.5424 duration_sec=13.83
346
+ 2026-04-04 03:33:00,116 [INFO] new_opacus_codex.train_steps: epoch=7 step=1155 loss=7.8236
347
+ 2026-04-04 03:33:43,811 [INFO] new_opacus_codex.train_steps: epoch=7 step=1160 loss=8.0157
348
+ 2026-04-04 03:33:57,653 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1160 eval_loss=7.5370 duration_sec=13.82
349
+ 2026-04-04 03:34:40,444 [INFO] new_opacus_codex.train_steps: epoch=7 step=1165 loss=8.0337
350
+ 2026-04-04 03:35:24,890 [INFO] new_opacus_codex.train_steps: epoch=7 step=1170 loss=7.8524
351
+ 2026-04-04 03:35:38,955 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1170 eval_loss=7.5323 duration_sec=14.05
352
+ 2026-04-04 03:36:22,272 [INFO] new_opacus_codex.train_steps: epoch=7 step=1175 loss=7.8862
353
+ 2026-04-04 03:37:05,633 [INFO] new_opacus_codex.train_steps: epoch=7 step=1180 loss=8.0950
354
+ 2026-04-04 03:37:19,491 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1180 eval_loss=7.5269 duration_sec=13.85
355
+ 2026-04-04 03:38:03,195 [INFO] new_opacus_codex.train_steps: epoch=7 step=1185 loss=8.1494
356
+ 2026-04-04 03:38:46,724 [INFO] new_opacus_codex.train_steps: epoch=7 step=1190 loss=7.9626
357
+ 2026-04-04 03:39:01,016 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1190 eval_loss=7.5220 duration_sec=14.27
358
+ 2026-04-04 03:39:43,683 [INFO] new_opacus_codex.train_steps: epoch=7 step=1195 loss=7.7957
359
+ 2026-04-04 03:40:26,815 [INFO] new_opacus_codex.train_steps: epoch=7 step=1200 loss=7.9577
360
+ 2026-04-04 03:40:40,738 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1200 eval_loss=7.5165 duration_sec=13.92
361
+ 2026-04-04 03:41:23,845 [INFO] new_opacus_codex.train_steps: epoch=7 step=1205 loss=8.0731
362
+ 2026-04-04 03:42:06,387 [INFO] new_opacus_codex.train_steps: epoch=7 step=1210 loss=8.0248
363
+ 2026-04-04 03:42:20,420 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1210 eval_loss=7.5110 duration_sec=14.02
364
+ 2026-04-04 03:43:04,396 [INFO] new_opacus_codex.train_steps: epoch=7 step=1215 loss=8.0109
365
+ 2026-04-04 03:43:47,544 [INFO] new_opacus_codex.train_steps: epoch=7 step=1220 loss=7.9315
366
+ 2026-04-04 03:44:02,006 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1220 eval_loss=7.5064 duration_sec=14.45
367
+ 2026-04-04 03:44:45,915 [INFO] new_opacus_codex.train_steps: epoch=7 step=1225 loss=7.9213
368
+ 2026-04-04 03:45:27,907 [INFO] new_opacus_codex.train_steps: epoch=7 step=1230 loss=8.0083
369
+ 2026-04-04 03:45:41,986 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1230 eval_loss=7.5019 duration_sec=14.07
370
+ 2026-04-04 03:46:25,141 [INFO] new_opacus_codex.train_steps: epoch=7 step=1235 loss=7.9879
371
+ 2026-04-04 03:47:08,756 [INFO] new_opacus_codex.train_steps: epoch=7 step=1240 loss=7.8809
372
+ 2026-04-04 03:47:22,837 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1240 eval_loss=7.4976 duration_sec=13.90
373
+ 2026-04-04 03:48:04,135 [INFO] new_opacus_codex.train_steps: epoch=7 step=1245 loss=7.8408
374
+ 2026-04-04 03:48:47,625 [INFO] new_opacus_codex.train_steps: epoch=7 step=1250 loss=7.7889
375
+ 2026-04-04 03:49:01,546 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1250 eval_loss=7.4931 duration_sec=13.91
376
+ 2026-04-04 03:49:45,538 [INFO] new_opacus_codex.train_steps: epoch=7 step=1255 loss=7.7489
377
+ 2026-04-04 03:50:28,980 [INFO] new_opacus_codex.train_steps: epoch=7 step=1260 loss=7.8477
378
+ 2026-04-04 03:50:42,891 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1260 eval_loss=7.4875 duration_sec=13.89
379
+ 2026-04-04 03:51:26,813 [INFO] new_opacus_codex.train_steps: epoch=7 step=1265 loss=7.9028
380
+ 2026-04-04 03:52:10,971 [INFO] new_opacus_codex.train_steps: epoch=7 step=1270 loss=7.8012
381
+ 2026-04-04 03:52:24,855 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1270 eval_loss=7.4830 duration_sec=13.86
382
+ 2026-04-04 03:53:07,299 [INFO] new_opacus_codex.train_steps: epoch=7 step=1275 loss=7.8290
383
+ 2026-04-04 03:53:50,855 [INFO] new_opacus_codex.train_steps: epoch=7 step=1280 loss=7.9845
384
+ 2026-04-04 03:54:04,834 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=7 step=1280 eval_loss=7.4788 duration_sec=13.96
385
+ 2026-04-04 03:54:47,416 [INFO] new_opacus_codex.train_steps: epoch=7 step=1285 loss=7.9911
386
+ 2026-04-04 03:55:50,327 [INFO] new_opacus_codex.train_steps: epoch=8 step=1290 loss=7.9830
387
+ 2026-04-04 03:56:04,184 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1290 eval_loss=7.4748 duration_sec=13.85
388
+ 2026-04-04 03:56:48,524 [INFO] new_opacus_codex.train_steps: epoch=8 step=1295 loss=8.0071
389
+ 2026-04-04 03:57:32,409 [INFO] new_opacus_codex.train_steps: epoch=8 step=1300 loss=7.9990
390
+ 2026-04-04 03:57:46,717 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1300 eval_loss=7.4710 duration_sec=14.29
391
+ 2026-04-04 03:58:30,043 [INFO] new_opacus_codex.train_steps: epoch=8 step=1305 loss=7.9741
392
+ 2026-04-04 03:59:12,220 [INFO] new_opacus_codex.train_steps: epoch=8 step=1310 loss=7.9464
393
+ 2026-04-04 03:59:26,111 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1310 eval_loss=7.4674 duration_sec=13.88
394
+ 2026-04-04 04:00:08,779 [INFO] new_opacus_codex.train_steps: epoch=8 step=1315 loss=7.8838
395
+ 2026-04-04 04:00:52,143 [INFO] new_opacus_codex.train_steps: epoch=8 step=1320 loss=7.9046
396
+ 2026-04-04 04:01:05,996 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1320 eval_loss=7.4638 duration_sec=13.84
397
+ 2026-04-04 04:01:50,835 [INFO] new_opacus_codex.train_steps: epoch=8 step=1325 loss=7.9094
398
+ 2026-04-04 04:02:34,209 [INFO] new_opacus_codex.train_steps: epoch=8 step=1330 loss=7.9835
399
+ 2026-04-04 04:02:48,404 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1330 eval_loss=7.4606 duration_sec=14.18
400
+ 2026-04-04 04:03:32,515 [INFO] new_opacus_codex.train_steps: epoch=8 step=1335 loss=7.9677
401
+ 2026-04-04 04:04:14,387 [INFO] new_opacus_codex.train_steps: epoch=8 step=1340 loss=7.9598
402
+ 2026-04-04 04:04:28,265 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1340 eval_loss=7.4575 duration_sec=13.86
403
+ 2026-04-04 04:05:12,476 [INFO] new_opacus_codex.train_steps: epoch=8 step=1345 loss=7.9358
404
+ 2026-04-04 04:05:55,037 [INFO] new_opacus_codex.train_steps: epoch=8 step=1350 loss=7.9013
405
+ 2026-04-04 04:06:09,408 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1350 eval_loss=7.4545 duration_sec=14.36
406
+ 2026-04-04 04:06:52,741 [INFO] new_opacus_codex.train_steps: epoch=8 step=1355 loss=7.9375
407
+ 2026-04-04 04:07:36,616 [INFO] new_opacus_codex.train_steps: epoch=8 step=1360 loss=7.9138
408
+ 2026-04-04 04:07:50,695 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1360 eval_loss=7.4505 duration_sec=14.07
409
+ 2026-04-04 04:08:33,789 [INFO] new_opacus_codex.train_steps: epoch=8 step=1365 loss=7.9890
410
+ 2026-04-04 04:09:16,838 [INFO] new_opacus_codex.train_steps: epoch=8 step=1370 loss=7.9265
411
+ 2026-04-04 04:09:30,701 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1370 eval_loss=7.4466 duration_sec=13.85
412
+ 2026-04-04 04:10:13,724 [INFO] new_opacus_codex.train_steps: epoch=8 step=1375 loss=7.8448
413
+ 2026-04-04 04:10:57,538 [INFO] new_opacus_codex.train_steps: epoch=8 step=1380 loss=7.8301
414
+ 2026-04-04 04:11:11,442 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1380 eval_loss=7.4432 duration_sec=13.90
415
+ 2026-04-04 04:11:54,960 [INFO] new_opacus_codex.train_steps: epoch=8 step=1385 loss=7.8886
416
+ 2026-04-04 04:12:38,430 [INFO] new_opacus_codex.train_steps: epoch=8 step=1390 loss=8.0215
417
+ 2026-04-04 04:12:52,573 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1390 eval_loss=7.4393 duration_sec=14.13
418
+ 2026-04-04 04:13:35,438 [INFO] new_opacus_codex.train_steps: epoch=8 step=1395 loss=7.9096
419
+ 2026-04-04 04:14:16,922 [INFO] new_opacus_codex.train_steps: epoch=8 step=1400 loss=7.8305
420
+ 2026-04-04 04:14:30,782 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1400 eval_loss=7.4354 duration_sec=13.84
421
+ 2026-04-04 04:15:14,431 [INFO] new_opacus_codex.train_steps: epoch=8 step=1405 loss=7.7752
422
+ 2026-04-04 04:15:58,156 [INFO] new_opacus_codex.train_steps: epoch=8 step=1410 loss=7.7383
423
+ 2026-04-04 04:16:12,290 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1410 eval_loss=7.4321 duration_sec=14.12
424
+ 2026-04-04 04:16:55,026 [INFO] new_opacus_codex.train_steps: epoch=8 step=1415 loss=7.7265
425
+ 2026-04-04 04:17:38,675 [INFO] new_opacus_codex.train_steps: epoch=8 step=1420 loss=7.7253
426
+ 2026-04-04 04:17:52,552 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1420 eval_loss=7.4285 duration_sec=13.87
427
+ 2026-04-04 04:18:36,383 [INFO] new_opacus_codex.train_steps: epoch=8 step=1425 loss=7.8654
428
+ 2026-04-04 04:19:18,678 [INFO] new_opacus_codex.train_steps: epoch=8 step=1430 loss=7.8701
429
+ 2026-04-04 04:19:32,530 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1430 eval_loss=7.4250 duration_sec=13.84
430
+ 2026-04-04 04:20:15,240 [INFO] new_opacus_codex.train_steps: epoch=8 step=1435 loss=7.9211
431
+ 2026-04-04 04:20:58,926 [INFO] new_opacus_codex.train_steps: epoch=8 step=1440 loss=7.8688
432
+ 2026-04-04 04:21:12,829 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1440 eval_loss=7.4220 duration_sec=13.89
433
+ 2026-04-04 04:21:56,666 [INFO] new_opacus_codex.train_steps: epoch=8 step=1445 loss=7.7763
434
+ 2026-04-04 04:22:40,251 [INFO] new_opacus_codex.train_steps: epoch=8 step=1450 loss=7.8217
435
+ 2026-04-04 04:22:54,490 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1450 eval_loss=7.4184 duration_sec=14.23
436
+ 2026-04-04 04:23:35,932 [INFO] new_opacus_codex.train_steps: epoch=8 step=1455 loss=7.7890
437
+ 2026-04-04 04:24:20,271 [INFO] new_opacus_codex.train_steps: epoch=8 step=1460 loss=7.8089
438
+ 2026-04-04 04:24:34,148 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1460 eval_loss=7.4153 duration_sec=13.86
439
+ 2026-04-04 04:25:16,931 [INFO] new_opacus_codex.train_steps: epoch=8 step=1465 loss=7.8729
440
+ 2026-04-04 04:25:59,329 [INFO] new_opacus_codex.train_steps: epoch=8 step=1470 loss=7.8296
441
+ 2026-04-04 04:26:13,189 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=8 step=1470 eval_loss=7.4122 duration_sec=13.85
qwen3-8b-base/base/adapter/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen/Qwen3-8B-Base
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:Qwen/Qwen3-8B-Base
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.18.1
qwen3-8b-base/base/adapter/adapter_config.json ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": true,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 32,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "megatron_config": null,
23
+ "megatron_core": "megatron.core",
24
+ "modules_to_save": [
25
+ "lm_head",
26
+ "embed_tokens"
27
+ ],
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.18.1",
30
+ "qalora_group_size": 16,
31
+ "r": 16,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "v_proj",
36
+ "o_proj",
37
+ "q_proj",
38
+ "k_proj"
39
+ ],
40
+ "target_parameters": null,
41
+ "task_type": "CAUSAL_LM",
42
+ "trainable_token_indices": null,
43
+ "use_dora": false,
44
+ "use_qalora": false,
45
+ "use_rslora": false
46
+ }
qwen3-8b-base/base/tokenizer/chat_template.jinja ADDED
@@ -0,0 +1,85 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0].role == 'system' %}
4
+ {{- messages[0].content + '\n\n' }}
5
+ {%- endif %}
6
+ {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
7
+ {%- for tool in tools %}
8
+ {{- "\n" }}
9
+ {{- tool | tojson }}
10
+ {%- endfor %}
11
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
+ {%- else %}
13
+ {%- if messages[0].role == 'system' %}
14
+ {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
+ {%- endif %}
16
+ {%- endif %}
17
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
+ {%- for message in messages[::-1] %}
19
+ {%- set index = (messages|length - 1) - loop.index0 %}
20
+ {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
21
+ {%- set ns.multi_step_tool = false %}
22
+ {%- set ns.last_query_index = index %}
23
+ {%- endif %}
24
+ {%- endfor %}
25
+ {%- for message in messages %}
26
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
27
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
28
+ {%- elif message.role == "assistant" %}
29
+ {%- set content = message.content %}
30
+ {%- set reasoning_content = '' %}
31
+ {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
32
+ {%- set reasoning_content = message.reasoning_content %}
33
+ {%- else %}
34
+ {%- if '</think>' in message.content %}
35
+ {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
36
+ {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
37
+ {%- endif %}
38
+ {%- endif %}
39
+ {%- if loop.index0 > ns.last_query_index %}
40
+ {%- if loop.last or (not loop.last and reasoning_content) %}
41
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
42
+ {%- else %}
43
+ {{- '<|im_start|>' + message.role + '\n' + content }}
44
+ {%- endif %}
45
+ {%- else %}
46
+ {{- '<|im_start|>' + message.role + '\n' + content }}
47
+ {%- endif %}
48
+ {%- if message.tool_calls %}
49
+ {%- for tool_call in message.tool_calls %}
50
+ {%- if (loop.first and content) or (not loop.first) %}
51
+ {{- '\n' }}
52
+ {%- endif %}
53
+ {%- if tool_call.function %}
54
+ {%- set tool_call = tool_call.function %}
55
+ {%- endif %}
56
+ {{- '<tool_call>\n{"name": "' }}
57
+ {{- tool_call.name }}
58
+ {{- '", "arguments": ' }}
59
+ {%- if tool_call.arguments is string %}
60
+ {{- tool_call.arguments }}
61
+ {%- else %}
62
+ {{- tool_call.arguments | tojson }}
63
+ {%- endif %}
64
+ {{- '}\n</tool_call>' }}
65
+ {%- endfor %}
66
+ {%- endif %}
67
+ {{- '<|im_end|>\n' }}
68
+ {%- elif message.role == "tool" %}
69
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
70
+ {{- '<|im_start|>user' }}
71
+ {%- endif %}
72
+ {{- '\n<tool_response>\n' }}
73
+ {{- message.content }}
74
+ {{- '\n</tool_response>' }}
75
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
76
+ {{- '<|im_end|>\n' }}
77
+ {%- endif %}
78
+ {%- endif %}
79
+ {%- endfor %}
80
+ {%- if add_generation_prompt %}
81
+ {{- '<|im_start|>assistant\n' }}
82
+ {%- if enable_thinking is defined and enable_thinking is false %}
83
+ {{- '<think>\n\n</think>\n\n' }}
84
+ {%- endif %}
85
+ {%- endif %}
qwen3-8b-base/base/tokenizer/tokenizer_config.json ADDED
@@ -0,0 +1,516 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|endoftext|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": {
9
+ "extra_special_token_0": "865331112869",
10
+ "extra_special_token_1": "569765693871",
11
+ "extra_special_token_2": "485177821815",
12
+ "extra_special_token_3": "135441121756",
13
+ "extra_special_token_4": "367459894796",
14
+ "extra_special_token_5": "877482678543",
15
+ "extra_special_token_6": "457919547633",
16
+ "extra_special_token_7": "765474393376",
17
+ "extra_special_token_8": "114848338811",
18
+ "extra_special_token_9": "746285987371",
19
+ "extra_special_token_10": "649291669397",
20
+ "extra_special_token_11": "927914615679",
21
+ "extra_special_token_12": "445925149649",
22
+ "extra_special_token_13": "691587454538",
23
+ "extra_special_token_14": "143777992227",
24
+ "extra_special_token_15": "997981281989",
25
+ "extra_special_token_16": "425949483533",
26
+ "extra_special_token_17": "982993456429",
27
+ "extra_special_token_18": "718726519731",
28
+ "extra_special_token_19": "172599315861",
29
+ "extra_special_token_20": "643489267333",
30
+ "extra_special_token_21": "282322838685",
31
+ "extra_special_token_22": "781653545886",
32
+ "extra_special_token_23": "796415361892",
33
+ "extra_special_token_24": "841991688488",
34
+ "extra_special_token_25": "211411365397",
35
+ "extra_special_token_26": "698218415444",
36
+ "extra_special_token_27": "355977139358",
37
+ "extra_special_token_28": "682564697312",
38
+ "extra_special_token_29": "383837596997",
39
+ "extra_special_token_30": "689362171782",
40
+ "extra_special_token_31": "749966767285",
41
+ "extra_special_token_32": "753159165157",
42
+ "extra_special_token_33": "795693824762",
43
+ "extra_special_token_34": "669689115557",
44
+ "extra_special_token_35": "327491773134",
45
+ "extra_special_token_36": "983569279932",
46
+ "extra_special_token_37": "612128769512",
47
+ "extra_special_token_38": "374327157578",
48
+ "extra_special_token_39": "311632789559",
49
+ "extra_special_token_40": "523918658846",
50
+ "extra_special_token_41": "765981581453",
51
+ "extra_special_token_42": "794825141891",
52
+ "extra_special_token_43": "873898736873",
53
+ "extra_special_token_44": "447445629421",
54
+ "extra_special_token_45": "473822473819",
55
+ "extra_special_token_46": "181439694557",
56
+ "extra_special_token_47": "592538279337",
57
+ "extra_special_token_48": "668134915514",
58
+ "extra_special_token_49": "643692393748",
59
+ "extra_special_token_50": "696651276628",
60
+ "extra_special_token_51": "853859348234",
61
+ "extra_special_token_52": "778466723723",
62
+ "extra_special_token_53": "929826356991",
63
+ "extra_special_token_54": "272362973463",
64
+ "extra_special_token_55": "694235616268",
65
+ "extra_special_token_56": "281673864127",
66
+ "extra_special_token_57": "479676316326",
67
+ "extra_special_token_58": "646979124677",
68
+ "extra_special_token_59": "922327493433",
69
+ "extra_special_token_60": "883685933161",
70
+ "extra_special_token_61": "264259917554",
71
+ "extra_special_token_62": "836746273134",
72
+ "extra_special_token_63": "658481324922",
73
+ "extra_special_token_64": "481884157827",
74
+ "extra_special_token_65": "587787496812",
75
+ "extra_special_token_66": "579184949249",
76
+ "extra_special_token_67": "912193598348",
77
+ "extra_special_token_68": "529679678956",
78
+ "extra_special_token_69": "795838284624",
79
+ "extra_special_token_70": "159337222655",
80
+ "extra_special_token_71": "173781362446",
81
+ "extra_special_token_72": "773687856563",
82
+ "extra_special_token_73": "535787224917",
83
+ "extra_special_token_74": "351885857332",
84
+ "extra_special_token_75": "578827344666",
85
+ "extra_special_token_76": "198462689911",
86
+ "extra_special_token_77": "722618266242",
87
+ "extra_special_token_78": "952872416512",
88
+ "extra_special_token_79": "517778845323",
89
+ "extra_special_token_80": "749665846687",
90
+ "extra_special_token_81": "661436365453",
91
+ "extra_special_token_82": "259666844669",
92
+ "extra_special_token_83": "242851284913",
93
+ "extra_special_token_84": "514532995959",
94
+ "extra_special_token_85": "161588262349",
95
+ "extra_special_token_86": "742765629356",
96
+ "extra_special_token_87": "225164373623",
97
+ "extra_special_token_88": "676539973863",
98
+ "extra_special_token_89": "826214551218",
99
+ "extra_special_token_90": "182345464792",
100
+ "extra_special_token_91": "232776999554",
101
+ "extra_special_token_92": "337326533813",
102
+ "extra_special_token_93": "676676697292",
103
+ "extra_special_token_94": "929185622831",
104
+ "extra_special_token_95": "545512344383",
105
+ "extra_special_token_96": "499444466686",
106
+ "extra_special_token_97": "314697386682",
107
+ "extra_special_token_98": "517379856925",
108
+ "extra_special_token_99": "379557332953",
109
+ "extra_special_token_100": "614797267726",
110
+ "extra_special_token_101": "429781429464",
111
+ "extra_special_token_102": "922466849763",
112
+ "extra_special_token_103": "721737645236",
113
+ "extra_special_token_104": "479227349997",
114
+ "extra_special_token_105": "136931728327",
115
+ "extra_special_token_106": "259533577263",
116
+ "extra_special_token_107": "488538864842",
117
+ "extra_special_token_108": "937495658852",
118
+ "extra_special_token_109": "489991411364",
119
+ "extra_special_token_110": "499148455254",
120
+ "extra_special_token_111": "441373944925",
121
+ "extra_special_token_112": "899151413682",
122
+ "extra_special_token_113": "467893531755",
123
+ "extra_special_token_114": "527117488925",
124
+ "extra_special_token_115": "928335588653",
125
+ "extra_special_token_116": "374439448821",
126
+ "extra_special_token_117": "879425227932",
127
+ "extra_special_token_118": "867678158885",
128
+ "extra_special_token_119": "399749397872",
129
+ "extra_special_token_120": "129693547287",
130
+ "extra_special_token_121": "689285841825",
131
+ "extra_special_token_122": "771619544974",
132
+ "extra_special_token_123": "724883568652",
133
+ "extra_special_token_124": "516968424863",
134
+ "extra_special_token_125": "733737988257",
135
+ "extra_special_token_126": "852347289392",
136
+ "extra_special_token_127": "296953381169",
137
+ "extra_special_token_128": "377273562477",
138
+ "extra_special_token_129": "262296912232",
139
+ "extra_special_token_130": "547149832394",
140
+ "extra_special_token_131": "298464134954",
141
+ "extra_special_token_132": "216667245274",
142
+ "extra_special_token_133": "843998562287",
143
+ "extra_special_token_134": "572154333646",
144
+ "extra_special_token_135": "124589118494",
145
+ "extra_special_token_136": "841824384614",
146
+ "extra_special_token_137": "232896526252",
147
+ "extra_special_token_138": "295448593321",
148
+ "extra_special_token_139": "123741461297",
149
+ "extra_special_token_140": "653573457168",
150
+ "extra_special_token_141": "196735786156",
151
+ "extra_special_token_142": "377338713663",
152
+ "extra_special_token_143": "964342468552",
153
+ "extra_special_token_144": "586855179568",
154
+ "extra_special_token_145": "484773717614",
155
+ "extra_special_token_146": "894885246797",
156
+ "extra_special_token_147": "677896358599",
157
+ "extra_special_token_148": "848845611563",
158
+ "extra_special_token_149": "851852651677",
159
+ "extra_special_token_150": "398549545767",
160
+ "extra_special_token_151": "454244839926",
161
+ "extra_special_token_152": "799364566435",
162
+ "extra_special_token_153": "967114116556",
163
+ "extra_special_token_154": "817378986438",
164
+ "extra_special_token_155": "233795848681",
165
+ "extra_special_token_156": "824387273757",
166
+ "extra_special_token_157": "916198946615",
167
+ "extra_special_token_158": "563117729724",
168
+ "extra_special_token_159": "951794811935",
169
+ "extra_special_token_160": "374598961236",
170
+ "extra_special_token_161": "922867396683",
171
+ "extra_special_token_162": "765737843639",
172
+ "extra_special_token_163": "175469284871",
173
+ "extra_special_token_164": "231853711778",
174
+ "extra_special_token_165": "662426712668",
175
+ "extra_special_token_166": "711412347158",
176
+ "extra_special_token_167": "753466987363",
177
+ "extra_special_token_168": "513361312532",
178
+ "extra_special_token_169": "712992815957",
179
+ "extra_special_token_170": "971621888444",
180
+ "extra_special_token_171": "829235161526",
181
+ "extra_special_token_172": "585544633356",
182
+ "extra_special_token_173": "582471228164",
183
+ "extra_special_token_174": "678666359123",
184
+ "extra_special_token_175": "557533689478",
185
+ "extra_special_token_176": "632962475133",
186
+ "extra_special_token_177": "484489193824",
187
+ "extra_special_token_178": "489562189822",
188
+ "extra_special_token_179": "589547936288",
189
+ "extra_special_token_180": "363214487524",
190
+ "extra_special_token_181": "244885399387",
191
+ "extra_special_token_182": "431751228368",
192
+ "extra_special_token_183": "433581868192",
193
+ "extra_special_token_184": "486391569221",
194
+ "extra_special_token_185": "185438575221",
195
+ "extra_special_token_186": "126574388585",
196
+ "extra_special_token_187": "741757479784",
197
+ "extra_special_token_188": "529854679937",
198
+ "extra_special_token_189": "996116119839",
199
+ "extra_special_token_190": "616248973917",
200
+ "extra_special_token_191": "763531783491",
201
+ "extra_special_token_192": "955456118295",
202
+ "extra_special_token_193": "364196983365",
203
+ "extra_special_token_194": "195792996468",
204
+ "extra_special_token_195": "151859598873",
205
+ "extra_special_token_196": "399223169721",
206
+ "extra_special_token_197": "938488813964",
207
+ "extra_special_token_198": "961981959227",
208
+ "extra_special_token_199": "183368827562",
209
+ "extra_special_token_200": "533417736566",
210
+ "extra_special_token_201": "786391632558",
211
+ "extra_special_token_202": "665661658354",
212
+ "extra_special_token_203": "693281533643",
213
+ "extra_special_token_204": "475794684356",
214
+ "extra_special_token_205": "652154162978",
215
+ "extra_special_token_206": "753233719644",
216
+ "extra_special_token_207": "668514843129",
217
+ "extra_special_token_208": "819162623892",
218
+ "extra_special_token_209": "941169431859",
219
+ "extra_special_token_210": "877385381798",
220
+ "extra_special_token_211": "752644929761",
221
+ "extra_special_token_212": "881136466196",
222
+ "extra_special_token_213": "275597777299",
223
+ "extra_special_token_214": "731681792655",
224
+ "extra_special_token_215": "961133895172",
225
+ "extra_special_token_216": "864718285734",
226
+ "extra_special_token_217": "963852916563",
227
+ "extra_special_token_218": "319584985416",
228
+ "extra_special_token_219": "563365646341",
229
+ "extra_special_token_220": "811371928234",
230
+ "extra_special_token_221": "837131396371",
231
+ "extra_special_token_222": "267514771964",
232
+ "extra_special_token_223": "944513428457",
233
+ "extra_special_token_224": "117298239631",
234
+ "extra_special_token_225": "158142752582",
235
+ "extra_special_token_226": "252867443568",
236
+ "extra_special_token_227": "839269684865",
237
+ "extra_special_token_228": "612788593128",
238
+ "extra_special_token_229": "145669731981",
239
+ "extra_special_token_230": "121557291859",
240
+ "extra_special_token_231": "245416776926",
241
+ "extra_special_token_232": "799417897197",
242
+ "extra_special_token_233": "997958836435",
243
+ "extra_special_token_234": "892336777248",
244
+ "extra_special_token_235": "158929292238",
245
+ "extra_special_token_236": "581976444672",
246
+ "extra_special_token_237": "897784492783",
247
+ "extra_special_token_238": "492373714791",
248
+ "extra_special_token_239": "512659818733",
249
+ "extra_special_token_240": "881112998642",
250
+ "extra_special_token_241": "619454958782",
251
+ "extra_special_token_242": "431149748713",
252
+ "extra_special_token_243": "624221476921",
253
+ "extra_special_token_244": "125866399464",
254
+ "extra_special_token_245": "339882449689",
255
+ "extra_special_token_246": "186198784585",
256
+ "extra_special_token_247": "943193294691",
257
+ "extra_special_token_248": "955668961269",
258
+ "extra_special_token_249": "232787996724",
259
+ "extra_special_token_250": "215671314196",
260
+ "extra_special_token_251": "286173241916",
261
+ "extra_special_token_252": "745977673725",
262
+ "extra_special_token_253": "556976448182",
263
+ "extra_special_token_254": "599961512792",
264
+ "extra_special_token_255": "766294538337",
265
+ "extra_special_token_256": "934912591213",
266
+ "extra_special_token_257": "295118729589",
267
+ "extra_special_token_258": "529455466433",
268
+ "extra_special_token_259": "196119929397",
269
+ "extra_special_token_260": "379571934299",
270
+ "extra_special_token_261": "251789649997",
271
+ "extra_special_token_262": "564544131355",
272
+ "extra_special_token_263": "244371196654",
273
+ "extra_special_token_264": "384598329253",
274
+ "extra_special_token_265": "887753195844",
275
+ "extra_special_token_266": "364947325679",
276
+ "extra_special_token_267": "655517954651",
277
+ "extra_special_token_268": "673948786567",
278
+ "extra_special_token_269": "857231548835",
279
+ "extra_special_token_270": "816115936673",
280
+ "extra_special_token_271": "644234165531",
281
+ "extra_special_token_272": "182782912224",
282
+ "extra_special_token_273": "234316622259",
283
+ "extra_special_token_274": "421369185549",
284
+ "extra_special_token_275": "434632855397",
285
+ "extra_special_token_276": "921889371893",
286
+ "extra_special_token_277": "415956914763",
287
+ "extra_special_token_278": "598916996413",
288
+ "extra_special_token_279": "773671349113",
289
+ "extra_special_token_280": "952465217972",
290
+ "extra_special_token_281": "117657531962",
291
+ "extra_special_token_282": "729825168745",
292
+ "extra_special_token_283": "691315125346",
293
+ "extra_special_token_284": "768461952319",
294
+ "extra_special_token_285": "664847713559",
295
+ "extra_special_token_286": "953267689786",
296
+ "extra_special_token_287": "886464195129",
297
+ "extra_special_token_288": "824488329416",
298
+ "extra_special_token_289": "837873762491",
299
+ "extra_special_token_290": "532833541879",
300
+ "extra_special_token_291": "669183782449",
301
+ "extra_special_token_292": "941976537588",
302
+ "extra_special_token_293": "739394546916",
303
+ "extra_special_token_294": "267954879268",
304
+ "extra_special_token_295": "637551427887",
305
+ "extra_special_token_296": "217756494954",
306
+ "extra_special_token_297": "524444658383",
307
+ "extra_special_token_298": "117783274348",
308
+ "extra_special_token_299": "138218735276",
309
+ "extra_special_token_300": "814611949491",
310
+ "extra_special_token_301": "711641973413",
311
+ "extra_special_token_302": "499156317423",
312
+ "extra_special_token_303": "515856611931",
313
+ "extra_special_token_304": "454164859837",
314
+ "extra_special_token_305": "345271433112",
315
+ "extra_special_token_306": "462294118988",
316
+ "extra_special_token_307": "511785788222",
317
+ "extra_special_token_308": "497294727353",
318
+ "extra_special_token_309": "866519986723",
319
+ "extra_special_token_310": "334513529294",
320
+ "extra_special_token_311": "549946382131",
321
+ "extra_special_token_312": "284445431422",
322
+ "extra_special_token_313": "396521188476",
323
+ "extra_special_token_314": "421435255895",
324
+ "extra_special_token_315": "133373659361",
325
+ "extra_special_token_316": "322683334381",
326
+ "extra_special_token_317": "228358422847",
327
+ "extra_special_token_318": "291762694874",
328
+ "extra_special_token_319": "143182978129",
329
+ "extra_special_token_320": "511923256573",
330
+ "extra_special_token_321": "327158398268",
331
+ "extra_special_token_322": "879764613759",
332
+ "extra_special_token_323": "564395222747",
333
+ "extra_special_token_324": "451161679736",
334
+ "extra_special_token_325": "538631466654",
335
+ "extra_special_token_326": "221762325616",
336
+ "extra_special_token_327": "218391991184",
337
+ "extra_special_token_328": "322589379462",
338
+ "extra_special_token_329": "876537814263",
339
+ "extra_special_token_330": "152676556624",
340
+ "extra_special_token_331": "332522971941",
341
+ "extra_special_token_332": "884354318946",
342
+ "extra_special_token_333": "513349618943",
343
+ "extra_special_token_334": "116639746413",
344
+ "extra_special_token_335": "635185846287",
345
+ "extra_special_token_336": "993832498489",
346
+ "extra_special_token_337": "813981174797",
347
+ "extra_special_token_338": "438745114173",
348
+ "extra_special_token_339": "983493951323",
349
+ "extra_special_token_340": "724492262421",
350
+ "extra_special_token_341": "622553389126",
351
+ "extra_special_token_342": "889965243135",
352
+ "extra_special_token_343": "364492359246",
353
+ "extra_special_token_344": "154962668224",
354
+ "extra_special_token_345": "179564995814",
355
+ "extra_special_token_346": "418412875665",
356
+ "extra_special_token_347": "718951851413",
357
+ "extra_special_token_348": "699446724178",
358
+ "extra_special_token_349": "624266421831",
359
+ "extra_special_token_350": "815458725125",
360
+ "extra_special_token_351": "455423278865",
361
+ "extra_special_token_352": "393741199486",
362
+ "extra_special_token_353": "328552864359",
363
+ "extra_special_token_354": "211662639865",
364
+ "extra_special_token_355": "218784516525",
365
+ "extra_special_token_356": "762486672996",
366
+ "extra_special_token_357": "142799718159",
367
+ "extra_special_token_358": "858146415154",
368
+ "extra_special_token_359": "767858144912",
369
+ "extra_special_token_360": "571317457151",
370
+ "extra_special_token_361": "635127952696",
371
+ "extra_special_token_362": "116427191984",
372
+ "extra_special_token_363": "268921994538",
373
+ "extra_special_token_364": "523937669294",
374
+ "extra_special_token_365": "165429152138",
375
+ "extra_special_token_366": "739246183345",
376
+ "extra_special_token_367": "591464355756",
377
+ "extra_special_token_368": "212985874612",
378
+ "extra_special_token_369": "191887635211",
379
+ "extra_special_token_370": "967214577653",
380
+ "extra_special_token_371": "119342152414",
381
+ "extra_special_token_372": "946444632795",
382
+ "extra_special_token_373": "618423867817",
383
+ "extra_special_token_374": "228565148417",
384
+ "extra_special_token_375": "729116422489",
385
+ "extra_special_token_376": "527874729936",
386
+ "extra_special_token_377": "739784153482",
387
+ "extra_special_token_378": "387763951128",
388
+ "extra_special_token_379": "331369926711",
389
+ "extra_special_token_380": "562716493614",
390
+ "extra_special_token_381": "739667844957",
391
+ "extra_special_token_382": "562389434565",
392
+ "extra_special_token_383": "256497188281",
393
+ "extra_special_token_384": "859927364588",
394
+ "extra_special_token_385": "417668946583",
395
+ "extra_special_token_386": "357621613582",
396
+ "extra_special_token_387": "438435178228",
397
+ "extra_special_token_388": "485692541169",
398
+ "extra_special_token_389": "825815739116",
399
+ "extra_special_token_390": "342221452223",
400
+ "extra_special_token_391": "697747991249",
401
+ "extra_special_token_392": "716763689965",
402
+ "extra_special_token_393": "141499982867",
403
+ "extra_special_token_394": "818479319499",
404
+ "extra_special_token_395": "336813343298",
405
+ "extra_special_token_396": "594688742928",
406
+ "extra_special_token_397": "472129283475",
407
+ "extra_special_token_398": "514354144759",
408
+ "extra_special_token_399": "349249721685",
409
+ "extra_special_token_400": "546276298359",
410
+ "extra_special_token_401": "353755529131",
411
+ "extra_special_token_402": "315534574435",
412
+ "extra_special_token_403": "523723475786",
413
+ "extra_special_token_404": "215826764872",
414
+ "extra_special_token_405": "367968398551",
415
+ "extra_special_token_406": "569853653352",
416
+ "extra_special_token_407": "389715484387",
417
+ "extra_special_token_408": "293847485454",
418
+ "extra_special_token_409": "714738141818",
419
+ "extra_special_token_410": "178478368922",
420
+ "extra_special_token_411": "581493616981",
421
+ "extra_special_token_412": "589439538674",
422
+ "extra_special_token_413": "846657726193",
423
+ "extra_special_token_414": "722339992679",
424
+ "extra_special_token_415": "138154781148",
425
+ "extra_special_token_416": "757785319772",
426
+ "extra_special_token_417": "492516914298",
427
+ "extra_special_token_418": "919181521716",
428
+ "extra_special_token_419": "985781138935",
429
+ "extra_special_token_420": "476969195485",
430
+ "extra_special_token_421": "313145133463",
431
+ "extra_special_token_422": "758963111966",
432
+ "extra_special_token_423": "147541537162",
433
+ "extra_special_token_424": "557163366873",
434
+ "extra_special_token_425": "144373897488",
435
+ "extra_special_token_426": "522515164754",
436
+ "extra_special_token_427": "724964923582",
437
+ "extra_special_token_428": "284776712475",
438
+ "extra_special_token_429": "375429755114",
439
+ "extra_special_token_430": "181233596124",
440
+ "extra_special_token_431": "948585673431",
441
+ "extra_special_token_432": "243165586174",
442
+ "extra_special_token_433": "396847976144",
443
+ "extra_special_token_434": "997724962668",
444
+ "extra_special_token_435": "558837194455",
445
+ "extra_special_token_436": "163165456396",
446
+ "extra_special_token_437": "378749551722",
447
+ "extra_special_token_438": "161238482259",
448
+ "extra_special_token_439": "754978243758",
449
+ "extra_special_token_440": "195388849133",
450
+ "extra_special_token_441": "229775525672",
451
+ "extra_special_token_442": "262437452884",
452
+ "extra_special_token_443": "441377892146",
453
+ "extra_special_token_444": "451885565366",
454
+ "extra_special_token_445": "981277526855",
455
+ "extra_special_token_446": "762495822823",
456
+ "extra_special_token_447": "368763327262",
457
+ "extra_special_token_448": "757422791351",
458
+ "extra_special_token_449": "636324136426",
459
+ "extra_special_token_450": "214193645583",
460
+ "extra_special_token_451": "412843856172",
461
+ "extra_special_token_452": "179386156569",
462
+ "extra_special_token_453": "756916173536",
463
+ "extra_special_token_454": "892697125149",
464
+ "extra_special_token_455": "625334487352",
465
+ "extra_special_token_456": "941861857715",
466
+ "extra_special_token_457": "887417525236",
467
+ "extra_special_token_458": "649516938598",
468
+ "extra_special_token_459": "717628619782",
469
+ "extra_special_token_460": "438124184139",
470
+ "extra_special_token_461": "547563892268",
471
+ "extra_special_token_462": "856317483891",
472
+ "extra_special_token_463": "313313831273",
473
+ "extra_special_token_464": "371496153876",
474
+ "extra_special_token_465": "587541149322",
475
+ "extra_special_token_466": "265847332563",
476
+ "extra_special_token_467": "449549215429",
477
+ "extra_special_token_468": "163497196769",
478
+ "extra_special_token_469": "861342291298",
479
+ "extra_special_token_470": "268433315926",
480
+ "extra_special_token_471": "774679513717",
481
+ "extra_special_token_472": "851254219729",
482
+ "extra_special_token_473": "583527834464",
483
+ "extra_special_token_474": "488496781997",
484
+ "extra_special_token_475": "556814553861",
485
+ "extra_special_token_476": "482829231639",
486
+ "extra_special_token_477": "618878266619",
487
+ "extra_special_token_478": "147444452794",
488
+ "extra_special_token_479": "949235426629",
489
+ "extra_special_token_480": "357299947518",
490
+ "extra_special_token_481": "175528632226",
491
+ "extra_special_token_482": "645527857972",
492
+ "extra_special_token_483": "186872457894",
493
+ "extra_special_token_484": "552738847828",
494
+ "extra_special_token_485": "626748382482",
495
+ "extra_special_token_486": "921894985642",
496
+ "extra_special_token_487": "943878645871",
497
+ "extra_special_token_488": "859289776479",
498
+ "extra_special_token_489": "614583493135",
499
+ "extra_special_token_490": "933775286797",
500
+ "extra_special_token_491": "332234613346",
501
+ "extra_special_token_492": "325196781219",
502
+ "extra_special_token_493": "142526557681",
503
+ "extra_special_token_494": "356722692178",
504
+ "extra_special_token_495": "449318681694",
505
+ "extra_special_token_496": "687284547244",
506
+ "extra_special_token_497": "947262995132",
507
+ "extra_special_token_498": "893974619684",
508
+ "extra_special_token_499": "797238311233"
509
+ },
510
+ "is_local": false,
511
+ "model_max_length": 131072,
512
+ "pad_token": "<|endoftext|>",
513
+ "split_special_tokens": false,
514
+ "tokenizer_class": "Qwen2Tokenizer",
515
+ "unk_token": null
516
+ }
qwen3-8b-base/dp3/adapter/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen/Qwen3-8B-Base
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:Qwen/Qwen3-8B-Base
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.18.1
qwen3-8b-base/dp3/adapter/adapter_config.json ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": true,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 32,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "megatron_config": null,
23
+ "megatron_core": "megatron.core",
24
+ "modules_to_save": [
25
+ "lm_head",
26
+ "embed_tokens"
27
+ ],
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.18.1",
30
+ "qalora_group_size": 16,
31
+ "r": 16,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "o_proj",
36
+ "v_proj",
37
+ "q_proj",
38
+ "k_proj"
39
+ ],
40
+ "target_parameters": null,
41
+ "task_type": "CAUSAL_LM",
42
+ "trainable_token_indices": null,
43
+ "use_dora": false,
44
+ "use_qalora": false,
45
+ "use_rslora": false
46
+ }
qwen3-8b-base/dp3/tokenizer/chat_template.jinja ADDED
@@ -0,0 +1,85 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0].role == 'system' %}
4
+ {{- messages[0].content + '\n\n' }}
5
+ {%- endif %}
6
+ {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
7
+ {%- for tool in tools %}
8
+ {{- "\n" }}
9
+ {{- tool | tojson }}
10
+ {%- endfor %}
11
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
+ {%- else %}
13
+ {%- if messages[0].role == 'system' %}
14
+ {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
+ {%- endif %}
16
+ {%- endif %}
17
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
+ {%- for message in messages[::-1] %}
19
+ {%- set index = (messages|length - 1) - loop.index0 %}
20
+ {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
21
+ {%- set ns.multi_step_tool = false %}
22
+ {%- set ns.last_query_index = index %}
23
+ {%- endif %}
24
+ {%- endfor %}
25
+ {%- for message in messages %}
26
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
27
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
28
+ {%- elif message.role == "assistant" %}
29
+ {%- set content = message.content %}
30
+ {%- set reasoning_content = '' %}
31
+ {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
32
+ {%- set reasoning_content = message.reasoning_content %}
33
+ {%- else %}
34
+ {%- if '</think>' in message.content %}
35
+ {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
36
+ {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
37
+ {%- endif %}
38
+ {%- endif %}
39
+ {%- if loop.index0 > ns.last_query_index %}
40
+ {%- if loop.last or (not loop.last and reasoning_content) %}
41
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
42
+ {%- else %}
43
+ {{- '<|im_start|>' + message.role + '\n' + content }}
44
+ {%- endif %}
45
+ {%- else %}
46
+ {{- '<|im_start|>' + message.role + '\n' + content }}
47
+ {%- endif %}
48
+ {%- if message.tool_calls %}
49
+ {%- for tool_call in message.tool_calls %}
50
+ {%- if (loop.first and content) or (not loop.first) %}
51
+ {{- '\n' }}
52
+ {%- endif %}
53
+ {%- if tool_call.function %}
54
+ {%- set tool_call = tool_call.function %}
55
+ {%- endif %}
56
+ {{- '<tool_call>\n{"name": "' }}
57
+ {{- tool_call.name }}
58
+ {{- '", "arguments": ' }}
59
+ {%- if tool_call.arguments is string %}
60
+ {{- tool_call.arguments }}
61
+ {%- else %}
62
+ {{- tool_call.arguments | tojson }}
63
+ {%- endif %}
64
+ {{- '}\n</tool_call>' }}
65
+ {%- endfor %}
66
+ {%- endif %}
67
+ {{- '<|im_end|>\n' }}
68
+ {%- elif message.role == "tool" %}
69
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
70
+ {{- '<|im_start|>user' }}
71
+ {%- endif %}
72
+ {{- '\n<tool_response>\n' }}
73
+ {{- message.content }}
74
+ {{- '\n</tool_response>' }}
75
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
76
+ {{- '<|im_end|>\n' }}
77
+ {%- endif %}
78
+ {%- endif %}
79
+ {%- endfor %}
80
+ {%- if add_generation_prompt %}
81
+ {{- '<|im_start|>assistant\n' }}
82
+ {%- if enable_thinking is defined and enable_thinking is false %}
83
+ {{- '<think>\n\n</think>\n\n' }}
84
+ {%- endif %}
85
+ {%- endif %}
qwen3-8b-base/dp3/tokenizer/tokenizer_config.json ADDED
@@ -0,0 +1,516 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|endoftext|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": {
9
+ "extra_special_token_0": "865331112869",
10
+ "extra_special_token_1": "569765693871",
11
+ "extra_special_token_2": "485177821815",
12
+ "extra_special_token_3": "135441121756",
13
+ "extra_special_token_4": "367459894796",
14
+ "extra_special_token_5": "877482678543",
15
+ "extra_special_token_6": "457919547633",
16
+ "extra_special_token_7": "765474393376",
17
+ "extra_special_token_8": "114848338811",
18
+ "extra_special_token_9": "746285987371",
19
+ "extra_special_token_10": "649291669397",
20
+ "extra_special_token_11": "927914615679",
21
+ "extra_special_token_12": "445925149649",
22
+ "extra_special_token_13": "691587454538",
23
+ "extra_special_token_14": "143777992227",
24
+ "extra_special_token_15": "997981281989",
25
+ "extra_special_token_16": "425949483533",
26
+ "extra_special_token_17": "982993456429",
27
+ "extra_special_token_18": "718726519731",
28
+ "extra_special_token_19": "172599315861",
29
+ "extra_special_token_20": "643489267333",
30
+ "extra_special_token_21": "282322838685",
31
+ "extra_special_token_22": "781653545886",
32
+ "extra_special_token_23": "796415361892",
33
+ "extra_special_token_24": "841991688488",
34
+ "extra_special_token_25": "211411365397",
35
+ "extra_special_token_26": "698218415444",
36
+ "extra_special_token_27": "355977139358",
37
+ "extra_special_token_28": "682564697312",
38
+ "extra_special_token_29": "383837596997",
39
+ "extra_special_token_30": "689362171782",
40
+ "extra_special_token_31": "749966767285",
41
+ "extra_special_token_32": "753159165157",
42
+ "extra_special_token_33": "795693824762",
43
+ "extra_special_token_34": "669689115557",
44
+ "extra_special_token_35": "327491773134",
45
+ "extra_special_token_36": "983569279932",
46
+ "extra_special_token_37": "612128769512",
47
+ "extra_special_token_38": "374327157578",
48
+ "extra_special_token_39": "311632789559",
49
+ "extra_special_token_40": "523918658846",
50
+ "extra_special_token_41": "765981581453",
51
+ "extra_special_token_42": "794825141891",
52
+ "extra_special_token_43": "873898736873",
53
+ "extra_special_token_44": "447445629421",
54
+ "extra_special_token_45": "473822473819",
55
+ "extra_special_token_46": "181439694557",
56
+ "extra_special_token_47": "592538279337",
57
+ "extra_special_token_48": "668134915514",
58
+ "extra_special_token_49": "643692393748",
59
+ "extra_special_token_50": "696651276628",
60
+ "extra_special_token_51": "853859348234",
61
+ "extra_special_token_52": "778466723723",
62
+ "extra_special_token_53": "929826356991",
63
+ "extra_special_token_54": "272362973463",
64
+ "extra_special_token_55": "694235616268",
65
+ "extra_special_token_56": "281673864127",
66
+ "extra_special_token_57": "479676316326",
67
+ "extra_special_token_58": "646979124677",
68
+ "extra_special_token_59": "922327493433",
69
+ "extra_special_token_60": "883685933161",
70
+ "extra_special_token_61": "264259917554",
71
+ "extra_special_token_62": "836746273134",
72
+ "extra_special_token_63": "658481324922",
73
+ "extra_special_token_64": "481884157827",
74
+ "extra_special_token_65": "587787496812",
75
+ "extra_special_token_66": "579184949249",
76
+ "extra_special_token_67": "912193598348",
77
+ "extra_special_token_68": "529679678956",
78
+ "extra_special_token_69": "795838284624",
79
+ "extra_special_token_70": "159337222655",
80
+ "extra_special_token_71": "173781362446",
81
+ "extra_special_token_72": "773687856563",
82
+ "extra_special_token_73": "535787224917",
83
+ "extra_special_token_74": "351885857332",
84
+ "extra_special_token_75": "578827344666",
85
+ "extra_special_token_76": "198462689911",
86
+ "extra_special_token_77": "722618266242",
87
+ "extra_special_token_78": "952872416512",
88
+ "extra_special_token_79": "517778845323",
89
+ "extra_special_token_80": "749665846687",
90
+ "extra_special_token_81": "661436365453",
91
+ "extra_special_token_82": "259666844669",
92
+ "extra_special_token_83": "242851284913",
93
+ "extra_special_token_84": "514532995959",
94
+ "extra_special_token_85": "161588262349",
95
+ "extra_special_token_86": "742765629356",
96
+ "extra_special_token_87": "225164373623",
97
+ "extra_special_token_88": "676539973863",
98
+ "extra_special_token_89": "826214551218",
99
+ "extra_special_token_90": "182345464792",
100
+ "extra_special_token_91": "232776999554",
101
+ "extra_special_token_92": "337326533813",
102
+ "extra_special_token_93": "676676697292",
103
+ "extra_special_token_94": "929185622831",
104
+ "extra_special_token_95": "545512344383",
105
+ "extra_special_token_96": "499444466686",
106
+ "extra_special_token_97": "314697386682",
107
+ "extra_special_token_98": "517379856925",
108
+ "extra_special_token_99": "379557332953",
109
+ "extra_special_token_100": "614797267726",
110
+ "extra_special_token_101": "429781429464",
111
+ "extra_special_token_102": "922466849763",
112
+ "extra_special_token_103": "721737645236",
113
+ "extra_special_token_104": "479227349997",
114
+ "extra_special_token_105": "136931728327",
115
+ "extra_special_token_106": "259533577263",
116
+ "extra_special_token_107": "488538864842",
117
+ "extra_special_token_108": "937495658852",
118
+ "extra_special_token_109": "489991411364",
119
+ "extra_special_token_110": "499148455254",
120
+ "extra_special_token_111": "441373944925",
121
+ "extra_special_token_112": "899151413682",
122
+ "extra_special_token_113": "467893531755",
123
+ "extra_special_token_114": "527117488925",
124
+ "extra_special_token_115": "928335588653",
125
+ "extra_special_token_116": "374439448821",
126
+ "extra_special_token_117": "879425227932",
127
+ "extra_special_token_118": "867678158885",
128
+ "extra_special_token_119": "399749397872",
129
+ "extra_special_token_120": "129693547287",
130
+ "extra_special_token_121": "689285841825",
131
+ "extra_special_token_122": "771619544974",
132
+ "extra_special_token_123": "724883568652",
133
+ "extra_special_token_124": "516968424863",
134
+ "extra_special_token_125": "733737988257",
135
+ "extra_special_token_126": "852347289392",
136
+ "extra_special_token_127": "296953381169",
137
+ "extra_special_token_128": "377273562477",
138
+ "extra_special_token_129": "262296912232",
139
+ "extra_special_token_130": "547149832394",
140
+ "extra_special_token_131": "298464134954",
141
+ "extra_special_token_132": "216667245274",
142
+ "extra_special_token_133": "843998562287",
143
+ "extra_special_token_134": "572154333646",
144
+ "extra_special_token_135": "124589118494",
145
+ "extra_special_token_136": "841824384614",
146
+ "extra_special_token_137": "232896526252",
147
+ "extra_special_token_138": "295448593321",
148
+ "extra_special_token_139": "123741461297",
149
+ "extra_special_token_140": "653573457168",
150
+ "extra_special_token_141": "196735786156",
151
+ "extra_special_token_142": "377338713663",
152
+ "extra_special_token_143": "964342468552",
153
+ "extra_special_token_144": "586855179568",
154
+ "extra_special_token_145": "484773717614",
155
+ "extra_special_token_146": "894885246797",
156
+ "extra_special_token_147": "677896358599",
157
+ "extra_special_token_148": "848845611563",
158
+ "extra_special_token_149": "851852651677",
159
+ "extra_special_token_150": "398549545767",
160
+ "extra_special_token_151": "454244839926",
161
+ "extra_special_token_152": "799364566435",
162
+ "extra_special_token_153": "967114116556",
163
+ "extra_special_token_154": "817378986438",
164
+ "extra_special_token_155": "233795848681",
165
+ "extra_special_token_156": "824387273757",
166
+ "extra_special_token_157": "916198946615",
167
+ "extra_special_token_158": "563117729724",
168
+ "extra_special_token_159": "951794811935",
169
+ "extra_special_token_160": "374598961236",
170
+ "extra_special_token_161": "922867396683",
171
+ "extra_special_token_162": "765737843639",
172
+ "extra_special_token_163": "175469284871",
173
+ "extra_special_token_164": "231853711778",
174
+ "extra_special_token_165": "662426712668",
175
+ "extra_special_token_166": "711412347158",
176
+ "extra_special_token_167": "753466987363",
177
+ "extra_special_token_168": "513361312532",
178
+ "extra_special_token_169": "712992815957",
179
+ "extra_special_token_170": "971621888444",
180
+ "extra_special_token_171": "829235161526",
181
+ "extra_special_token_172": "585544633356",
182
+ "extra_special_token_173": "582471228164",
183
+ "extra_special_token_174": "678666359123",
184
+ "extra_special_token_175": "557533689478",
185
+ "extra_special_token_176": "632962475133",
186
+ "extra_special_token_177": "484489193824",
187
+ "extra_special_token_178": "489562189822",
188
+ "extra_special_token_179": "589547936288",
189
+ "extra_special_token_180": "363214487524",
190
+ "extra_special_token_181": "244885399387",
191
+ "extra_special_token_182": "431751228368",
192
+ "extra_special_token_183": "433581868192",
193
+ "extra_special_token_184": "486391569221",
194
+ "extra_special_token_185": "185438575221",
195
+ "extra_special_token_186": "126574388585",
196
+ "extra_special_token_187": "741757479784",
197
+ "extra_special_token_188": "529854679937",
198
+ "extra_special_token_189": "996116119839",
199
+ "extra_special_token_190": "616248973917",
200
+ "extra_special_token_191": "763531783491",
201
+ "extra_special_token_192": "955456118295",
202
+ "extra_special_token_193": "364196983365",
203
+ "extra_special_token_194": "195792996468",
204
+ "extra_special_token_195": "151859598873",
205
+ "extra_special_token_196": "399223169721",
206
+ "extra_special_token_197": "938488813964",
207
+ "extra_special_token_198": "961981959227",
208
+ "extra_special_token_199": "183368827562",
209
+ "extra_special_token_200": "533417736566",
210
+ "extra_special_token_201": "786391632558",
211
+ "extra_special_token_202": "665661658354",
212
+ "extra_special_token_203": "693281533643",
213
+ "extra_special_token_204": "475794684356",
214
+ "extra_special_token_205": "652154162978",
215
+ "extra_special_token_206": "753233719644",
216
+ "extra_special_token_207": "668514843129",
217
+ "extra_special_token_208": "819162623892",
218
+ "extra_special_token_209": "941169431859",
219
+ "extra_special_token_210": "877385381798",
220
+ "extra_special_token_211": "752644929761",
221
+ "extra_special_token_212": "881136466196",
222
+ "extra_special_token_213": "275597777299",
223
+ "extra_special_token_214": "731681792655",
224
+ "extra_special_token_215": "961133895172",
225
+ "extra_special_token_216": "864718285734",
226
+ "extra_special_token_217": "963852916563",
227
+ "extra_special_token_218": "319584985416",
228
+ "extra_special_token_219": "563365646341",
229
+ "extra_special_token_220": "811371928234",
230
+ "extra_special_token_221": "837131396371",
231
+ "extra_special_token_222": "267514771964",
232
+ "extra_special_token_223": "944513428457",
233
+ "extra_special_token_224": "117298239631",
234
+ "extra_special_token_225": "158142752582",
235
+ "extra_special_token_226": "252867443568",
236
+ "extra_special_token_227": "839269684865",
237
+ "extra_special_token_228": "612788593128",
238
+ "extra_special_token_229": "145669731981",
239
+ "extra_special_token_230": "121557291859",
240
+ "extra_special_token_231": "245416776926",
241
+ "extra_special_token_232": "799417897197",
242
+ "extra_special_token_233": "997958836435",
243
+ "extra_special_token_234": "892336777248",
244
+ "extra_special_token_235": "158929292238",
245
+ "extra_special_token_236": "581976444672",
246
+ "extra_special_token_237": "897784492783",
247
+ "extra_special_token_238": "492373714791",
248
+ "extra_special_token_239": "512659818733",
249
+ "extra_special_token_240": "881112998642",
250
+ "extra_special_token_241": "619454958782",
251
+ "extra_special_token_242": "431149748713",
252
+ "extra_special_token_243": "624221476921",
253
+ "extra_special_token_244": "125866399464",
254
+ "extra_special_token_245": "339882449689",
255
+ "extra_special_token_246": "186198784585",
256
+ "extra_special_token_247": "943193294691",
257
+ "extra_special_token_248": "955668961269",
258
+ "extra_special_token_249": "232787996724",
259
+ "extra_special_token_250": "215671314196",
260
+ "extra_special_token_251": "286173241916",
261
+ "extra_special_token_252": "745977673725",
262
+ "extra_special_token_253": "556976448182",
263
+ "extra_special_token_254": "599961512792",
264
+ "extra_special_token_255": "766294538337",
265
+ "extra_special_token_256": "934912591213",
266
+ "extra_special_token_257": "295118729589",
267
+ "extra_special_token_258": "529455466433",
268
+ "extra_special_token_259": "196119929397",
269
+ "extra_special_token_260": "379571934299",
270
+ "extra_special_token_261": "251789649997",
271
+ "extra_special_token_262": "564544131355",
272
+ "extra_special_token_263": "244371196654",
273
+ "extra_special_token_264": "384598329253",
274
+ "extra_special_token_265": "887753195844",
275
+ "extra_special_token_266": "364947325679",
276
+ "extra_special_token_267": "655517954651",
277
+ "extra_special_token_268": "673948786567",
278
+ "extra_special_token_269": "857231548835",
279
+ "extra_special_token_270": "816115936673",
280
+ "extra_special_token_271": "644234165531",
281
+ "extra_special_token_272": "182782912224",
282
+ "extra_special_token_273": "234316622259",
283
+ "extra_special_token_274": "421369185549",
284
+ "extra_special_token_275": "434632855397",
285
+ "extra_special_token_276": "921889371893",
286
+ "extra_special_token_277": "415956914763",
287
+ "extra_special_token_278": "598916996413",
288
+ "extra_special_token_279": "773671349113",
289
+ "extra_special_token_280": "952465217972",
290
+ "extra_special_token_281": "117657531962",
291
+ "extra_special_token_282": "729825168745",
292
+ "extra_special_token_283": "691315125346",
293
+ "extra_special_token_284": "768461952319",
294
+ "extra_special_token_285": "664847713559",
295
+ "extra_special_token_286": "953267689786",
296
+ "extra_special_token_287": "886464195129",
297
+ "extra_special_token_288": "824488329416",
298
+ "extra_special_token_289": "837873762491",
299
+ "extra_special_token_290": "532833541879",
300
+ "extra_special_token_291": "669183782449",
301
+ "extra_special_token_292": "941976537588",
302
+ "extra_special_token_293": "739394546916",
303
+ "extra_special_token_294": "267954879268",
304
+ "extra_special_token_295": "637551427887",
305
+ "extra_special_token_296": "217756494954",
306
+ "extra_special_token_297": "524444658383",
307
+ "extra_special_token_298": "117783274348",
308
+ "extra_special_token_299": "138218735276",
309
+ "extra_special_token_300": "814611949491",
310
+ "extra_special_token_301": "711641973413",
311
+ "extra_special_token_302": "499156317423",
312
+ "extra_special_token_303": "515856611931",
313
+ "extra_special_token_304": "454164859837",
314
+ "extra_special_token_305": "345271433112",
315
+ "extra_special_token_306": "462294118988",
316
+ "extra_special_token_307": "511785788222",
317
+ "extra_special_token_308": "497294727353",
318
+ "extra_special_token_309": "866519986723",
319
+ "extra_special_token_310": "334513529294",
320
+ "extra_special_token_311": "549946382131",
321
+ "extra_special_token_312": "284445431422",
322
+ "extra_special_token_313": "396521188476",
323
+ "extra_special_token_314": "421435255895",
324
+ "extra_special_token_315": "133373659361",
325
+ "extra_special_token_316": "322683334381",
326
+ "extra_special_token_317": "228358422847",
327
+ "extra_special_token_318": "291762694874",
328
+ "extra_special_token_319": "143182978129",
329
+ "extra_special_token_320": "511923256573",
330
+ "extra_special_token_321": "327158398268",
331
+ "extra_special_token_322": "879764613759",
332
+ "extra_special_token_323": "564395222747",
333
+ "extra_special_token_324": "451161679736",
334
+ "extra_special_token_325": "538631466654",
335
+ "extra_special_token_326": "221762325616",
336
+ "extra_special_token_327": "218391991184",
337
+ "extra_special_token_328": "322589379462",
338
+ "extra_special_token_329": "876537814263",
339
+ "extra_special_token_330": "152676556624",
340
+ "extra_special_token_331": "332522971941",
341
+ "extra_special_token_332": "884354318946",
342
+ "extra_special_token_333": "513349618943",
343
+ "extra_special_token_334": "116639746413",
344
+ "extra_special_token_335": "635185846287",
345
+ "extra_special_token_336": "993832498489",
346
+ "extra_special_token_337": "813981174797",
347
+ "extra_special_token_338": "438745114173",
348
+ "extra_special_token_339": "983493951323",
349
+ "extra_special_token_340": "724492262421",
350
+ "extra_special_token_341": "622553389126",
351
+ "extra_special_token_342": "889965243135",
352
+ "extra_special_token_343": "364492359246",
353
+ "extra_special_token_344": "154962668224",
354
+ "extra_special_token_345": "179564995814",
355
+ "extra_special_token_346": "418412875665",
356
+ "extra_special_token_347": "718951851413",
357
+ "extra_special_token_348": "699446724178",
358
+ "extra_special_token_349": "624266421831",
359
+ "extra_special_token_350": "815458725125",
360
+ "extra_special_token_351": "455423278865",
361
+ "extra_special_token_352": "393741199486",
362
+ "extra_special_token_353": "328552864359",
363
+ "extra_special_token_354": "211662639865",
364
+ "extra_special_token_355": "218784516525",
365
+ "extra_special_token_356": "762486672996",
366
+ "extra_special_token_357": "142799718159",
367
+ "extra_special_token_358": "858146415154",
368
+ "extra_special_token_359": "767858144912",
369
+ "extra_special_token_360": "571317457151",
370
+ "extra_special_token_361": "635127952696",
371
+ "extra_special_token_362": "116427191984",
372
+ "extra_special_token_363": "268921994538",
373
+ "extra_special_token_364": "523937669294",
374
+ "extra_special_token_365": "165429152138",
375
+ "extra_special_token_366": "739246183345",
376
+ "extra_special_token_367": "591464355756",
377
+ "extra_special_token_368": "212985874612",
378
+ "extra_special_token_369": "191887635211",
379
+ "extra_special_token_370": "967214577653",
380
+ "extra_special_token_371": "119342152414",
381
+ "extra_special_token_372": "946444632795",
382
+ "extra_special_token_373": "618423867817",
383
+ "extra_special_token_374": "228565148417",
384
+ "extra_special_token_375": "729116422489",
385
+ "extra_special_token_376": "527874729936",
386
+ "extra_special_token_377": "739784153482",
387
+ "extra_special_token_378": "387763951128",
388
+ "extra_special_token_379": "331369926711",
389
+ "extra_special_token_380": "562716493614",
390
+ "extra_special_token_381": "739667844957",
391
+ "extra_special_token_382": "562389434565",
392
+ "extra_special_token_383": "256497188281",
393
+ "extra_special_token_384": "859927364588",
394
+ "extra_special_token_385": "417668946583",
395
+ "extra_special_token_386": "357621613582",
396
+ "extra_special_token_387": "438435178228",
397
+ "extra_special_token_388": "485692541169",
398
+ "extra_special_token_389": "825815739116",
399
+ "extra_special_token_390": "342221452223",
400
+ "extra_special_token_391": "697747991249",
401
+ "extra_special_token_392": "716763689965",
402
+ "extra_special_token_393": "141499982867",
403
+ "extra_special_token_394": "818479319499",
404
+ "extra_special_token_395": "336813343298",
405
+ "extra_special_token_396": "594688742928",
406
+ "extra_special_token_397": "472129283475",
407
+ "extra_special_token_398": "514354144759",
408
+ "extra_special_token_399": "349249721685",
409
+ "extra_special_token_400": "546276298359",
410
+ "extra_special_token_401": "353755529131",
411
+ "extra_special_token_402": "315534574435",
412
+ "extra_special_token_403": "523723475786",
413
+ "extra_special_token_404": "215826764872",
414
+ "extra_special_token_405": "367968398551",
415
+ "extra_special_token_406": "569853653352",
416
+ "extra_special_token_407": "389715484387",
417
+ "extra_special_token_408": "293847485454",
418
+ "extra_special_token_409": "714738141818",
419
+ "extra_special_token_410": "178478368922",
420
+ "extra_special_token_411": "581493616981",
421
+ "extra_special_token_412": "589439538674",
422
+ "extra_special_token_413": "846657726193",
423
+ "extra_special_token_414": "722339992679",
424
+ "extra_special_token_415": "138154781148",
425
+ "extra_special_token_416": "757785319772",
426
+ "extra_special_token_417": "492516914298",
427
+ "extra_special_token_418": "919181521716",
428
+ "extra_special_token_419": "985781138935",
429
+ "extra_special_token_420": "476969195485",
430
+ "extra_special_token_421": "313145133463",
431
+ "extra_special_token_422": "758963111966",
432
+ "extra_special_token_423": "147541537162",
433
+ "extra_special_token_424": "557163366873",
434
+ "extra_special_token_425": "144373897488",
435
+ "extra_special_token_426": "522515164754",
436
+ "extra_special_token_427": "724964923582",
437
+ "extra_special_token_428": "284776712475",
438
+ "extra_special_token_429": "375429755114",
439
+ "extra_special_token_430": "181233596124",
440
+ "extra_special_token_431": "948585673431",
441
+ "extra_special_token_432": "243165586174",
442
+ "extra_special_token_433": "396847976144",
443
+ "extra_special_token_434": "997724962668",
444
+ "extra_special_token_435": "558837194455",
445
+ "extra_special_token_436": "163165456396",
446
+ "extra_special_token_437": "378749551722",
447
+ "extra_special_token_438": "161238482259",
448
+ "extra_special_token_439": "754978243758",
449
+ "extra_special_token_440": "195388849133",
450
+ "extra_special_token_441": "229775525672",
451
+ "extra_special_token_442": "262437452884",
452
+ "extra_special_token_443": "441377892146",
453
+ "extra_special_token_444": "451885565366",
454
+ "extra_special_token_445": "981277526855",
455
+ "extra_special_token_446": "762495822823",
456
+ "extra_special_token_447": "368763327262",
457
+ "extra_special_token_448": "757422791351",
458
+ "extra_special_token_449": "636324136426",
459
+ "extra_special_token_450": "214193645583",
460
+ "extra_special_token_451": "412843856172",
461
+ "extra_special_token_452": "179386156569",
462
+ "extra_special_token_453": "756916173536",
463
+ "extra_special_token_454": "892697125149",
464
+ "extra_special_token_455": "625334487352",
465
+ "extra_special_token_456": "941861857715",
466
+ "extra_special_token_457": "887417525236",
467
+ "extra_special_token_458": "649516938598",
468
+ "extra_special_token_459": "717628619782",
469
+ "extra_special_token_460": "438124184139",
470
+ "extra_special_token_461": "547563892268",
471
+ "extra_special_token_462": "856317483891",
472
+ "extra_special_token_463": "313313831273",
473
+ "extra_special_token_464": "371496153876",
474
+ "extra_special_token_465": "587541149322",
475
+ "extra_special_token_466": "265847332563",
476
+ "extra_special_token_467": "449549215429",
477
+ "extra_special_token_468": "163497196769",
478
+ "extra_special_token_469": "861342291298",
479
+ "extra_special_token_470": "268433315926",
480
+ "extra_special_token_471": "774679513717",
481
+ "extra_special_token_472": "851254219729",
482
+ "extra_special_token_473": "583527834464",
483
+ "extra_special_token_474": "488496781997",
484
+ "extra_special_token_475": "556814553861",
485
+ "extra_special_token_476": "482829231639",
486
+ "extra_special_token_477": "618878266619",
487
+ "extra_special_token_478": "147444452794",
488
+ "extra_special_token_479": "949235426629",
489
+ "extra_special_token_480": "357299947518",
490
+ "extra_special_token_481": "175528632226",
491
+ "extra_special_token_482": "645527857972",
492
+ "extra_special_token_483": "186872457894",
493
+ "extra_special_token_484": "552738847828",
494
+ "extra_special_token_485": "626748382482",
495
+ "extra_special_token_486": "921894985642",
496
+ "extra_special_token_487": "943878645871",
497
+ "extra_special_token_488": "859289776479",
498
+ "extra_special_token_489": "614583493135",
499
+ "extra_special_token_490": "933775286797",
500
+ "extra_special_token_491": "332234613346",
501
+ "extra_special_token_492": "325196781219",
502
+ "extra_special_token_493": "142526557681",
503
+ "extra_special_token_494": "356722692178",
504
+ "extra_special_token_495": "449318681694",
505
+ "extra_special_token_496": "687284547244",
506
+ "extra_special_token_497": "947262995132",
507
+ "extra_special_token_498": "893974619684",
508
+ "extra_special_token_499": "797238311233"
509
+ },
510
+ "is_local": false,
511
+ "model_max_length": 131072,
512
+ "pad_token": "<|endoftext|>",
513
+ "split_special_tokens": false,
514
+ "tokenizer_class": "Qwen2Tokenizer",
515
+ "unk_token": null
516
+ }
qwen3-8b-base/dp3_v2/adapter/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen/Qwen3-8B-Base
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:Qwen/Qwen3-8B-Base
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.18.1
qwen3-8b-base/dp3_v2/adapter/adapter_config.json ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": true,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 32,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "megatron_config": null,
23
+ "megatron_core": "megatron.core",
24
+ "modules_to_save": [
25
+ "lm_head",
26
+ "embed_tokens"
27
+ ],
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.18.1",
30
+ "qalora_group_size": 16,
31
+ "r": 16,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "q_proj",
36
+ "k_proj",
37
+ "v_proj",
38
+ "o_proj"
39
+ ],
40
+ "target_parameters": null,
41
+ "task_type": "CAUSAL_LM",
42
+ "trainable_token_indices": null,
43
+ "use_dora": false,
44
+ "use_qalora": false,
45
+ "use_rslora": false
46
+ }
qwen3-8b-base/dp3_v2/audit_results.json ADDED
@@ -0,0 +1,137 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "delta": 1e-05,
3
+ "num_canaries": 500,
4
+ "num_members": 250,
5
+ "paper_guess_fraction": 0.2,
6
+ "paper_guess_steps": 20,
7
+ "loss": {
8
+ "auc": 0.508168,
9
+ "empirical_epsilon": {
10
+ "0.05": 0.23387643229216337,
11
+ "0.01": 0.0
12
+ },
13
+ "empirical_epsilon_details": {
14
+ "0.05": {
15
+ "epsilon": 0.23387643229216337,
16
+ "num_guesses": 15,
17
+ "correct_guesses": 12,
18
+ "candidate_num_guesses": [
19
+ 5,
20
+ 10,
21
+ 15,
22
+ 20,
23
+ 25,
24
+ 30,
25
+ 35,
26
+ 40,
27
+ 45,
28
+ 50,
29
+ 55,
30
+ 60,
31
+ 65,
32
+ 70,
33
+ 75,
34
+ 80,
35
+ 85,
36
+ 90,
37
+ 95,
38
+ 100
39
+ ],
40
+ "direction": "lower"
41
+ },
42
+ "0.01": {
43
+ "epsilon": 0.0,
44
+ "num_guesses": 0,
45
+ "correct_guesses": 0,
46
+ "candidate_num_guesses": [
47
+ 5,
48
+ 10,
49
+ 15,
50
+ 20,
51
+ 25,
52
+ 30,
53
+ 35,
54
+ 40,
55
+ 45,
56
+ 50,
57
+ 55,
58
+ 60,
59
+ 65,
60
+ 70,
61
+ 75,
62
+ 80,
63
+ 85,
64
+ 90,
65
+ 95,
66
+ 100
67
+ ],
68
+ "direction": "lower"
69
+ }
70
+ }
71
+ },
72
+ "embedding": {
73
+ "auc": 0.518896,
74
+ "empirical_epsilon": {
75
+ "0.05": 0.09125839080661535,
76
+ "0.01": 0.0
77
+ },
78
+ "empirical_epsilon_details": {
79
+ "0.05": {
80
+ "epsilon": 0.09125839080661535,
81
+ "num_guesses": 100,
82
+ "correct_guesses": 61,
83
+ "candidate_num_guesses": [
84
+ 5,
85
+ 10,
86
+ 15,
87
+ 20,
88
+ 25,
89
+ 30,
90
+ 35,
91
+ 40,
92
+ 45,
93
+ 50,
94
+ 55,
95
+ 60,
96
+ 65,
97
+ 70,
98
+ 75,
99
+ 80,
100
+ 85,
101
+ 90,
102
+ 95,
103
+ 100
104
+ ],
105
+ "direction": "higher"
106
+ },
107
+ "0.01": {
108
+ "epsilon": 0.0,
109
+ "num_guesses": 0,
110
+ "correct_guesses": 0,
111
+ "candidate_num_guesses": [
112
+ 5,
113
+ 10,
114
+ 15,
115
+ 20,
116
+ 25,
117
+ 30,
118
+ 35,
119
+ 40,
120
+ 45,
121
+ 50,
122
+ 55,
123
+ 60,
124
+ 65,
125
+ 70,
126
+ 75,
127
+ 80,
128
+ 85,
129
+ 90,
130
+ 95,
131
+ 100
132
+ ],
133
+ "direction": "lower"
134
+ }
135
+ }
136
+ }
137
+ }
qwen3-8b-base/dp3_v2/canary_meta.json ADDED
The diff for this file is too large to render. See raw diff
 
qwen3-8b-base/dp3_v2/metrics.jsonl ADDED
The diff for this file is too large to render. See raw diff