Upload folder using huggingface_hub

README.md

@@ -0,0 +1,209 @@
+---
+base_model: allenai/OLMoE-1B-7B-0125-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:allenai/OLMoE-1B-7B-0125-Instruct
+- grpo
+- lora
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

adapter_config.json

@@ -0,0 +1,54 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    ".*\\.gate_up_proj": 64
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "allenai/OLMoE-1B-7B-0125-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    ".*\\.gate_up_proj": 32
+  },
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "router",
+    "k_proj",
+    "q_proj",
+    "o_proj"
+  ],
+  "target_parameters": [
+    "down_proj",
+    "gate_up_proj",
+    "gate.weight"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66bef69b001859cf0288bdc857607d5934a8be30de5a31cd7614630254efa479
+size 757167808

chat_template.jinja

@@ -0,0 +1,9 @@
+{{ bos_token }}{% for message in messages %}{% if message['role'] == 'system' %}{{ '<|system|>
+' + message['content'] + '
+' }}{% elif message['role'] == 'user' %}{{ '<|user|>
+' + message['content'] + '
+' }}{% elif message['role'] == 'assistant' %}{% if not loop.last %}{{ '<|assistant|>
+'  + message['content'] + eos_token + '
+' }}{% else %}{{ '<|assistant|>
+'  + message['content'] + eos_token }}{% endif %}{% endif %}{% if loop.last and add_generation_prompt %}{{ '<|assistant|>
+' }}{% endif %}{% endfor %}

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82b85f459254dae235e4a682ab240cea1b9c74d09f2bb842ef6a71572f13dd30
+size 1514466827

rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:570e333df026be0afe4258c1e7ef27fc9eb5459fa566c051a4ae4d3d266dff93
+size 14645

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4dcd0fcafdf0cac56c6781c9554680e5d2ba9bfee0d823975791d1d648a00172
+size 1465

tokenizer_config.json

@@ -0,0 +1,15 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "|||IP_ADDRESS|||",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "|||IP_ADDRESS|||",
+  "errors": "replace",
+  "is_local": false,
+  "local_files_only": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "tokenizer_class": "GPTNeoXTokenizer",
+  "trim_offsets": true,
+  "unk_token": null
+}

trainer_state.json

@@ -0,0 +1,674 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.041736227045075125,
+  "eval_steps": 500,
+  "global_step": 200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "cache_hit_rate": 0.6749276161193848,
+      "cache_hit_rate_size16": 0.6550344030062357,
+      "cache_hit_rate_size32": 0.8710664212703705,
+      "cache_hit_rate_size8": 0.4304385483264923,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.2875,
+      "completions/max_length": 1024.0,
+      "completions/max_terminated_length": 926.0,
+      "completions/mean_length": 732.10625,
+      "completions/mean_terminated_length": 595.8285858154297,
+      "completions/min_length": 208.4,
+      "completions/min_terminated_length": 208.4,
+      "entropy": 0.6143155984580517,
+      "epoch": 0.002086811352253756,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.1999109834432602,
+      "kl": 0.002709078643238172,
+      "learning_rate": 9e-05,
+      "loss": -0.020685632526874543,
+      "num_tokens": 148025.0,
+      "reward": 0.6688930571079255,
+      "reward_std": 0.10492694256827235,
+      "rewards/cache_reward/mean": 0.6688930571079255,
+      "rewards/cache_reward/std": 0.10492694256827235,
+      "step": 10,
+      "step_time": 109.66132322684862
+    },
+    {
+      "cache_hit_rate": 0.6733510851860046,
+      "cache_hit_rate_size16": 0.7422660708427429,
+      "cache_hit_rate_size32": 0.8451784491539002,
+      "cache_hit_rate_size8": 0.4291795551776886,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.38125,
+      "completions/max_length": 1024.0,
+      "completions/max_terminated_length": 988.3,
+      "completions/mean_length": 870.26875,
+      "completions/mean_terminated_length": 780.20380859375,
+      "completions/min_length": 517.0,
+      "completions/min_terminated_length": 517.0,
+      "entropy": 0.5574822902679444,
+      "epoch": 0.004173622704507512,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.27785834670066833,
+      "kl": 0.008998260926455259,
+      "learning_rate": 0.0001,
+      "loss": -0.04589595198631287,
+      "num_tokens": 324300.0,
+      "reward": 0.6649930059909821,
+      "reward_std": 0.09102208893746137,
+      "rewards/cache_reward/mean": 0.6649930059909821,
+      "rewards/cache_reward/std": 0.09102208893746137,
+      "step": 20,
+      "step_time": 109.77420157198794
+    },
+    {
+      "cache_hit_rate": 0.7234689474105835,
+      "cache_hit_rate_size16": 0.7119116663932801,
+      "cache_hit_rate_size32": 0.8416047170758247,
+      "cache_hit_rate_size8": 0.3245641191800435,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.2625,
+      "completions/max_length": 989.8,
+      "completions/max_terminated_length": 914.9,
+      "completions/mean_length": 701.85625,
+      "completions/mean_terminated_length": 626.0504425048828,
+      "completions/min_length": 341.9,
+      "completions/min_terminated_length": 341.9,
+      "entropy": 0.6420590102672576,
+      "epoch": 0.006260434056761269,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.2189197838306427,
+      "kl": 0.015754866227507592,
+      "learning_rate": 0.0001,
+      "loss": -0.057462894916534425,
+      "num_tokens": 461973.0,
+      "reward": 0.7092925369739532,
+      "reward_std": 0.09992160452529789,
+      "rewards/cache_reward/mean": 0.7092925369739532,
+      "rewards/cache_reward/std": 0.09992160452529789,
+      "step": 30,
+      "step_time": 105.50118747297674
+    },
+    {
+      "cache_hit_rate": 0.6422885358333588,
+      "cache_hit_rate_size16": 0.7278285367148263,
+      "cache_hit_rate_size32": 0.8580973446369171,
+      "cache_hit_rate_size8": 0.37648922204971313,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.4125,
+      "completions/max_length": 1017.0,
+      "completions/max_terminated_length": 930.3,
+      "completions/mean_length": 831.81875,
+      "completions/mean_terminated_length": 727.4821502685547,
+      "completions/min_length": 484.1,
+      "completions/min_terminated_length": 484.1,
+      "entropy": 0.5546563148498536,
+      "epoch": 0.008347245409015025,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.2441343367099762,
+      "kl": 0.021256055543199183,
+      "learning_rate": 0.0001,
+      "loss": -0.06346478462219238,
+      "num_tokens": 636696.0,
+      "reward": 0.6391598552465438,
+      "reward_std": 0.13217019997537136,
+      "rewards/cache_reward/mean": 0.6391598552465438,
+      "rewards/cache_reward/std": 0.13217019997537136,
+      "step": 40,
+      "step_time": 109.29803132868838
+    },
+    {
+      "cache_hit_rate": 0.6705451369285583,
+      "cache_hit_rate_size16": 0.6974249720573426,
+      "cache_hit_rate_size32": 0.8658615776470729,
+      "cache_hit_rate_size8": 0.4653993273774783,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.34375,
+      "completions/max_length": 1021.8,
+      "completions/max_terminated_length": 879.7,
+      "completions/mean_length": 794.4875,
+      "completions/mean_terminated_length": 615.1027160644531,
+      "completions/min_length": 435.1,
+      "completions/min_terminated_length": 332.7,
+      "entropy": 0.6008389577269554,
+      "epoch": 0.010434056761268781,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.23809342086315155,
+      "kl": 0.03263230957090855,
+      "learning_rate": 0.0001,
+      "loss": -0.04350569248199463,
+      "num_tokens": 798422.0,
+      "reward": 0.688891452550888,
+      "reward_std": 0.1476906482130289,
+      "rewards/cache_reward/mean": 0.688891452550888,
+      "rewards/cache_reward/std": 0.1476906482130289,
+      "step": 50,
+      "step_time": 109.32008387609385
+    },
+    {
+      "cache_hit_rate": 0.5735559463500977,
+      "cache_hit_rate_size16": 0.6513433158397675,
+      "cache_hit_rate_size32": 0.8832110166549683,
+      "cache_hit_rate_size8": 0.4417416916953193,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.39375,
+      "completions/max_length": 1024.0,
+      "completions/max_terminated_length": 950.8,
+      "completions/mean_length": 820.71875,
+      "completions/mean_terminated_length": 705.5888244628907,
+      "completions/min_length": 393.1,
+      "completions/min_terminated_length": 393.1,
+      "entropy": 0.5146959632635116,
+      "epoch": 0.012520868113522538,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.2036655694246292,
+      "kl": 0.03261517193168402,
+      "learning_rate": 0.0001,
+      "loss": -0.06676955223083496,
+      "num_tokens": 977289.0,
+      "reward": 0.5551395505666733,
+      "reward_std": 0.130782650411129,
+      "rewards/cache_reward/mean": 0.5551395505666733,
+      "rewards/cache_reward/std": 0.130782650411129,
+      "step": 60,
+      "step_time": 110.19092689147219
+    },
+    {
+      "cache_hit_rate": 0.7626929998397827,
+      "cache_hit_rate_size16": 0.6949937641620636,
+      "cache_hit_rate_size32": 0.8773939145935906,
+      "cache_hit_rate_size8": 0.5052446524302164,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.375,
+      "completions/max_length": 1024.0,
+      "completions/max_terminated_length": 960.0,
+      "completions/mean_length": 831.425,
+      "completions/mean_terminated_length": 736.7794067382813,
+      "completions/min_length": 410.3,
+      "completions/min_terminated_length": 410.3,
+      "entropy": 0.550393845140934,
+      "epoch": 0.014607679465776294,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.2976628541946411,
+      "kl": 0.04141218531876802,
+      "learning_rate": 0.0001,
+      "loss": -0.028155168890953063,
+      "num_tokens": 1149221.0,
+      "reward": 0.765302586555481,
+      "reward_std": 0.10612610904499889,
+      "rewards/cache_reward/mean": 0.765302586555481,
+      "rewards/cache_reward/std": 0.10612610904499889,
+      "step": 70,
+      "step_time": 109.80838733934797
+    },
+    {
+      "cache_hit_rate": 0.6532172083854675,
+      "cache_hit_rate_size16": 0.7496759465762547,
+      "cache_hit_rate_size32": 0.8642965952555338,
+      "cache_hit_rate_size8": 0.4807915687561035,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.475,
+      "completions/max_length": 1024.0,
+      "completions/max_terminated_length": 958.0,
+      "completions/mean_length": 828.0625,
+      "completions/mean_terminated_length": 696.7795776367187,
+      "completions/min_length": 460.7,
+      "completions/min_terminated_length": 460.7,
+      "entropy": 0.5280586190521717,
+      "epoch": 0.01669449081803005,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.21478967368602753,
+      "kl": 0.05440526399761438,
+      "learning_rate": 0.0001,
+      "loss": -0.047751826047897336,
+      "num_tokens": 1317623.0,
+      "reward": 0.6507435321807862,
+      "reward_std": 0.1329113408923149,
+      "rewards/cache_reward/mean": 0.6507435321807862,
+      "rewards/cache_reward/std": 0.1329113408923149,
+      "step": 80,
+      "step_time": 109.61795915367547
+    },
+    {
+      "cache_hit_rate": 0.628938740491867,
+      "cache_hit_rate_size16": 0.6589138408501943,
+      "cache_hit_rate_size32": 0.8045709133148193,
+      "cache_hit_rate_size8": 0.4631062958921705,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.39375,
+      "completions/max_length": 1024.0,
+      "completions/max_terminated_length": 922.1,
+      "completions/mean_length": 763.0125,
+      "completions/mean_terminated_length": 620.8271697998047,
+      "completions/min_length": 354.5,
+      "completions/min_terminated_length": 354.5,
+      "entropy": 0.6081107705831528,
+      "epoch": 0.018781302170283808,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.2587888240814209,
+      "kl": 0.06952012721449137,
+      "learning_rate": 0.0001,
+      "loss": -0.006095578521490097,
+      "num_tokens": 1485377.0,
+      "reward": 0.6154629409313201,
+      "reward_std": 0.10154437553137541,
+      "rewards/cache_reward/mean": 0.6154629409313201,
+      "rewards/cache_reward/std": 0.10154437553137541,
+      "step": 90,
+      "step_time": 110.01140693970956
+    },
+    {
+      "cache_hit_rate": 0.6613744765520095,
+      "cache_hit_rate_size16": 0.7402904033660889,
+      "cache_hit_rate_size32": 0.8814258575439453,
+      "cache_hit_rate_size8": 0.42935203909873965,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.425,
+      "completions/max_length": 1024.0,
+      "completions/max_terminated_length": 943.8,
+      "completions/mean_length": 829.99375,
+      "completions/mean_terminated_length": 697.3199035644532,
+      "completions/min_length": 391.9,
+      "completions/min_terminated_length": 391.9,
+      "entropy": 0.5821917943656445,
+      "epoch": 0.020868113522537562,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.23848511278629303,
+      "kl": 0.07018006853759289,
+      "learning_rate": 0.0001,
+      "loss": -0.031068474054336548,
+      "num_tokens": 1647416.0,
+      "reward": 0.6516854614019394,
+      "reward_std": 0.08088633958250284,
+      "rewards/cache_reward/mean": 0.6516854614019394,
+      "rewards/cache_reward/std": 0.08088633958250284,
+      "step": 100,
+      "step_time": 109.22561302329413
+    },
+    {
+      "cache_hit_rate": 0.7659814059734344,
+      "cache_hit_rate_size16": 0.7210325002670288,
+      "cache_hit_rate_size32": 0.8733292288250394,
+      "cache_hit_rate_size8": 0.5074326276779175,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.39375,
+      "completions/max_length": 1000.1,
+      "completions/max_terminated_length": 903.8,
+      "completions/mean_length": 738.875,
+      "completions/mean_terminated_length": 597.613003540039,
+      "completions/min_length": 244.4,
+      "completions/min_terminated_length": 244.4,
+      "entropy": 0.6063698619604111,
+      "epoch": 0.022954924874791317,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.22822512686252594,
+      "kl": 0.0519987877458334,
+      "learning_rate": 0.0001,
+      "loss": -0.004398582130670547,
+      "num_tokens": 1809740.0,
+      "reward": 0.7583289802074432,
+      "reward_std": 0.11587669281288981,
+      "rewards/cache_reward/mean": 0.7583289802074432,
+      "rewards/cache_reward/std": 0.11587669281288981,
+      "step": 110,
+      "step_time": 107.61979042319581
+    },
+    {
+      "cache_hit_rate": 0.667035973072052,
+      "cache_hit_rate_size16": 0.6942717552185058,
+      "cache_hit_rate_size32": 0.8527272343635559,
+      "cache_hit_rate_size8": 0.40293918251991273,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.175,
+      "completions/max_length": 1003.6,
+      "completions/max_terminated_length": 930.3,
+      "completions/mean_length": 665.3625,
+      "completions/mean_terminated_length": 598.4001220703125,
+      "completions/min_length": 247.0,
+      "completions/min_terminated_length": 247.0,
+      "entropy": 0.7441994078457356,
+      "epoch": 0.025041736227045076,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.21977545320987701,
+      "kl": 0.06097110863775015,
+      "learning_rate": 0.0001,
+      "loss": -0.05008636116981506,
+      "num_tokens": 1947502.0,
+      "reward": 0.6475550562143326,
+      "reward_std": 0.11349433939903975,
+      "rewards/cache_reward/mean": 0.6475550562143326,
+      "rewards/cache_reward/std": 0.11349433939903975,
+      "step": 120,
+      "step_time": 107.30077534515877
+    },
+    {
+      "cache_hit_rate": 0.686054265499115,
+      "cache_hit_rate_size16": 0.6798208653926849,
+      "cache_hit_rate_size32": 0.871903528769811,
+      "cache_hit_rate_size8": 0.443097785115242,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.2875,
+      "completions/max_length": 986.6,
+      "completions/max_terminated_length": 909.5,
+      "completions/mean_length": 661.2125,
+      "completions/mean_terminated_length": 539.6267791748047,
+      "completions/min_length": 202.2,
+      "completions/min_terminated_length": 202.2,
+      "entropy": 0.685629041492939,
+      "epoch": 0.02712854757929883,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.2505522072315216,
+      "kl": 0.06346707474440336,
+      "learning_rate": 0.0001,
+      "loss": -0.08610177040100098,
+      "num_tokens": 2106072.0,
+      "reward": 0.6472916662693023,
+      "reward_std": 0.10670399386435747,
+      "rewards/cache_reward/mean": 0.6472916662693023,
+      "rewards/cache_reward/std": 0.10670399386435747,
+      "step": 130,
+      "step_time": 106.4104538091924
+    },
+    {
+      "cache_hit_rate": 0.7193193137645721,
+      "cache_hit_rate_size16": 0.6384188731511434,
+      "cache_hit_rate_size32": 0.9035206958651543,
+      "cache_hit_rate_size8": 0.45546102098056246,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.49375,
+      "completions/max_length": 1011.1,
+      "completions/max_terminated_length": 949.7,
+      "completions/mean_length": 824.2125,
+      "completions/mean_terminated_length": 655.7207336425781,
+      "completions/min_length": 250.9,
+      "completions/min_terminated_length": 250.9,
+      "entropy": 0.5345787741243839,
+      "epoch": 0.02921535893155259,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.24551904201507568,
+      "kl": 0.05889502316713333,
+      "learning_rate": 0.0001,
+      "loss": -0.10549832582473755,
+      "num_tokens": 2277234.0,
+      "reward": 0.6919173955917358,
+      "reward_std": 0.18916533272713423,
+      "rewards/cache_reward/mean": 0.6919173955917358,
+      "rewards/cache_reward/std": 0.18916533272713423,
+      "step": 140,
+      "step_time": 108.70422725854442
+    },
+    {
+      "cache_hit_rate": 0.6837528288364411,
+      "cache_hit_rate_size16": 0.743753457069397,
+      "cache_hit_rate_size32": 0.8899944126605988,
+      "cache_hit_rate_size8": 0.43498979695141315,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.29375,
+      "completions/max_length": 997.1,
+      "completions/max_terminated_length": 943.5,
+      "completions/mean_length": 775.30625,
+      "completions/mean_terminated_length": 693.9155212402344,
+      "completions/min_length": 411.5,
+      "completions/min_terminated_length": 411.5,
+      "entropy": 0.5689274027943612,
+      "epoch": 0.031302170283806344,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.22718407213687897,
+      "kl": 0.06529534254223109,
+      "learning_rate": 0.0001,
+      "loss": -0.04616514444351196,
+      "num_tokens": 2444147.0,
+      "reward": 0.6700104534626007,
+      "reward_std": 0.17748181223869325,
+      "rewards/cache_reward/mean": 0.6700104534626007,
+      "rewards/cache_reward/std": 0.17748181223869325,
+      "step": 150,
+      "step_time": 106.98494991599583
+    },
+    {
+      "cache_hit_rate": 0.6439329028129578,
+      "cache_hit_rate_size16": 0.7306425826890128,
+      "cache_hit_rate_size32": 0.865531712770462,
+      "cache_hit_rate_size8": 0.4455534170071284,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.24375,
+      "completions/max_length": 1010.9,
+      "completions/max_terminated_length": 928.8,
+      "completions/mean_length": 701.30625,
+      "completions/mean_terminated_length": 617.9401885986329,
+      "completions/min_length": 373.5,
+      "completions/min_terminated_length": 373.5,
+      "entropy": 0.6062380790710449,
+      "epoch": 0.0333889816360601,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.311809241771698,
+      "kl": 0.06560975145548582,
+      "learning_rate": 0.0001,
+      "loss": -0.11374449729919434,
+      "num_tokens": 2588116.0,
+      "reward": 0.629342782497406,
+      "reward_std": 0.09073512144386768,
+      "rewards/cache_reward/mean": 0.629342782497406,
+      "rewards/cache_reward/std": 0.09073512144386768,
+      "step": 160,
+      "step_time": 107.92681602598168
+    },
+    {
+      "cache_hit_rate": 0.737691605091095,
+      "cache_hit_rate_size16": 0.7616987451910973,
+      "cache_hit_rate_size32": 0.8484096378087997,
+      "cache_hit_rate_size8": 0.47391005357106525,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.5375,
+      "completions/max_length": 1015.0,
+      "completions/max_terminated_length": 968.0,
+      "completions/mean_length": 876.60625,
+      "completions/mean_terminated_length": 778.4466735839844,
+      "completions/min_length": 541.3,
+      "completions/min_terminated_length": 541.3,
+      "entropy": 0.5039393544197083,
+      "epoch": 0.03547579298831385,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.22193852066993713,
+      "kl": 0.07721416000276804,
+      "learning_rate": 0.0001,
+      "loss": 0.009571240842342376,
+      "num_tokens": 2774853.0,
+      "reward": 0.7360832095146179,
+      "reward_std": 0.09169260896742344,
+      "rewards/cache_reward/mean": 0.7360832095146179,
+      "rewards/cache_reward/std": 0.09169260896742344,
+      "step": 170,
+      "step_time": 108.83163746409119
+    },
+    {
+      "cache_hit_rate": 0.6702417641878128,
+      "cache_hit_rate_size16": 0.7594628930091858,
+      "cache_hit_rate_size32": 0.8569953143596649,
+      "cache_hit_rate_size8": 0.4382890040675799,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.3625,
+      "completions/max_length": 1016.3,
+      "completions/max_terminated_length": 938.8,
+      "completions/mean_length": 702.325,
+      "completions/mean_terminated_length": 549.5508026123047,
+      "completions/min_length": 199.8,
+      "completions/min_terminated_length": 199.8,
+      "entropy": 0.6205699190497398,
+      "epoch": 0.037562604340567615,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.28704333305358887,
+      "kl": 0.08045898843556643,
+      "learning_rate": 0.0001,
+      "loss": -0.04709939360618591,
+      "num_tokens": 2935177.0,
+      "reward": 0.6992827504873276,
+      "reward_std": 0.131203200109303,
+      "rewards/cache_reward/mean": 0.6992827504873276,
+      "rewards/cache_reward/std": 0.131203200109303,
+      "step": 180,
+      "step_time": 109.50919925523922
+    },
+    {
+      "cache_hit_rate": 0.6852899014949798,
+      "cache_hit_rate_size16": 0.6899512580462864,
+      "cache_hit_rate_size32": 0.8868090361356735,
+      "cache_hit_rate_size8": 0.4937669684489568,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.40625,
+      "completions/max_length": 1024.0,
+      "completions/max_terminated_length": 867.3,
+      "completions/mean_length": 734.975,
+      "completions/mean_terminated_length": 540.3156127929688,
+      "completions/min_length": 192.5,
+      "completions/min_terminated_length": 192.5,
+      "entropy": 0.5207937769591808,
+      "epoch": 0.03964941569282137,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.2835640013217926,
+      "kl": 0.0956730455160141,
+      "learning_rate": 0.0001,
+      "loss": -0.044486325979232785,
+      "num_tokens": 3104349.0,
+      "reward": 0.6792330205440521,
+      "reward_std": 0.14855806417763234,
+      "rewards/cache_reward/mean": 0.6792330205440521,
+      "rewards/cache_reward/std": 0.14855806417763234,
+      "step": 190,
+      "step_time": 110.20991073080805
+    },
+    {
+      "cache_hit_rate": 0.6421143919229507,
+      "cache_hit_rate_size16": 0.6826490312814713,
+      "cache_hit_rate_size32": 0.8723512291908264,
+      "cache_hit_rate_size8": 0.44266503552595776,
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.425,
+      "completions/max_length": 1024.0,
+      "completions/max_terminated_length": 871.2,
+      "completions/mean_length": 670.93125,
+      "completions/mean_terminated_length": 457.2080528259277,
+      "completions/min_length": 191.1,
+      "completions/min_terminated_length": 191.1,
+      "entropy": 0.6987772159278393,
+      "epoch": 0.041736227045075125,
+      "frac_reward_zero_std": 0.0,
+      "grad_norm": 0.28543853759765625,
+      "kl": 0.09522749911993741,
+      "learning_rate": 0.0001,
+      "loss": -0.05127257704734802,
+      "num_tokens": 3267042.0,
+      "reward": 0.6385201156139374,
+      "reward_std": 0.12131488006561994,
+      "rewards/cache_reward/mean": 0.6385201156139374,
+      "rewards/cache_reward/std": 0.12131488006561994,
+      "step": 200,
+      "step_time": 110.4871686702827
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 200,
+  "num_input_tokens_seen": 3267042,
+  "num_train_epochs": 1,
+  "save_steps": 50,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d990e886140bc05f15c566f6832c8487051786c46e57df5ffd57605bdf6d2cf1
+size 7825