Delete checkpoint-500

checkpoint-500/README.md

@@ -1,209 +0,0 @@
----
-base_model: HuggingFaceTB/SmolLM2-135M-Instruct
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:HuggingFaceTB/SmolLM2-135M-Instruct
-- lora
-- sft
-- transformers
-- trl
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1

checkpoint-500/adapter_config.json

@@ -1,43 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "HuggingFaceTB/SmolLM2-135M-Instruct",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": false,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": null,
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "o_proj",
-    "k_proj",
-    "q_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}

checkpoint-500/adapter_model.safetensors

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:fa7990f97f806a5ca35e6c6fbeb73528ef072b484aa5b018c59c13c78e90f90e
-size 7404368

checkpoint-500/chat_template.jinja

@@ -1,6 +0,0 @@
-{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system
-You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>
-' }}{% endif %}{{'<|im_start|>' + message['role'] + '
-' + message['content'] + '<|im_end|>' + '
-'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
-' }}{% endif %}

checkpoint-500/optimizer.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:40159c5fd3460141b13169d253859137073f9b816cb234a93ddaebb9cced807a
-size 14950667

checkpoint-500/rng_state.pth

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:71fdddcc073255adb0886fcc6bc840ad5473c35f42a4a6358de3fb1ebfc0b168
-size 14645

checkpoint-500/scaler.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f77569c2e850b04af982cc8c1389f1430851448915c593b69e5da36ce05b71d7
-size 1383

checkpoint-500/scheduler.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f93153ebd2954a5fdb05744476d618441b613584136cf9a87571f2c05d8f3e7c
-size 1465

checkpoint-500/tokenizer_config.json

@@ -1,17 +0,0 @@
-{
-  "add_prefix_space": false,
-  "backend": "tokenizers",
-  "bos_token": "<|im_start|>",
-  "clean_up_tokenization_spaces": false,
-  "eos_token": "<|im_end|>",
-  "extra_special_tokens": [
-    "<|im_start|>",
-    "<|im_end|>"
-  ],
-  "is_local": false,
-  "model_max_length": 8192,
-  "pad_token": "<|im_end|>",
-  "tokenizer_class": "TokenizersBackend",
-  "unk_token": "<|endoftext|>",
-  "vocab_size": 49152
-}

checkpoint-500/trainer_state.json

@@ -1,534 +0,0 @@
-{
-  "best_global_step": null,
-  "best_metric": null,
-  "best_model_checkpoint": null,
-  "epoch": 0.42955326460481097,
-  "eval_steps": 500,
-  "global_step": 500,
-  "is_hyper_param_search": false,
-  "is_local_process_zero": true,
-  "is_world_process_zero": true,
-  "log_history": [
-    {
-      "entropy": 1.1566081464290618,
-      "epoch": 0.00859106529209622,
-      "grad_norm": 0.15214066207408905,
-      "learning_rate": 0.00029884020618556697,
-      "loss": 1.226055908203125,
-      "mean_token_accuracy": 0.7206311166286469,
-      "num_tokens": 26541.0,
-      "step": 10
-    },
-    {
-      "entropy": 1.2002840995788575,
-      "epoch": 0.01718213058419244,
-      "grad_norm": 0.1833968460559845,
-      "learning_rate": 0.00029755154639175255,
-      "loss": 1.2007878303527832,
-      "mean_token_accuracy": 0.7304173231124877,
-      "num_tokens": 52280.0,
-      "step": 20
-    },
-    {
-      "entropy": 1.083837980031967,
-      "epoch": 0.02577319587628866,
-      "grad_norm": 0.15183115005493164,
-      "learning_rate": 0.0002962628865979381,
-      "loss": 1.1582960128784179,
-      "mean_token_accuracy": 0.7341346919536591,
-      "num_tokens": 79603.0,
-      "step": 30
-    },
-    {
-      "entropy": 0.9830838143825531,
-      "epoch": 0.03436426116838488,
-      "grad_norm": 0.1640024185180664,
-      "learning_rate": 0.00029497422680412364,
-      "loss": 0.989743423461914,
-      "mean_token_accuracy": 0.7652481079101563,
-      "num_tokens": 106301.0,
-      "step": 40
-    },
-    {
-      "entropy": 0.9237875938415527,
-      "epoch": 0.0429553264604811,
-      "grad_norm": 0.1496010422706604,
-      "learning_rate": 0.0002936855670103092,
-      "loss": 0.9489053726196289,
-      "mean_token_accuracy": 0.7682253539562225,
-      "num_tokens": 131791.0,
-      "step": 50
-    },
-    {
-      "entropy": 0.945674192905426,
-      "epoch": 0.05154639175257732,
-      "grad_norm": 0.16326577961444855,
-      "learning_rate": 0.0002923969072164948,
-      "loss": 0.9549709320068359,
-      "mean_token_accuracy": 0.7691607773303986,
-      "num_tokens": 156376.0,
-      "step": 60
-    },
-    {
-      "entropy": 0.9946551442146301,
-      "epoch": 0.06013745704467354,
-      "grad_norm": 0.14451684057712555,
-      "learning_rate": 0.00029110824742268037,
-      "loss": 0.9979846954345704,
-      "mean_token_accuracy": 0.7634965121746063,
-      "num_tokens": 183753.0,
-      "step": 70
-    },
-    {
-      "entropy": 0.9941512286663056,
-      "epoch": 0.06872852233676977,
-      "grad_norm": 0.15088772773742676,
-      "learning_rate": 0.00028981958762886595,
-      "loss": 0.9917948722839356,
-      "mean_token_accuracy": 0.7609466493129731,
-      "num_tokens": 209154.0,
-      "step": 80
-    },
-    {
-      "entropy": 0.899951022863388,
-      "epoch": 0.07731958762886598,
-      "grad_norm": 0.1635286509990692,
-      "learning_rate": 0.0002885309278350515,
-      "loss": 0.9084607124328613,
-      "mean_token_accuracy": 0.773688817024231,
-      "num_tokens": 232285.0,
-      "step": 90
-    },
-    {
-      "entropy": 0.9840596318244934,
-      "epoch": 0.0859106529209622,
-      "grad_norm": 0.1687513142824173,
-      "learning_rate": 0.0002872422680412371,
-      "loss": 1.005988883972168,
-      "mean_token_accuracy": 0.7565966248512268,
-      "num_tokens": 258165.0,
-      "step": 100
-    },
-    {
-      "entropy": 0.9256654202938079,
-      "epoch": 0.09450171821305842,
-      "grad_norm": 0.15977086126804352,
-      "learning_rate": 0.0002859536082474227,
-      "loss": 0.8968193054199218,
-      "mean_token_accuracy": 0.7728870630264282,
-      "num_tokens": 283613.0,
-      "step": 110
-    },
-    {
-      "entropy": 0.9430408120155335,
-      "epoch": 0.10309278350515463,
-      "grad_norm": 0.14840301871299744,
-      "learning_rate": 0.0002846649484536082,
-      "loss": 0.9234983444213867,
-      "mean_token_accuracy": 0.7652122378349304,
-      "num_tokens": 308257.0,
-      "step": 120
-    },
-    {
-      "entropy": 0.9537142395973206,
-      "epoch": 0.11168384879725086,
-      "grad_norm": 0.1754530817270279,
-      "learning_rate": 0.0002833762886597938,
-      "loss": 0.949033260345459,
-      "mean_token_accuracy": 0.7616437077522278,
-      "num_tokens": 333700.0,
-      "step": 130
-    },
-    {
-      "entropy": 0.9385378897190094,
-      "epoch": 0.12027491408934708,
-      "grad_norm": 0.16817958652973175,
-      "learning_rate": 0.00028208762886597935,
-      "loss": 0.9222366333007812,
-      "mean_token_accuracy": 0.7721138775348664,
-      "num_tokens": 359746.0,
-      "step": 140
-    },
-    {
-      "entropy": 0.8476657152175904,
-      "epoch": 0.12886597938144329,
-      "grad_norm": 0.15769453346729279,
-      "learning_rate": 0.0002807989690721649,
-      "loss": 0.8225071907043457,
-      "mean_token_accuracy": 0.7880932271480561,
-      "num_tokens": 383468.0,
-      "step": 150
-    },
-    {
-      "entropy": 0.8862757325172425,
-      "epoch": 0.13745704467353953,
-      "grad_norm": 0.17620904743671417,
-      "learning_rate": 0.0002795103092783505,
-      "loss": 0.8871653556823731,
-      "mean_token_accuracy": 0.7726393282413483,
-      "num_tokens": 409660.0,
-      "step": 160
-    },
-    {
-      "entropy": 0.9064472615718842,
-      "epoch": 0.14604810996563575,
-      "grad_norm": 0.15342549979686737,
-      "learning_rate": 0.0002782216494845361,
-      "loss": 0.9092922210693359,
-      "mean_token_accuracy": 0.7713103413581848,
-      "num_tokens": 437071.0,
-      "step": 170
-    },
-    {
-      "entropy": 0.9095455348491669,
-      "epoch": 0.15463917525773196,
-      "grad_norm": 0.17525391280651093,
-      "learning_rate": 0.00027693298969072165,
-      "loss": 0.8956180572509765,
-      "mean_token_accuracy": 0.7772489190101624,
-      "num_tokens": 460852.0,
-      "step": 180
-    },
-    {
-      "entropy": 0.8514094173908233,
-      "epoch": 0.16323024054982818,
-      "grad_norm": 0.15829823911190033,
-      "learning_rate": 0.0002756443298969072,
-      "loss": 0.8513413429260254,
-      "mean_token_accuracy": 0.779003256559372,
-      "num_tokens": 486114.0,
-      "step": 190
-    },
-    {
-      "entropy": 0.8995061337947845,
-      "epoch": 0.1718213058419244,
-      "grad_norm": 0.15622206032276154,
-      "learning_rate": 0.00027435567010309275,
-      "loss": 0.898340892791748,
-      "mean_token_accuracy": 0.7708433449268342,
-      "num_tokens": 512397.0,
-      "step": 200
-    },
-    {
-      "entropy": 0.8732269763946533,
-      "epoch": 0.18041237113402062,
-      "grad_norm": 0.17759552597999573,
-      "learning_rate": 0.0002730670103092783,
-      "loss": 0.8695596694946289,
-      "mean_token_accuracy": 0.7793173313140869,
-      "num_tokens": 536554.0,
-      "step": 210
-    },
-    {
-      "entropy": 0.9366106212139129,
-      "epoch": 0.18900343642611683,
-      "grad_norm": 0.1767597794532776,
-      "learning_rate": 0.0002717783505154639,
-      "loss": 0.9249121665954589,
-      "mean_token_accuracy": 0.7724991559982299,
-      "num_tokens": 561637.0,
-      "step": 220
-    },
-    {
-      "entropy": 0.8857321858406066,
-      "epoch": 0.19759450171821305,
-      "grad_norm": 0.20788735151290894,
-      "learning_rate": 0.0002704896907216495,
-      "loss": 0.8790401458740235,
-      "mean_token_accuracy": 0.7778670430183411,
-      "num_tokens": 586614.0,
-      "step": 230
-    },
-    {
-      "entropy": 0.9273143649101258,
-      "epoch": 0.20618556701030927,
-      "grad_norm": 0.16100187599658966,
-      "learning_rate": 0.00026920103092783505,
-      "loss": 0.9232232093811035,
-      "mean_token_accuracy": 0.7677364766597747,
-      "num_tokens": 612428.0,
-      "step": 240
-    },
-    {
-      "entropy": 0.8786617696285248,
-      "epoch": 0.21477663230240548,
-      "grad_norm": 0.1608886867761612,
-      "learning_rate": 0.00026791237113402063,
-      "loss": 0.8883329391479492,
-      "mean_token_accuracy": 0.7775621056556702,
-      "num_tokens": 638564.0,
-      "step": 250
-    },
-    {
-      "entropy": 0.9227964103221893,
-      "epoch": 0.22336769759450173,
-      "grad_norm": 0.1510680615901947,
-      "learning_rate": 0.00026662371134020615,
-      "loss": 0.9296038627624512,
-      "mean_token_accuracy": 0.7714014649391174,
-      "num_tokens": 665263.0,
-      "step": 260
-    },
-    {
-      "entropy": 0.9126189887523651,
-      "epoch": 0.23195876288659795,
-      "grad_norm": 0.16434459388256073,
-      "learning_rate": 0.00026533505154639173,
-      "loss": 0.9191689491271973,
-      "mean_token_accuracy": 0.7705935597419739,
-      "num_tokens": 691956.0,
-      "step": 270
-    },
-    {
-      "entropy": 0.8797955513000488,
-      "epoch": 0.24054982817869416,
-      "grad_norm": 0.1714707911014557,
-      "learning_rate": 0.0002640463917525773,
-      "loss": 0.8856119155883789,
-      "mean_token_accuracy": 0.7766761302947998,
-      "num_tokens": 716820.0,
-      "step": 280
-    },
-    {
-      "entropy": 0.877093505859375,
-      "epoch": 0.24914089347079038,
-      "grad_norm": 0.15326625108718872,
-      "learning_rate": 0.0002627577319587629,
-      "loss": 0.8566678047180176,
-      "mean_token_accuracy": 0.7812119901180268,
-      "num_tokens": 742221.0,
-      "step": 290
-    },
-    {
-      "entropy": 0.87042076587677,
-      "epoch": 0.25773195876288657,
-      "grad_norm": 0.1710847020149231,
-      "learning_rate": 0.00026146907216494846,
-      "loss": 0.8829298973083496,
-      "mean_token_accuracy": 0.7770332932472229,
-      "num_tokens": 765839.0,
-      "step": 300
-    },
-    {
-      "entropy": 0.8417458057403564,
-      "epoch": 0.2663230240549828,
-      "grad_norm": 0.1496821790933609,
-      "learning_rate": 0.00026018041237113403,
-      "loss": 0.8433545112609864,
-      "mean_token_accuracy": 0.7857938230037689,
-      "num_tokens": 791131.0,
-      "step": 310
-    },
-    {
-      "entropy": 0.9158999443054199,
-      "epoch": 0.27491408934707906,
-      "grad_norm": 0.17929035425186157,
-      "learning_rate": 0.00025889175257731955,
-      "loss": 0.9123490333557129,
-      "mean_token_accuracy": 0.775556480884552,
-      "num_tokens": 816984.0,
-      "step": 320
-    },
-    {
-      "entropy": 0.8314016401767731,
-      "epoch": 0.28350515463917525,
-      "grad_norm": 0.16051091253757477,
-      "learning_rate": 0.00025760309278350513,
-      "loss": 0.8279628753662109,
-      "mean_token_accuracy": 0.7862712442874908,
-      "num_tokens": 842220.0,
-      "step": 330
-    },
-    {
-      "entropy": 0.8677021145820618,
-      "epoch": 0.2920962199312715,
-      "grad_norm": 0.19157563149929047,
-      "learning_rate": 0.0002563144329896907,
-      "loss": 0.8552864074707032,
-      "mean_token_accuracy": 0.7835110187530517,
-      "num_tokens": 867868.0,
-      "step": 340
-    },
-    {
-      "entropy": 0.8919401407241822,
-      "epoch": 0.3006872852233677,
-      "grad_norm": 0.14258626103401184,
-      "learning_rate": 0.0002550257731958763,
-      "loss": 0.9019416809082031,
-      "mean_token_accuracy": 0.7760707855224609,
-      "num_tokens": 894676.0,
-      "step": 350
-    },
-    {
-      "entropy": 0.8415017545223236,
-      "epoch": 0.30927835051546393,
-      "grad_norm": 0.14830808341503143,
-      "learning_rate": 0.00025373711340206186,
-      "loss": 0.8072587013244629,
-      "mean_token_accuracy": 0.7912742376327515,
-      "num_tokens": 921632.0,
-      "step": 360
-    },
-    {
-      "entropy": 0.8371863842010498,
-      "epoch": 0.3178694158075601,
-      "grad_norm": 0.15538977086544037,
-      "learning_rate": 0.0002524484536082474,
-      "loss": 0.8308550834655761,
-      "mean_token_accuracy": 0.7849089920520782,
-      "num_tokens": 945955.0,
-      "step": 370
-    },
-    {
-      "entropy": 0.8338176369667053,
-      "epoch": 0.32646048109965636,
-      "grad_norm": 0.1711844801902771,
-      "learning_rate": 0.00025115979381443295,
-      "loss": 0.8479556083679199,
-      "mean_token_accuracy": 0.7850408136844635,
-      "num_tokens": 971508.0,
-      "step": 380
-    },
-    {
-      "entropy": 0.8876297354698182,
-      "epoch": 0.33505154639175255,
-      "grad_norm": 0.14902909100055695,
-      "learning_rate": 0.00024987113402061853,
-      "loss": 0.8821809768676758,
-      "mean_token_accuracy": 0.7764640510082245,
-      "num_tokens": 995124.0,
-      "step": 390
-    },
-    {
-      "entropy": 0.8212514638900756,
-      "epoch": 0.3436426116838488,
-      "grad_norm": 0.17950233817100525,
-      "learning_rate": 0.0002485824742268041,
-      "loss": 0.8255527496337891,
-      "mean_token_accuracy": 0.7870513379573822,
-      "num_tokens": 1019735.0,
-      "step": 400
-    },
-    {
-      "entropy": 0.9294387817382812,
-      "epoch": 0.35223367697594504,
-      "grad_norm": 0.15237703919410706,
-      "learning_rate": 0.0002472938144329897,
-      "loss": 0.9235431671142578,
-      "mean_token_accuracy": 0.7703655660152435,
-      "num_tokens": 1046295.0,
-      "step": 410
-    },
-    {
-      "entropy": 0.8429355442523956,
-      "epoch": 0.36082474226804123,
-      "grad_norm": 0.1589658558368683,
-      "learning_rate": 0.0002460051546391752,
-      "loss": 0.8329957008361817,
-      "mean_token_accuracy": 0.7897172749042511,
-      "num_tokens": 1072020.0,
-      "step": 420
-    },
-    {
-      "entropy": 0.8407810032367706,
-      "epoch": 0.3694158075601375,
-      "grad_norm": 0.17521116137504578,
-      "learning_rate": 0.0002447164948453608,
-      "loss": 0.8535200119018554,
-      "mean_token_accuracy": 0.7809976935386658,
-      "num_tokens": 1098420.0,
-      "step": 430
-    },
-    {
-      "entropy": 0.8515082120895385,
-      "epoch": 0.37800687285223367,
-      "grad_norm": 0.15218117833137512,
-      "learning_rate": 0.00024342783505154638,
-      "loss": 0.8459153175354004,
-      "mean_token_accuracy": 0.7833445191383361,
-      "num_tokens": 1124235.0,
-      "step": 440
-    },
-    {
-      "entropy": 0.8506015419960022,
-      "epoch": 0.3865979381443299,
-      "grad_norm": 0.1809423863887787,
-      "learning_rate": 0.00024213917525773193,
-      "loss": 0.8384982109069824,
-      "mean_token_accuracy": 0.7841372966766358,
-      "num_tokens": 1149636.0,
-      "step": 450
-    },
-    {
-      "entropy": 0.9110018074512481,
-      "epoch": 0.3951890034364261,
-      "grad_norm": 0.15085963904857635,
-      "learning_rate": 0.0002408505154639175,
-      "loss": 0.9191832542419434,
-      "mean_token_accuracy": 0.7757422685623169,
-      "num_tokens": 1173711.0,
-      "step": 460
-    },
-    {
-      "entropy": 0.9000201225280762,
-      "epoch": 0.40378006872852235,
-      "grad_norm": 0.16748838126659393,
-      "learning_rate": 0.00023956185567010308,
-      "loss": 0.9107230186462403,
-      "mean_token_accuracy": 0.7768173336982727,
-      "num_tokens": 1199340.0,
-      "step": 470
-    },
-    {
-      "entropy": 0.8751251935958863,
-      "epoch": 0.41237113402061853,
-      "grad_norm": 0.1593722254037857,
-      "learning_rate": 0.00023827319587628863,
-      "loss": 0.8804635047912598,
-      "mean_token_accuracy": 0.7814023733139038,
-      "num_tokens": 1226032.0,
-      "step": 480
-    },
-    {
-      "entropy": 0.8843371510505676,
-      "epoch": 0.4209621993127148,
-      "grad_norm": 0.12278851121664047,
-      "learning_rate": 0.0002369845360824742,
-      "loss": 0.8643080711364746,
-      "mean_token_accuracy": 0.7809630572795868,
-      "num_tokens": 1252769.0,
-      "step": 490
-    },
-    {
-      "entropy": 0.8834336459636688,
-      "epoch": 0.42955326460481097,
-      "grad_norm": 0.1367531716823578,
-      "learning_rate": 0.00023569587628865976,
-      "loss": 0.8869522094726563,
-      "mean_token_accuracy": 0.7775909185409546,
-      "num_tokens": 1278979.0,
-      "step": 500
-    }
-  ],
-  "logging_steps": 10,
-  "max_steps": 2328,
-  "num_input_tokens_seen": 0,
-  "num_train_epochs": 2,
-  "save_steps": 500,
-  "stateful_callbacks": {
-    "TrainerControl": {
-      "args": {
-        "should_epoch_stop": false,
-        "should_evaluate": false,
-        "should_log": false,
-        "should_save": true,
-        "should_training_stop": false
-      },
-      "attributes": {}
-    }
-  },
-  "total_flos": 2129667010873344.0,
-  "train_batch_size": 16,
-  "trial_name": null,
-  "trial_params": null
-}

checkpoint-500/training_args.bin

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f2af45870b4c4d611381846df6bdd4a32cb292a7b5897b79b7e5f196be57c596
-size 5585