Instructions to use madhuHuggingface/functiongemma-ec2-finetuned with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use madhuHuggingface/functiongemma-ec2-finetuned with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("madhuHuggingface/functiongemma-ec2-finetuned", dtype="auto")

llama-cpp-python

How to use madhuHuggingface/functiongemma-ec2-finetuned with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="madhuHuggingface/functiongemma-ec2-finetuned",
	filename="gguf/functiongemma-270m-it.Q8_0.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use madhuHuggingface/functiongemma-ec2-finetuned with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf madhuHuggingface/functiongemma-ec2-finetuned:Q8_0
# Run inference directly in the terminal:
llama-cli -hf madhuHuggingface/functiongemma-ec2-finetuned:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf madhuHuggingface/functiongemma-ec2-finetuned:Q8_0
# Run inference directly in the terminal:
llama-cli -hf madhuHuggingface/functiongemma-ec2-finetuned:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf madhuHuggingface/functiongemma-ec2-finetuned:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf madhuHuggingface/functiongemma-ec2-finetuned:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf madhuHuggingface/functiongemma-ec2-finetuned:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf madhuHuggingface/functiongemma-ec2-finetuned:Q8_0

Use Docker

docker model run hf.co/madhuHuggingface/functiongemma-ec2-finetuned:Q8_0

LM Studio
Jan
Ollama
How to use madhuHuggingface/functiongemma-ec2-finetuned with Ollama:
```
ollama run hf.co/madhuHuggingface/functiongemma-ec2-finetuned:Q8_0
```

Unsloth Studio

How to use madhuHuggingface/functiongemma-ec2-finetuned with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for madhuHuggingface/functiongemma-ec2-finetuned to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for madhuHuggingface/functiongemma-ec2-finetuned to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for madhuHuggingface/functiongemma-ec2-finetuned to start chatting

How to use madhuHuggingface/functiongemma-ec2-finetuned with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf madhuHuggingface/functiongemma-ec2-finetuned:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "madhuHuggingface/functiongemma-ec2-finetuned:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use madhuHuggingface/functiongemma-ec2-finetuned with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf madhuHuggingface/functiongemma-ec2-finetuned:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default madhuHuggingface/functiongemma-ec2-finetuned:Q8_0

Run Hermes

hermes

Docker Model Runner
How to use madhuHuggingface/functiongemma-ec2-finetuned with Docker Model Runner:
```
docker model run hf.co/madhuHuggingface/functiongemma-ec2-finetuned:Q8_0
```

Lemonade

How to use madhuHuggingface/functiongemma-ec2-finetuned with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull madhuHuggingface/functiongemma-ec2-finetuned:Q8_0

Run and chat with the model

lemonade run user.functiongemma-ec2-finetuned-Q8_0

List all available models

lemonade list

madhuHuggingface commited on Apr 27

Commit

93c3581

verified ·

1 Parent(s): 02e84c9

Training in progress, step 3000, checkpoint

Browse files

Files changed (7) hide show

last-checkpoint/adapter_config.json +5 -5
last-checkpoint/adapter_model.safetensors +1 -1
last-checkpoint/optimizer.pt +1 -1
last-checkpoint/rng_state.pth +1 -1
last-checkpoint/scheduler.pt +1 -1
last-checkpoint/trainer_state.json +1952 -202
last-checkpoint/training_args.bin +1 -1

last-checkpoint/adapter_config.json CHANGED Viewed

@@ -33,13 +33,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "gate_proj",
     "down_proj",
-    "up_proj",
-    "q_proj",
-    "v_proj",
     "k_proj",
-    "o_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
+    "gate_proj",
     "k_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

last-checkpoint/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:018cb6d9db46f0ce13f349c696bdff67ba6a901dfa5aed13a0d047c7e30d4f9c
 size 121537408

 version https://git-lfs.github.com/spec/v1
+oid sha256:a9439369cdca91e9821590c85c5a4c1357c77a70eb9ab25f75b0f4e50cf95bb1
 size 121537408

last-checkpoint/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:68e7ed425766c35922189444ff63fdd81e7ebecc193c7c6e5fc44ea98e0cc541
 size 62000725

 version https://git-lfs.github.com/spec/v1
+oid sha256:1328e3e149b947f6884678f87ea27f348aec465791c71944f2b44b43e70ced61
 size 62000725

last-checkpoint/rng_state.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f1d565802a8e26c4e8a31328752b7a7fdc186d9401aa008e65697d0ad8c22e33
 size 14645

 version https://git-lfs.github.com/spec/v1
+oid sha256:7c800b778fa7e115e4c34de8529902de8b61c9a1b4bab3eb8295d06dafff030e
 size 14645

last-checkpoint/scheduler.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4b9811df9ea96a4280268b28843ba2166267f887cc556319f28a0df33b75a25a
 size 1465

 version https://git-lfs.github.com/spec/v1
+oid sha256:fcbab70503ebd427552c6d55065506bf9fdc83718e701b1734d49f171ccd8158
 size 1465

last-checkpoint/trainer_state.json CHANGED Viewed

@@ -2,366 +2,2116 @@
   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 0.32,
   "eval_steps": 500,
-  "global_step": 500,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
   "log_history": [
     {
-      "epoch": 0.0064,
-      "grad_norm": 40.911293029785156,
       "learning_rate": 9e-05,
-      "loss": 1.1173,
       "step": 10
     },
     {
-      "epoch": 0.0128,
-      "grad_norm": 4.927021026611328,
       "learning_rate": 0.00019,
-      "loss": 0.9803,
       "step": 20
     },
     {
-      "epoch": 0.0192,
-      "grad_norm": 13.638679504394531,
-      "learning_rate": 0.00019999816639538432,
-      "loss": 0.4325,
       "step": 30
     },
     {
-      "epoch": 0.0256,
-      "grad_norm": 3.560319423675537,
-      "learning_rate": 0.00019999182809538713,
-      "loss": 0.1852,
       "step": 40
     },
     {
-      "epoch": 0.032,
-      "grad_norm": 0.6141765117645264,
-      "learning_rate": 0.00019998096274980728,
-      "loss": 0.1316,
       "step": 50
     },
     {
-      "epoch": 0.0384,
-      "grad_norm": 12.074165344238281,
-      "learning_rate": 0.00019996557085056414,
-      "loss": 0.1166,
       "step": 60
     },
     {
-      "epoch": 0.0448,
-      "grad_norm": 3.718775749206543,
-      "learning_rate": 0.00019994565309451292,
-      "loss": 0.1073,
       "step": 70
     },
     {
-      "epoch": 0.0512,
-      "grad_norm": 3.271156072616577,
-      "learning_rate": 0.0001999212103834132,
-      "loss": 0.2006,
       "step": 80
     },
     {
-      "epoch": 0.0576,
-      "grad_norm": 1.295325756072998,
-      "learning_rate": 0.00019989224382388813,
-      "loss": 0.0938,
       "step": 90
     },
     {
-      "epoch": 0.064,
-      "grad_norm": 1.3511019945144653,
-      "learning_rate": 0.00019985875472737415,
-      "loss": 0.0729,
       "step": 100
     },
     {
-      "epoch": 0.0704,
-      "grad_norm": 0.43467989563941956,
-      "learning_rate": 0.00019982074461006194,
-      "loss": 0.0511,
       "step": 110
     },
     {
-      "epoch": 0.0768,
-      "grad_norm": 0.451219767332077,
-      "learning_rate": 0.0001997782151928274,
-      "loss": 0.0431,
       "step": 120
     },
     {
-      "epoch": 0.0832,
-      "grad_norm": 0.5558948516845703,
-      "learning_rate": 0.00019973116840115397,
-      "loss": 0.1314,
       "step": 130
     },
     {
-      "epoch": 0.0896,
-      "grad_norm": 0.31716081500053406,
-      "learning_rate": 0.00019967960636504547,
-      "loss": 0.058,
       "step": 140
     },
     {
-      "epoch": 0.096,
-      "grad_norm": 0.08058512210845947,
-      "learning_rate": 0.00019962353141892946,
-      "loss": 0.0373,
       "step": 150
     },
     {
-      "epoch": 0.1024,
-      "grad_norm": 0.0707520842552185,
-      "learning_rate": 0.00019956294610155186,
-      "loss": 0.0288,
       "step": 160
     },
     {
-      "epoch": 0.1088,
-      "grad_norm": 0.06796549260616302,
-      "learning_rate": 0.00019949785315586172,
-      "loss": 0.026,
       "step": 170
     },
     {
-      "epoch": 0.1152,
-      "grad_norm": 1.5500516891479492,
-      "learning_rate": 0.00019942825552888717,
-      "loss": 0.029,
       "step": 180
     },
     {
-      "epoch": 0.1216,
-      "grad_norm": 0.6438320279121399,
-      "learning_rate": 0.00019935415637160203,
-      "loss": 0.0321,
       "step": 190
     },
     {
-      "epoch": 0.128,
-      "grad_norm": 0.7204006314277649,
-      "learning_rate": 0.0001992755590387831,
-      "loss": 0.0359,
       "step": 200
     },
     {
-      "epoch": 0.1344,
-      "grad_norm": 0.28932827711105347,
-      "learning_rate": 0.0001991924670888582,
-      "loss": 0.0252,
       "step": 210
     },
     {
-      "epoch": 0.1408,
-      "grad_norm": 0.46170133352279663,
-      "learning_rate": 0.0001991048842837452,
-      "loss": 0.0191,
       "step": 220
     },
     {
-      "epoch": 0.1472,
-      "grad_norm": 13.702939987182617,
-      "learning_rate": 0.00019901281458868173,
-      "loss": 0.0851,
       "step": 230
     },
     {
-      "epoch": 0.1536,
-      "grad_norm": 0.1703188419342041,
-      "learning_rate": 0.00019891626217204538,
-      "loss": 0.032,
       "step": 240
     },
     {
-      "epoch": 0.16,
-      "grad_norm": 0.1269959956407547,
-      "learning_rate": 0.00019881523140516535,
-      "loss": 0.0254,
       "step": 250
     },
     {
-      "epoch": 0.1664,
-      "grad_norm": 0.39395102858543396,
-      "learning_rate": 0.00019870972686212427,
-      "loss": 0.0465,
       "step": 260
     },
     {
-      "epoch": 0.1728,
-      "grad_norm": 0.7702052593231201,
-      "learning_rate": 0.00019859975331955126,
-      "loss": 0.0332,
       "step": 270
     },
     {
-      "epoch": 0.1792,
-      "grad_norm": 0.12844760715961456,
-      "learning_rate": 0.00019848531575640557,
-      "loss": 0.044,
       "step": 280
     },
     {
-      "epoch": 0.1856,
-      "grad_norm": 0.36863207817077637,
-      "learning_rate": 0.00019836641935375125,
-      "loss": 0.0233,
       "step": 290
     },
     {
-      "epoch": 0.192,
-      "grad_norm": 0.9188430309295654,
-      "learning_rate": 0.00019824306949452254,
-      "loss": 0.0276,
       "step": 300
     },
     {
-      "epoch": 0.1984,
-      "grad_norm": 0.49796220660209656,
-      "learning_rate": 0.00019811527176328016,
-      "loss": 0.0303,
       "step": 310
     },
     {
-      "epoch": 0.2048,
-      "grad_norm": 0.29771795868873596,
-      "learning_rate": 0.00019798303194595846,
-      "loss": 0.0148,
       "step": 320
     },
     {
-      "epoch": 0.2112,
-      "grad_norm": 0.08168808370828629,
-      "learning_rate": 0.00019784635602960353,
-      "loss": 0.0147,
       "step": 330
     },
     {
-      "epoch": 0.2176,
-      "grad_norm": 0.0375661738216877,
-      "learning_rate": 0.00019770525020210204,
-      "loss": 0.0181,
       "step": 340
     },
     {
-      "epoch": 0.224,
-      "grad_norm": 0.032354097813367844,
-      "learning_rate": 0.00019755972085190124,
-      "loss": 0.0136,
       "step": 350
     },
     {
-      "epoch": 0.2304,
-      "grad_norm": 0.03650612756609917,
-      "learning_rate": 0.0001974097745677195,
-      "loss": 0.0134,
       "step": 360
     },
     {
-      "epoch": 0.2368,
-      "grad_norm": 0.04585101455450058,
-      "learning_rate": 0.00019725541813824825,
-      "loss": 0.018,
       "step": 370
     },
     {
-      "epoch": 0.2432,
-      "grad_norm": 0.02587795816361904,
-      "learning_rate": 0.00019709665855184447,
-      "loss": 0.0171,
       "step": 380
     },
     {
-      "epoch": 0.2496,
-      "grad_norm": 0.019718684256076813,
-      "learning_rate": 0.00019693350299621442,
-      "loss": 0.0126,
       "step": 390
     },
     {
-      "epoch": 0.256,
-      "grad_norm": 0.01756865903735161,
-      "learning_rate": 0.0001967659588580881,
-      "loss": 0.0107,
       "step": 400
     },
     {
-      "epoch": 0.2624,
-      "grad_norm": 0.017915094271302223,
-      "learning_rate": 0.00019659403372288489,
-      "loss": 0.0102,
       "step": 410
     },
     {
-      "epoch": 0.2688,
-      "grad_norm": 0.01841767504811287,
-      "learning_rate": 0.00019641773537437004,
-      "loss": 0.0108,
       "step": 420
     },
     {
-      "epoch": 0.2752,
-      "grad_norm": 0.01479355152696371,
-      "learning_rate": 0.00019623707179430253,
-      "loss": 0.0116,
       "step": 430
     },
     {
-      "epoch": 0.2816,
-      "grad_norm": 0.01765090599656105,
-      "learning_rate": 0.00019605205116207333,
-      "loss": 0.0119,
       "step": 440
     },
     {
-      "epoch": 0.288,
-      "grad_norm": 0.014900445006787777,
-      "learning_rate": 0.0001958626818543353,
-      "loss": 0.0128,
       "step": 450
     },
     {
-      "epoch": 0.2944,
-      "grad_norm": 0.01522466167807579,
-      "learning_rate": 0.00019566897244462398,
-      "loss": 0.0129,
       "step": 460
     },
     {
-      "epoch": 0.3008,
-      "grad_norm": 0.014997781254351139,
-      "learning_rate": 0.0001954709317029693,
-      "loss": 0.0129,
       "step": 470
     },
     {
-      "epoch": 0.3072,
-      "grad_norm": 0.01686343364417553,
-      "learning_rate": 0.00019526856859549863,
-      "loss": 0.0135,
       "step": 480
     },
     {
-      "epoch": 0.3136,
-      "grad_norm": 0.012961529195308685,
-      "learning_rate": 0.00019506189228403082,
-      "loss": 0.0127,
       "step": 490
     },
     {
-      "epoch": 0.32,
-      "grad_norm": 0.0174432210624218,
-      "learning_rate": 0.00019485091212566132,
-      "loss": 0.0117,
       "step": 500
     }
   ],
   "logging_steps": 10,
-  "max_steps": 4689,
   "num_input_tokens_seen": 0,
   "num_train_epochs": 3,
   "save_steps": 100,
@@ -377,7 +2127,7 @@
       "attributes": {}
     }
   },
-  "total_flos": 6424262344704000.0,
   "train_batch_size": 2,
   "trial_name": null,
   "trial_params": null

   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
+  "epoch": 1.8717628705148206,
   "eval_steps": 500,
+  "global_step": 3000,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
   "log_history": [
     {
+      "epoch": 0.0062402496099844,
+      "grad_norm": 18.16543197631836,
       "learning_rate": 9e-05,
+      "loss": 2.5416,
       "step": 10
     },
     {
+      "epoch": 0.0124804992199688,
+      "grad_norm": 4.823841094970703,
       "learning_rate": 0.00019,
+      "loss": 0.2597,
       "step": 20
     },
     {
+      "epoch": 0.0187207488299532,
+      "grad_norm": 2.6858906745910645,
+      "learning_rate": 0.00019999825713466112,
+      "loss": 0.1064,
       "step": 30
     },
     {
+      "epoch": 0.0249609984399376,
+      "grad_norm": 1.4995951652526855,
+      "learning_rate": 0.00019999223249296698,
+      "loss": 0.074,
       "step": 40
     },
     {
+      "epoch": 0.031201248049921998,
+      "grad_norm": 4.664715766906738,
+      "learning_rate": 0.00019998190481726233,
+      "loss": 0.0804,
       "step": 50
     },
     {
+      "epoch": 0.0374414976599064,
+      "grad_norm": 0.7267784476280212,
+      "learning_rate": 0.00019996727455198513,
+      "loss": 0.0427,
       "step": 60
     },
     {
+      "epoch": 0.0436817472698908,
+      "grad_norm": 0.29998767375946045,
+      "learning_rate": 0.00019994834232672947,
+      "loss": 0.0267,
       "step": 70
     },
     {
+      "epoch": 0.0499219968798752,
+      "grad_norm": 0.43115338683128357,
+      "learning_rate": 0.00019992510895621885,
+      "loss": 0.0193,
       "step": 80
     },
     {
+      "epoch": 0.056162246489859596,
+      "grad_norm": 0.339687705039978,
+      "learning_rate": 0.00019989757544027072,
+      "loss": 0.0324,
       "step": 90
     },
     {
+      "epoch": 0.062402496099843996,
+      "grad_norm": 0.648891806602478,
+      "learning_rate": 0.00019986574296375364,
+      "loss": 0.0339,
       "step": 100
     },
     {
+      "epoch": 0.0686427457098284,
+      "grad_norm": 1.3054920434951782,
+      "learning_rate": 0.0001998296128965363,
+      "loss": 0.0243,
       "step": 110
     },
     {
+      "epoch": 0.0748829953198128,
+      "grad_norm": 1.6219518184661865,
+      "learning_rate": 0.00019978918679342865,
+      "loss": 0.0357,
       "step": 120
     },
     {
+      "epoch": 0.0811232449297972,
+      "grad_norm": 0.8563578128814697,
+      "learning_rate": 0.00019974446639411464,
+      "loss": 0.0224,
       "step": 130
     },
     {
+      "epoch": 0.0873634945397816,
+      "grad_norm": 0.16929180920124054,
+      "learning_rate": 0.00019969545362307785,
+      "loss": 0.0115,
       "step": 140
     },
     {
+      "epoch": 0.093603744149766,
+      "grad_norm": 1.1288150548934937,
+      "learning_rate": 0.0001996421505895183,
+      "loss": 0.0134,
       "step": 150
     },
     {
+      "epoch": 0.0998439937597504,
+      "grad_norm": 0.6310799717903137,
+      "learning_rate": 0.0001995845595872618,
+      "loss": 0.0242,
       "step": 160
     },
     {
+      "epoch": 0.1060842433697348,
+      "grad_norm": 1.2807273864746094,
+      "learning_rate": 0.0001995226830946613,
+      "loss": 0.0163,
       "step": 170
     },
     {
+      "epoch": 0.11232449297971919,
+      "grad_norm": 0.5181924700737,
+      "learning_rate": 0.00019945652377449012,
+      "loss": 0.0314,
       "step": 180
     },
     {
+      "epoch": 0.11856474258970359,
+      "grad_norm": 2.7124929428100586,
+      "learning_rate": 0.0001993860844738275,
+      "loss": 0.0241,
       "step": 190
     },
     {
+      "epoch": 0.12480499219968799,
+      "grad_norm": 0.7487115859985352,
+      "learning_rate": 0.0001993113682239359,
+      "loss": 0.0206,
       "step": 200
     },
     {
+      "epoch": 0.1310452418096724,
+      "grad_norm": 0.9230497479438782,
+      "learning_rate": 0.00019923237824013076,
+      "loss": 0.011,
       "step": 210
     },
     {
+      "epoch": 0.1372854914196568,
+      "grad_norm": 4.362138271331787,
+      "learning_rate": 0.000199149117921642,
+      "loss": 0.0182,
       "step": 220
     },
     {
+      "epoch": 0.1435257410296412,
+      "grad_norm": 2.675947904586792,
+      "learning_rate": 0.00019906159085146768,
+      "loss": 0.0205,
       "step": 230
     },
     {
+      "epoch": 0.1497659906396256,
+      "grad_norm": 0.9578750729560852,
+      "learning_rate": 0.00019896980079622,
+      "loss": 0.0142,
       "step": 240
     },
     {
+      "epoch": 0.15600624024961,
+      "grad_norm": 0.5006669759750366,
+      "learning_rate": 0.00019887375170596312,
+      "loss": 0.0296,
       "step": 250
     },
     {
+      "epoch": 0.1622464898595944,
+      "grad_norm": 1.6358978748321533,
+      "learning_rate": 0.00019877344771404307,
+      "loss": 0.0261,
       "step": 260
     },
     {
+      "epoch": 0.1684867394695788,
+      "grad_norm": 5.082851886749268,
+      "learning_rate": 0.00019866889313691,
+      "loss": 0.0169,
       "step": 270
     },
     {
+      "epoch": 0.1747269890795632,
+      "grad_norm": 0.11479347199201584,
+      "learning_rate": 0.0001985600924739325,
+      "loss": 0.0229,
       "step": 280
     },
     {
+      "epoch": 0.1809672386895476,
+      "grad_norm": 0.9521530866622925,
+      "learning_rate": 0.0001984470504072037,
+      "loss": 0.0193,
       "step": 290
     },
     {
+      "epoch": 0.187207488299532,
+      "grad_norm": 2.097628593444824,
+      "learning_rate": 0.0001983297718013401,
+      "loss": 0.0217,
       "step": 300
     },
     {
+      "epoch": 0.1934477379095164,
+      "grad_norm": 0.75714111328125,
+      "learning_rate": 0.00019820826170327194,
+      "loss": 0.0176,
       "step": 310
     },
     {
+      "epoch": 0.1996879875195008,
+      "grad_norm": 1.0679302215576172,
+      "learning_rate": 0.0001980825253420263,
+      "loss": 0.0145,
       "step": 320
     },
     {
+      "epoch": 0.2059282371294852,
+      "grad_norm": 0.7506147623062134,
+      "learning_rate": 0.0001979525681285018,
+      "loss": 0.0132,
       "step": 330
     },
     {
+      "epoch": 0.2121684867394696,
+      "grad_norm": 0.606416642665863,
+      "learning_rate": 0.000197818395655236,
+      "loss": 0.0231,
       "step": 340
     },
     {
+      "epoch": 0.21840873634945399,
+      "grad_norm": 0.4706982672214508,
+      "learning_rate": 0.0001976800136961645,
+      "loss": 0.0304,
       "step": 350
     },
     {
+      "epoch": 0.22464898595943839,
+      "grad_norm": 0.04006557539105415,
+      "learning_rate": 0.0001975374282063727,
+      "loss": 0.0249,
       "step": 360
     },
     {
+      "epoch": 0.23088923556942278,
+      "grad_norm": 0.1513279676437378,
+      "learning_rate": 0.0001973906453218392,
+      "loss": 0.0173,
       "step": 370
     },
     {
+      "epoch": 0.23712948517940718,
+      "grad_norm": 0.34040647745132446,
+      "learning_rate": 0.00019723967135917223,
+      "loss": 0.0121,
       "step": 380
     },
     {
+      "epoch": 0.24336973478939158,
+      "grad_norm": 0.6798901557922363,
+      "learning_rate": 0.0001970845128153373,
+      "loss": 0.0265,
       "step": 390
     },
     {
+      "epoch": 0.24960998439937598,
+      "grad_norm": 0.1554585099220276,
+      "learning_rate": 0.000196925176367378,
+      "loss": 0.0105,
       "step": 400
     },
     {
+      "epoch": 0.25585023400936036,
+      "grad_norm": 0.2256622165441513,
+      "learning_rate": 0.0001967616688721285,
+      "loss": 0.0128,
       "step": 410
     },
     {
+      "epoch": 0.2620904836193448,
+      "grad_norm": 0.2520933151245117,
+      "learning_rate": 0.00019659399736591843,
+      "loss": 0.017,
       "step": 420
     },
     {
+      "epoch": 0.26833073322932915,
+      "grad_norm": 0.6182873249053955,
+      "learning_rate": 0.00019642216906427026,
+      "loss": 0.0128,
       "step": 430
     },
     {
+      "epoch": 0.2745709828393136,
+      "grad_norm": 0.028039440512657166,
+      "learning_rate": 0.00019624619136158854,
+      "loss": 0.0155,
       "step": 440
     },
     {
+      "epoch": 0.28081123244929795,
+      "grad_norm": 0.018251167610287666,
+      "learning_rate": 0.00019606607183084192,
+      "loss": 0.0107,
       "step": 450
     },
     {
+      "epoch": 0.2870514820592824,
+      "grad_norm": 0.1820351779460907,
+      "learning_rate": 0.00019588181822323717,
+      "loss": 0.0063,
       "step": 460
     },
     {
+      "epoch": 0.29329173166926675,
+      "grad_norm": 0.4565224051475525,
+      "learning_rate": 0.00019569343846788557,
+      "loss": 0.012,
       "step": 470
     },
     {
+      "epoch": 0.2995319812792512,
+      "grad_norm": 0.01261977944523096,
+      "learning_rate": 0.00019550094067146174,
+      "loss": 0.0076,
       "step": 480
     },
     {
+      "epoch": 0.30577223088923555,
+      "grad_norm": 0.038973432034254074,
+      "learning_rate": 0.0001953043331178548,
+      "loss": 0.0048,
       "step": 490
     },
     {
+      "epoch": 0.31201248049922,
+      "grad_norm": 0.2936696410179138,
+      "learning_rate": 0.0001951036242678118,
+      "loss": 0.0062,
       "step": 500
+    },
+    {
+      "epoch": 0.31825273010920435,
+      "grad_norm": 0.6721827387809753,
+      "learning_rate": 0.00019489882275857377,
+      "loss": 0.0152,
+      "step": 510
+    },
+    {
+      "epoch": 0.3244929797191888,
+      "grad_norm": 0.0886274203658104,
+      "learning_rate": 0.00019468993740350387,
+      "loss": 0.0073,
+      "step": 520
+    },
+    {
+      "epoch": 0.33073322932917315,
+      "grad_norm": 0.03001420944929123,
+      "learning_rate": 0.00019447697719170815,
+      "loss": 0.0091,
+      "step": 530
+    },
+    {
+      "epoch": 0.3369734789391576,
+      "grad_norm": 0.3404795229434967,
+      "learning_rate": 0.00019425995128764887,
+      "loss": 0.0155,
+      "step": 540
+    },
+    {
+      "epoch": 0.34321372854914195,
+      "grad_norm": 0.07943115383386612,
+      "learning_rate": 0.00019403886903074987,
+      "loss": 0.0076,
+      "step": 550
+    },
+    {
+      "epoch": 0.3494539781591264,
+      "grad_norm": 0.018820637837052345,
+      "learning_rate": 0.00019381373993499493,
+      "loss": 0.0112,
+      "step": 560
+    },
+    {
+      "epoch": 0.35569422776911075,
+      "grad_norm": 0.49909457564353943,
+      "learning_rate": 0.0001935845736885181,
+      "loss": 0.018,
+      "step": 570
+    },
+    {
+      "epoch": 0.3619344773790952,
+      "grad_norm": 0.30171993374824524,
+      "learning_rate": 0.0001933513801531869,
+      "loss": 0.0076,
+      "step": 580
+    },
+    {
+      "epoch": 0.36817472698907955,
+      "grad_norm": 1.3735274076461792,
+      "learning_rate": 0.000193114169364178,
+      "loss": 0.0112,
+      "step": 590
+    },
+    {
+      "epoch": 0.374414976599064,
+      "grad_norm": 0.22438661754131317,
+      "learning_rate": 0.0001928729515295452,
+      "loss": 0.0117,
+      "step": 600
+    },
+    {
+      "epoch": 0.38065522620904835,
+      "grad_norm": 0.005140058230608702,
+      "learning_rate": 0.00019262773702978026,
+      "loss": 0.0085,
+      "step": 610
+    },
+    {
+      "epoch": 0.3868954758190328,
+      "grad_norm": 0.33452799916267395,
+      "learning_rate": 0.0001923785364173662,
+      "loss": 0.0295,
+      "step": 620
+    },
+    {
+      "epoch": 0.39313572542901715,
+      "grad_norm": 0.2755588889122009,
+      "learning_rate": 0.0001921253604163231,
+      "loss": 0.0106,
+      "step": 630
+    },
+    {
+      "epoch": 0.3993759750390016,
+      "grad_norm": 0.36959072947502136,
+      "learning_rate": 0.00019186821992174665,
+      "loss": 0.0085,
+      "step": 640
+    },
+    {
+      "epoch": 0.40561622464898595,
+      "grad_norm": 0.2535085082054138,
+      "learning_rate": 0.00019160712599933938,
+      "loss": 0.017,
+      "step": 650
+    },
+    {
+      "epoch": 0.4118564742589704,
+      "grad_norm": 0.06113250553607941,
+      "learning_rate": 0.00019134208988493424,
+      "loss": 0.0089,
+      "step": 660
+    },
+    {
+      "epoch": 0.41809672386895474,
+      "grad_norm": 0.12086120992898941,
+      "learning_rate": 0.00019107312298401137,
+      "loss": 0.0037,
+      "step": 670
+    },
+    {
+      "epoch": 0.4243369734789392,
+      "grad_norm": 0.2163705676794052,
+      "learning_rate": 0.00019080023687120698,
+      "loss": 0.0125,
+      "step": 680
+    },
+    {
+      "epoch": 0.43057722308892354,
+      "grad_norm": 0.904460608959198,
+      "learning_rate": 0.00019052344328981553,
+      "loss": 0.0155,
+      "step": 690
+    },
+    {
+      "epoch": 0.43681747269890797,
+      "grad_norm": 0.5320896506309509,
+      "learning_rate": 0.00019024275415128412,
+      "loss": 0.0175,
+      "step": 700
+    },
+    {
+      "epoch": 0.44305772230889234,
+      "grad_norm": 0.31751132011413574,
+      "learning_rate": 0.00018995818153470017,
+      "loss": 0.0447,
+      "step": 710
+    },
+    {
+      "epoch": 0.44929797191887677,
+      "grad_norm": 0.023313090205192566,
+      "learning_rate": 0.00018966973768627134,
+      "loss": 0.0147,
+      "step": 720
+    },
+    {
+      "epoch": 0.45553822152886114,
+      "grad_norm": 0.3516482710838318,
+      "learning_rate": 0.0001893774350187987,
+      "loss": 0.0117,
+      "step": 730
+    },
+    {
+      "epoch": 0.46177847113884557,
+      "grad_norm": 0.8473337292671204,
+      "learning_rate": 0.0001890812861111425,
+      "loss": 0.0154,
+      "step": 740
+    },
+    {
+      "epoch": 0.46801872074882994,
+      "grad_norm": 0.50248122215271,
+      "learning_rate": 0.0001887813037076809,
+      "loss": 0.0093,
+      "step": 750
+    },
+    {
+      "epoch": 0.47425897035881437,
+      "grad_norm": 1.4992247819900513,
+      "learning_rate": 0.00018847750071776143,
+      "loss": 0.0264,
+      "step": 760
+    },
+    {
+      "epoch": 0.48049921996879874,
+      "grad_norm": 1.2603468894958496,
+      "learning_rate": 0.00018816989021514568,
+      "loss": 0.0111,
+      "step": 770
+    },
+    {
+      "epoch": 0.48673946957878317,
+      "grad_norm": 0.49606853723526,
+      "learning_rate": 0.00018785848543744632,
+      "loss": 0.0125,
+      "step": 780
+    },
+    {
+      "epoch": 0.49297971918876754,
+      "grad_norm": 0.9916170835494995,
+      "learning_rate": 0.00018754329978555782,
+      "loss": 0.01,
+      "step": 790
+    },
+    {
+      "epoch": 0.49921996879875197,
+      "grad_norm": 1.2155410051345825,
+      "learning_rate": 0.0001872243468230795,
+      "loss": 0.0126,
+      "step": 800
+    },
+    {
+      "epoch": 0.5054602184087363,
+      "grad_norm": 2.0329480171203613,
+      "learning_rate": 0.000186901640275732,
+      "loss": 0.0165,
+      "step": 810
+    },
+    {
+      "epoch": 0.5117004680187207,
+      "grad_norm": 0.0864962488412857,
+      "learning_rate": 0.00018657519403076644,
+      "loss": 0.0161,
+      "step": 820
+    },
+    {
+      "epoch": 0.5179407176287052,
+      "grad_norm": 0.1404924988746643,
+      "learning_rate": 0.00018624502213636702,
+      "loss": 0.0216,
+      "step": 830
+    },
+    {
+      "epoch": 0.5241809672386896,
+      "grad_norm": 1.004807472229004,
+      "learning_rate": 0.00018591113880104626,
+      "loss": 0.0135,
+      "step": 840
+    },
+    {
+      "epoch": 0.5304212168486739,
+      "grad_norm": 0.3545820116996765,
+      "learning_rate": 0.0001855735583930337,
+      "loss": 0.0124,
+      "step": 850
+    },
+    {
+      "epoch": 0.5366614664586583,
+      "grad_norm": 1.5320320129394531,
+      "learning_rate": 0.0001852322954396575,
+      "loss": 0.0096,
+      "step": 860
+    },
+    {
+      "epoch": 0.5429017160686428,
+      "grad_norm": 0.41443562507629395,
+      "learning_rate": 0.0001848873646267194,
+      "loss": 0.0164,
+      "step": 870
+    },
+    {
+      "epoch": 0.5491419656786272,
+      "grad_norm": 0.3798370063304901,
+      "learning_rate": 0.0001845387807978625,
+      "loss": 0.008,
+      "step": 880
+    },
+    {
+      "epoch": 0.5553822152886115,
+      "grad_norm": 0.16353817284107208,
+      "learning_rate": 0.0001841865589539327,
+      "loss": 0.0094,
+      "step": 890
+    },
+    {
+      "epoch": 0.5616224648985959,
+      "grad_norm": 0.03544636815786362,
+      "learning_rate": 0.00018383071425233315,
+      "loss": 0.0054,
+      "step": 900
+    },
+    {
+      "epoch": 0.5678627145085804,
+      "grad_norm": 0.6518317461013794,
+      "learning_rate": 0.0001834712620063718,
+      "loss": 0.0089,
+      "step": 910
+    },
+    {
+      "epoch": 0.5741029641185648,
+      "grad_norm": 2.557168960571289,
+      "learning_rate": 0.00018310821768460257,
+      "loss": 0.0142,
+      "step": 920
+    },
+    {
+      "epoch": 0.5803432137285491,
+      "grad_norm": 0.38063621520996094,
+      "learning_rate": 0.00018274159691015966,
+      "loss": 0.0087,
+      "step": 930
+    },
+    {
+      "epoch": 0.5865834633385335,
+      "grad_norm": 0.10065897554159164,
+      "learning_rate": 0.0001823714154600851,
+      "loss": 0.0072,
+      "step": 940
+    },
+    {
+      "epoch": 0.592823712948518,
+      "grad_norm": 0.2000608593225479,
+      "learning_rate": 0.00018199768926465003,
+      "loss": 0.0041,
+      "step": 950
+    },
+    {
+      "epoch": 0.5990639625585024,
+      "grad_norm": 0.5174477696418762,
+      "learning_rate": 0.000181620434406669,
+      "loss": 0.0117,
+      "step": 960
+    },
+    {
+      "epoch": 0.6053042121684867,
+      "grad_norm": 0.14495687186717987,
+      "learning_rate": 0.00018123966712080782,
+      "loss": 0.0052,
+      "step": 970
+    },
+    {
+      "epoch": 0.6115444617784711,
+      "grad_norm": 0.00905101839452982,
+      "learning_rate": 0.0001808554037928851,
+      "loss": 0.0053,
+      "step": 980
+    },
+    {
+      "epoch": 0.6177847113884556,
+      "grad_norm": 0.7881221175193787,
+      "learning_rate": 0.00018046766095916697,
+      "loss": 0.0051,
+      "step": 990
+    },
+    {
+      "epoch": 0.62402496099844,
+      "grad_norm": 0.008211757056415081,
+      "learning_rate": 0.00018007645530565555,
+      "loss": 0.0054,
+      "step": 1000
+    },
+    {
+      "epoch": 0.6302652106084243,
+      "grad_norm": 0.19711071252822876,
+      "learning_rate": 0.00017968180366737083,
+      "loss": 0.0108,
+      "step": 1010
+    },
+    {
+      "epoch": 0.6365054602184087,
+      "grad_norm": 0.4079477787017822,
+      "learning_rate": 0.00017928372302762616,
+      "loss": 0.0091,
+      "step": 1020
+    },
+    {
+      "epoch": 0.6427457098283932,
+      "grad_norm": 0.38131341338157654,
+      "learning_rate": 0.00017888223051729755,
+      "loss": 0.0197,
+      "step": 1030
+    },
+    {
+      "epoch": 0.6489859594383776,
+      "grad_norm": 0.2548066973686218,
+      "learning_rate": 0.00017847734341408628,
+      "loss": 0.0221,
+      "step": 1040
+    },
+    {
+      "epoch": 0.6552262090483619,
+      "grad_norm": 0.016785508021712303,
+      "learning_rate": 0.00017806907914177554,
+      "loss": 0.0082,
+      "step": 1050
+    },
+    {
+      "epoch": 0.6614664586583463,
+      "grad_norm": 0.17338435351848602,
+      "learning_rate": 0.00017765745526948046,
+      "loss": 0.0141,
+      "step": 1060
+    },
+    {
+      "epoch": 0.6677067082683308,
+      "grad_norm": 1.0973267555236816,
+      "learning_rate": 0.0001772424895108922,
+      "loss": 0.0133,
+      "step": 1070
+    },
+    {
+      "epoch": 0.6739469578783152,
+      "grad_norm": 0.015839748084545135,
+      "learning_rate": 0.00017682419972351554,
+      "loss": 0.0083,
+      "step": 1080
+    },
+    {
+      "epoch": 0.6801872074882995,
+      "grad_norm": 0.014509073458611965,
+      "learning_rate": 0.00017640260390790053,
+      "loss": 0.0085,
+      "step": 1090
+    },
+    {
+      "epoch": 0.6864274570982839,
+      "grad_norm": 0.03248611092567444,
+      "learning_rate": 0.00017597772020686768,
+      "loss": 0.0037,
+      "step": 1100
+    },
+    {
+      "epoch": 0.6926677067082684,
+      "grad_norm": 0.06335289031267166,
+      "learning_rate": 0.00017554956690472748,
+      "loss": 0.0071,
+      "step": 1110
+    },
+    {
+      "epoch": 0.6989079563182528,
+      "grad_norm": 0.04545082896947861,
+      "learning_rate": 0.00017511816242649322,
+      "loss": 0.0125,
+      "step": 1120
+    },
+    {
+      "epoch": 0.7051482059282371,
+      "grad_norm": 1.186499834060669,
+      "learning_rate": 0.00017468352533708836,
+      "loss": 0.0142,
+      "step": 1130
+    },
+    {
+      "epoch": 0.7113884555382215,
+      "grad_norm": 0.7652639746665955,
+      "learning_rate": 0.00017424567434054758,
+      "loss": 0.0061,
+      "step": 1140
+    },
+    {
+      "epoch": 0.717628705148206,
+      "grad_norm": 0.2659410238265991,
+      "learning_rate": 0.0001738046282792117,
+      "loss": 0.0097,
+      "step": 1150
+    },
+    {
+      "epoch": 0.7238689547581904,
+      "grad_norm": 0.14636874198913574,
+      "learning_rate": 0.000173360406132917,
+      "loss": 0.0056,
+      "step": 1160
+    },
+    {
+      "epoch": 0.7301092043681747,
+      "grad_norm": 0.26678094267845154,
+      "learning_rate": 0.00017291302701817843,
+      "loss": 0.01,
+      "step": 1170
+    },
+    {
+      "epoch": 0.7363494539781591,
+      "grad_norm": 0.11483141034841537,
+      "learning_rate": 0.00017246251018736687,
+      "loss": 0.0114,
+      "step": 1180
+    },
+    {
+      "epoch": 0.7425897035881436,
+      "grad_norm": 0.010227694176137447,
+      "learning_rate": 0.0001720088750278807,
+      "loss": 0.0073,
+      "step": 1190
+    },
+    {
+      "epoch": 0.748829953198128,
+      "grad_norm": 0.15639427304267883,
+      "learning_rate": 0.0001715521410613114,
+      "loss": 0.0132,
+      "step": 1200
+    },
+    {
+      "epoch": 0.7550702028081123,
+      "grad_norm": 0.1970348358154297,
+      "learning_rate": 0.00017109232794260362,
+      "loss": 0.0062,
+      "step": 1210
+    },
+    {
+      "epoch": 0.7613104524180967,
+      "grad_norm": 0.13912296295166016,
+      "learning_rate": 0.00017062945545920927,
+      "loss": 0.0112,
+      "step": 1220
+    },
+    {
+      "epoch": 0.7675507020280812,
+      "grad_norm": 0.0700615793466568,
+      "learning_rate": 0.00017016354353023593,
+      "loss": 0.0086,
+      "step": 1230
+    },
+    {
+      "epoch": 0.7737909516380655,
+      "grad_norm": 0.08227328956127167,
+      "learning_rate": 0.00016969461220558976,
+      "loss": 0.005,
+      "step": 1240
+    },
+    {
+      "epoch": 0.7800312012480499,
+      "grad_norm": 0.5434132814407349,
+      "learning_rate": 0.00016922268166511255,
+      "loss": 0.0078,
+      "step": 1250
+    },
+    {
+      "epoch": 0.7862714508580343,
+      "grad_norm": 0.14495456218719482,
+      "learning_rate": 0.00016874777221771354,
+      "loss": 0.0053,
+      "step": 1260
+    },
+    {
+      "epoch": 0.7925117004680188,
+      "grad_norm": 0.6543499827384949,
+      "learning_rate": 0.00016826990430049513,
+      "loss": 0.0021,
+      "step": 1270
+    },
+    {
+      "epoch": 0.7987519500780031,
+      "grad_norm": 1.2169020175933838,
+      "learning_rate": 0.00016778909847787378,
+      "loss": 0.0109,
+      "step": 1280
+    },
+    {
+      "epoch": 0.8049921996879875,
+      "grad_norm": 0.059403009712696075,
+      "learning_rate": 0.00016730537544069464,
+      "loss": 0.0083,
+      "step": 1290
+    },
+    {
+      "epoch": 0.8112324492979719,
+      "grad_norm": 1.1702568531036377,
+      "learning_rate": 0.0001668187560053415,
+      "loss": 0.012,
+      "step": 1300
+    },
+    {
+      "epoch": 0.8174726989079563,
+      "grad_norm": 0.35837844014167786,
+      "learning_rate": 0.00016632926111284066,
+      "loss": 0.0059,
+      "step": 1310
+    },
+    {
+      "epoch": 0.8237129485179407,
+      "grad_norm": 0.26777470111846924,
+      "learning_rate": 0.00016583691182796013,
+      "loss": 0.0079,
+      "step": 1320
+    },
+    {
+      "epoch": 0.8299531981279251,
+      "grad_norm": 0.6792253255844116,
+      "learning_rate": 0.00016534172933830275,
+      "loss": 0.0116,
+      "step": 1330
+    },
+    {
+      "epoch": 0.8361934477379095,
+      "grad_norm": 0.23119303584098816,
+      "learning_rate": 0.00016484373495339477,
+      "loss": 0.0048,
+      "step": 1340
+    },
+    {
+      "epoch": 0.8424336973478939,
+      "grad_norm": 0.2668294608592987,
+      "learning_rate": 0.0001643429501037685,
+      "loss": 0.0106,
+      "step": 1350
+    },
+    {
+      "epoch": 0.8486739469578783,
+      "grad_norm": 0.18860620260238647,
+      "learning_rate": 0.00016383939634004034,
+      "loss": 0.0111,
+      "step": 1360
+    },
+    {
+      "epoch": 0.8549141965678627,
+      "grad_norm": 0.2770467698574066,
+      "learning_rate": 0.00016333309533198316,
+      "loss": 0.0094,
+      "step": 1370
+    },
+    {
+      "epoch": 0.8611544461778471,
+      "grad_norm": 0.14996962249279022,
+      "learning_rate": 0.00016282406886759392,
+      "loss": 0.006,
+      "step": 1380
+    },
+    {
+      "epoch": 0.8673946957878315,
+      "grad_norm": 0.0023365733213722706,
+      "learning_rate": 0.000162312338852156,
+      "loss": 0.0062,
+      "step": 1390
+    },
+    {
+      "epoch": 0.8736349453978159,
+      "grad_norm": 0.06916924566030502,
+      "learning_rate": 0.00016179792730729652,
+      "loss": 0.0013,
+      "step": 1400
+    },
+    {
+      "epoch": 0.8798751950078003,
+      "grad_norm": 0.5850982666015625,
+      "learning_rate": 0.0001612808563700388,
+      "loss": 0.011,
+      "step": 1410
+    },
+    {
+      "epoch": 0.8861154446177847,
+      "grad_norm": 0.3427271246910095,
+      "learning_rate": 0.00016076114829184938,
+      "loss": 0.0122,
+      "step": 1420
+    },
+    {
+      "epoch": 0.8923556942277691,
+      "grad_norm": 0.300658643245697,
+      "learning_rate": 0.00016023882543768093,
+      "loss": 0.0129,
+      "step": 1430
+    },
+    {
+      "epoch": 0.8985959438377535,
+      "grad_norm": 0.20494520664215088,
+      "learning_rate": 0.00015971391028500934,
+      "loss": 0.0098,
+      "step": 1440
+    },
+    {
+      "epoch": 0.9048361934477379,
+      "grad_norm": 0.2109510898590088,
+      "learning_rate": 0.00015918642542286687,
+      "loss": 0.0222,
+      "step": 1450
+    },
+    {
+      "epoch": 0.9110764430577223,
+      "grad_norm": 0.05611221864819527,
+      "learning_rate": 0.00015865639355086967,
+      "loss": 0.0145,
+      "step": 1460
+    },
+    {
+      "epoch": 0.9173166926677067,
+      "grad_norm": 0.3887752592563629,
+      "learning_rate": 0.00015812383747824116,
+      "loss": 0.0111,
+      "step": 1470
+    },
+    {
+      "epoch": 0.9235569422776911,
+      "grad_norm": 1.0526059865951538,
+      "learning_rate": 0.00015758878012283044,
+      "loss": 0.0126,
+      "step": 1480
+    },
+    {
+      "epoch": 0.9297971918876755,
+      "grad_norm": 0.2805418074131012,
+      "learning_rate": 0.00015705124451012602,
+      "loss": 0.0076,
+      "step": 1490
+    },
+    {
+      "epoch": 0.9360374414976599,
+      "grad_norm": 0.014235242269933224,
+      "learning_rate": 0.0001565112537722649,
+      "loss": 0.01,
+      "step": 1500
+    },
+    {
+      "epoch": 0.9422776911076443,
+      "grad_norm": 0.3192802965641022,
+      "learning_rate": 0.00015596883114703714,
+      "loss": 0.015,
+      "step": 1510
+    },
+    {
+      "epoch": 0.9485179407176287,
+      "grad_norm": 0.06423596292734146,
+      "learning_rate": 0.00015542399997688598,
+      "loss": 0.0145,
+      "step": 1520
+    },
+    {
+      "epoch": 0.9547581903276131,
+      "grad_norm": 0.06878869235515594,
+      "learning_rate": 0.00015487678370790307,
+      "loss": 0.0129,
+      "step": 1530
+    },
+    {
+      "epoch": 0.9609984399375975,
+      "grad_norm": 0.3116801381111145,
+      "learning_rate": 0.0001543272058888198,
+      "loss": 0.0071,
+      "step": 1540
+    },
+    {
+      "epoch": 0.9672386895475819,
+      "grad_norm": 0.40218546986579895,
+      "learning_rate": 0.00015377529016999365,
+      "loss": 0.0117,
+      "step": 1550
+    },
+    {
+      "epoch": 0.9734789391575663,
+      "grad_norm": 0.9296953678131104,
+      "learning_rate": 0.00015322106030239054,
+      "loss": 0.0125,
+      "step": 1560
+    },
+    {
+      "epoch": 0.9797191887675507,
+      "grad_norm": 0.10756753385066986,
+      "learning_rate": 0.00015266454013656278,
+      "loss": 0.0119,
+      "step": 1570
+    },
+    {
+      "epoch": 0.9859594383775351,
+      "grad_norm": 0.09447533637285233,
+      "learning_rate": 0.0001521057536216227,
+      "loss": 0.0112,
+      "step": 1580
+    },
+    {
+      "epoch": 0.9921996879875195,
+      "grad_norm": 0.5510380268096924,
+      "learning_rate": 0.0001515447248042118,
+      "loss": 0.0126,
+      "step": 1590
+    },
+    {
+      "epoch": 0.9984399375975039,
+      "grad_norm": 0.004704979248344898,
+      "learning_rate": 0.00015098147782746633,
+      "loss": 0.0057,
+      "step": 1600
+    },
+    {
+      "epoch": 1.004368174726989,
+      "grad_norm": 0.24134884774684906,
+      "learning_rate": 0.00015041603692997788,
+      "loss": 0.006,
+      "step": 1610
+    },
+    {
+      "epoch": 1.0106084243369735,
+      "grad_norm": 0.23953481018543243,
+      "learning_rate": 0.00014984842644475075,
+      "loss": 0.0048,
+      "step": 1620
+    },
+    {
+      "epoch": 1.0168486739469578,
+      "grad_norm": 0.008176998235285282,
+      "learning_rate": 0.0001492786707981544,
+      "loss": 0.006,
+      "step": 1630
+    },
+    {
+      "epoch": 1.0230889235569423,
+      "grad_norm": 0.046635717153549194,
+      "learning_rate": 0.00014870679450887261,
+      "loss": 0.0087,
+      "step": 1640
+    },
+    {
+      "epoch": 1.0293291731669267,
+      "grad_norm": 0.03800414875149727,
+      "learning_rate": 0.00014813282218684826,
+      "loss": 0.0037,
+      "step": 1650
+    },
+    {
+      "epoch": 1.035569422776911,
+      "grad_norm": 0.10656531155109406,
+      "learning_rate": 0.000147556778532224,
+      "loss": 0.0058,
+      "step": 1660
+    },
+    {
+      "epoch": 1.0418096723868955,
+      "grad_norm": 0.28034937381744385,
+      "learning_rate": 0.00014697868833427985,
+      "loss": 0.0052,
+      "step": 1670
+    },
+    {
+      "epoch": 1.04804992199688,
+      "grad_norm": 0.09066982567310333,
+      "learning_rate": 0.0001463985764703659,
+      "loss": 0.0058,
+      "step": 1680
+    },
+    {
+      "epoch": 1.0542901716068642,
+      "grad_norm": 0.024250924587249756,
+      "learning_rate": 0.00014581646790483218,
+      "loss": 0.0098,
+      "step": 1690
+    },
+    {
+      "epoch": 1.0605304212168487,
+      "grad_norm": 0.20950108766555786,
+      "learning_rate": 0.00014523238768795386,
+      "loss": 0.0085,
+      "step": 1700
+    },
+    {
+      "epoch": 1.0667706708268332,
+      "grad_norm": 0.26374754309654236,
+      "learning_rate": 0.00014464636095485385,
+      "loss": 0.0096,
+      "step": 1710
+    },
+    {
+      "epoch": 1.0730109204368175,
+      "grad_norm": 0.004857131280004978,
+      "learning_rate": 0.00014405841292442054,
+      "loss": 0.0089,
+      "step": 1720
+    },
+    {
+      "epoch": 1.079251170046802,
+      "grad_norm": 0.1881384700536728,
+      "learning_rate": 0.00014346856889822305,
+      "loss": 0.0097,
+      "step": 1730
+    },
+    {
+      "epoch": 1.0854914196567862,
+      "grad_norm": 0.3700352609157562,
+      "learning_rate": 0.000142876854259422,
+      "loss": 0.0098,
+      "step": 1740
+    },
+    {
+      "epoch": 1.0917316692667707,
+      "grad_norm": 0.029609568417072296,
+      "learning_rate": 0.00014228329447167742,
+      "loss": 0.0068,
+      "step": 1750
+    },
+    {
+      "epoch": 1.0979719188767552,
+      "grad_norm": 0.6126700043678284,
+      "learning_rate": 0.000141687915078053,
+      "loss": 0.0201,
+      "step": 1760
+    },
+    {
+      "epoch": 1.1042121684867394,
+      "grad_norm": 0.16146229207515717,
+      "learning_rate": 0.00014109074169991662,
+      "loss": 0.014,
+      "step": 1770
+    },
+    {
+      "epoch": 1.110452418096724,
+      "grad_norm": 0.13777479529380798,
+      "learning_rate": 0.000140491800035838,
+      "loss": 0.0115,
+      "step": 1780
+    },
+    {
+      "epoch": 1.1166926677067082,
+      "grad_norm": 0.006505697965621948,
+      "learning_rate": 0.00013989111586048265,
+      "loss": 0.0044,
+      "step": 1790
+    },
+    {
+      "epoch": 1.1229329173166926,
+      "grad_norm": 0.02420825883746147,
+      "learning_rate": 0.00013928871502350286,
+      "loss": 0.0021,
+      "step": 1800
+    },
+    {
+      "epoch": 1.1291731669266771,
+      "grad_norm": 1.3016235828399658,
+      "learning_rate": 0.00013868462344842513,
+      "loss": 0.0082,
+      "step": 1810
+    },
+    {
+      "epoch": 1.1354134165366614,
+      "grad_norm": 0.35244765877723694,
+      "learning_rate": 0.00013807886713153463,
+      "loss": 0.0148,
+      "step": 1820
+    },
+    {
+      "epoch": 1.1416536661466459,
+      "grad_norm": 0.011646454222500324,
+      "learning_rate": 0.00013747147214075652,
+      "loss": 0.0072,
+      "step": 1830
+    },
+    {
+      "epoch": 1.1478939157566304,
+      "grad_norm": 0.3862789571285248,
+      "learning_rate": 0.00013686246461453417,
+      "loss": 0.0171,
+      "step": 1840
+    },
+    {
+      "epoch": 1.1541341653666146,
+      "grad_norm": 0.8616077899932861,
+      "learning_rate": 0.00013625187076070423,
+      "loss": 0.0092,
+      "step": 1850
+    },
+    {
+      "epoch": 1.160374414976599,
+      "grad_norm": 0.04771306738257408,
+      "learning_rate": 0.00013563971685536894,
+      "loss": 0.0062,
+      "step": 1860
+    },
+    {
+      "epoch": 1.1666146645865836,
+      "grad_norm": 0.26229849457740784,
+      "learning_rate": 0.00013502602924176516,
+      "loss": 0.0059,
+      "step": 1870
+    },
+    {
+      "epoch": 1.1728549141965678,
+      "grad_norm": 0.017036737874150276,
+      "learning_rate": 0.00013441083432913107,
+      "loss": 0.0059,
+      "step": 1880
+    },
+    {
+      "epoch": 1.1790951638065523,
+      "grad_norm": 0.10029294341802597,
+      "learning_rate": 0.0001337941585915693,
+      "loss": 0.0061,
+      "step": 1890
+    },
+    {
+      "epoch": 1.1853354134165366,
+      "grad_norm": 0.15829603374004364,
+      "learning_rate": 0.00013317602856690789,
+      "loss": 0.0045,
+      "step": 1900
+    },
+    {
+      "epoch": 1.191575663026521,
+      "grad_norm": 0.12337626516819,
+      "learning_rate": 0.0001325564708555583,
+      "loss": 0.0042,
+      "step": 1910
+    },
+    {
+      "epoch": 1.1978159126365056,
+      "grad_norm": 0.1467108130455017,
+      "learning_rate": 0.00013193551211937042,
+      "loss": 0.0072,
+      "step": 1920
+    },
+    {
+      "epoch": 1.2040561622464898,
+      "grad_norm": 0.12583035230636597,
+      "learning_rate": 0.0001313131790804856,
+      "loss": 0.0037,
+      "step": 1930
+    },
+    {
+      "epoch": 1.2102964118564743,
+      "grad_norm": 0.07843368500471115,
+      "learning_rate": 0.00013068949852018634,
+      "loss": 0.0077,
+      "step": 1940
+    },
+    {
+      "epoch": 1.2165366614664586,
+      "grad_norm": 0.0029602251015603542,
+      "learning_rate": 0.00013006449727774401,
+      "loss": 0.0058,
+      "step": 1950
+    },
+    {
+      "epoch": 1.222776911076443,
+      "grad_norm": 0.049546901136636734,
+      "learning_rate": 0.0001294382022492638,
+      "loss": 0.0061,
+      "step": 1960
+    },
+    {
+      "epoch": 1.2290171606864275,
+      "grad_norm": 0.16197310388088226,
+      "learning_rate": 0.00012881064038652734,
+      "loss": 0.0058,
+      "step": 1970
+    },
+    {
+      "epoch": 1.2352574102964118,
+      "grad_norm": 0.11641169339418411,
+      "learning_rate": 0.00012818183869583264,
+      "loss": 0.0027,
+      "step": 1980
+    },
+    {
+      "epoch": 1.2414976599063963,
+      "grad_norm": 0.001966765383258462,
+      "learning_rate": 0.00012755182423683224,
+      "loss": 0.0014,
+      "step": 1990
+    },
+    {
+      "epoch": 1.2477379095163807,
+      "grad_norm": 0.0025216981302946806,
+      "learning_rate": 0.00012692062412136842,
+      "loss": 0.0037,
+      "step": 2000
+    },
+    {
+      "epoch": 1.253978159126365,
+      "grad_norm": 0.23342859745025635,
+      "learning_rate": 0.00012628826551230676,
+      "loss": 0.0043,
+      "step": 2010
+    },
+    {
+      "epoch": 1.2602184087363495,
+      "grad_norm": 0.0014470007736235857,
+      "learning_rate": 0.00012565477562236706,
+      "loss": 0.0024,
+      "step": 2020
+    },
+    {
+      "epoch": 1.266458658346334,
+      "grad_norm": 0.09222376346588135,
+      "learning_rate": 0.00012502018171295216,
+      "loss": 0.0027,
+      "step": 2030
+    },
+    {
+      "epoch": 1.2726989079563182,
+      "grad_norm": 0.08314360678195953,
+      "learning_rate": 0.0001243845110929751,
+      "loss": 0.0054,
+      "step": 2040
+    },
+    {
+      "epoch": 1.2789391575663027,
+      "grad_norm": 0.03709840402007103,
+      "learning_rate": 0.0001237477911176837,
+      "loss": 0.0064,
+      "step": 2050
+    },
+    {
+      "epoch": 1.285179407176287,
+      "grad_norm": 0.1064552441239357,
+      "learning_rate": 0.00012311004918748333,
+      "loss": 0.0044,
+      "step": 2060
+    },
+    {
+      "epoch": 1.2914196567862715,
+      "grad_norm": 0.028765583410859108,
+      "learning_rate": 0.00012247131274675788,
+      "loss": 0.0098,
+      "step": 2070
+    },
+    {
+      "epoch": 1.2976599063962557,
+      "grad_norm": 0.14255695044994354,
+      "learning_rate": 0.00012183160928268875,
+      "loss": 0.0063,
+      "step": 2080
+    },
+    {
+      "epoch": 1.3039001560062402,
+      "grad_norm": 0.352822482585907,
+      "learning_rate": 0.00012119096632407185,
+      "loss": 0.0116,
+      "step": 2090
+    },
+    {
+      "epoch": 1.3101404056162247,
+      "grad_norm": 0.08606994897127151,
+      "learning_rate": 0.00012054941144013298,
+      "loss": 0.0102,
+      "step": 2100
+    },
+    {
+      "epoch": 1.316380655226209,
+      "grad_norm": 0.1337183713912964,
+      "learning_rate": 0.00011990697223934151,
+      "loss": 0.0088,
+      "step": 2110
+    },
+    {
+      "epoch": 1.3226209048361934,
+      "grad_norm": 0.0092279938980937,
+      "learning_rate": 0.00011926367636822223,
+      "loss": 0.0049,
+      "step": 2120
+    },
+    {
+      "epoch": 1.328861154446178,
+      "grad_norm": 0.003310105064883828,
+      "learning_rate": 0.00011861955151016548,
+      "loss": 0.0059,
+      "step": 2130
+    },
+    {
+      "epoch": 1.3351014040561622,
+      "grad_norm": 0.06122155115008354,
+      "learning_rate": 0.00011797462538423614,
+      "loss": 0.0074,
+      "step": 2140
+    },
+    {
+      "epoch": 1.3413416536661467,
+      "grad_norm": 0.16545473039150238,
+      "learning_rate": 0.00011732892574398047,
+      "loss": 0.0092,
+      "step": 2150
+    },
+    {
+      "epoch": 1.3475819032761311,
+      "grad_norm": 0.004430962726473808,
+      "learning_rate": 0.00011668248037623194,
+      "loss": 0.0037,
+      "step": 2160
+    },
+    {
+      "epoch": 1.3538221528861154,
+      "grad_norm": 0.009139200672507286,
+      "learning_rate": 0.00011603531709991544,
+      "loss": 0.0037,
+      "step": 2170
+    },
+    {
+      "epoch": 1.3600624024960999,
+      "grad_norm": 0.010984491556882858,
+      "learning_rate": 0.00011538746376485003,
+      "loss": 0.0087,
+      "step": 2180
+    },
+    {
+      "epoch": 1.3663026521060844,
+      "grad_norm": 0.05745226517319679,
+      "learning_rate": 0.00011473894825055071,
+      "loss": 0.0058,
+      "step": 2190
+    },
+    {
+      "epoch": 1.3725429017160686,
+      "grad_norm": 0.047253601253032684,
+      "learning_rate": 0.00011408979846502828,
+      "loss": 0.0068,
+      "step": 2200
+    },
+    {
+      "epoch": 1.3787831513260531,
+      "grad_norm": 0.0017802199581637979,
+      "learning_rate": 0.00011344004234358872,
+      "loss": 0.0045,
+      "step": 2210
+    },
+    {
+      "epoch": 1.3850234009360374,
+      "grad_norm": 0.07970303297042847,
+      "learning_rate": 0.00011278970784763079,
+      "loss": 0.004,
+      "step": 2220
+    },
+    {
+      "epoch": 1.3912636505460219,
+      "grad_norm": 0.1246202290058136,
+      "learning_rate": 0.00011213882296344289,
+      "loss": 0.0065,
+      "step": 2230
+    },
+    {
+      "epoch": 1.3975039001560061,
+      "grad_norm": 0.01450931467115879,
+      "learning_rate": 0.00011148741570099863,
+      "loss": 0.0046,
+      "step": 2240
+    },
+    {
+      "epoch": 1.4037441497659906,
+      "grad_norm": 0.4136829078197479,
+      "learning_rate": 0.0001108355140927515,
+      "loss": 0.0067,
+      "step": 2250
+    },
+    {
+      "epoch": 1.409984399375975,
+      "grad_norm": 0.0039209057576954365,
+      "learning_rate": 0.00011018314619242848,
+      "loss": 0.005,
+      "step": 2260
+    },
+    {
+      "epoch": 1.4162246489859593,
+      "grad_norm": 0.0021903226152062416,
+      "learning_rate": 0.00010953034007382281,
+      "loss": 0.0089,
+      "step": 2270
+    },
+    {
+      "epoch": 1.4224648985959438,
+      "grad_norm": 0.0021560548339039087,
+      "learning_rate": 0.0001088771238295859,
+      "loss": 0.0059,
+      "step": 2280
+    },
+    {
+      "epoch": 1.4287051482059283,
+      "grad_norm": 0.16054560244083405,
+      "learning_rate": 0.0001082235255700184,
+      "loss": 0.0037,
+      "step": 2290
+    },
+    {
+      "epoch": 1.4349453978159126,
+      "grad_norm": 0.09998723119497299,
+      "learning_rate": 0.00010756957342186042,
+      "loss": 0.004,
+      "step": 2300
+    },
+    {
+      "epoch": 1.441185647425897,
+      "grad_norm": 0.08871794492006302,
+      "learning_rate": 0.00010691529552708128,
+      "loss": 0.0067,
+      "step": 2310
+    },
+    {
+      "epoch": 1.4474258970358815,
+      "grad_norm": 0.005393527913838625,
+      "learning_rate": 0.00010626072004166829,
+      "loss": 0.0061,
+      "step": 2320
+    },
+    {
+      "epoch": 1.4536661466458658,
+      "grad_norm": 0.20176663994789124,
+      "learning_rate": 0.00010560587513441524,
+      "loss": 0.0084,
+      "step": 2330
+    },
+    {
+      "epoch": 1.4599063962558503,
+      "grad_norm": 0.25814759731292725,
+      "learning_rate": 0.00010495078898571014,
+      "loss": 0.0046,
+      "step": 2340
+    },
+    {
+      "epoch": 1.4661466458658348,
+      "grad_norm": 0.0187391247600317,
+      "learning_rate": 0.00010429548978632246,
+      "loss": 0.0041,
+      "step": 2350
+    },
+    {
+      "epoch": 1.472386895475819,
+      "grad_norm": 0.0017654044786468148,
+      "learning_rate": 0.00010364000573619008,
+      "loss": 0.0025,
+      "step": 2360
+    },
+    {
+      "epoch": 1.4786271450858035,
+      "grad_norm": 0.04109564423561096,
+      "learning_rate": 0.00010298436504320561,
+      "loss": 0.0031,
+      "step": 2370
+    },
+    {
+      "epoch": 1.4848673946957878,
+      "grad_norm": 0.0455956868827343,
+      "learning_rate": 0.0001023285959220027,
+      "loss": 0.007,
+      "step": 2380
+    },
+    {
+      "epoch": 1.4911076443057723,
+      "grad_norm": 0.0018099998123943806,
+      "learning_rate": 0.0001016727265927416,
+      "loss": 0.0026,
+      "step": 2390
+    },
+    {
+      "epoch": 1.4973478939157565,
+      "grad_norm": 0.10770522803068161,
+      "learning_rate": 0.000101016785279895,
+      "loss": 0.0062,
+      "step": 2400
+    },
+    {
+      "epoch": 1.503588143525741,
+      "grad_norm": 0.04892813041806221,
+      "learning_rate": 0.00010036080021103323,
+      "loss": 0.003,
+      "step": 2410
+    },
+    {
+      "epoch": 1.5098283931357255,
+      "grad_norm": 0.02330232597887516,
+      "learning_rate": 9.970479961560964e-05,
+      "loss": 0.0055,
+      "step": 2420
+    },
+    {
+      "epoch": 1.5160686427457097,
+      "grad_norm": 0.03708835691213608,
+      "learning_rate": 9.904881172374576e-05,
+      "loss": 0.0029,
+      "step": 2430
+    },
+    {
+      "epoch": 1.5223088923556942,
+      "grad_norm": 0.01838928461074829,
+      "learning_rate": 9.839286476501635e-05,
+      "loss": 0.0038,
+      "step": 2440
+    },
+    {
+      "epoch": 1.5285491419656787,
+      "grad_norm": 1.0069528818130493,
+      "learning_rate": 9.773698696723481e-05,
+      "loss": 0.0063,
+      "step": 2450
+    },
+    {
+      "epoch": 1.534789391575663,
+      "grad_norm": 1.22986900806427,
+      "learning_rate": 9.70812065552382e-05,
+      "loss": 0.0069,
+      "step": 2460
+    },
+    {
+      "epoch": 1.5410296411856474,
+      "grad_norm": 0.04500283673405647,
+      "learning_rate": 9.642555174967268e-05,
+      "loss": 0.011,
+      "step": 2470
+    },
+    {
+      "epoch": 1.547269890795632,
+      "grad_norm": 0.21551202237606049,
+      "learning_rate": 9.577005076577915e-05,
+      "loss": 0.007,
+      "step": 2480
+    },
+    {
+      "epoch": 1.5535101404056162,
+      "grad_norm": 0.2890082895755768,
+      "learning_rate": 9.511473181217904e-05,
+      "loss": 0.0046,
+      "step": 2490
+    },
+    {
+      "epoch": 1.5597503900156007,
+      "grad_norm": 0.014139365404844284,
+      "learning_rate": 9.445962308966028e-05,
+      "loss": 0.0046,
+      "step": 2500
+    },
+    {
+      "epoch": 1.5659906396255852,
+      "grad_norm": 0.0018333615735173225,
+      "learning_rate": 9.380475278996375e-05,
+      "loss": 0.0042,
+      "step": 2510
+    },
+    {
+      "epoch": 1.5722308892355694,
+      "grad_norm": 0.01185754593461752,
+      "learning_rate": 9.315014909457023e-05,
+      "loss": 0.0032,
+      "step": 2520
+    },
+    {
+      "epoch": 1.5784711388455537,
+      "grad_norm": 0.21326202154159546,
+      "learning_rate": 9.249584017348748e-05,
+      "loss": 0.0055,
+      "step": 2530
+    },
+    {
+      "epoch": 1.5847113884555384,
+      "grad_norm": 0.0019281799905002117,
+      "learning_rate": 9.184185418403799e-05,
+      "loss": 0.0042,
+      "step": 2540
+    },
+    {
+      "epoch": 1.5909516380655226,
+      "grad_norm": 0.0017161122523248196,
+      "learning_rate": 9.118821926964733e-05,
+      "loss": 0.0038,
+      "step": 2550
+    },
+    {
+      "epoch": 1.597191887675507,
+      "grad_norm": 0.050887156277894974,
+      "learning_rate": 9.053496355863311e-05,
+      "loss": 0.0044,
+      "step": 2560
+    },
+    {
+      "epoch": 1.6034321372854914,
+      "grad_norm": 0.0016990028088912368,
+      "learning_rate": 8.988211516299434e-05,
+      "loss": 0.0024,
+      "step": 2570
+    },
+    {
+      "epoch": 1.6096723868954759,
+      "grad_norm": 0.1756485253572464,
+      "learning_rate": 8.92297021772017e-05,
+      "loss": 0.005,
+      "step": 2580
+    },
+    {
+      "epoch": 1.6159126365054601,
+      "grad_norm": 0.0019257533131167293,
+      "learning_rate": 8.857775267698869e-05,
+      "loss": 0.004,
+      "step": 2590
+    },
+    {
+      "epoch": 1.6221528861154446,
+      "grad_norm": 0.047810349613428116,
+      "learning_rate": 8.792629471814325e-05,
+      "loss": 0.0028,
+      "step": 2600
+    },
+    {
+      "epoch": 1.628393135725429,
+      "grad_norm": 0.0032332607079297304,
+      "learning_rate": 8.727535633530057e-05,
+      "loss": 0.003,
+      "step": 2610
+    },
+    {
+      "epoch": 1.6346333853354134,
+      "grad_norm": 0.0024718448985368013,
+      "learning_rate": 8.662496554073642e-05,
+      "loss": 0.0044,
+      "step": 2620
+    },
+    {
+      "epoch": 1.6408736349453978,
+      "grad_norm": 0.0009530312963761389,
+      "learning_rate": 8.597515032316197e-05,
+      "loss": 0.0022,
+      "step": 2630
+    },
+    {
+      "epoch": 1.6471138845553823,
+      "grad_norm": 0.05391823127865791,
+      "learning_rate": 8.532593864651913e-05,
+      "loss": 0.0039,
+      "step": 2640
+    },
+    {
+      "epoch": 1.6533541341653666,
+      "grad_norm": 0.20347720384597778,
+      "learning_rate": 8.467735844877717e-05,
+      "loss": 0.0048,
+      "step": 2650
+    },
+    {
+      "epoch": 1.659594383775351,
+      "grad_norm": 0.215189591050148,
+      "learning_rate": 8.402943764073064e-05,
+      "loss": 0.0059,
+      "step": 2660
+    },
+    {
+      "epoch": 1.6658346333853356,
+      "grad_norm": 0.008820041082799435,
+      "learning_rate": 8.338220410479799e-05,
+      "loss": 0.0047,
+      "step": 2670
+    },
+    {
+      "epoch": 1.6720748829953198,
+      "grad_norm": 0.048564255237579346,
+      "learning_rate": 8.273568569382193e-05,
+      "loss": 0.0031,
+      "step": 2680
+    },
+    {
+      "epoch": 1.678315132605304,
+      "grad_norm": 0.0026437535416334867,
+      "learning_rate": 8.208991022987057e-05,
+      "loss": 0.0042,
+      "step": 2690
+    },
+    {
+      "epoch": 1.6845553822152888,
+      "grad_norm": 0.19647392630577087,
+      "learning_rate": 8.144490550304038e-05,
+      "loss": 0.0024,
+      "step": 2700
+    },
+    {
+      "epoch": 1.690795631825273,
+      "grad_norm": 0.0007143426337279379,
+      "learning_rate": 8.080069927026018e-05,
+      "loss": 0.0038,
+      "step": 2710
+    },
+    {
+      "epoch": 1.6970358814352573,
+      "grad_norm": 0.1795455664396286,
+      "learning_rate": 8.015731925409661e-05,
+      "loss": 0.0061,
+      "step": 2720
+    },
+    {
+      "epoch": 1.7032761310452418,
+      "grad_norm": 0.0010726724285632372,
+      "learning_rate": 7.951479314156113e-05,
+      "loss": 0.005,
+      "step": 2730
+    },
+    {
+      "epoch": 1.7095163806552263,
+      "grad_norm": 0.1190568059682846,
+      "learning_rate": 7.887314858291865e-05,
+      "loss": 0.0038,
+      "step": 2740
+    },
+    {
+      "epoch": 1.7157566302652105,
+      "grad_norm": 0.07289066165685654,
+      "learning_rate": 7.823241319049756e-05,
+      "loss": 0.0028,
+      "step": 2750
+    },
+    {
+      "epoch": 1.721996879875195,
+      "grad_norm": 0.060338374227285385,
+      "learning_rate": 7.759261453750146e-05,
+      "loss": 0.0039,
+      "step": 2760
+    },
+    {
+      "epoch": 1.7282371294851795,
+      "grad_norm": 0.22540676593780518,
+      "learning_rate": 7.695378015682256e-05,
+      "loss": 0.007,
+      "step": 2770
+    },
+    {
+      "epoch": 1.7344773790951638,
+      "grad_norm": 0.11157722026109695,
+      "learning_rate": 7.631593753985699e-05,
+      "loss": 0.0029,
+      "step": 2780
+    },
+    {
+      "epoch": 1.7407176287051482,
+      "grad_norm": 0.11412902176380157,
+      "learning_rate": 7.567911413532156e-05,
+      "loss": 0.0038,
+      "step": 2790
+    },
+    {
+      "epoch": 1.7469578783151327,
+      "grad_norm": 0.03782040998339653,
+      "learning_rate": 7.504333734807258e-05,
+      "loss": 0.0038,
+      "step": 2800
+    },
+    {
+      "epoch": 1.753198127925117,
+      "grad_norm": 0.10296830534934998,
+      "learning_rate": 7.440863453792665e-05,
+      "loss": 0.0045,
+      "step": 2810
+    },
+    {
+      "epoch": 1.7594383775351012,
+      "grad_norm": 0.0008850370650179684,
+      "learning_rate": 7.377503301848315e-05,
+      "loss": 0.0029,
+      "step": 2820
+    },
+    {
+      "epoch": 1.765678627145086,
+      "grad_norm": 0.0012563092168420553,
+      "learning_rate": 7.314256005594885e-05,
+      "loss": 0.0068,
+      "step": 2830
+    },
+    {
+      "epoch": 1.7719188767550702,
+      "grad_norm": 0.08966930955648422,
+      "learning_rate": 7.251124286796455e-05,
+      "loss": 0.0036,
+      "step": 2840
+    },
+    {
+      "epoch": 1.7781591263650545,
+      "grad_norm": 0.006013158708810806,
+      "learning_rate": 7.188110862243384e-05,
+      "loss": 0.002,
+      "step": 2850
+    },
+    {
+      "epoch": 1.784399375975039,
+      "grad_norm": 0.07801369577646255,
+      "learning_rate": 7.125218443635393e-05,
+      "loss": 0.0045,
+      "step": 2860
+    },
+    {
+      "epoch": 1.7906396255850234,
+      "grad_norm": 0.04352996125817299,
+      "learning_rate": 7.062449737464877e-05,
+      "loss": 0.0047,
+      "step": 2870
+    },
+    {
+      "epoch": 1.7968798751950077,
+      "grad_norm": 0.011117925867438316,
+      "learning_rate": 6.999807444900416e-05,
+      "loss": 0.0019,
+      "step": 2880
+    },
+    {
+      "epoch": 1.8031201248049922,
+      "grad_norm": 0.0019138542702421546,
+      "learning_rate": 6.937294261670558e-05,
+      "loss": 0.0021,
+      "step": 2890
+    },
+    {
+      "epoch": 1.8093603744149767,
+      "grad_norm": 0.0006595162558369339,
+      "learning_rate": 6.874912877947801e-05,
+      "loss": 0.0023,
+      "step": 2900
+    },
+    {
+      "epoch": 1.815600624024961,
+      "grad_norm": 0.0925321877002716,
+      "learning_rate": 6.812665978232814e-05,
+      "loss": 0.0043,
+      "step": 2910
+    },
+    {
+      "epoch": 1.8218408736349454,
+      "grad_norm": 0.0005468401359394193,
+      "learning_rate": 6.750556241238938e-05,
+      "loss": 0.0033,
+      "step": 2920
+    },
+    {
+      "epoch": 1.8280811232449299,
+      "grad_norm": 0.037381552159786224,
+      "learning_rate": 6.688586339776886e-05,
+      "loss": 0.0058,
+      "step": 2930
+    },
+    {
+      "epoch": 1.8343213728549141,
+      "grad_norm": 0.002739663003012538,
+      "learning_rate": 6.626758940639745e-05,
+      "loss": 0.0042,
+      "step": 2940
+    },
+    {
+      "epoch": 1.8405616224648986,
+      "grad_norm": 0.11427415907382965,
+      "learning_rate": 6.565076704488187e-05,
+      "loss": 0.004,
+      "step": 2950
+    },
+    {
+      "epoch": 1.846801872074883,
+      "grad_norm": 0.03485134243965149,
+      "learning_rate": 6.503542285735994e-05,
+      "loss": 0.0048,
+      "step": 2960
+    },
+    {
+      "epoch": 1.8530421216848674,
+      "grad_norm": 0.009942654520273209,
+      "learning_rate": 6.442158332435826e-05,
+      "loss": 0.004,
+      "step": 2970
+    },
+    {
+      "epoch": 1.8592823712948516,
+      "grad_norm": 0.06687472760677338,
+      "learning_rate": 6.380927486165253e-05,
+      "loss": 0.0059,
+      "step": 2980
+    },
+    {
+      "epoch": 1.8655226209048363,
+      "grad_norm": 0.002326112939044833,
+      "learning_rate": 6.319852381913087e-05,
+      "loss": 0.0041,
+      "step": 2990
+    },
+    {
+      "epoch": 1.8717628705148206,
+      "grad_norm": 0.002811886603012681,
+      "learning_rate": 6.258935647965987e-05,
+      "loss": 0.0054,
+      "step": 3000
     }
   ],
   "logging_steps": 10,
+  "max_steps": 4809,
   "num_input_tokens_seen": 0,
   "num_train_epochs": 3,
   "save_steps": 100,
       "attributes": {}
     }
   },
+  "total_flos": 2.4257904823962624e+16,
   "train_batch_size": 2,
   "trial_name": null,
   "trial_params": null

last-checkpoint/training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6df47478e677d9e0d8880cf9d4aa39fd45fc494558b25d955602aea050586dbf
 size 6353

 version https://git-lfs.github.com/spec/v1
+oid sha256:a7b86e3819a1931d176f3cf8aa240c5ae52991ad2bb53a0e4063d88fb1d5392e
 size 6353