Instructions to use Ba2han/l-cpt-1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ba2han/l-cpt-1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Ba2han/l-cpt-1")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("Ba2han/l-cpt-1")
model = AutoModelForImageTextToText.from_pretrained("Ba2han/l-cpt-1")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Ba2han/l-cpt-1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ba2han/l-cpt-1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/l-cpt-1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Ba2han/l-cpt-1

SGLang

How to use Ba2han/l-cpt-1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ba2han/l-cpt-1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/l-cpt-1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ba2han/l-cpt-1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/l-cpt-1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Unsloth Studio new

How to use Ba2han/l-cpt-1 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ba2han/l-cpt-1 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ba2han/l-cpt-1 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Ba2han/l-cpt-1 to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="Ba2han/l-cpt-1",
    max_seq_length=2048,
)

Docker Model Runner
How to use Ba2han/l-cpt-1 with Docker Model Runner:
```
docker model run hf.co/Ba2han/l-cpt-1
```

Ba2han commited on Apr 14

Commit

3ca3c61

verified ·

1 Parent(s): e5d5a36

Training in progress, step 500, checkpoint

Browse files

Files changed (11) hide show

last-checkpoint/chat_template.jinja +65 -0
last-checkpoint/config.json +107 -0
last-checkpoint/generation_config.json +10 -0
last-checkpoint/model.safetensors +3 -0
last-checkpoint/optimizer.pt +3 -0
last-checkpoint/rng_state.pth +3 -0
last-checkpoint/scheduler.pt +3 -0
last-checkpoint/tokenizer.json +0 -0
last-checkpoint/tokenizer_config.json +27 -0
last-checkpoint/trainer_state.json +3534 -0
last-checkpoint/training_args.bin +3 -0

last-checkpoint/chat_template.jinja ADDED Viewed

	@@ -0,0 +1,65 @@

+{{- bos_token -}}
+{%- set keep_past_thinking = keep_past_thinking | default(false) -%}
+{%- set ns = namespace(system_prompt="") -%}
+{%- if messages[0]["role"] == "system" -%}
+    {%- set sys_content = messages[0]["content"] -%}
+    {%- if sys_content is not string -%}
+        {%- for item in sys_content -%}
+            {%- if item["type"] == "text" -%}
+                {%- set ns.system_prompt = ns.system_prompt + item["text"] -%}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {%- set ns.system_prompt = sys_content -%}
+    {%- endif -%}
+    {%- set messages = messages[1:] -%}
+{%- endif -%}
+{%- if tools -%}
+    {%- set ns.system_prompt = ns.system_prompt + ("\n" if ns.system_prompt else "") + "List of tools: [" -%}
+    {%- for tool in tools -%}
+        {%- if tool is not string -%}
+            {%- set tool = tool | tojson -%}
+        {%- endif -%}
+        {%- set ns.system_prompt = ns.system_prompt + tool -%}
+        {%- if not loop.last -%}
+            {%- set ns.system_prompt = ns.system_prompt + ", " -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- set ns.system_prompt = ns.system_prompt + "]" -%}
+{%- endif -%}
+{%- if ns.system_prompt -%}
+    {{- "<|im_start|>system\n" + ns.system_prompt + "<|im_end|>\n" -}}
+{%- endif -%}
+{%- set ns.last_assistant_index = -1 -%}
+{%- for message in messages -%}
+    {%- if message["role"] == "assistant" -%}
+        {%- set ns.last_assistant_index = loop.index0 -%}
+    {%- endif -%}
+{%- endfor -%}
+{%- for message in messages -%}
+    {{- "<|im_start|>" + message["role"] + "\n" -}}
+    {%- if message["content"] is not string -%}
+        {%- set ns.content = "" -%}
+        {%- for item in message["content"] -%}
+            {%- if item["type"] == "image" -%}
+                {%- set ns.content = ns.content + "<image>" -%}
+            {%- elif item["type"] == "text" -%}
+                {%- set ns.content = ns.content + item["text"] -%}
+            {%- else -%}
+                {%- set ns.content = ns.content + item | tojson -%}
+            {%- endif -%}
+        {%- endfor -%}
+        {%- set content = ns.content -%}
+    {%- else -%}
+        {%- set content = message["content"] -%}
+    {%- endif -%}
+    {%- if message["role"] == "assistant" and not keep_past_thinking and loop.index0 != ns.last_assistant_index -%}
+        {%- if "</think>" in content -%}
+            {%- set content = content.split("</think>")[-1] | trim -%}
+        {%- endif -%}
+    {%- endif -%}
+    {{- content + "<|im_end|>\n" -}}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{- "<|im_start|>assistant\n" -}}
+{%- endif -%}

last-checkpoint/config.json ADDED Viewed

	@@ -0,0 +1,107 @@

+{
+  "architectures": [
+    "Lfm2VlForConditionalGeneration"
+  ],
+  "bos_token_id": 1,
+  "do_image_splitting": true,
+  "downsample_factor": 2,
+  "dtype": "bfloat16",
+  "encoder_patch_size": 16,
+  "eos_token_id": 7,
+  "image_token_id": 396,
+  "max_image_tokens": 256,
+  "max_pixels_tolerance": 2.0,
+  "max_tiles": 10,
+  "min_image_tokens": 64,
+  "min_tiles": 2,
+  "model_name": "LiquidAI/LFM2.5-VL-1.6B",
+  "model_type": "lfm2_vl",
+  "pad_token_id": 0,
+  "projector_bias": true,
+  "projector_hidden_act": "gelu",
+  "projector_hidden_size": 2048,
+  "projector_use_layernorm": false,
+  "text_config": {
+    "_name_or_path": "LiquidAI/LFM2-1.2B",
+    "architectures": [
+      "Lfm2ForCausalLM"
+    ],
+    "block_auto_adjust_ff_dim": true,
+    "block_dim": 2048,
+    "block_ffn_dim_multiplier": 1.0,
+    "block_mlp_init_scale": 1.0,
+    "block_multiple_of": 256,
+    "block_norm_eps": 1e-05,
+    "block_out_init_scale": 1.0,
+    "block_use_swiglu": true,
+    "block_use_xavier_init": true,
+    "bos_token_id": 1,
+    "conv_L_cache": 3,
+    "conv_bias": false,
+    "conv_dim": 2048,
+    "conv_dim_out": 2048,
+    "conv_use_xavier_init": true,
+    "dtype": "bfloat16",
+    "eos_token_id": 7,
+    "full_attn_idxs": null,
+    "hidden_size": 2048,
+    "initializer_range": 0.02,
+    "intermediate_size": 12288,
+    "layer_types": [
+      "conv",
+      "conv",
+      "full_attention",
+      "conv",
+      "conv",
+      "full_attention",
+      "conv",
+      "conv",
+      "full_attention",
+      "conv",
+      "full_attention",
+      "conv",
+      "full_attention",
+      "conv",
+      "full_attention",
+      "conv"
+    ],
+    "max_position_embeddings": 128000,
+    "model_type": "lfm2",
+    "norm_eps": 1e-05,
+    "num_attention_heads": 32,
+    "num_heads": 32,
+    "num_hidden_layers": 16,
+    "num_key_value_heads": 8,
+    "pad_token_id": 0,
+    "rope_parameters": {
+      "rope_theta": 1000000.0,
+      "rope_type": "default"
+    },
+    "tie_word_embeddings": true,
+    "use_cache": true,
+    "use_pos_enc": true,
+    "vocab_size": 65536
+  },
+  "tie_word_embeddings": true,
+  "tile_size": 512,
+  "transformers_version": "5.5.4",
+  "unsloth_version": "2026.4.4",
+  "use_cache": false,
+  "use_image_special_tokens": true,
+  "use_thumbnail": true,
+  "vision_config": {
+    "attention_dropout": 0.0,
+    "dtype": "bfloat16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "intermediate_size": 4304,
+    "layer_norm_eps": 1e-06,
+    "model_type": "siglip2_vision_model",
+    "num_attention_heads": 16,
+    "num_channels": 3,
+    "num_hidden_layers": 27,
+    "num_patches": 256,
+    "patch_size": 16,
+    "vision_use_head": false
+  }
+}

last-checkpoint/generation_config.json ADDED Viewed

	@@ -0,0 +1,10 @@

+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": [
+    7,
+    7
+  ],
+  "pad_token_id": 0,
+  "transformers_version": "5.5.4"
+}

last-checkpoint/model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5677a2173a33d6f66237aa196624808c2b78e97d8144c6961fe447aacd25cdcf
+size 3193334216

last-checkpoint/optimizer.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:446bbd9048afe33cbf1538a14a5932be07294ed52bc87a492feed52801fac519
+size 3464388157

last-checkpoint/rng_state.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0e831a926501c469a9775d4b96fe9f086ccb9933946219ebf4fe959ebd9224d3
+size 14709

last-checkpoint/scheduler.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7a1d514a74775420a0bc0eb69e376ced5433951b8aed35df81a76d745d37898f
+size 1465

last-checkpoint/tokenizer.json ADDED Viewed

The diff for this file is too large to render. See raw diff

last-checkpoint/tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,27 @@

+{
+  "backend": "tokenizers",
+  "bos_token": "<|startoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|im_end|>",
+  "image_end_token": "<|image_end|>",
+  "image_start_token": "<|image_start|>",
+  "image_thumbnail": "<|img_thumbnail|>",
+  "image_token": "<image>",
+  "is_local": false,
+  "legacy": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "image_end_token": "<|image_end|>",
+    "image_start_token": "<|image_start|>",
+    "image_token": "<image>"
+  },
+  "pad_token": "<|pad|>",
+  "padding_side": "right",
+  "processor_class": "Lfm2VlProcessor",
+  "return_token_type_ids": false,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "use_default_system_prompt": false,
+  "use_fast": true
+}

last-checkpoint/trainer_state.json ADDED Viewed

	@@ -0,0 +1,3534 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.04357470542818276,
+  "eval_steps": 500,
+  "global_step": 500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 8.714941085636552e-05,
+      "grad_norm": 16.375,
+      "learning_rate": 1e-06,
+      "loss": 2.980116844177246,
+      "step": 1
+    },
+    {
+      "epoch": 0.00017429882171273104,
+      "grad_norm": 19.875,
+      "learning_rate": 7.912745179406311e-06,
+      "loss": 3.0092227458953857,
+      "step": 2
+    },
+    {
+      "epoch": 0.00026144823256909653,
+      "grad_norm": 22.5,
+      "learning_rate": 1.4825490358812626e-05,
+      "loss": 3.037306785583496,
+      "step": 3
+    },
+    {
+      "epoch": 0.0003485976434254621,
+      "grad_norm": 15.0625,
+      "learning_rate": 2.1738235538218935e-05,
+      "loss": 2.9077248573303223,
+      "step": 4
+    },
+    {
+      "epoch": 0.00043574705428182757,
+      "grad_norm": 16.5,
+      "learning_rate": 2.8650980717625252e-05,
+      "loss": 3.111506223678589,
+      "step": 5
+    },
+    {
+      "epoch": 0.0005228964651381931,
+      "grad_norm": 18.75,
+      "learning_rate": 3.5563725897031555e-05,
+      "loss": 2.9358785152435303,
+      "step": 6
+    },
+    {
+      "epoch": 0.0006100458759945586,
+      "grad_norm": 17.0,
+      "learning_rate": 4.2476471076437866e-05,
+      "loss": 2.9959819316864014,
+      "step": 7
+    },
+    {
+      "epoch": 0.0006971952868509242,
+      "grad_norm": 19.875,
+      "learning_rate": 4.938921625584418e-05,
+      "loss": 3.071417808532715,
+      "step": 8
+    },
+    {
+      "epoch": 0.0007843446977072896,
+      "grad_norm": 19.25,
+      "learning_rate": 5.63019614352505e-05,
+      "loss": 2.969663381576538,
+      "step": 9
+    },
+    {
+      "epoch": 0.0008714941085636551,
+      "grad_norm": 15.25,
+      "learning_rate": 6.321470661465681e-05,
+      "loss": 2.907989740371704,
+      "step": 10
+    },
+    {
+      "epoch": 0.0009586435194200207,
+      "grad_norm": 16.125,
+      "learning_rate": 7.012745179406311e-05,
+      "loss": 2.910642385482788,
+      "step": 11
+    },
+    {
+      "epoch": 0.0010457929302763861,
+      "grad_norm": 16.875,
+      "learning_rate": 7.704019697346946e-05,
+      "loss": 2.9582762718200684,
+      "step": 12
+    },
+    {
+      "epoch": 0.0011329423411327517,
+      "grad_norm": 17.375,
+      "learning_rate": 8.395294215287573e-05,
+      "loss": 2.97444748878479,
+      "step": 13
+    },
+    {
+      "epoch": 0.0012200917519891172,
+      "grad_norm": 12.8125,
+      "learning_rate": 9.086568733228206e-05,
+      "loss": 2.9798922538757324,
+      "step": 14
+    },
+    {
+      "epoch": 0.0013072411628454828,
+      "grad_norm": 18.125,
+      "learning_rate": 9.777843251168837e-05,
+      "loss": 2.890429973602295,
+      "step": 15
+    },
+    {
+      "epoch": 0.0013943905737018483,
+      "grad_norm": 12.0,
+      "learning_rate": 0.00010469117769109469,
+      "loss": 2.9602813720703125,
+      "step": 16
+    },
+    {
+      "epoch": 0.0014815399845582139,
+      "grad_norm": 8.6875,
+      "learning_rate": 0.000111603922870501,
+      "loss": 2.978081226348877,
+      "step": 17
+    },
+    {
+      "epoch": 0.0015686893954145792,
+      "grad_norm": 7.90625,
+      "learning_rate": 0.00011851666804990732,
+      "loss": 2.899472713470459,
+      "step": 18
+    },
+    {
+      "epoch": 0.0016558388062709447,
+      "grad_norm": 9.375,
+      "learning_rate": 0.00012542941322931362,
+      "loss": 2.990980386734009,
+      "step": 19
+    },
+    {
+      "epoch": 0.0017429882171273103,
+      "grad_norm": 7.59375,
+      "learning_rate": 0.00013234215840871993,
+      "loss": 2.841034412384033,
+      "step": 20
+    },
+    {
+      "epoch": 0.0018301376279836758,
+      "grad_norm": 8.375,
+      "learning_rate": 0.00013925490358812623,
+      "loss": 2.9813711643218994,
+      "step": 21
+    },
+    {
+      "epoch": 0.0019172870388400414,
+      "grad_norm": 9.4375,
+      "learning_rate": 0.00014616764876753256,
+      "loss": 2.945570945739746,
+      "step": 22
+    },
+    {
+      "epoch": 0.002004436449696407,
+      "grad_norm": 6.0,
+      "learning_rate": 0.0001530803939469389,
+      "loss": 3.026611089706421,
+      "step": 23
+    },
+    {
+      "epoch": 0.0020915858605527722,
+      "grad_norm": 6.59375,
+      "learning_rate": 0.0001599931391263452,
+      "loss": 3.0398619174957275,
+      "step": 24
+    },
+    {
+      "epoch": 0.002178735271409138,
+      "grad_norm": 4.9375,
+      "learning_rate": 0.00016690588430575147,
+      "loss": 2.729804277420044,
+      "step": 25
+    },
+    {
+      "epoch": 0.0022658846822655033,
+      "grad_norm": 4.4375,
+      "learning_rate": 0.0001738186294851578,
+      "loss": 2.8671164512634277,
+      "step": 26
+    },
+    {
+      "epoch": 0.002353034093121869,
+      "grad_norm": 4.75,
+      "learning_rate": 0.00018073137466456413,
+      "loss": 2.892765760421753,
+      "step": 27
+    },
+    {
+      "epoch": 0.0024401835039782344,
+      "grad_norm": 3.34375,
+      "learning_rate": 0.00018764411984397044,
+      "loss": 2.7960426807403564,
+      "step": 28
+    },
+    {
+      "epoch": 0.0025273329148345998,
+      "grad_norm": 3.265625,
+      "learning_rate": 0.00019455686502337674,
+      "loss": 2.8209526538848877,
+      "step": 29
+    },
+    {
+      "epoch": 0.0026144823256909655,
+      "grad_norm": 2.890625,
+      "learning_rate": 0.00020146961020278307,
+      "loss": 2.950427293777466,
+      "step": 30
+    },
+    {
+      "epoch": 0.002701631736547331,
+      "grad_norm": 2.625,
+      "learning_rate": 0.00020838235538218935,
+      "loss": 2.760995626449585,
+      "step": 31
+    },
+    {
+      "epoch": 0.0027887811474036966,
+      "grad_norm": 2.5625,
+      "learning_rate": 0.00021529510056159568,
+      "loss": 2.905595064163208,
+      "step": 32
+    },
+    {
+      "epoch": 0.002875930558260062,
+      "grad_norm": 2.484375,
+      "learning_rate": 0.000222207845741002,
+      "loss": 2.885206460952759,
+      "step": 33
+    },
+    {
+      "epoch": 0.0029630799691164277,
+      "grad_norm": 2.5625,
+      "learning_rate": 0.00022912059092040834,
+      "loss": 3.003300666809082,
+      "step": 34
+    },
+    {
+      "epoch": 0.003050229379972793,
+      "grad_norm": 2.375,
+      "learning_rate": 0.00023603333609981464,
+      "loss": 2.8469698429107666,
+      "step": 35
+    },
+    {
+      "epoch": 0.0031373787908291584,
+      "grad_norm": 4.0625,
+      "learning_rate": 0.00024294608127922094,
+      "loss": 2.7171757221221924,
+      "step": 36
+    },
+    {
+      "epoch": 0.003224528201685524,
+      "grad_norm": 2.1875,
+      "learning_rate": 0.0002498588264586273,
+      "loss": 2.888355255126953,
+      "step": 37
+    },
+    {
+      "epoch": 0.0033116776125418895,
+      "grad_norm": 2.125,
+      "learning_rate": 0.0002567715716380336,
+      "loss": 2.792030096054077,
+      "step": 38
+    },
+    {
+      "epoch": 0.0033988270233982552,
+      "grad_norm": 2.28125,
+      "learning_rate": 0.0002636843168174399,
+      "loss": 2.853731393814087,
+      "step": 39
+    },
+    {
+      "epoch": 0.0034859764342546205,
+      "grad_norm": 2.1875,
+      "learning_rate": 0.0002705970619968462,
+      "loss": 3.0276989936828613,
+      "step": 40
+    },
+    {
+      "epoch": 0.0035731258451109863,
+      "grad_norm": 2.359375,
+      "learning_rate": 0.0002775098071762525,
+      "loss": 3.0183589458465576,
+      "step": 41
+    },
+    {
+      "epoch": 0.0036602752559673516,
+      "grad_norm": 2.171875,
+      "learning_rate": 0.0002844225523556588,
+      "loss": 2.8405275344848633,
+      "step": 42
+    },
+    {
+      "epoch": 0.003747424666823717,
+      "grad_norm": 2.25,
+      "learning_rate": 0.00029133529753506515,
+      "loss": 2.770927667617798,
+      "step": 43
+    },
+    {
+      "epoch": 0.0038345740776800827,
+      "grad_norm": 2.171875,
+      "learning_rate": 0.0002982480427144715,
+      "loss": 2.7248220443725586,
+      "step": 44
+    },
+    {
+      "epoch": 0.0039217234885364485,
+      "grad_norm": 2.109375,
+      "learning_rate": 0.0003051607878938778,
+      "loss": 2.926846504211426,
+      "step": 45
+    },
+    {
+      "epoch": 0.004008872899392814,
+      "grad_norm": 2.0625,
+      "learning_rate": 0.0003120735330732841,
+      "loss": 2.7914741039276123,
+      "step": 46
+    },
+    {
+      "epoch": 0.004096022310249179,
+      "grad_norm": 1.9609375,
+      "learning_rate": 0.0003189862782526904,
+      "loss": 2.8833417892456055,
+      "step": 47
+    },
+    {
+      "epoch": 0.0041831717211055445,
+      "grad_norm": 2.171875,
+      "learning_rate": 0.0003258990234320967,
+      "loss": 2.822157382965088,
+      "step": 48
+    },
+    {
+      "epoch": 0.004270321131961911,
+      "grad_norm": 2.21875,
+      "learning_rate": 0.00033281176861150297,
+      "loss": 2.8196861743927,
+      "step": 49
+    },
+    {
+      "epoch": 0.004357470542818276,
+      "grad_norm": 1.921875,
+      "learning_rate": 0.0003397245137909093,
+      "loss": 2.7460930347442627,
+      "step": 50
+    },
+    {
+      "epoch": 0.004444619953674641,
+      "grad_norm": 1.828125,
+      "learning_rate": 0.00034663725897031563,
+      "loss": 2.8679468631744385,
+      "step": 51
+    },
+    {
+      "epoch": 0.004531769364531007,
+      "grad_norm": 1.9765625,
+      "learning_rate": 0.00035355000414972196,
+      "loss": 2.670346975326538,
+      "step": 52
+    },
+    {
+      "epoch": 0.004618918775387372,
+      "grad_norm": 1.9453125,
+      "learning_rate": 0.0003604627493291283,
+      "loss": 2.8237805366516113,
+      "step": 53
+    },
+    {
+      "epoch": 0.004706068186243738,
+      "grad_norm": 1.7734375,
+      "learning_rate": 0.00036737549450853457,
+      "loss": 2.686187505722046,
+      "step": 54
+    },
+    {
+      "epoch": 0.0047932175971001035,
+      "grad_norm": 1.8125,
+      "learning_rate": 0.0003742882396879409,
+      "loss": 2.8298659324645996,
+      "step": 55
+    },
+    {
+      "epoch": 0.004880367007956469,
+      "grad_norm": 2.109375,
+      "learning_rate": 0.0003812009848673472,
+      "loss": 2.807030439376831,
+      "step": 56
+    },
+    {
+      "epoch": 0.004967516418812834,
+      "grad_norm": 1.828125,
+      "learning_rate": 0.0003881137300467535,
+      "loss": 2.7621939182281494,
+      "step": 57
+    },
+    {
+      "epoch": 0.0050546658296691995,
+      "grad_norm": 1.8828125,
+      "learning_rate": 0.00039502647522615984,
+      "loss": 2.792243003845215,
+      "step": 58
+    },
+    {
+      "epoch": 0.005141815240525566,
+      "grad_norm": 1.796875,
+      "learning_rate": 0.00040193922040556617,
+      "loss": 2.6842851638793945,
+      "step": 59
+    },
+    {
+      "epoch": 0.005228964651381931,
+      "grad_norm": 2.0625,
+      "learning_rate": 0.0004088519655849725,
+      "loss": 2.967288017272949,
+      "step": 60
+    },
+    {
+      "epoch": 0.005316114062238296,
+      "grad_norm": 1.859375,
+      "learning_rate": 0.0004157647107643788,
+      "loss": 2.664837121963501,
+      "step": 61
+    },
+    {
+      "epoch": 0.005403263473094662,
+      "grad_norm": 1.8984375,
+      "learning_rate": 0.0004226774559437851,
+      "loss": 2.790057420730591,
+      "step": 62
+    },
+    {
+      "epoch": 0.005490412883951028,
+      "grad_norm": 1.7578125,
+      "learning_rate": 0.0004295902011231914,
+      "loss": 2.6879723072052,
+      "step": 63
+    },
+    {
+      "epoch": 0.005577562294807393,
+      "grad_norm": 1.7265625,
+      "learning_rate": 0.0004365029463025977,
+      "loss": 2.6767494678497314,
+      "step": 64
+    },
+    {
+      "epoch": 0.0056647117056637586,
+      "grad_norm": 1.8984375,
+      "learning_rate": 0.00044341569148200404,
+      "loss": 2.7523298263549805,
+      "step": 65
+    },
+    {
+      "epoch": 0.005751861116520124,
+      "grad_norm": 1.765625,
+      "learning_rate": 0.0004503284366614104,
+      "loss": 2.6057050228118896,
+      "step": 66
+    },
+    {
+      "epoch": 0.005839010527376489,
+      "grad_norm": 1.78125,
+      "learning_rate": 0.0004572411818408167,
+      "loss": 2.6737048625946045,
+      "step": 67
+    },
+    {
+      "epoch": 0.005926159938232855,
+      "grad_norm": 1.765625,
+      "learning_rate": 0.000464153927020223,
+      "loss": 2.755410671234131,
+      "step": 68
+    },
+    {
+      "epoch": 0.006013309349089221,
+      "grad_norm": 1.6953125,
+      "learning_rate": 0.0004710666721996293,
+      "loss": 2.7767393589019775,
+      "step": 69
+    },
+    {
+      "epoch": 0.006100458759945586,
+      "grad_norm": 1.6875,
+      "learning_rate": 0.0004779794173790356,
+      "loss": 2.5728745460510254,
+      "step": 70
+    },
+    {
+      "epoch": 0.006187608170801951,
+      "grad_norm": 1.6796875,
+      "learning_rate": 0.0004848921625584419,
+      "loss": 2.5615997314453125,
+      "step": 71
+    },
+    {
+      "epoch": 0.006274757581658317,
+      "grad_norm": 1.875,
+      "learning_rate": 0.0004918049077378483,
+      "loss": 2.6013240814208984,
+      "step": 72
+    },
+    {
+      "epoch": 0.006361906992514683,
+      "grad_norm": 1.921875,
+      "learning_rate": 0.0004987176529172545,
+      "loss": 2.709704637527466,
+      "step": 73
+    },
+    {
+      "epoch": 0.006449056403371048,
+      "grad_norm": 1.765625,
+      "learning_rate": 0.0005056303980966609,
+      "loss": 2.5971601009368896,
+      "step": 74
+    },
+    {
+      "epoch": 0.006536205814227414,
+      "grad_norm": 1.734375,
+      "learning_rate": 0.0005125431432760672,
+      "loss": 2.578047513961792,
+      "step": 75
+    },
+    {
+      "epoch": 0.006623355225083779,
+      "grad_norm": 1.6171875,
+      "learning_rate": 0.0005194558884554734,
+      "loss": 2.471486806869507,
+      "step": 76
+    },
+    {
+      "epoch": 0.006710504635940145,
+      "grad_norm": 1.625,
+      "learning_rate": 0.0005263686336348797,
+      "loss": 2.6826093196868896,
+      "step": 77
+    },
+    {
+      "epoch": 0.0067976540467965104,
+      "grad_norm": 1.8046875,
+      "learning_rate": 0.0005332813788142861,
+      "loss": 2.600142240524292,
+      "step": 78
+    },
+    {
+      "epoch": 0.006884803457652876,
+      "grad_norm": 1.6328125,
+      "learning_rate": 0.0005401941239936924,
+      "loss": 2.487628221511841,
+      "step": 79
+    },
+    {
+      "epoch": 0.006971952868509241,
+      "grad_norm": 1.7734375,
+      "learning_rate": 0.0005471068691730987,
+      "loss": 2.510748863220215,
+      "step": 80
+    },
+    {
+      "epoch": 0.007059102279365606,
+      "grad_norm": 1.703125,
+      "learning_rate": 0.000554019614352505,
+      "loss": 2.5530645847320557,
+      "step": 81
+    },
+    {
+      "epoch": 0.007146251690221973,
+      "grad_norm": 1.6640625,
+      "learning_rate": 0.0005609323595319114,
+      "loss": 2.6373367309570312,
+      "step": 82
+    },
+    {
+      "epoch": 0.007233401101078338,
+      "grad_norm": 1.71875,
+      "learning_rate": 0.0005678451047113176,
+      "loss": 2.556617498397827,
+      "step": 83
+    },
+    {
+      "epoch": 0.007320550511934703,
+      "grad_norm": 1.8046875,
+      "learning_rate": 0.0005747578498907239,
+      "loss": 2.5635764598846436,
+      "step": 84
+    },
+    {
+      "epoch": 0.007407699922791069,
+      "grad_norm": 1.765625,
+      "learning_rate": 0.0005816705950701303,
+      "loss": 2.5362768173217773,
+      "step": 85
+    },
+    {
+      "epoch": 0.007494849333647434,
+      "grad_norm": 1.7109375,
+      "learning_rate": 0.0005885833402495366,
+      "loss": 2.5362229347229004,
+      "step": 86
+    },
+    {
+      "epoch": 0.0075819987445038,
+      "grad_norm": 1.859375,
+      "learning_rate": 0.0005954960854289429,
+      "loss": 2.6001806259155273,
+      "step": 87
+    },
+    {
+      "epoch": 0.0076691481553601655,
+      "grad_norm": 1.8671875,
+      "learning_rate": 0.0006024088306083492,
+      "loss": 2.541794776916504,
+      "step": 88
+    },
+    {
+      "epoch": 0.007756297566216531,
+      "grad_norm": 1.90625,
+      "learning_rate": 0.0006093215757877556,
+      "loss": 2.6054954528808594,
+      "step": 89
+    },
+    {
+      "epoch": 0.007843446977072897,
+      "grad_norm": 1.6796875,
+      "learning_rate": 0.0006162343209671618,
+      "loss": 2.5833845138549805,
+      "step": 90
+    },
+    {
+      "epoch": 0.007930596387929261,
+      "grad_norm": 1.8046875,
+      "learning_rate": 0.0006231470661465682,
+      "loss": 2.623051881790161,
+      "step": 91
+    },
+    {
+      "epoch": 0.008017745798785628,
+      "grad_norm": 1.8359375,
+      "learning_rate": 0.0006300598113259745,
+      "loss": 2.5758605003356934,
+      "step": 92
+    },
+    {
+      "epoch": 0.008104895209641992,
+      "grad_norm": 1.671875,
+      "learning_rate": 0.0006369725565053808,
+      "loss": 2.433521270751953,
+      "step": 93
+    },
+    {
+      "epoch": 0.008192044620498358,
+      "grad_norm": 1.5859375,
+      "learning_rate": 0.0006438853016847871,
+      "loss": 2.5033862590789795,
+      "step": 94
+    },
+    {
+      "epoch": 0.008279194031354725,
+      "grad_norm": 1.65625,
+      "learning_rate": 0.0006507980468641934,
+      "loss": 2.486121416091919,
+      "step": 95
+    },
+    {
+      "epoch": 0.008366343442211089,
+      "grad_norm": 1.5859375,
+      "learning_rate": 0.0006577107920435997,
+      "loss": 2.301229953765869,
+      "step": 96
+    },
+    {
+      "epoch": 0.008453492853067455,
+      "grad_norm": 1.578125,
+      "learning_rate": 0.0006646235372230059,
+      "loss": 2.4249179363250732,
+      "step": 97
+    },
+    {
+      "epoch": 0.008540642263923821,
+      "grad_norm": 1.6328125,
+      "learning_rate": 0.0006715362824024124,
+      "loss": 2.4232752323150635,
+      "step": 98
+    },
+    {
+      "epoch": 0.008627791674780186,
+      "grad_norm": 1.7890625,
+      "learning_rate": 0.0006784490275818186,
+      "loss": 2.5611414909362793,
+      "step": 99
+    },
+    {
+      "epoch": 0.008714941085636552,
+      "grad_norm": 1.6953125,
+      "learning_rate": 0.0006853617727612249,
+      "loss": 2.3588075637817383,
+      "step": 100
+    },
+    {
+      "epoch": 0.008802090496492916,
+      "grad_norm": 1.59375,
+      "learning_rate": 0.0006922745179406312,
+      "loss": 2.506977081298828,
+      "step": 101
+    },
+    {
+      "epoch": 0.008889239907349283,
+      "grad_norm": 1.6640625,
+      "learning_rate": 0.0006991872631200376,
+      "loss": 2.470222234725952,
+      "step": 102
+    },
+    {
+      "epoch": 0.008976389318205649,
+      "grad_norm": 1.5234375,
+      "learning_rate": 0.0007061000082994439,
+      "loss": 2.3257453441619873,
+      "step": 103
+    },
+    {
+      "epoch": 0.009063538729062013,
+      "grad_norm": 1.796875,
+      "learning_rate": 0.0007130127534788501,
+      "loss": 2.4208428859710693,
+      "step": 104
+    },
+    {
+      "epoch": 0.00915068813991838,
+      "grad_norm": 1.703125,
+      "learning_rate": 0.0007199254986582566,
+      "loss": 2.469883441925049,
+      "step": 105
+    },
+    {
+      "epoch": 0.009237837550774744,
+      "grad_norm": 1.765625,
+      "learning_rate": 0.0007268382438376628,
+      "loss": 2.4200963973999023,
+      "step": 106
+    },
+    {
+      "epoch": 0.00932498696163111,
+      "grad_norm": 1.703125,
+      "learning_rate": 0.0007337509890170691,
+      "loss": 2.3745367527008057,
+      "step": 107
+    },
+    {
+      "epoch": 0.009412136372487476,
+      "grad_norm": 1.7734375,
+      "learning_rate": 0.0007406637341964754,
+      "loss": 2.3642737865448,
+      "step": 108
+    },
+    {
+      "epoch": 0.00949928578334384,
+      "grad_norm": 1.625,
+      "learning_rate": 0.0007475764793758818,
+      "loss": 2.4235646724700928,
+      "step": 109
+    },
+    {
+      "epoch": 0.009586435194200207,
+      "grad_norm": 1.59375,
+      "learning_rate": 0.0007544892245552881,
+      "loss": 2.322707414627075,
+      "step": 110
+    },
+    {
+      "epoch": 0.009673584605056572,
+      "grad_norm": 1.796875,
+      "learning_rate": 0.0007614019697346943,
+      "loss": 2.4208385944366455,
+      "step": 111
+    },
+    {
+      "epoch": 0.009760734015912938,
+      "grad_norm": 1.6796875,
+      "learning_rate": 0.0007683147149141008,
+      "loss": 2.3120336532592773,
+      "step": 112
+    },
+    {
+      "epoch": 0.009847883426769304,
+      "grad_norm": 1.7265625,
+      "learning_rate": 0.000775227460093507,
+      "loss": 2.3327786922454834,
+      "step": 113
+    },
+    {
+      "epoch": 0.009935032837625668,
+      "grad_norm": 1.71875,
+      "learning_rate": 0.0007821402052729133,
+      "loss": 2.3989481925964355,
+      "step": 114
+    },
+    {
+      "epoch": 0.010022182248482035,
+      "grad_norm": 1.765625,
+      "learning_rate": 0.0007890529504523197,
+      "loss": 2.408418893814087,
+      "step": 115
+    },
+    {
+      "epoch": 0.010109331659338399,
+      "grad_norm": 1.6484375,
+      "learning_rate": 0.000795965695631726,
+      "loss": 2.494696855545044,
+      "step": 116
+    },
+    {
+      "epoch": 0.010196481070194765,
+      "grad_norm": 1.65625,
+      "learning_rate": 0.0008028784408111323,
+      "loss": 2.403883457183838,
+      "step": 117
+    },
+    {
+      "epoch": 0.010283630481051131,
+      "grad_norm": 1.6484375,
+      "learning_rate": 0.0008097911859905385,
+      "loss": 2.322655439376831,
+      "step": 118
+    },
+    {
+      "epoch": 0.010370779891907496,
+      "grad_norm": 1.6484375,
+      "learning_rate": 0.000816703931169945,
+      "loss": 2.389268636703491,
+      "step": 119
+    },
+    {
+      "epoch": 0.010457929302763862,
+      "grad_norm": 1.84375,
+      "learning_rate": 0.0008236166763493512,
+      "loss": 2.360283851623535,
+      "step": 120
+    },
+    {
+      "epoch": 0.010545078713620227,
+      "grad_norm": 1.5703125,
+      "learning_rate": 0.0008305294215287575,
+      "loss": 2.3794872760772705,
+      "step": 121
+    },
+    {
+      "epoch": 0.010632228124476593,
+      "grad_norm": 1.7578125,
+      "learning_rate": 0.000837442166708164,
+      "loss": 2.305957317352295,
+      "step": 122
+    },
+    {
+      "epoch": 0.010719377535332959,
+      "grad_norm": 1.7265625,
+      "learning_rate": 0.0008443549118875702,
+      "loss": 2.2807626724243164,
+      "step": 123
+    },
+    {
+      "epoch": 0.010806526946189323,
+      "grad_norm": 1.5625,
+      "learning_rate": 0.0008512676570669765,
+      "loss": 2.303670883178711,
+      "step": 124
+    },
+    {
+      "epoch": 0.01089367635704569,
+      "grad_norm": 1.6953125,
+      "learning_rate": 0.0008581804022463827,
+      "loss": 2.3687870502471924,
+      "step": 125
+    },
+    {
+      "epoch": 0.010980825767902056,
+      "grad_norm": 1.6484375,
+      "learning_rate": 0.0008650931474257892,
+      "loss": 2.2691609859466553,
+      "step": 126
+    },
+    {
+      "epoch": 0.01106797517875842,
+      "grad_norm": 1.71875,
+      "learning_rate": 0.0008720058926051954,
+      "loss": 2.1968977451324463,
+      "step": 127
+    },
+    {
+      "epoch": 0.011155124589614786,
+      "grad_norm": 1.609375,
+      "learning_rate": 0.0008789186377846017,
+      "loss": 2.261756181716919,
+      "step": 128
+    },
+    {
+      "epoch": 0.011242274000471151,
+      "grad_norm": 1.6484375,
+      "learning_rate": 0.0008858313829640081,
+      "loss": 2.457672595977783,
+      "step": 129
+    },
+    {
+      "epoch": 0.011329423411327517,
+      "grad_norm": 1.5546875,
+      "learning_rate": 0.0008927441281434144,
+      "loss": 2.3836045265197754,
+      "step": 130
+    },
+    {
+      "epoch": 0.011416572822183883,
+      "grad_norm": 1.71875,
+      "learning_rate": 0.0008996568733228207,
+      "loss": 2.319143056869507,
+      "step": 131
+    },
+    {
+      "epoch": 0.011503722233040248,
+      "grad_norm": 1.734375,
+      "learning_rate": 0.000906569618502227,
+      "loss": 2.393441677093506,
+      "step": 132
+    },
+    {
+      "epoch": 0.011590871643896614,
+      "grad_norm": 1.6328125,
+      "learning_rate": 0.0009134823636816334,
+      "loss": 2.285923957824707,
+      "step": 133
+    },
+    {
+      "epoch": 0.011678021054752978,
+      "grad_norm": 1.6328125,
+      "learning_rate": 0.0009203951088610396,
+      "loss": 2.176638603210449,
+      "step": 134
+    },
+    {
+      "epoch": 0.011765170465609345,
+      "grad_norm": 1.734375,
+      "learning_rate": 0.0009273078540404459,
+      "loss": 2.3897297382354736,
+      "step": 135
+    },
+    {
+      "epoch": 0.01185231987646571,
+      "grad_norm": 1.6875,
+      "learning_rate": 0.0009342205992198523,
+      "loss": 2.421382427215576,
+      "step": 136
+    },
+    {
+      "epoch": 0.011939469287322075,
+      "grad_norm": 1.5859375,
+      "learning_rate": 0.0009411333443992586,
+      "loss": 2.3119120597839355,
+      "step": 137
+    },
+    {
+      "epoch": 0.012026618698178441,
+      "grad_norm": 1.65625,
+      "learning_rate": 0.0009480460895786649,
+      "loss": 2.43278169631958,
+      "step": 138
+    },
+    {
+      "epoch": 0.012113768109034806,
+      "grad_norm": 1.640625,
+      "learning_rate": 0.0009549588347580712,
+      "loss": 2.324469566345215,
+      "step": 139
+    },
+    {
+      "epoch": 0.012200917519891172,
+      "grad_norm": 1.65625,
+      "learning_rate": 0.0009618715799374775,
+      "loss": 2.2620527744293213,
+      "step": 140
+    },
+    {
+      "epoch": 0.012288066930747538,
+      "grad_norm": 1.671875,
+      "learning_rate": 0.0009687843251168838,
+      "loss": 2.3233892917633057,
+      "step": 141
+    },
+    {
+      "epoch": 0.012375216341603903,
+      "grad_norm": 1.6015625,
+      "learning_rate": 0.0009756970702962901,
+      "loss": 2.3079047203063965,
+      "step": 142
+    },
+    {
+      "epoch": 0.012462365752460269,
+      "grad_norm": 1.5859375,
+      "learning_rate": 0.0009826098154756965,
+      "loss": 2.406510353088379,
+      "step": 143
+    },
+    {
+      "epoch": 0.012549515163316633,
+      "grad_norm": 1.6484375,
+      "learning_rate": 0.0009895225606551027,
+      "loss": 2.3381083011627197,
+      "step": 144
+    },
+    {
+      "epoch": 0.012636664574173,
+      "grad_norm": 1.6640625,
+      "learning_rate": 0.000996435305834509,
+      "loss": 2.2969248294830322,
+      "step": 145
+    },
+    {
+      "epoch": 0.012723813985029366,
+      "grad_norm": 1.796875,
+      "learning_rate": 0.0010033480510139154,
+      "loss": 2.1769518852233887,
+      "step": 146
+    },
+    {
+      "epoch": 0.01281096339588573,
+      "grad_norm": 1.734375,
+      "learning_rate": 0.0010102607961933216,
+      "loss": 2.3034377098083496,
+      "step": 147
+    },
+    {
+      "epoch": 0.012898112806742097,
+      "grad_norm": 1.65625,
+      "learning_rate": 0.001017173541372728,
+      "loss": 2.28684139251709,
+      "step": 148
+    },
+    {
+      "epoch": 0.012985262217598461,
+      "grad_norm": 1.59375,
+      "learning_rate": 0.0010240862865521342,
+      "loss": 2.258565664291382,
+      "step": 149
+    },
+    {
+      "epoch": 0.013072411628454827,
+      "grad_norm": 1.5703125,
+      "learning_rate": 0.0010309990317315405,
+      "loss": 2.114553451538086,
+      "step": 150
+    },
+    {
+      "epoch": 0.013159561039311193,
+      "grad_norm": 1.5625,
+      "learning_rate": 0.0010379117769109467,
+      "loss": 2.260165214538574,
+      "step": 151
+    },
+    {
+      "epoch": 0.013246710450167558,
+      "grad_norm": 1.5859375,
+      "learning_rate": 0.0010448245220903531,
+      "loss": 2.2665607929229736,
+      "step": 152
+    },
+    {
+      "epoch": 0.013333859861023924,
+      "grad_norm": 1.4921875,
+      "learning_rate": 0.0010517372672697593,
+      "loss": 2.0330801010131836,
+      "step": 153
+    },
+    {
+      "epoch": 0.01342100927188029,
+      "grad_norm": 1.6796875,
+      "learning_rate": 0.0010586500124491656,
+      "loss": 2.2008140087127686,
+      "step": 154
+    },
+    {
+      "epoch": 0.013508158682736655,
+      "grad_norm": 1.515625,
+      "learning_rate": 0.001065562757628572,
+      "loss": 2.1958272457122803,
+      "step": 155
+    },
+    {
+      "epoch": 0.013595308093593021,
+      "grad_norm": 1.4921875,
+      "learning_rate": 0.0010724755028079784,
+      "loss": 2.3595407009124756,
+      "step": 156
+    },
+    {
+      "epoch": 0.013682457504449385,
+      "grad_norm": 1.515625,
+      "learning_rate": 0.0010793882479873847,
+      "loss": 2.205357789993286,
+      "step": 157
+    },
+    {
+      "epoch": 0.013769606915305752,
+      "grad_norm": 1.7421875,
+      "learning_rate": 0.0010863009931667909,
+      "loss": 2.1484756469726562,
+      "step": 158
+    },
+    {
+      "epoch": 0.013856756326162118,
+      "grad_norm": 1.8671875,
+      "learning_rate": 0.0010932137383461973,
+      "loss": 2.309096336364746,
+      "step": 159
+    },
+    {
+      "epoch": 0.013943905737018482,
+      "grad_norm": 1.6953125,
+      "learning_rate": 0.0011001264835256036,
+      "loss": 2.185431718826294,
+      "step": 160
+    },
+    {
+      "epoch": 0.014031055147874848,
+      "grad_norm": 1.5625,
+      "learning_rate": 0.0011070392287050098,
+      "loss": 2.1881372928619385,
+      "step": 161
+    },
+    {
+      "epoch": 0.014118204558731213,
+      "grad_norm": 1.671875,
+      "learning_rate": 0.0011139519738844162,
+      "loss": 2.2584939002990723,
+      "step": 162
+    },
+    {
+      "epoch": 0.014205353969587579,
+      "grad_norm": 1.4921875,
+      "learning_rate": 0.0011208647190638227,
+      "loss": 2.2748982906341553,
+      "step": 163
+    },
+    {
+      "epoch": 0.014292503380443945,
+      "grad_norm": 1.5625,
+      "learning_rate": 0.0011277774642432289,
+      "loss": 1.9929231405258179,
+      "step": 164
+    },
+    {
+      "epoch": 0.01437965279130031,
+      "grad_norm": 1.59375,
+      "learning_rate": 0.001134690209422635,
+      "loss": 2.2797374725341797,
+      "step": 165
+    },
+    {
+      "epoch": 0.014466802202156676,
+      "grad_norm": 1.5078125,
+      "learning_rate": 0.0011416029546020415,
+      "loss": 2.250164031982422,
+      "step": 166
+    },
+    {
+      "epoch": 0.01455395161301304,
+      "grad_norm": 1.453125,
+      "learning_rate": 0.0011485156997814478,
+      "loss": 2.274890899658203,
+      "step": 167
+    },
+    {
+      "epoch": 0.014641101023869407,
+      "grad_norm": 1.5390625,
+      "learning_rate": 0.001155428444960854,
+      "loss": 2.1506783962249756,
+      "step": 168
+    },
+    {
+      "epoch": 0.014728250434725773,
+      "grad_norm": 1.640625,
+      "learning_rate": 0.0011623411901402604,
+      "loss": 2.2475764751434326,
+      "step": 169
+    },
+    {
+      "epoch": 0.014815399845582137,
+      "grad_norm": 1.609375,
+      "learning_rate": 0.0011692539353196669,
+      "loss": 2.2267816066741943,
+      "step": 170
+    },
+    {
+      "epoch": 0.014902549256438503,
+      "grad_norm": 1.578125,
+      "learning_rate": 0.001176166680499073,
+      "loss": 2.070962905883789,
+      "step": 171
+    },
+    {
+      "epoch": 0.014989698667294868,
+      "grad_norm": 1.6015625,
+      "learning_rate": 0.0011830794256784793,
+      "loss": 2.1214816570281982,
+      "step": 172
+    },
+    {
+      "epoch": 0.015076848078151234,
+      "grad_norm": 1.5078125,
+      "learning_rate": 0.0011899921708578857,
+      "loss": 2.2736213207244873,
+      "step": 173
+    },
+    {
+      "epoch": 0.0151639974890076,
+      "grad_norm": 1.4453125,
+      "learning_rate": 0.001196904916037292,
+      "loss": 2.1768054962158203,
+      "step": 174
+    },
+    {
+      "epoch": 0.015251146899863965,
+      "grad_norm": 1.6484375,
+      "learning_rate": 0.0012038176612166982,
+      "loss": 2.353447675704956,
+      "step": 175
+    },
+    {
+      "epoch": 0.015338296310720331,
+      "grad_norm": 1.515625,
+      "learning_rate": 0.0012107304063961046,
+      "loss": 2.247162342071533,
+      "step": 176
+    },
+    {
+      "epoch": 0.015425445721576695,
+      "grad_norm": 1.4375,
+      "learning_rate": 0.001217643151575511,
+      "loss": 2.1556575298309326,
+      "step": 177
+    },
+    {
+      "epoch": 0.015512595132433062,
+      "grad_norm": 1.5546875,
+      "learning_rate": 0.0012245558967549173,
+      "loss": 2.1065351963043213,
+      "step": 178
+    },
+    {
+      "epoch": 0.015599744543289428,
+      "grad_norm": 1.5,
+      "learning_rate": 0.0012314686419343235,
+      "loss": 2.1535067558288574,
+      "step": 179
+    },
+    {
+      "epoch": 0.015686893954145794,
+      "grad_norm": 1.5078125,
+      "learning_rate": 0.00123838138711373,
+      "loss": 2.193218231201172,
+      "step": 180
+    },
+    {
+      "epoch": 0.01577404336500216,
+      "grad_norm": 1.640625,
+      "learning_rate": 0.0012452941322931362,
+      "loss": 2.1786725521087646,
+      "step": 181
+    },
+    {
+      "epoch": 0.015861192775858523,
+      "grad_norm": 1.59375,
+      "learning_rate": 0.0012522068774725424,
+      "loss": 2.1510398387908936,
+      "step": 182
+    },
+    {
+      "epoch": 0.01594834218671489,
+      "grad_norm": 1.828125,
+      "learning_rate": 0.0012591196226519488,
+      "loss": 2.194058895111084,
+      "step": 183
+    },
+    {
+      "epoch": 0.016035491597571255,
+      "grad_norm": 1.4921875,
+      "learning_rate": 0.0012660323678313553,
+      "loss": 2.1457104682922363,
+      "step": 184
+    },
+    {
+      "epoch": 0.01612264100842762,
+      "grad_norm": 1.5703125,
+      "learning_rate": 0.0012729451130107615,
+      "loss": 2.1425299644470215,
+      "step": 185
+    },
+    {
+      "epoch": 0.016209790419283984,
+      "grad_norm": 1.375,
+      "learning_rate": 0.0012798578581901677,
+      "loss": 2.2450737953186035,
+      "step": 186
+    },
+    {
+      "epoch": 0.016296939830140352,
+      "grad_norm": 1.6015625,
+      "learning_rate": 0.0012867706033695742,
+      "loss": 2.1825942993164062,
+      "step": 187
+    },
+    {
+      "epoch": 0.016384089240996717,
+      "grad_norm": 1.375,
+      "learning_rate": 0.0012936833485489804,
+      "loss": 2.076286554336548,
+      "step": 188
+    },
+    {
+      "epoch": 0.01647123865185308,
+      "grad_norm": 1.46875,
+      "learning_rate": 0.0013005960937283866,
+      "loss": 2.1608879566192627,
+      "step": 189
+    },
+    {
+      "epoch": 0.01655838806270945,
+      "grad_norm": 1.4765625,
+      "learning_rate": 0.001307508838907793,
+      "loss": 2.1806795597076416,
+      "step": 190
+    },
+    {
+      "epoch": 0.016645537473565813,
+      "grad_norm": 1.5625,
+      "learning_rate": 0.0013144215840871993,
+      "loss": 2.1682558059692383,
+      "step": 191
+    },
+    {
+      "epoch": 0.016732686884422178,
+      "grad_norm": 1.640625,
+      "learning_rate": 0.0013213343292666055,
+      "loss": 2.202089548110962,
+      "step": 192
+    },
+    {
+      "epoch": 0.016819836295278546,
+      "grad_norm": 1.484375,
+      "learning_rate": 0.0013282470744460117,
+      "loss": 2.1461832523345947,
+      "step": 193
+    },
+    {
+      "epoch": 0.01690698570613491,
+      "grad_norm": 1.4140625,
+      "learning_rate": 0.0013351598196254181,
+      "loss": 1.9571164846420288,
+      "step": 194
+    },
+    {
+      "epoch": 0.016994135116991275,
+      "grad_norm": 1.5546875,
+      "learning_rate": 0.0013420725648048246,
+      "loss": 2.198993682861328,
+      "step": 195
+    },
+    {
+      "epoch": 0.017081284527847643,
+      "grad_norm": 1.390625,
+      "learning_rate": 0.0013489853099842308,
+      "loss": 2.0970957279205322,
+      "step": 196
+    },
+    {
+      "epoch": 0.017168433938704007,
+      "grad_norm": 1.5,
+      "learning_rate": 0.001355898055163637,
+      "loss": 2.1195521354675293,
+      "step": 197
+    },
+    {
+      "epoch": 0.01725558334956037,
+      "grad_norm": 1.3984375,
+      "learning_rate": 0.0013628108003430435,
+      "loss": 2.0975658893585205,
+      "step": 198
+    },
+    {
+      "epoch": 0.017342732760416736,
+      "grad_norm": 1.3984375,
+      "learning_rate": 0.0013697235455224497,
+      "loss": 2.2034478187561035,
+      "step": 199
+    },
+    {
+      "epoch": 0.017429882171273104,
+      "grad_norm": 1.3671875,
+      "learning_rate": 0.001376636290701856,
+      "loss": 2.172551155090332,
+      "step": 200
+    },
+    {
+      "epoch": 0.01751703158212947,
+      "grad_norm": 1.4375,
+      "learning_rate": 0.0013835490358812623,
+      "loss": 2.036912679672241,
+      "step": 201
+    },
+    {
+      "epoch": 0.017604180992985833,
+      "grad_norm": 1.4375,
+      "learning_rate": 0.0013904617810606688,
+      "loss": 2.1607420444488525,
+      "step": 202
+    },
+    {
+      "epoch": 0.0176913304038422,
+      "grad_norm": 1.4453125,
+      "learning_rate": 0.001397374526240075,
+      "loss": 2.0542709827423096,
+      "step": 203
+    },
+    {
+      "epoch": 0.017778479814698565,
+      "grad_norm": 1.375,
+      "learning_rate": 0.0014042872714194812,
+      "loss": 2.043264150619507,
+      "step": 204
+    },
+    {
+      "epoch": 0.01786562922555493,
+      "grad_norm": 1.4140625,
+      "learning_rate": 0.0014112000165988877,
+      "loss": 2.129647731781006,
+      "step": 205
+    },
+    {
+      "epoch": 0.017952778636411298,
+      "grad_norm": 1.390625,
+      "learning_rate": 0.001418112761778294,
+      "loss": 2.0812385082244873,
+      "step": 206
+    },
+    {
+      "epoch": 0.018039928047267662,
+      "grad_norm": 1.46875,
+      "learning_rate": 0.0014250255069577001,
+      "loss": 2.0312585830688477,
+      "step": 207
+    },
+    {
+      "epoch": 0.018127077458124027,
+      "grad_norm": 1.53125,
+      "learning_rate": 0.0014319382521371066,
+      "loss": 2.110504388809204,
+      "step": 208
+    },
+    {
+      "epoch": 0.01821422686898039,
+      "grad_norm": 1.5859375,
+      "learning_rate": 0.001438850997316513,
+      "loss": 2.1393139362335205,
+      "step": 209
+    },
+    {
+      "epoch": 0.01830137627983676,
+      "grad_norm": 1.4609375,
+      "learning_rate": 0.0014457637424959192,
+      "loss": 2.062541961669922,
+      "step": 210
+    },
+    {
+      "epoch": 0.018388525690693124,
+      "grad_norm": 1.5078125,
+      "learning_rate": 0.0014526764876753254,
+      "loss": 2.1736390590667725,
+      "step": 211
+    },
+    {
+      "epoch": 0.018475675101549488,
+      "grad_norm": 1.390625,
+      "learning_rate": 0.0014595892328547319,
+      "loss": 1.993078589439392,
+      "step": 212
+    },
+    {
+      "epoch": 0.018562824512405856,
+      "grad_norm": 1.6171875,
+      "learning_rate": 0.0014665019780341383,
+      "loss": 2.123861312866211,
+      "step": 213
+    },
+    {
+      "epoch": 0.01864997392326222,
+      "grad_norm": 1.5,
+      "learning_rate": 0.0014734147232135443,
+      "loss": 2.265376567840576,
+      "step": 214
+    },
+    {
+      "epoch": 0.018737123334118585,
+      "grad_norm": 1.4296875,
+      "learning_rate": 0.0014803274683929508,
+      "loss": 2.060065984725952,
+      "step": 215
+    },
+    {
+      "epoch": 0.018824272744974953,
+      "grad_norm": 1.4453125,
+      "learning_rate": 0.0014872402135723572,
+      "loss": 2.0445477962493896,
+      "step": 216
+    },
+    {
+      "epoch": 0.018911422155831317,
+      "grad_norm": 1.3828125,
+      "learning_rate": 0.0014941529587517634,
+      "loss": 2.004782199859619,
+      "step": 217
+    },
+    {
+      "epoch": 0.01899857156668768,
+      "grad_norm": 1.390625,
+      "learning_rate": 0.0015010657039311696,
+      "loss": 2.1203224658966064,
+      "step": 218
+    },
+    {
+      "epoch": 0.019085720977544046,
+      "grad_norm": 1.2890625,
+      "learning_rate": 0.001507978449110576,
+      "loss": 2.0167741775512695,
+      "step": 219
+    },
+    {
+      "epoch": 0.019172870388400414,
+      "grad_norm": 1.4140625,
+      "learning_rate": 0.0015148911942899823,
+      "loss": 1.9482148885726929,
+      "step": 220
+    },
+    {
+      "epoch": 0.01926001979925678,
+      "grad_norm": 1.859375,
+      "learning_rate": 0.0015218039394693885,
+      "loss": 2.1183648109436035,
+      "step": 221
+    },
+    {
+      "epoch": 0.019347169210113143,
+      "grad_norm": 1.4921875,
+      "learning_rate": 0.001528716684648795,
+      "loss": 2.272049903869629,
+      "step": 222
+    },
+    {
+      "epoch": 0.01943431862096951,
+      "grad_norm": 1.71875,
+      "learning_rate": 0.0015356294298282014,
+      "loss": 2.0988314151763916,
+      "step": 223
+    },
+    {
+      "epoch": 0.019521468031825875,
+      "grad_norm": 1.4375,
+      "learning_rate": 0.0015425421750076076,
+      "loss": 2.056494951248169,
+      "step": 224
+    },
+    {
+      "epoch": 0.01960861744268224,
+      "grad_norm": 1.453125,
+      "learning_rate": 0.0015494549201870138,
+      "loss": 2.1593847274780273,
+      "step": 225
+    },
+    {
+      "epoch": 0.019695766853538608,
+      "grad_norm": 1.40625,
+      "learning_rate": 0.0015563676653664203,
+      "loss": 2.065830707550049,
+      "step": 226
+    },
+    {
+      "epoch": 0.019782916264394972,
+      "grad_norm": 1.4140625,
+      "learning_rate": 0.0015632804105458265,
+      "loss": 2.0604701042175293,
+      "step": 227
+    },
+    {
+      "epoch": 0.019870065675251337,
+      "grad_norm": 1.515625,
+      "learning_rate": 0.0015701931557252327,
+      "loss": 2.2560975551605225,
+      "step": 228
+    },
+    {
+      "epoch": 0.019957215086107705,
+      "grad_norm": 1.328125,
+      "learning_rate": 0.0015771059009046392,
+      "loss": 2.0184080600738525,
+      "step": 229
+    },
+    {
+      "epoch": 0.02004436449696407,
+      "grad_norm": 1.4609375,
+      "learning_rate": 0.0015840186460840456,
+      "loss": 2.0684657096862793,
+      "step": 230
+    },
+    {
+      "epoch": 0.020131513907820434,
+      "grad_norm": 1.4375,
+      "learning_rate": 0.0015909313912634518,
+      "loss": 2.0670692920684814,
+      "step": 231
+    },
+    {
+      "epoch": 0.020218663318676798,
+      "grad_norm": 1.421875,
+      "learning_rate": 0.001597844136442858,
+      "loss": 2.03656268119812,
+      "step": 232
+    },
+    {
+      "epoch": 0.020305812729533166,
+      "grad_norm": 1.4296875,
+      "learning_rate": 0.0016047568816222645,
+      "loss": 2.1474218368530273,
+      "step": 233
+    },
+    {
+      "epoch": 0.02039296214038953,
+      "grad_norm": 1.3203125,
+      "learning_rate": 0.0016116696268016707,
+      "loss": 2.187220335006714,
+      "step": 234
+    },
+    {
+      "epoch": 0.020480111551245895,
+      "grad_norm": 1.34375,
+      "learning_rate": 0.0016185823719810767,
+      "loss": 2.1362173557281494,
+      "step": 235
+    },
+    {
+      "epoch": 0.020567260962102263,
+      "grad_norm": 1.3515625,
+      "learning_rate": 0.0016254951171604834,
+      "loss": 1.9493178129196167,
+      "step": 236
+    },
+    {
+      "epoch": 0.020654410372958627,
+      "grad_norm": 1.3828125,
+      "learning_rate": 0.0016324078623398898,
+      "loss": 2.105910301208496,
+      "step": 237
+    },
+    {
+      "epoch": 0.020741559783814992,
+      "grad_norm": 1.2578125,
+      "learning_rate": 0.001639320607519296,
+      "loss": 1.9420387744903564,
+      "step": 238
+    },
+    {
+      "epoch": 0.02082870919467136,
+      "grad_norm": 1.375,
+      "learning_rate": 0.0016462333526987023,
+      "loss": 2.0721983909606934,
+      "step": 239
+    },
+    {
+      "epoch": 0.020915858605527724,
+      "grad_norm": 1.3515625,
+      "learning_rate": 0.0016531460978781087,
+      "loss": 2.1015870571136475,
+      "step": 240
+    },
+    {
+      "epoch": 0.02100300801638409,
+      "grad_norm": 1.2265625,
+      "learning_rate": 0.001660058843057515,
+      "loss": 2.103015422821045,
+      "step": 241
+    },
+    {
+      "epoch": 0.021090157427240453,
+      "grad_norm": 1.4296875,
+      "learning_rate": 0.0016669715882369211,
+      "loss": 2.080415964126587,
+      "step": 242
+    },
+    {
+      "epoch": 0.02117730683809682,
+      "grad_norm": 1.359375,
+      "learning_rate": 0.0016738843334163278,
+      "loss": 2.0258567333221436,
+      "step": 243
+    },
+    {
+      "epoch": 0.021264456248953185,
+      "grad_norm": 1.4296875,
+      "learning_rate": 0.001680797078595734,
+      "loss": 2.1504578590393066,
+      "step": 244
+    },
+    {
+      "epoch": 0.02135160565980955,
+      "grad_norm": 1.3359375,
+      "learning_rate": 0.0016877098237751402,
+      "loss": 2.0242486000061035,
+      "step": 245
+    },
+    {
+      "epoch": 0.021438755070665918,
+      "grad_norm": 1.34375,
+      "learning_rate": 0.0016946225689545465,
+      "loss": 2.0960941314697266,
+      "step": 246
+    },
+    {
+      "epoch": 0.021525904481522282,
+      "grad_norm": 1.328125,
+      "learning_rate": 0.001701535314133953,
+      "loss": 2.0462875366210938,
+      "step": 247
+    },
+    {
+      "epoch": 0.021613053892378647,
+      "grad_norm": 1.234375,
+      "learning_rate": 0.0017084480593133591,
+      "loss": 1.8613609075546265,
+      "step": 248
+    },
+    {
+      "epoch": 0.021700203303235015,
+      "grad_norm": 1.234375,
+      "learning_rate": 0.0017153608044927653,
+      "loss": 2.0070531368255615,
+      "step": 249
+    },
+    {
+      "epoch": 0.02178735271409138,
+      "grad_norm": 1.328125,
+      "learning_rate": 0.001722273549672172,
+      "loss": 2.0557873249053955,
+      "step": 250
+    },
+    {
+      "epoch": 0.021874502124947744,
+      "grad_norm": 1.4140625,
+      "learning_rate": 0.0017291862948515782,
+      "loss": 2.0736894607543945,
+      "step": 251
+    },
+    {
+      "epoch": 0.02196165153580411,
+      "grad_norm": 1.359375,
+      "learning_rate": 0.0017360990400309844,
+      "loss": 2.0420143604278564,
+      "step": 252
+    },
+    {
+      "epoch": 0.022048800946660476,
+      "grad_norm": 1.328125,
+      "learning_rate": 0.0017430117852103907,
+      "loss": 2.047842264175415,
+      "step": 253
+    },
+    {
+      "epoch": 0.02213595035751684,
+      "grad_norm": 1.34375,
+      "learning_rate": 0.0017499245303897971,
+      "loss": 1.9989928007125854,
+      "step": 254
+    },
+    {
+      "epoch": 0.022223099768373205,
+      "grad_norm": 1.3203125,
+      "learning_rate": 0.0017568372755692033,
+      "loss": 1.9565446376800537,
+      "step": 255
+    },
+    {
+      "epoch": 0.022310249179229573,
+      "grad_norm": 1.2578125,
+      "learning_rate": 0.0017637500207486096,
+      "loss": 2.1068029403686523,
+      "step": 256
+    },
+    {
+      "epoch": 0.022397398590085937,
+      "grad_norm": 1.2734375,
+      "learning_rate": 0.001770662765928016,
+      "loss": 2.0168914794921875,
+      "step": 257
+    },
+    {
+      "epoch": 0.022484548000942302,
+      "grad_norm": 1.34375,
+      "learning_rate": 0.0017775755111074224,
+      "loss": 2.1034085750579834,
+      "step": 258
+    },
+    {
+      "epoch": 0.02257169741179867,
+      "grad_norm": 1.203125,
+      "learning_rate": 0.0017844882562868287,
+      "loss": 2.069204330444336,
+      "step": 259
+    },
+    {
+      "epoch": 0.022658846822655034,
+      "grad_norm": 1.328125,
+      "learning_rate": 0.0017914010014662349,
+      "loss": 2.0968642234802246,
+      "step": 260
+    },
+    {
+      "epoch": 0.0227459962335114,
+      "grad_norm": 1.234375,
+      "learning_rate": 0.0017983137466456413,
+      "loss": 1.967484712600708,
+      "step": 261
+    },
+    {
+      "epoch": 0.022833145644367767,
+      "grad_norm": 1.2265625,
+      "learning_rate": 0.0018052264918250475,
+      "loss": 2.0483593940734863,
+      "step": 262
+    },
+    {
+      "epoch": 0.02292029505522413,
+      "grad_norm": 1.1484375,
+      "learning_rate": 0.0018121392370044538,
+      "loss": 1.8896889686584473,
+      "step": 263
+    },
+    {
+      "epoch": 0.023007444466080496,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.0018190519821838602,
+      "loss": 2.069145917892456,
+      "step": 264
+    },
+    {
+      "epoch": 0.02309459387693686,
+      "grad_norm": 1.3984375,
+      "learning_rate": 0.0018259647273632666,
+      "loss": 2.109785318374634,
+      "step": 265
+    },
+    {
+      "epoch": 0.023181743287793228,
+      "grad_norm": 1.2578125,
+      "learning_rate": 0.0018328774725426729,
+      "loss": 2.097126007080078,
+      "step": 266
+    },
+    {
+      "epoch": 0.023268892698649592,
+      "grad_norm": 1.3671875,
+      "learning_rate": 0.001839790217722079,
+      "loss": 2.030325174331665,
+      "step": 267
+    },
+    {
+      "epoch": 0.023356042109505957,
+      "grad_norm": 1.390625,
+      "learning_rate": 0.0018467029629014855,
+      "loss": 2.169196128845215,
+      "step": 268
+    },
+    {
+      "epoch": 0.023443191520362325,
+      "grad_norm": 1.3671875,
+      "learning_rate": 0.0018536157080808917,
+      "loss": 2.1001839637756348,
+      "step": 269
+    },
+    {
+      "epoch": 0.02353034093121869,
+      "grad_norm": 1.359375,
+      "learning_rate": 0.001860528453260298,
+      "loss": 2.0785632133483887,
+      "step": 270
+    },
+    {
+      "epoch": 0.023617490342075054,
+      "grad_norm": 1.296875,
+      "learning_rate": 0.0018674411984397044,
+      "loss": 2.0410194396972656,
+      "step": 271
+    },
+    {
+      "epoch": 0.02370463975293142,
+      "grad_norm": 1.328125,
+      "learning_rate": 0.0018743539436191108,
+      "loss": 2.073333740234375,
+      "step": 272
+    },
+    {
+      "epoch": 0.023791789163787786,
+      "grad_norm": 1.328125,
+      "learning_rate": 0.001881266688798517,
+      "loss": 2.003096103668213,
+      "step": 273
+    },
+    {
+      "epoch": 0.02387893857464415,
+      "grad_norm": 1.2890625,
+      "learning_rate": 0.0018881794339779233,
+      "loss": 2.098187208175659,
+      "step": 274
+    },
+    {
+      "epoch": 0.02396608798550052,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.0018950921791573297,
+      "loss": 2.1531052589416504,
+      "step": 275
+    },
+    {
+      "epoch": 0.024053237396356883,
+      "grad_norm": 1.21875,
+      "learning_rate": 0.001902004924336736,
+      "loss": 1.9294949769973755,
+      "step": 276
+    },
+    {
+      "epoch": 0.024140386807213247,
+      "grad_norm": 1.3125,
+      "learning_rate": 0.0019089176695161422,
+      "loss": 2.0394935607910156,
+      "step": 277
+    },
+    {
+      "epoch": 0.024227536218069612,
+      "grad_norm": 1.265625,
+      "learning_rate": 0.0019158304146955486,
+      "loss": 2.1297898292541504,
+      "step": 278
+    },
+    {
+      "epoch": 0.02431468562892598,
+      "grad_norm": 1.3359375,
+      "learning_rate": 0.001922743159874955,
+      "loss": 2.0463480949401855,
+      "step": 279
+    },
+    {
+      "epoch": 0.024401835039782344,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.0019296559050543613,
+      "loss": 2.062194347381592,
+      "step": 280
+    },
+    {
+      "epoch": 0.02448898445063871,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.0019365686502337677,
+      "loss": 1.9813222885131836,
+      "step": 281
+    },
+    {
+      "epoch": 0.024576133861495077,
+      "grad_norm": 1.3125,
+      "learning_rate": 0.001943481395413174,
+      "loss": 1.9294824600219727,
+      "step": 282
+    },
+    {
+      "epoch": 0.02466328327235144,
+      "grad_norm": 1.265625,
+      "learning_rate": 0.0019503941405925802,
+      "loss": 2.099668502807617,
+      "step": 283
+    },
+    {
+      "epoch": 0.024750432683207806,
+      "grad_norm": 1.28125,
+      "learning_rate": 0.001957306885771987,
+      "loss": 1.9695556163787842,
+      "step": 284
+    },
+    {
+      "epoch": 0.024837582094064174,
+      "grad_norm": 1.3125,
+      "learning_rate": 0.0019642196309513933,
+      "loss": 2.177349328994751,
+      "step": 285
+    },
+    {
+      "epoch": 0.024924731504920538,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.0019711323761307993,
+      "loss": 2.044966697692871,
+      "step": 286
+    },
+    {
+      "epoch": 0.025011880915776902,
+      "grad_norm": 1.2734375,
+      "learning_rate": 0.0019780451213102057,
+      "loss": 2.0162243843078613,
+      "step": 287
+    },
+    {
+      "epoch": 0.025099030326633267,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.001984957866489612,
+      "loss": 2.0284998416900635,
+      "step": 288
+    },
+    {
+      "epoch": 0.025186179737489635,
+      "grad_norm": 1.1875,
+      "learning_rate": 0.001991870611669018,
+      "loss": 1.9222617149353027,
+      "step": 289
+    },
+    {
+      "epoch": 0.025273329148346,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.0019987833568484246,
+      "loss": 2.029639482498169,
+      "step": 290
+    },
+    {
+      "epoch": 0.025360478559202364,
+      "grad_norm": 1.1484375,
+      "learning_rate": 0.002,
+      "loss": 2.001166582107544,
+      "step": 291
+    },
+    {
+      "epoch": 0.02544762797005873,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.002,
+      "loss": 2.0644993782043457,
+      "step": 292
+    },
+    {
+      "epoch": 0.025534777380915096,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.002,
+      "loss": 2.013618230819702,
+      "step": 293
+    },
+    {
+      "epoch": 0.02562192679177146,
+      "grad_norm": 1.234375,
+      "learning_rate": 0.002,
+      "loss": 1.9548147916793823,
+      "step": 294
+    },
+    {
+      "epoch": 0.02570907620262783,
+      "grad_norm": 1.28125,
+      "learning_rate": 0.002,
+      "loss": 2.0976831912994385,
+      "step": 295
+    },
+    {
+      "epoch": 0.025796225613484193,
+      "grad_norm": 1.28125,
+      "learning_rate": 0.002,
+      "loss": 2.043471574783325,
+      "step": 296
+    },
+    {
+      "epoch": 0.025883375024340557,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.002,
+      "loss": 2.0143167972564697,
+      "step": 297
+    },
+    {
+      "epoch": 0.025970524435196922,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.002,
+      "loss": 1.9477717876434326,
+      "step": 298
+    },
+    {
+      "epoch": 0.02605767384605329,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.002,
+      "loss": 2.046755313873291,
+      "step": 299
+    },
+    {
+      "epoch": 0.026144823256909654,
+      "grad_norm": 1.1640625,
+      "learning_rate": 0.002,
+      "loss": 2.00517201423645,
+      "step": 300
+    },
+    {
+      "epoch": 0.02623197266776602,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.002,
+      "loss": 1.8747062683105469,
+      "step": 301
+    },
+    {
+      "epoch": 0.026319122078622387,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.002,
+      "loss": 1.9339944124221802,
+      "step": 302
+    },
+    {
+      "epoch": 0.02640627148947875,
+      "grad_norm": 1.2109375,
+      "learning_rate": 0.002,
+      "loss": 2.032811403274536,
+      "step": 303
+    },
+    {
+      "epoch": 0.026493420900335116,
+      "grad_norm": 1.21875,
+      "learning_rate": 0.002,
+      "loss": 2.098724126815796,
+      "step": 304
+    },
+    {
+      "epoch": 0.026580570311191484,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.002,
+      "loss": 1.916012167930603,
+      "step": 305
+    },
+    {
+      "epoch": 0.026667719722047848,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.002,
+      "loss": 2.1294262409210205,
+      "step": 306
+    },
+    {
+      "epoch": 0.026754869132904213,
+      "grad_norm": 1.2265625,
+      "learning_rate": 0.002,
+      "loss": 2.059067487716675,
+      "step": 307
+    },
+    {
+      "epoch": 0.02684201854376058,
+      "grad_norm": 1.1484375,
+      "learning_rate": 0.002,
+      "loss": 2.064976692199707,
+      "step": 308
+    },
+    {
+      "epoch": 0.026929167954616945,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.002,
+      "loss": 1.9300205707550049,
+      "step": 309
+    },
+    {
+      "epoch": 0.02701631736547331,
+      "grad_norm": 1.21875,
+      "learning_rate": 0.002,
+      "loss": 1.9407846927642822,
+      "step": 310
+    },
+    {
+      "epoch": 0.027103466776329674,
+      "grad_norm": 1.21875,
+      "learning_rate": 0.002,
+      "loss": 2.019566059112549,
+      "step": 311
+    },
+    {
+      "epoch": 0.027190616187186042,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.002,
+      "loss": 2.1206910610198975,
+      "step": 312
+    },
+    {
+      "epoch": 0.027277765598042406,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.002,
+      "loss": 2.05499005317688,
+      "step": 313
+    },
+    {
+      "epoch": 0.02736491500889877,
+      "grad_norm": 1.2109375,
+      "learning_rate": 0.002,
+      "loss": 2.132664680480957,
+      "step": 314
+    },
+    {
+      "epoch": 0.02745206441975514,
+      "grad_norm": 1.2578125,
+      "learning_rate": 0.002,
+      "loss": 2.0348358154296875,
+      "step": 315
+    },
+    {
+      "epoch": 0.027539213830611503,
+      "grad_norm": 1.265625,
+      "learning_rate": 0.002,
+      "loss": 1.94222092628479,
+      "step": 316
+    },
+    {
+      "epoch": 0.027626363241467868,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.002,
+      "loss": 1.9610005617141724,
+      "step": 317
+    },
+    {
+      "epoch": 0.027713512652324235,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.002,
+      "loss": 2.1766140460968018,
+      "step": 318
+    },
+    {
+      "epoch": 0.0278006620631806,
+      "grad_norm": 1.2734375,
+      "learning_rate": 0.002,
+      "loss": 2.109349489212036,
+      "step": 319
+    },
+    {
+      "epoch": 0.027887811474036964,
+      "grad_norm": 1.125,
+      "learning_rate": 0.002,
+      "loss": 2.074075222015381,
+      "step": 320
+    },
+    {
+      "epoch": 0.02797496088489333,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.002,
+      "loss": 1.8542454242706299,
+      "step": 321
+    },
+    {
+      "epoch": 0.028062110295749697,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.002,
+      "loss": 2.072103500366211,
+      "step": 322
+    },
+    {
+      "epoch": 0.02814925970660606,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.002,
+      "loss": 1.8817863464355469,
+      "step": 323
+    },
+    {
+      "epoch": 0.028236409117462426,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.002,
+      "loss": 2.1079821586608887,
+      "step": 324
+    },
+    {
+      "epoch": 0.028323558528318794,
+      "grad_norm": 1.2109375,
+      "learning_rate": 0.002,
+      "loss": 2.1467366218566895,
+      "step": 325
+    },
+    {
+      "epoch": 0.028410707939175158,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.002,
+      "loss": 2.0277698040008545,
+      "step": 326
+    },
+    {
+      "epoch": 0.028497857350031523,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.002,
+      "loss": 1.950154185295105,
+      "step": 327
+    },
+    {
+      "epoch": 0.02858500676088789,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.002,
+      "loss": 2.015425443649292,
+      "step": 328
+    },
+    {
+      "epoch": 0.028672156171744255,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.002,
+      "loss": 1.872285008430481,
+      "step": 329
+    },
+    {
+      "epoch": 0.02875930558260062,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.002,
+      "loss": 2.0059897899627686,
+      "step": 330
+    },
+    {
+      "epoch": 0.028846454993456987,
+      "grad_norm": 1.1171875,
+      "learning_rate": 0.002,
+      "loss": 1.9773547649383545,
+      "step": 331
+    },
+    {
+      "epoch": 0.028933604404313352,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.002,
+      "loss": 2.0682241916656494,
+      "step": 332
+    },
+    {
+      "epoch": 0.029020753815169716,
+      "grad_norm": 1.0390625,
+      "learning_rate": 0.002,
+      "loss": 1.9896056652069092,
+      "step": 333
+    },
+    {
+      "epoch": 0.02910790322602608,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.002,
+      "loss": 2.179992437362671,
+      "step": 334
+    },
+    {
+      "epoch": 0.02919505263688245,
+      "grad_norm": 1.125,
+      "learning_rate": 0.002,
+      "loss": 1.9344980716705322,
+      "step": 335
+    },
+    {
+      "epoch": 0.029282202047738813,
+      "grad_norm": 0.98828125,
+      "learning_rate": 0.002,
+      "loss": 1.9128037691116333,
+      "step": 336
+    },
+    {
+      "epoch": 0.029369351458595178,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.002,
+      "loss": 1.9457592964172363,
+      "step": 337
+    },
+    {
+      "epoch": 0.029456500869451546,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.002,
+      "loss": 2.0802388191223145,
+      "step": 338
+    },
+    {
+      "epoch": 0.02954365028030791,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.002,
+      "loss": 1.976395606994629,
+      "step": 339
+    },
+    {
+      "epoch": 0.029630799691164274,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.002,
+      "loss": 2.0161664485931396,
+      "step": 340
+    },
+    {
+      "epoch": 0.029717949102020642,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.002,
+      "loss": 1.996329426765442,
+      "step": 341
+    },
+    {
+      "epoch": 0.029805098512877007,
+      "grad_norm": 1.375,
+      "learning_rate": 0.002,
+      "loss": 1.9847618341445923,
+      "step": 342
+    },
+    {
+      "epoch": 0.02989224792373337,
+      "grad_norm": 1.1171875,
+      "learning_rate": 0.002,
+      "loss": 2.025886297225952,
+      "step": 343
+    },
+    {
+      "epoch": 0.029979397334589736,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.002,
+      "loss": 2.0314159393310547,
+      "step": 344
+    },
+    {
+      "epoch": 0.030066546745446104,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.002,
+      "loss": 1.9757808446884155,
+      "step": 345
+    },
+    {
+      "epoch": 0.030153696156302468,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.002,
+      "loss": 2.00164794921875,
+      "step": 346
+    },
+    {
+      "epoch": 0.030240845567158833,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.002,
+      "loss": 1.920770525932312,
+      "step": 347
+    },
+    {
+      "epoch": 0.0303279949780152,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.002,
+      "loss": 2.0524423122406006,
+      "step": 348
+    },
+    {
+      "epoch": 0.030415144388871565,
+      "grad_norm": 1.53125,
+      "learning_rate": 0.002,
+      "loss": 2.0547049045562744,
+      "step": 349
+    },
+    {
+      "epoch": 0.03050229379972793,
+      "grad_norm": 0.98828125,
+      "learning_rate": 0.002,
+      "loss": 1.9880136251449585,
+      "step": 350
+    },
+    {
+      "epoch": 0.030589443210584297,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.002,
+      "loss": 1.987094759941101,
+      "step": 351
+    },
+    {
+      "epoch": 0.030676592621440662,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.002,
+      "loss": 1.9624769687652588,
+      "step": 352
+    },
+    {
+      "epoch": 0.030763742032297026,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.002,
+      "loss": 2.0789482593536377,
+      "step": 353
+    },
+    {
+      "epoch": 0.03085089144315339,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.002,
+      "loss": 2.166853666305542,
+      "step": 354
+    },
+    {
+      "epoch": 0.03093804085400976,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.002,
+      "loss": 2.0598955154418945,
+      "step": 355
+    },
+    {
+      "epoch": 0.031025190264866123,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.002,
+      "loss": 1.9756646156311035,
+      "step": 356
+    },
+    {
+      "epoch": 0.031112339675722488,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.002,
+      "loss": 1.9314295053482056,
+      "step": 357
+    },
+    {
+      "epoch": 0.031199489086578856,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.002,
+      "loss": 1.911449670791626,
+      "step": 358
+    },
+    {
+      "epoch": 0.031286638497435224,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.002,
+      "loss": 2.096069812774658,
+      "step": 359
+    },
+    {
+      "epoch": 0.03137378790829159,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.002,
+      "loss": 1.9302388429641724,
+      "step": 360
+    },
+    {
+      "epoch": 0.03146093731914795,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.002,
+      "loss": 1.9593164920806885,
+      "step": 361
+    },
+    {
+      "epoch": 0.03154808673000432,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.002,
+      "loss": 1.9172489643096924,
+      "step": 362
+    },
+    {
+      "epoch": 0.03163523614086068,
+      "grad_norm": 2.375,
+      "learning_rate": 0.002,
+      "loss": 2.0493578910827637,
+      "step": 363
+    },
+    {
+      "epoch": 0.031722385551717046,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.002,
+      "loss": 1.9867511987686157,
+      "step": 364
+    },
+    {
+      "epoch": 0.03180953496257341,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.002,
+      "loss": 2.0832464694976807,
+      "step": 365
+    },
+    {
+      "epoch": 0.03189668437342978,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.002,
+      "loss": 1.886700987815857,
+      "step": 366
+    },
+    {
+      "epoch": 0.031983833784286146,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.002,
+      "loss": 1.9116026163101196,
+      "step": 367
+    },
+    {
+      "epoch": 0.03207098319514251,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.002,
+      "loss": 1.8263874053955078,
+      "step": 368
+    },
+    {
+      "epoch": 0.032158132605998875,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.002,
+      "loss": 2.0412628650665283,
+      "step": 369
+    },
+    {
+      "epoch": 0.03224528201685524,
+      "grad_norm": 1.125,
+      "learning_rate": 0.002,
+      "loss": 2.1048011779785156,
+      "step": 370
+    },
+    {
+      "epoch": 0.032332431427711604,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.002,
+      "loss": 2.0405256748199463,
+      "step": 371
+    },
+    {
+      "epoch": 0.03241958083856797,
+      "grad_norm": 1.0,
+      "learning_rate": 0.002,
+      "loss": 1.9497419595718384,
+      "step": 372
+    },
+    {
+      "epoch": 0.03250673024942434,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.002,
+      "loss": 2.084228515625,
+      "step": 373
+    },
+    {
+      "epoch": 0.032593879660280704,
+      "grad_norm": 1.0,
+      "learning_rate": 0.002,
+      "loss": 1.9401715993881226,
+      "step": 374
+    },
+    {
+      "epoch": 0.03268102907113707,
+      "grad_norm": 1.296875,
+      "learning_rate": 0.002,
+      "loss": 2.0952723026275635,
+      "step": 375
+    },
+    {
+      "epoch": 0.03276817848199343,
+      "grad_norm": 0.921875,
+      "learning_rate": 0.002,
+      "loss": 1.8625407218933105,
+      "step": 376
+    },
+    {
+      "epoch": 0.0328553278928498,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.002,
+      "loss": 2.0086042881011963,
+      "step": 377
+    },
+    {
+      "epoch": 0.03294247730370616,
+      "grad_norm": 0.97265625,
+      "learning_rate": 0.002,
+      "loss": 1.9601489305496216,
+      "step": 378
+    },
+    {
+      "epoch": 0.033029626714562534,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.002,
+      "loss": 2.102181911468506,
+      "step": 379
+    },
+    {
+      "epoch": 0.0331167761254189,
+      "grad_norm": 0.91015625,
+      "learning_rate": 0.002,
+      "loss": 1.9650415182113647,
+      "step": 380
+    },
+    {
+      "epoch": 0.03320392553627526,
+      "grad_norm": 1.0390625,
+      "learning_rate": 0.002,
+      "loss": 1.9646992683410645,
+      "step": 381
+    },
+    {
+      "epoch": 0.03329107494713163,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.002,
+      "loss": 1.9591023921966553,
+      "step": 382
+    },
+    {
+      "epoch": 0.03337822435798799,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.002,
+      "loss": 2.010939359664917,
+      "step": 383
+    },
+    {
+      "epoch": 0.033465373768844356,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.002,
+      "loss": 2.059732675552368,
+      "step": 384
+    },
+    {
+      "epoch": 0.03355252317970072,
+      "grad_norm": 0.98046875,
+      "learning_rate": 0.002,
+      "loss": 2.010468006134033,
+      "step": 385
+    },
+    {
+      "epoch": 0.03363967259055709,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.002,
+      "loss": 2.05426287651062,
+      "step": 386
+    },
+    {
+      "epoch": 0.033726822001413456,
+      "grad_norm": 1.0,
+      "learning_rate": 0.002,
+      "loss": 2.0237786769866943,
+      "step": 387
+    },
+    {
+      "epoch": 0.03381397141226982,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.002,
+      "loss": 1.9790992736816406,
+      "step": 388
+    },
+    {
+      "epoch": 0.033901120823126185,
+      "grad_norm": 0.98828125,
+      "learning_rate": 0.002,
+      "loss": 1.950234293937683,
+      "step": 389
+    },
+    {
+      "epoch": 0.03398827023398255,
+      "grad_norm": 0.953125,
+      "learning_rate": 0.002,
+      "loss": 1.907360315322876,
+      "step": 390
+    },
+    {
+      "epoch": 0.034075419644838914,
+      "grad_norm": 0.91796875,
+      "learning_rate": 0.002,
+      "loss": 1.8177075386047363,
+      "step": 391
+    },
+    {
+      "epoch": 0.034162569055695285,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.002,
+      "loss": 2.075497627258301,
+      "step": 392
+    },
+    {
+      "epoch": 0.03424971846655165,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.002,
+      "loss": 2.132970094680786,
+      "step": 393
+    },
+    {
+      "epoch": 0.034336867877408014,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.002,
+      "loss": 1.8559272289276123,
+      "step": 394
+    },
+    {
+      "epoch": 0.03442401728826438,
+      "grad_norm": 1.1640625,
+      "learning_rate": 0.002,
+      "loss": 2.099686861038208,
+      "step": 395
+    },
+    {
+      "epoch": 0.03451116669912074,
+      "grad_norm": 0.95703125,
+      "learning_rate": 0.002,
+      "loss": 1.87687087059021,
+      "step": 396
+    },
+    {
+      "epoch": 0.03459831610997711,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.002,
+      "loss": 1.945698618888855,
+      "step": 397
+    },
+    {
+      "epoch": 0.03468546552083347,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.002,
+      "loss": 1.9908119440078735,
+      "step": 398
+    },
+    {
+      "epoch": 0.034772614931689844,
+      "grad_norm": 1.0,
+      "learning_rate": 0.002,
+      "loss": 2.079252004623413,
+      "step": 399
+    },
+    {
+      "epoch": 0.03485976434254621,
+      "grad_norm": 0.92578125,
+      "learning_rate": 0.002,
+      "loss": 1.9281474351882935,
+      "step": 400
+    },
+    {
+      "epoch": 0.03494691375340257,
+      "grad_norm": 0.94140625,
+      "learning_rate": 0.002,
+      "loss": 1.8578988313674927,
+      "step": 401
+    },
+    {
+      "epoch": 0.03503406316425894,
+      "grad_norm": 0.94921875,
+      "learning_rate": 0.002,
+      "loss": 1.8781170845031738,
+      "step": 402
+    },
+    {
+      "epoch": 0.0351212125751153,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.002,
+      "loss": 1.936561107635498,
+      "step": 403
+    },
+    {
+      "epoch": 0.035208361985971666,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.002,
+      "loss": 2.086552381515503,
+      "step": 404
+    },
+    {
+      "epoch": 0.03529551139682803,
+      "grad_norm": 0.96484375,
+      "learning_rate": 0.002,
+      "loss": 1.947652816772461,
+      "step": 405
+    },
+    {
+      "epoch": 0.0353826608076844,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.002,
+      "loss": 2.0454230308532715,
+      "step": 406
+    },
+    {
+      "epoch": 0.035469810218540766,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.002,
+      "loss": 1.8672873973846436,
+      "step": 407
+    },
+    {
+      "epoch": 0.03555695962939713,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.002,
+      "loss": 2.1776187419891357,
+      "step": 408
+    },
+    {
+      "epoch": 0.035644109040253495,
+      "grad_norm": 0.92578125,
+      "learning_rate": 0.002,
+      "loss": 1.873447060585022,
+      "step": 409
+    },
+    {
+      "epoch": 0.03573125845110986,
+      "grad_norm": 0.91796875,
+      "learning_rate": 0.002,
+      "loss": 2.0407629013061523,
+      "step": 410
+    },
+    {
+      "epoch": 0.035818407861966224,
+      "grad_norm": 0.890625,
+      "learning_rate": 0.002,
+      "loss": 2.0063130855560303,
+      "step": 411
+    },
+    {
+      "epoch": 0.035905557272822596,
+      "grad_norm": 0.98046875,
+      "learning_rate": 0.002,
+      "loss": 1.9252933263778687,
+      "step": 412
+    },
+    {
+      "epoch": 0.03599270668367896,
+      "grad_norm": 0.90234375,
+      "learning_rate": 0.002,
+      "loss": 2.0427045822143555,
+      "step": 413
+    },
+    {
+      "epoch": 0.036079856094535324,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.002,
+      "loss": 1.8753350973129272,
+      "step": 414
+    },
+    {
+      "epoch": 0.03616700550539169,
+      "grad_norm": 0.97265625,
+      "learning_rate": 0.002,
+      "loss": 1.813300609588623,
+      "step": 415
+    },
+    {
+      "epoch": 0.03625415491624805,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.002,
+      "loss": 1.9801394939422607,
+      "step": 416
+    },
+    {
+      "epoch": 0.03634130432710442,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.002,
+      "loss": 1.8856185674667358,
+      "step": 417
+    },
+    {
+      "epoch": 0.03642845373796078,
+      "grad_norm": 0.94140625,
+      "learning_rate": 0.002,
+      "loss": 1.8563461303710938,
+      "step": 418
+    },
+    {
+      "epoch": 0.036515603148817154,
+      "grad_norm": 0.9375,
+      "learning_rate": 0.002,
+      "loss": 1.8451026678085327,
+      "step": 419
+    },
+    {
+      "epoch": 0.03660275255967352,
+      "grad_norm": 0.85546875,
+      "learning_rate": 0.002,
+      "loss": 1.9298255443572998,
+      "step": 420
+    },
+    {
+      "epoch": 0.03668990197052988,
+      "grad_norm": 0.953125,
+      "learning_rate": 0.002,
+      "loss": 2.0476126670837402,
+      "step": 421
+    },
+    {
+      "epoch": 0.03677705138138625,
+      "grad_norm": 0.90234375,
+      "learning_rate": 0.002,
+      "loss": 1.8805210590362549,
+      "step": 422
+    },
+    {
+      "epoch": 0.03686420079224261,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.002,
+      "loss": 2.0011842250823975,
+      "step": 423
+    },
+    {
+      "epoch": 0.036951350203098976,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.002,
+      "loss": 1.971643328666687,
+      "step": 424
+    },
+    {
+      "epoch": 0.03703849961395535,
+      "grad_norm": 0.96484375,
+      "learning_rate": 0.002,
+      "loss": 1.9296948909759521,
+      "step": 425
+    },
+    {
+      "epoch": 0.03712564902481171,
+      "grad_norm": 0.953125,
+      "learning_rate": 0.002,
+      "loss": 1.8698842525482178,
+      "step": 426
+    },
+    {
+      "epoch": 0.037212798435668076,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.002,
+      "loss": 2.0018582344055176,
+      "step": 427
+    },
+    {
+      "epoch": 0.03729994784652444,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.002,
+      "loss": 1.9673770666122437,
+      "step": 428
+    },
+    {
+      "epoch": 0.037387097257380805,
+      "grad_norm": 0.94921875,
+      "learning_rate": 0.002,
+      "loss": 1.8905961513519287,
+      "step": 429
+    },
+    {
+      "epoch": 0.03747424666823717,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.002,
+      "loss": 2.1279525756835938,
+      "step": 430
+    },
+    {
+      "epoch": 0.037561396079093534,
+      "grad_norm": 0.98046875,
+      "learning_rate": 0.002,
+      "loss": 1.9549392461776733,
+      "step": 431
+    },
+    {
+      "epoch": 0.037648545489949906,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.002,
+      "loss": 2.018629789352417,
+      "step": 432
+    },
+    {
+      "epoch": 0.03773569490080627,
+      "grad_norm": 0.95703125,
+      "learning_rate": 0.002,
+      "loss": 2.0507311820983887,
+      "step": 433
+    },
+    {
+      "epoch": 0.037822844311662634,
+      "grad_norm": 0.96875,
+      "learning_rate": 0.002,
+      "loss": 1.8507345914840698,
+      "step": 434
+    },
+    {
+      "epoch": 0.037909993722519,
+      "grad_norm": 0.87109375,
+      "learning_rate": 0.002,
+      "loss": 2.018371105194092,
+      "step": 435
+    },
+    {
+      "epoch": 0.03799714313337536,
+      "grad_norm": 0.875,
+      "learning_rate": 0.002,
+      "loss": 1.9152356386184692,
+      "step": 436
+    },
+    {
+      "epoch": 0.03808429254423173,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.002,
+      "loss": 2.01894474029541,
+      "step": 437
+    },
+    {
+      "epoch": 0.03817144195508809,
+      "grad_norm": 1.0,
+      "learning_rate": 0.002,
+      "loss": 1.9083776473999023,
+      "step": 438
+    },
+    {
+      "epoch": 0.038258591365944464,
+      "grad_norm": 0.9921875,
+      "learning_rate": 0.002,
+      "loss": 1.8658249378204346,
+      "step": 439
+    },
+    {
+      "epoch": 0.03834574077680083,
+      "grad_norm": 0.96875,
+      "learning_rate": 0.002,
+      "loss": 1.910384178161621,
+      "step": 440
+    },
+    {
+      "epoch": 0.03843289018765719,
+      "grad_norm": 0.92578125,
+      "learning_rate": 0.002,
+      "loss": 1.9150588512420654,
+      "step": 441
+    },
+    {
+      "epoch": 0.03852003959851356,
+      "grad_norm": 0.96484375,
+      "learning_rate": 0.002,
+      "loss": 1.8923720121383667,
+      "step": 442
+    },
+    {
+      "epoch": 0.03860718900936992,
+      "grad_norm": 0.9453125,
+      "learning_rate": 0.002,
+      "loss": 2.0249733924865723,
+      "step": 443
+    },
+    {
+      "epoch": 0.038694338420226286,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.002,
+      "loss": 1.897134780883789,
+      "step": 444
+    },
+    {
+      "epoch": 0.03878148783108266,
+      "grad_norm": 0.984375,
+      "learning_rate": 0.002,
+      "loss": 1.9412795305252075,
+      "step": 445
+    },
+    {
+      "epoch": 0.03886863724193902,
+      "grad_norm": 0.91796875,
+      "learning_rate": 0.002,
+      "loss": 1.893125295639038,
+      "step": 446
+    },
+    {
+      "epoch": 0.038955786652795386,
+      "grad_norm": 0.94140625,
+      "learning_rate": 0.002,
+      "loss": 1.9204654693603516,
+      "step": 447
+    },
+    {
+      "epoch": 0.03904293606365175,
+      "grad_norm": 0.9609375,
+      "learning_rate": 0.002,
+      "loss": 1.9805623292922974,
+      "step": 448
+    },
+    {
+      "epoch": 0.039130085474508115,
+      "grad_norm": 0.9375,
+      "learning_rate": 0.002,
+      "loss": 1.8250699043273926,
+      "step": 449
+    },
+    {
+      "epoch": 0.03921723488536448,
+      "grad_norm": 0.92578125,
+      "learning_rate": 0.002,
+      "loss": 1.8739783763885498,
+      "step": 450
+    },
+    {
+      "epoch": 0.039304384296220844,
+      "grad_norm": 0.88671875,
+      "learning_rate": 0.002,
+      "loss": 1.8047534227371216,
+      "step": 451
+    },
+    {
+      "epoch": 0.039391533707077216,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.002,
+      "loss": 1.983681321144104,
+      "step": 452
+    },
+    {
+      "epoch": 0.03947868311793358,
+      "grad_norm": 0.93359375,
+      "learning_rate": 0.002,
+      "loss": 2.0652642250061035,
+      "step": 453
+    },
+    {
+      "epoch": 0.039565832528789945,
+      "grad_norm": 0.99609375,
+      "learning_rate": 0.002,
+      "loss": 1.9812188148498535,
+      "step": 454
+    },
+    {
+      "epoch": 0.03965298193964631,
+      "grad_norm": 0.96875,
+      "learning_rate": 0.002,
+      "loss": 1.8507156372070312,
+      "step": 455
+    },
+    {
+      "epoch": 0.039740131350502673,
+      "grad_norm": 0.9921875,
+      "learning_rate": 0.002,
+      "loss": 2.0216448307037354,
+      "step": 456
+    },
+    {
+      "epoch": 0.03982728076135904,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.002,
+      "loss": 1.928618311882019,
+      "step": 457
+    },
+    {
+      "epoch": 0.03991443017221541,
+      "grad_norm": 0.93359375,
+      "learning_rate": 0.002,
+      "loss": 1.8563501834869385,
+      "step": 458
+    },
+    {
+      "epoch": 0.040001579583071774,
+      "grad_norm": 0.9765625,
+      "learning_rate": 0.002,
+      "loss": 2.0219364166259766,
+      "step": 459
+    },
+    {
+      "epoch": 0.04008872899392814,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.002,
+      "loss": 1.9423730373382568,
+      "step": 460
+    },
+    {
+      "epoch": 0.0401758784047845,
+      "grad_norm": 0.87109375,
+      "learning_rate": 0.002,
+      "loss": 1.7398076057434082,
+      "step": 461
+    },
+    {
+      "epoch": 0.04026302781564087,
+      "grad_norm": 0.875,
+      "learning_rate": 0.002,
+      "loss": 1.946607232093811,
+      "step": 462
+    },
+    {
+      "epoch": 0.04035017722649723,
+      "grad_norm": 0.8359375,
+      "learning_rate": 0.002,
+      "loss": 1.8465126752853394,
+      "step": 463
+    },
+    {
+      "epoch": 0.040437326637353596,
+      "grad_norm": 0.91015625,
+      "learning_rate": 0.002,
+      "loss": 1.9049490690231323,
+      "step": 464
+    },
+    {
+      "epoch": 0.04052447604820997,
+      "grad_norm": 0.87109375,
+      "learning_rate": 0.002,
+      "loss": 1.9343353509902954,
+      "step": 465
+    },
+    {
+      "epoch": 0.04061162545906633,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.002,
+      "loss": 1.88978910446167,
+      "step": 466
+    },
+    {
+      "epoch": 0.040698774869922696,
+      "grad_norm": 0.88671875,
+      "learning_rate": 0.002,
+      "loss": 1.9816972017288208,
+      "step": 467
+    },
+    {
+      "epoch": 0.04078592428077906,
+      "grad_norm": 0.9765625,
+      "learning_rate": 0.002,
+      "loss": 1.866693377494812,
+      "step": 468
+    },
+    {
+      "epoch": 0.040873073691635425,
+      "grad_norm": 0.875,
+      "learning_rate": 0.002,
+      "loss": 1.8631305694580078,
+      "step": 469
+    },
+    {
+      "epoch": 0.04096022310249179,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.002,
+      "loss": 1.9814085960388184,
+      "step": 470
+    },
+    {
+      "epoch": 0.04104737251334816,
+      "grad_norm": 0.9296875,
+      "learning_rate": 0.002,
+      "loss": 1.952948808670044,
+      "step": 471
+    },
+    {
+      "epoch": 0.041134521924204526,
+      "grad_norm": 0.93359375,
+      "learning_rate": 0.002,
+      "loss": 1.7778507471084595,
+      "step": 472
+    },
+    {
+      "epoch": 0.04122167133506089,
+      "grad_norm": 0.87109375,
+      "learning_rate": 0.002,
+      "loss": 1.9396259784698486,
+      "step": 473
+    },
+    {
+      "epoch": 0.041308820745917255,
+      "grad_norm": 0.87890625,
+      "learning_rate": 0.002,
+      "loss": 1.774814486503601,
+      "step": 474
+    },
+    {
+      "epoch": 0.04139597015677362,
+      "grad_norm": 0.9453125,
+      "learning_rate": 0.002,
+      "loss": 1.9689006805419922,
+      "step": 475
+    },
+    {
+      "epoch": 0.041483119567629984,
+      "grad_norm": 0.98046875,
+      "learning_rate": 0.002,
+      "loss": 1.9724104404449463,
+      "step": 476
+    },
+    {
+      "epoch": 0.04157026897848635,
+      "grad_norm": 0.95703125,
+      "learning_rate": 0.002,
+      "loss": 1.9812343120574951,
+      "step": 477
+    },
+    {
+      "epoch": 0.04165741838934272,
+      "grad_norm": 1.5546875,
+      "learning_rate": 0.002,
+      "loss": 2.028676986694336,
+      "step": 478
+    },
+    {
+      "epoch": 0.041744567800199084,
+      "grad_norm": 0.984375,
+      "learning_rate": 0.002,
+      "loss": 1.8658119440078735,
+      "step": 479
+    },
+    {
+      "epoch": 0.04183171721105545,
+      "grad_norm": 0.9765625,
+      "learning_rate": 0.002,
+      "loss": 1.9101450443267822,
+      "step": 480
+    },
+    {
+      "epoch": 0.04191886662191181,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.002,
+      "loss": 2.0230343341827393,
+      "step": 481
+    },
+    {
+      "epoch": 0.04200601603276818,
+      "grad_norm": 0.890625,
+      "learning_rate": 0.002,
+      "loss": 1.901646375656128,
+      "step": 482
+    },
+    {
+      "epoch": 0.04209316544362454,
+      "grad_norm": 0.96875,
+      "learning_rate": 0.002,
+      "loss": 1.91860032081604,
+      "step": 483
+    },
+    {
+      "epoch": 0.042180314854480906,
+      "grad_norm": 0.953125,
+      "learning_rate": 0.002,
+      "loss": 1.8415836095809937,
+      "step": 484
+    },
+    {
+      "epoch": 0.04226746426533728,
+      "grad_norm": 0.9921875,
+      "learning_rate": 0.002,
+      "loss": 1.9032201766967773,
+      "step": 485
+    },
+    {
+      "epoch": 0.04235461367619364,
+      "grad_norm": 0.98046875,
+      "learning_rate": 0.002,
+      "loss": 1.857143759727478,
+      "step": 486
+    },
+    {
+      "epoch": 0.042441763087050006,
+      "grad_norm": 0.8359375,
+      "learning_rate": 0.002,
+      "loss": 1.6680635213851929,
+      "step": 487
+    },
+    {
+      "epoch": 0.04252891249790637,
+      "grad_norm": 0.890625,
+      "learning_rate": 0.002,
+      "loss": 1.8327181339263916,
+      "step": 488
+    },
+    {
+      "epoch": 0.042616061908762735,
+      "grad_norm": 0.9921875,
+      "learning_rate": 0.002,
+      "loss": 1.9679945707321167,
+      "step": 489
+    },
+    {
+      "epoch": 0.0427032113196191,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.002,
+      "loss": 1.956008791923523,
+      "step": 490
+    },
+    {
+      "epoch": 0.04279036073047547,
+      "grad_norm": 0.99609375,
+      "learning_rate": 0.002,
+      "loss": 1.87904953956604,
+      "step": 491
+    },
+    {
+      "epoch": 0.042877510141331836,
+      "grad_norm": 0.921875,
+      "learning_rate": 0.002,
+      "loss": 1.9231213331222534,
+      "step": 492
+    },
+    {
+      "epoch": 0.0429646595521882,
+      "grad_norm": 1.2109375,
+      "learning_rate": 0.002,
+      "loss": 1.965065836906433,
+      "step": 493
+    },
+    {
+      "epoch": 0.043051808963044565,
+      "grad_norm": 0.9453125,
+      "learning_rate": 0.002,
+      "loss": 1.9918681383132935,
+      "step": 494
+    },
+    {
+      "epoch": 0.04313895837390093,
+      "grad_norm": 0.97265625,
+      "learning_rate": 0.002,
+      "loss": 1.8851016759872437,
+      "step": 495
+    },
+    {
+      "epoch": 0.043226107784757294,
+      "grad_norm": 0.92578125,
+      "learning_rate": 0.002,
+      "loss": 1.8246723413467407,
+      "step": 496
+    },
+    {
+      "epoch": 0.04331325719561366,
+      "grad_norm": 0.91796875,
+      "learning_rate": 0.002,
+      "loss": 1.8631030321121216,
+      "step": 497
+    },
+    {
+      "epoch": 0.04340040660647003,
+      "grad_norm": 0.94921875,
+      "learning_rate": 0.002,
+      "loss": 1.9828230142593384,
+      "step": 498
+    },
+    {
+      "epoch": 0.043487556017326394,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.002,
+      "loss": 1.9444931745529175,
+      "step": 499
+    },
+    {
+      "epoch": 0.04357470542818276,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.002,
+      "loss": 1.9800543785095215,
+      "step": 500
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 12049,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.8551763601680355e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:acf04464caccee8e76e5e2200736032322398960e516d6237e49519cfb58664e
+size 5777