Instructions to use kth8/gemma-3-1b-it-OpenCode-Title-Generator with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kth8/gemma-3-1b-it-OpenCode-Title-Generator with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kth8/gemma-3-1b-it-OpenCode-Title-Generator")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kth8/gemma-3-1b-it-OpenCode-Title-Generator")
model = AutoModelForCausalLM.from_pretrained("kth8/gemma-3-1b-it-OpenCode-Title-Generator")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use kth8/gemma-3-1b-it-OpenCode-Title-Generator with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kth8/gemma-3-1b-it-OpenCode-Title-Generator"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kth8/gemma-3-1b-it-OpenCode-Title-Generator",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/kth8/gemma-3-1b-it-OpenCode-Title-Generator

SGLang

How to use kth8/gemma-3-1b-it-OpenCode-Title-Generator with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kth8/gemma-3-1b-it-OpenCode-Title-Generator" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kth8/gemma-3-1b-it-OpenCode-Title-Generator",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kth8/gemma-3-1b-it-OpenCode-Title-Generator" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kth8/gemma-3-1b-it-OpenCode-Title-Generator",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio

How to use kth8/gemma-3-1b-it-OpenCode-Title-Generator with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for kth8/gemma-3-1b-it-OpenCode-Title-Generator to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for kth8/gemma-3-1b-it-OpenCode-Title-Generator to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for kth8/gemma-3-1b-it-OpenCode-Title-Generator to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="kth8/gemma-3-1b-it-OpenCode-Title-Generator",
    max_seq_length=2048,
)

Docker Model Runner
How to use kth8/gemma-3-1b-it-OpenCode-Title-Generator with Docker Model Runner:
```
docker model run hf.co/kth8/gemma-3-1b-it-OpenCode-Title-Generator
```

kth8 commited on Jun 1

Commit

bc34009

verified ·

1 Parent(s): 75ce68d

Upload folder using huggingface_hub

Browse files

Files changed (14) hide show

.gitattributes +1 -0
README.md +142 -0
added_tokens.json +3 -0
chat_template.jinja +47 -0
config.json +63 -0
generation_config.json +14 -0
model.safetensors +3 -0
special_tokens_map.json +33 -0
tokenizer.json +3 -0
tokenizer.model +3 -0
tokenizer_config.json +0 -0
train/log.json +1277 -0
train/training_loss.png +0 -0
train/validation_loss.png +0 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,142 @@

+---
+license: gemma
+language:
+- en
+base_model: unsloth/gemma-3-1b-it
+datasets:
+- kth8/title-generation-25000x
+pipeline_tag: text-generation
+library_name: transformers
+tags:
+- sft
+- trl
+- unsloth
+- gemma
+- gemma3
+- gemma3_text
+---
+![logo](https://storage.googleapis.com/gweb-developer-goog-blog-assets/images/gemma-3_2.original.png)
+A supervised fine-tune of [unsloth/gemma-3-1b-it](https://huggingface.co/unsloth/gemma-3-1b-it) on the [kth8/title-generation-25000x](https://huggingface.co/datasets/kth8/title-generation-25000x) dataset.
+Trained with the exact system prompt OpenCode's [title agent uses](https://raw.githubusercontent.com/anomalyco/opencode/refs/heads/dev/packages/opencode/src/agent/prompt/title.txt).
+## Usage example
+**System prompt**
+```
+You are a title generator. You output ONLY a thread title. Nothing else.
+<task>
+Generate a brief title that would help the user find this conversation later.
+Follow all rules in <rules>
+Use the <examples> so you know what a good title looks like.
+Your output must be:
+- A single line
+- ≤50 characters
+- No explanations
+</task>
+<rules>
+- you MUST use the same language as the user message you are summarizing
+- Title must be grammatically correct and read naturally - no word salad
+- Never include tool names in the title (e.g. "read tool", "bash tool", "edit tool")
+- Focus on the main topic or question the user needs to retrieve
+- Vary your phrasing - avoid repetitive patterns like always starting with "Analyzing"
+- When a file is mentioned, focus on WHAT the user wants to do WITH the file, not just that they shared it
+- Keep exact: technical terms, numbers, filenames, HTTP codes
+- Remove: the, this, my, a, an
+- Never assume tech stack
+- Never use tools
+- NEVER respond to questions, just generate a title for the conversation
+- The title should NEVER include "summarizing" or "generating" when generating a title
+- DO NOT SAY YOU CANNOT GENERATE A TITLE OR COMPLAIN ABOUT THE INPUT
+- Always output something meaningful, even if the input is minimal.
+- If the user message is short or conversational (e.g. "hello", "lol", "what's up", "hey"):
+  → create a title that reflects the user's tone or intent (such as Greeting, Quick check-in, Light chat, Intro message, etc.)
+</rules>
+<examples>
+"debug 500 errors in production" → Debugging production 500 errors
+"refactor user service" → Refactoring user service
+"why is app.js failing" → app.js failure investigation
+"implement rate limiting" → Rate limiting implementation
+"how do I connect postgres to my API" → Postgres API connection
+"best practices for React hooks" → React hooks best practices
+"@src/auth.ts can you add refresh token support" → Auth refresh token support
+"@utils/parser.ts this is broken" → Parser bug fix
+"look at @config.json" → Config review
+"@App.tsx add dark mode toggle" → Dark mode toggle in App
+</examples>
+```
+**User prompt**
+```
+If there were 200 students who passed an English course three years ago, and each subsequent year until the current one that number increased by 50% of the previous year's number, how many students will pass the course this year?
+```
+**Assistant response**
+```
+Student course passing growth calculation
+```
+## Model Details
+- Base Model: `unsloth/gemma-3-1b-it`
+- Parameter Count: 999,885,952
+- Precision: torch.bfloat16
+## Training Settings
+### PEFT
+- Rank: 32
+- LoRA alpha: 64
+- Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
+- Gradient checkpointing: unsloth
+### SFT
+- Epoch: 1
+- Batch size: 8
+- Gradient Accumulation steps: 2
+- Learning rate: 0.0002
+- Optimizer: adamw_torch_fused
+- Learning rate scheduler: cosine
+- Warmup steps: 100
+- Weight decay: 0.01
+## Training stats
+- Date: 2026-06-01T08:29:30.781636
+- GPU: NVIDIA A100-SXM4-40GB
+- Peak VRAM usage: 13.873 GB
+- Global step: 1588
+- Training runtime (seconds): 2468.3919
+- Best validation loss: 1.019551396369934
+| Step | Training Loss | Validation Loss |
+|------|---------------|-----------------|
+| 79   | 1.295400      | 1.364987        |
+| 158  | 1.269300      | 1.269067        |
+| 237  | 1.199000      | 1.264947        |
+| 316  | 1.249800      | 1.216193        |
+| 395  | 1.342900      | 1.188592        |
+| 474  | 1.212000      | 1.192388        |
+| 553  | 1.194300      | 1.155604        |
+| 632  | 1.174100      | 1.136098        |
+| 711  | 1.088800      | 1.133109        |
+| 790  | 1.148900      | 1.151859        |
+| 869  | 1.070300      | 1.112693        |
+| 948  | 1.005500      | 1.072988        |
+| 1027 | 1.035400      | 1.059929        |
+| 1106 | 1.075200      | 1.054300        |
+| 1185 | 1.078300      | 1.046455        |
+| 1264 | 1.063400      | 1.032251        |
+| 1343 | 0.980000      | 1.026014        |
+| 1422 | 1.040300      | 1.019870        |
+| 1501 | 0.987600      | 1.019551        |
+| 1580 | 0.988000      | 1.019595        |
+## Framework versions
+- Unsloth: 2026.5.9
+- TRL: 0.22.2
+- Transformers: 4.56.2
+- Pytorch: 2.11.0+cu128
+- Datasets: 4.8.5
+- Tokenizers: 0.22.2
+## License
+This model is released under the Gemma license. See the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) and [Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy) regarding the use of Gemma-generated content.

added_tokens.json ADDED Viewed

	@@ -0,0 +1,3 @@

+{
+  "<image_soft_token>": 262144
+}

chat_template.jinja ADDED Viewed

	@@ -0,0 +1,47 @@

+{{ bos_token }}
+{%- if messages[0]['role'] == 'system' -%}
+    {%- if messages[0]['content'] is string -%}
+        {%- set first_user_prefix = messages[0]['content'] + '
+' -%}
+    {%- else -%}
+        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
+' -%}
+    {%- endif -%}
+    {%- set loop_messages = messages[1:] -%}
+{%- else -%}
+    {%- set first_user_prefix = "" -%}
+    {%- set loop_messages = messages -%}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<start_of_turn>' + role + '
+' + (first_user_prefix if loop.first else "") }}
+    {%- if message['content'] is string -%}
+        {{ message['content'] | trim }}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'image' -%}
+                {{ '<start_of_image>' }}
+            {%- elif item['type'] == 'text' -%}
+                {{ item['text'] | trim }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<end_of_turn>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<start_of_turn>model
+'}}
+{%- endif -%}

config.json ADDED Viewed

	@@ -0,0 +1,63 @@

+{
+  "_sliding_window_pattern": 6,
+  "architectures": [
+    "Gemma3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attn_logit_softcapping": null,
+  "bos_token_id": 2,
+  "cache_implementation": "hybrid",
+  "dtype": "bfloat16",
+  "eos_token_id": 106,
+  "final_logit_softcapping": null,
+  "head_dim": 256,
+  "hidden_activation": "gelu_pytorch_tanh",
+  "hidden_size": 1152,
+  "initializer_range": 0.02,
+  "intermediate_size": 6912,
+  "layer_types": [
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "model_type": "gemma3_text",
+  "num_attention_heads": 4,
+  "num_hidden_layers": 26,
+  "num_key_value_heads": 1,
+  "pad_token_id": 0,
+  "query_pre_attn_scalar": 256,
+  "rms_norm_eps": 1e-06,
+  "rope_local_base_freq": 10000,
+  "rope_scaling": null,
+  "rope_theta": 1000000,
+  "sliding_window": 512,
+  "transformers_version": "4.56.2",
+  "unsloth_fixed": true,
+  "use_cache": true,
+  "vocab_size": 262144
+}

generation_config.json ADDED Viewed

	@@ -0,0 +1,14 @@

+{
+  "bos_token_id": 2,
+  "cache_implementation": "hybrid",
+  "do_sample": true,
+  "eos_token_id": [
+    1,
+    106
+  ],
+  "max_length": 32768,
+  "pad_token_id": 0,
+  "top_k": 64,
+  "top_p": 0.95,
+  "transformers_version": "4.56.2"
+}

model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c234d92585f5bf0ed57ed1d171010da2fc92ca7c78b054d8468273b1ac9cf8c8
+size 1999811208

special_tokens_map.json ADDED Viewed

	@@ -0,0 +1,33 @@

+{
+  "boi_token": "<start_of_image>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<end_of_image>",
+  "eos_token": {
+    "content": "<end_of_turn>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image_soft_token>",
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4667f2089529e8e7657cfb6d1c19910ae71ff5f28aa7ab2ff2763330affad795
+size 33384568

tokenizer.model ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
+size 4689074

tokenizer_config.json ADDED Viewed

The diff for this file is too large to render. See raw diff

train/log.json ADDED Viewed

	@@ -0,0 +1,1277 @@

+[
+  {
+    "loss": 4.261,
+    "grad_norm": 28.33258819580078,
+    "learning_rate": 1.8e-05,
+    "epoch": 0.006297229219143577,
+    "step": 10
+  },
+  {
+    "loss": 2.013,
+    "grad_norm": 8.625818252563477,
+    "learning_rate": 3.8e-05,
+    "epoch": 0.012594458438287154,
+    "step": 20
+  },
+  {
+    "loss": 1.5624,
+    "grad_norm": 7.547111511230469,
+    "learning_rate": 5.8e-05,
+    "epoch": 0.018891687657430732,
+    "step": 30
+  },
+  {
+    "loss": 1.4473,
+    "grad_norm": 9.756204605102539,
+    "learning_rate": 7.800000000000001e-05,
+    "epoch": 0.02518891687657431,
+    "step": 40
+  },
+  {
+    "loss": 1.3737,
+    "grad_norm": 5.382819652557373,
+    "learning_rate": 9.8e-05,
+    "epoch": 0.031486146095717885,
+    "step": 50
+  },
+  {
+    "loss": 1.4106,
+    "grad_norm": 9.088937759399414,
+    "learning_rate": 0.000118,
+    "epoch": 0.037783375314861464,
+    "step": 60
+  },
+  {
+    "loss": 1.2954,
+    "grad_norm": 6.663686275482178,
+    "learning_rate": 0.000138,
+    "epoch": 0.04408060453400504,
+    "step": 70
+  },
+  {
+    "eval_loss": 1.3649871349334717,
+    "eval_runtime": 37.2236,
+    "eval_samples_per_second": 13.943,
+    "eval_steps_per_second": 3.492,
+    "epoch": 0.04974811083123426,
+    "step": 79
+  },
+  {
+    "loss": 1.4254,
+    "grad_norm": 6.387028694152832,
+    "learning_rate": 0.00015800000000000002,
+    "epoch": 0.05037783375314862,
+    "step": 80
+  },
+  {
+    "loss": 1.2516,
+    "grad_norm": 4.3521552085876465,
+    "learning_rate": 0.00017800000000000002,
+    "epoch": 0.05667506297229219,
+    "step": 90
+  },
+  {
+    "loss": 1.2928,
+    "grad_norm": 5.566264629364014,
+    "learning_rate": 0.00019800000000000002,
+    "epoch": 0.06297229219143577,
+    "step": 100
+  },
+  {
+    "loss": 1.3505,
+    "grad_norm": 4.266009330749512,
+    "learning_rate": 0.0001999819475629623,
+    "epoch": 0.06926952141057935,
+    "step": 110
+  },
+  {
+    "loss": 1.3153,
+    "grad_norm": 6.01008939743042,
+    "learning_rate": 0.00019991955244474347,
+    "epoch": 0.07556675062972293,
+    "step": 120
+  },
+  {
+    "loss": 1.307,
+    "grad_norm": 3.635507583618164,
+    "learning_rate": 0.0001998126195808803,
+    "epoch": 0.0818639798488665,
+    "step": 130
+  },
+  {
+    "loss": 1.2197,
+    "grad_norm": 5.475412845611572,
+    "learning_rate": 0.00019966119663520412,
+    "epoch": 0.08816120906801007,
+    "step": 140
+  },
+  {
+    "loss": 1.2693,
+    "grad_norm": 4.274078845977783,
+    "learning_rate": 0.00019946535110237904,
+    "epoch": 0.09445843828715365,
+    "step": 150
+  },
+  {
+    "eval_loss": 1.269067406654358,
+    "eval_runtime": 18.4869,
+    "eval_samples_per_second": 28.074,
+    "eval_steps_per_second": 7.032,
+    "epoch": 0.09949622166246852,
+    "step": 158
+  },
+  {
+    "loss": 1.2633,
+    "grad_norm": 5.927584648132324,
+    "learning_rate": 0.00019922517027781746,
+    "epoch": 0.10075566750629723,
+    "step": 160
+  },
+  {
+    "loss": 1.2406,
+    "grad_norm": 4.917751789093018,
+    "learning_rate": 0.000198940761218769,
+    "epoch": 0.1070528967254408,
+    "step": 170
+  },
+  {
+    "loss": 1.2731,
+    "grad_norm": 4.493299961090088,
+    "learning_rate": 0.0001986122506966014,
+    "epoch": 0.11335012594458438,
+    "step": 180
+  },
+  {
+    "loss": 1.2295,
+    "grad_norm": 5.21290397644043,
+    "learning_rate": 0.000198239785140294,
+    "epoch": 0.11964735516372796,
+    "step": 190
+  },
+  {
+    "loss": 1.1721,
+    "grad_norm": 4.943316459655762,
+    "learning_rate": 0.000197823530571169,
+    "epoch": 0.12594458438287154,
+    "step": 200
+  },
+  {
+    "loss": 1.2901,
+    "grad_norm": 4.010244846343994,
+    "learning_rate": 0.00019736367252888974,
+    "epoch": 0.13224181360201512,
+    "step": 210
+  },
+  {
+    "loss": 1.2298,
+    "grad_norm": 4.7228779792785645,
+    "learning_rate": 0.000196860415988759,
+    "epoch": 0.1385390428211587,
+    "step": 220
+  },
+  {
+    "loss": 1.199,
+    "grad_norm": 4.324512004852295,
+    "learning_rate": 0.00019631398527035422,
+    "epoch": 0.14483627204030228,
+    "step": 230
+  },
+  {
+    "eval_loss": 1.2649470567703247,
+    "eval_runtime": 18.6336,
+    "eval_samples_per_second": 27.853,
+    "eval_steps_per_second": 6.977,
+    "epoch": 0.14924433249370278,
+    "step": 237
+  },
+  {
+    "loss": 1.2394,
+    "grad_norm": 4.480823993682861,
+    "learning_rate": 0.00019572462393754005,
+    "epoch": 0.15113350125944586,
+    "step": 240
+  },
+  {
+    "loss": 1.1254,
+    "grad_norm": 5.136316776275635,
+    "learning_rate": 0.00019509259468990323,
+    "epoch": 0.1574307304785894,
+    "step": 250
+  },
+  {
+    "loss": 1.2375,
+    "grad_norm": 4.789736747741699,
+    "learning_rate": 0.00019441817924565786,
+    "epoch": 0.163727959697733,
+    "step": 260
+  },
+  {
+    "loss": 1.1322,
+    "grad_norm": 3.7004570960998535,
+    "learning_rate": 0.00019370167821607353,
+    "epoch": 0.17002518891687657,
+    "step": 270
+  },
+  {
+    "loss": 1.2878,
+    "grad_norm": 4.034934997558594,
+    "learning_rate": 0.00019294341097148194,
+    "epoch": 0.17632241813602015,
+    "step": 280
+  },
+  {
+    "loss": 1.3235,
+    "grad_norm": 4.752628803253174,
+    "learning_rate": 0.0001921437154989221,
+    "epoch": 0.18261964735516373,
+    "step": 290
+  },
+  {
+    "loss": 1.3083,
+    "grad_norm": 5.0100483894348145,
+    "learning_rate": 0.0001913029482514873,
+    "epoch": 0.1889168765743073,
+    "step": 300
+  },
+  {
+    "loss": 1.2498,
+    "grad_norm": 4.2906036376953125,
+    "learning_rate": 0.00019042148398944116,
+    "epoch": 0.1952141057934509,
+    "step": 310
+  },
+  {
+    "eval_loss": 1.216193437576294,
+    "eval_runtime": 18.6046,
+    "eval_samples_per_second": 27.896,
+    "eval_steps_per_second": 6.988,
+    "epoch": 0.19899244332493704,
+    "step": 316
+  },
+  {
+    "loss": 1.1354,
+    "grad_norm": 4.017844200134277,
+    "learning_rate": 0.0001894997156131734,
+    "epoch": 0.20151133501259447,
+    "step": 320
+  },
+  {
+    "loss": 1.2565,
+    "grad_norm": 4.2938055992126465,
+    "learning_rate": 0.00018853805398807004,
+    "epoch": 0.20780856423173805,
+    "step": 330
+  },
+  {
+    "loss": 1.1638,
+    "grad_norm": 5.103464126586914,
+    "learning_rate": 0.00018753692776137584,
+    "epoch": 0.2141057934508816,
+    "step": 340
+  },
+  {
+    "loss": 1.2232,
+    "grad_norm": 4.511836528778076,
+    "learning_rate": 0.00018649678317113084,
+    "epoch": 0.22040302267002518,
+    "step": 350
+  },
+  {
+    "loss": 1.2183,
+    "grad_norm": 4.007864475250244,
+    "learning_rate": 0.0001854180838472659,
+    "epoch": 0.22670025188916876,
+    "step": 360
+  },
+  {
+    "loss": 1.2473,
+    "grad_norm": 5.685576438903809,
+    "learning_rate": 0.00018430131060494616,
+    "epoch": 0.23299748110831234,
+    "step": 370
+  },
+  {
+    "loss": 1.2598,
+    "grad_norm": 5.161244869232178,
+    "learning_rate": 0.00018314696123025454,
+    "epoch": 0.23929471032745592,
+    "step": 380
+  },
+  {
+    "loss": 1.3429,
+    "grad_norm": 4.944404125213623,
+    "learning_rate": 0.00018195555025831022,
+    "epoch": 0.2455919395465995,
+    "step": 390
+  },
+  {
+    "eval_loss": 1.1885915994644165,
+    "eval_runtime": 18.7194,
+    "eval_samples_per_second": 27.725,
+    "eval_steps_per_second": 6.945,
+    "epoch": 0.24874055415617127,
+    "step": 395
+  },
+  {
+    "loss": 1.1816,
+    "grad_norm": 3.236201047897339,
+    "learning_rate": 0.00018072760874392215,
+    "epoch": 0.2518891687657431,
+    "step": 400
+  },
+  {
+    "loss": 1.1671,
+    "grad_norm": 3.8253695964813232,
+    "learning_rate": 0.00017946368402487845,
+    "epoch": 0.25818639798488663,
+    "step": 410
+  },
+  {
+    "loss": 1.1089,
+    "grad_norm": 4.663104057312012,
+    "learning_rate": 0.00017816433947797858,
+    "epoch": 0.26448362720403024,
+    "step": 420
+  },
+  {
+    "loss": 1.157,
+    "grad_norm": 3.8281562328338623,
+    "learning_rate": 0.00017683015426791566,
+    "epoch": 0.2707808564231738,
+    "step": 430
+  },
+  {
+    "loss": 1.1627,
+    "grad_norm": 5.250668048858643,
+    "learning_rate": 0.00017546172308912213,
+    "epoch": 0.2770780856423174,
+    "step": 440
+  },
+  {
+    "loss": 1.1441,
+    "grad_norm": 5.779282569885254,
+    "learning_rate": 0.0001740596559006929,
+    "epoch": 0.28337531486146095,
+    "step": 450
+  },
+  {
+    "loss": 1.1383,
+    "grad_norm": 4.290262222290039,
+    "learning_rate": 0.00017262457765450457,
+    "epoch": 0.28967254408060455,
+    "step": 460
+  },
+  {
+    "loss": 1.212,
+    "grad_norm": 4.313045024871826,
+    "learning_rate": 0.000171157128016652,
+    "epoch": 0.2959697732997481,
+    "step": 470
+  },
+  {
+    "eval_loss": 1.1923881769180298,
+    "eval_runtime": 18.6387,
+    "eval_samples_per_second": 27.845,
+    "eval_steps_per_second": 6.975,
+    "epoch": 0.29848866498740556,
+    "step": 474
+  },
+  {
+    "loss": 1.0729,
+    "grad_norm": 3.9757778644561768,
+    "learning_rate": 0.00016965796108232598,
+    "epoch": 0.3022670025188917,
+    "step": 480
+  },
+  {
+    "loss": 1.1157,
+    "grad_norm": 5.032901287078857,
+    "learning_rate": 0.00016812774508425933,
+    "epoch": 0.30856423173803527,
+    "step": 490
+  },
+  {
+    "loss": 1.1882,
+    "grad_norm": 4.437510013580322,
+    "learning_rate": 0.00016656716209487174,
+    "epoch": 0.3148614609571788,
+    "step": 500
+  },
+  {
+    "loss": 1.2293,
+    "grad_norm": 4.184134006500244,
+    "learning_rate": 0.00016497690772224522,
+    "epoch": 0.3211586901763224,
+    "step": 510
+  },
+  {
+    "loss": 1.0951,
+    "grad_norm": 7.480218410491943,
+    "learning_rate": 0.00016335769080006674,
+    "epoch": 0.327455919395466,
+    "step": 520
+  },
+  {
+    "loss": 1.2974,
+    "grad_norm": 3.937443256378174,
+    "learning_rate": 0.00016171023307167545,
+    "epoch": 0.3337531486146096,
+    "step": 530
+  },
+  {
+    "loss": 1.0903,
+    "grad_norm": 5.319689750671387,
+    "learning_rate": 0.00016003526886835553,
+    "epoch": 0.34005037783375314,
+    "step": 540
+  },
+  {
+    "loss": 1.1943,
+    "grad_norm": 4.3774733543396,
+    "learning_rate": 0.00015833354478201844,
+    "epoch": 0.34634760705289674,
+    "step": 550
+  },
+  {
+    "eval_loss": 1.1556044816970825,
+    "eval_runtime": 18.5813,
+    "eval_samples_per_second": 27.931,
+    "eval_steps_per_second": 6.996,
+    "epoch": 0.3482367758186398,
+    "step": 553
+  },
+  {
+    "loss": 1.1484,
+    "grad_norm": 4.87918758392334,
+    "learning_rate": 0.00015660581933241993,
+    "epoch": 0.3526448362720403,
+    "step": 560
+  },
+  {
+    "loss": 1.1569,
+    "grad_norm": 5.953218936920166,
+    "learning_rate": 0.00015485286262906044,
+    "epoch": 0.3589420654911839,
+    "step": 570
+  },
+  {
+    "loss": 1.1309,
+    "grad_norm": 3.7059683799743652,
+    "learning_rate": 0.00015307545602791955,
+    "epoch": 0.36523929471032746,
+    "step": 580
+  },
+  {
+    "loss": 1.1818,
+    "grad_norm": 3.702540159225464,
+    "learning_rate": 0.00015127439178317745,
+    "epoch": 0.371536523929471,
+    "step": 590
+  },
+  {
+    "loss": 1.0851,
+    "grad_norm": 3.8618204593658447,
+    "learning_rate": 0.00014945047269407876,
+    "epoch": 0.3778337531486146,
+    "step": 600
+  },
+  {
+    "loss": 1.2035,
+    "grad_norm": 3.884420871734619,
+    "learning_rate": 0.0001476045117470958,
+    "epoch": 0.38413098236775817,
+    "step": 610
+  },
+  {
+    "loss": 1.1274,
+    "grad_norm": 4.641972064971924,
+    "learning_rate": 0.0001457373317535515,
+    "epoch": 0.3904282115869018,
+    "step": 620
+  },
+  {
+    "loss": 1.1741,
+    "grad_norm": 3.4928109645843506,
+    "learning_rate": 0.00014384976498286252,
+    "epoch": 0.3967254408060453,
+    "step": 630
+  },
+  {
+    "eval_loss": 1.1360982656478882,
+    "eval_runtime": 18.5975,
+    "eval_samples_per_second": 27.907,
+    "eval_steps_per_second": 6.99,
+    "epoch": 0.3979848866498741,
+    "step": 632
+  },
+  {
+    "loss": 1.0498,
+    "grad_norm": 5.098562240600586,
+    "learning_rate": 0.00014194265279156675,
+    "epoch": 0.40302267002518893,
+    "step": 640
+  },
+  {
+    "loss": 1.0625,
+    "grad_norm": 4.585139274597168,
+    "learning_rate": 0.00014001684524830057,
+    "epoch": 0.4093198992443325,
+    "step": 650
+  },
+  {
+    "loss": 1.3131,
+    "grad_norm": 4.212075233459473,
+    "learning_rate": 0.0001380732007548925,
+    "epoch": 0.4156171284634761,
+    "step": 660
+  },
+  {
+    "loss": 1.0974,
+    "grad_norm": 2.9548299312591553,
+    "learning_rate": 0.00013611258566374257,
+    "epoch": 0.42191435768261965,
+    "step": 670
+  },
+  {
+    "loss": 1.1007,
+    "grad_norm": 7.119769096374512,
+    "learning_rate": 0.00013413587389165784,
+    "epoch": 0.4282115869017632,
+    "step": 680
+  },
+  {
+    "loss": 1.1405,
+    "grad_norm": 3.9440793991088867,
+    "learning_rate": 0.00013214394653031616,
+    "epoch": 0.4345088161209068,
+    "step": 690
+  },
+  {
+    "loss": 1.1412,
+    "grad_norm": 3.7775681018829346,
+    "learning_rate": 0.000130137691453532,
+    "epoch": 0.44080604534005036,
+    "step": 700
+  },
+  {
+    "loss": 1.0888,
+    "grad_norm": 3.689419984817505,
+    "learning_rate": 0.0001281180029214988,
+    "epoch": 0.44710327455919396,
+    "step": 710
+  },
+  {
+    "eval_loss": 1.1331086158752441,
+    "eval_runtime": 18.9998,
+    "eval_samples_per_second": 27.316,
+    "eval_steps_per_second": 6.842,
+    "epoch": 0.4477329974811083,
+    "step": 711
+  },
+  {
+    "loss": 1.0959,
+    "grad_norm": 4.162503242492676,
+    "learning_rate": 0.00012608578118218535,
+    "epoch": 0.4534005037783375,
+    "step": 720
+  },
+  {
+    "loss": 1.0881,
+    "grad_norm": 4.154449939727783,
+    "learning_rate": 0.00012404193207006246,
+    "epoch": 0.4596977329974811,
+    "step": 730
+  },
+  {
+    "loss": 1.1664,
+    "grad_norm": 3.9392330646514893,
+    "learning_rate": 0.00012198736660234009,
+    "epoch": 0.4659949622166247,
+    "step": 740
+  },
+  {
+    "loss": 1.2067,
+    "grad_norm": 3.568701982498169,
+    "learning_rate": 0.0001199230005728942,
+    "epoch": 0.4722921914357683,
+    "step": 750
+  },
+  {
+    "loss": 1.085,
+    "grad_norm": 4.274581432342529,
+    "learning_rate": 0.00011784975414406426,
+    "epoch": 0.47858942065491183,
+    "step": 760
+  },
+  {
+    "loss": 1.1039,
+    "grad_norm": 4.135290145874023,
+    "learning_rate": 0.00011576855143650371,
+    "epoch": 0.48488664987405544,
+    "step": 770
+  },
+  {
+    "loss": 1.0403,
+    "grad_norm": 3.5335512161254883,
+    "learning_rate": 0.00011368032011726635,
+    "epoch": 0.491183879093199,
+    "step": 780
+  },
+  {
+    "loss": 1.1489,
+    "grad_norm": 3.251199722290039,
+    "learning_rate": 0.00011158599098631137,
+    "epoch": 0.49748110831234255,
+    "step": 790
+  },
+  {
+    "eval_loss": 1.1518590450286865,
+    "eval_runtime": 18.5131,
+    "eval_samples_per_second": 28.034,
+    "eval_steps_per_second": 7.022,
+    "epoch": 0.49748110831234255,
+    "step": 790
+  },
+  {
+    "loss": 1.0229,
+    "grad_norm": 3.9981064796447754,
+    "learning_rate": 0.00010948649756161246,
+    "epoch": 0.5037783375314862,
+    "step": 800
+  },
+  {
+    "loss": 1.0989,
+    "grad_norm": 4.600470066070557,
+    "learning_rate": 0.00010738277566305514,
+    "epoch": 0.5100755667506297,
+    "step": 810
+  },
+  {
+    "loss": 1.0495,
+    "grad_norm": 3.7388057708740234,
+    "learning_rate": 0.0001052757629953082,
+    "epoch": 0.5163727959697733,
+    "step": 820
+  },
+  {
+    "loss": 1.0471,
+    "grad_norm": 4.045319557189941,
+    "learning_rate": 0.00010316639872985472,
+    "epoch": 0.5226700251889169,
+    "step": 830
+  },
+  {
+    "loss": 1.1542,
+    "grad_norm": 3.8340413570404053,
+    "learning_rate": 0.00010105562308636973,
+    "epoch": 0.5289672544080605,
+    "step": 840
+  },
+  {
+    "loss": 1.0944,
+    "grad_norm": 3.6133155822753906,
+    "learning_rate": 9.894437691363028e-05,
+    "epoch": 0.535264483627204,
+    "step": 850
+  },
+  {
+    "loss": 1.0703,
+    "grad_norm": 3.6610817909240723,
+    "learning_rate": 9.683360127014529e-05,
+    "epoch": 0.5415617128463476,
+    "step": 860
+  },
+  {
+    "eval_loss": 1.1126927137374878,
+    "eval_runtime": 18.6703,
+    "eval_samples_per_second": 27.798,
+    "eval_steps_per_second": 6.963,
+    "epoch": 0.5472292191435768,
+    "step": 869
+  },
+  {
+    "loss": 1.0068,
+    "grad_norm": 3.7706260681152344,
+    "learning_rate": 9.47242370046918e-05,
+    "epoch": 0.5478589420654912,
+    "step": 870
+  },
+  {
+    "loss": 1.0582,
+    "grad_norm": 4.216574192047119,
+    "learning_rate": 9.261722433694485e-05,
+    "epoch": 0.5541561712846348,
+    "step": 880
+  },
+  {
+    "loss": 1.091,
+    "grad_norm": 4.351677894592285,
+    "learning_rate": 9.051350243838756e-05,
+    "epoch": 0.5604534005037783,
+    "step": 890
+  },
+  {
+    "loss": 1.0095,
+    "grad_norm": 3.1938083171844482,
+    "learning_rate": 8.841400901368864e-05,
+    "epoch": 0.5667506297229219,
+    "step": 900
+  },
+  {
+    "loss": 0.9896,
+    "grad_norm": 3.466660976409912,
+    "learning_rate": 8.631967988273366e-05,
+    "epoch": 0.5730478589420654,
+    "step": 910
+  },
+  {
+    "loss": 0.9416,
+    "grad_norm": 4.237432479858398,
+    "learning_rate": 8.423144856349631e-05,
+    "epoch": 0.5793450881612091,
+    "step": 920
+  },
+  {
+    "loss": 1.1146,
+    "grad_norm": 2.8684630393981934,
+    "learning_rate": 8.215024585593577e-05,
+    "epoch": 0.5856423173803527,
+    "step": 930
+  },
+  {
+    "loss": 1.0055,
+    "grad_norm": 3.612492322921753,
+    "learning_rate": 8.007699942710581e-05,
+    "epoch": 0.5919395465994962,
+    "step": 940
+  },
+  {
+    "eval_loss": 1.072987675666809,
+    "eval_runtime": 18.7293,
+    "eval_samples_per_second": 27.711,
+    "eval_steps_per_second": 6.941,
+    "epoch": 0.5969773299748111,
+    "step": 948
+  },
+  {
+    "loss": 1.0303,
+    "grad_norm": 4.036654472351074,
+    "learning_rate": 7.801263339765994e-05,
+    "epoch": 0.5982367758186398,
+    "step": 950
+  },
+  {
+    "loss": 1.0264,
+    "grad_norm": 3.446193218231201,
+    "learning_rate": 7.595806792993759e-05,
+    "epoch": 0.6045340050377834,
+    "step": 960
+  },
+  {
+    "loss": 1.1178,
+    "grad_norm": 4.124652862548828,
+    "learning_rate": 7.391421881781468e-05,
+    "epoch": 0.610831234256927,
+    "step": 970
+  },
+  {
+    "loss": 1.0875,
+    "grad_norm": 3.254272699356079,
+    "learning_rate": 7.188199707850122e-05,
+    "epoch": 0.6171284634760705,
+    "step": 980
+  },
+  {
+    "loss": 1.0247,
+    "grad_norm": 4.263841152191162,
+    "learning_rate": 6.986230854646804e-05,
+    "epoch": 0.6234256926952141,
+    "step": 990
+  },
+  {
+    "loss": 1.0394,
+    "grad_norm": 4.2250447273254395,
+    "learning_rate": 6.785605346968386e-05,
+    "epoch": 0.6297229219143576,
+    "step": 1000
+  },
+  {
+    "loss": 1.058,
+    "grad_norm": 4.4213128089904785,
+    "learning_rate": 6.586412610834221e-05,
+    "epoch": 0.6360201511335013,
+    "step": 1010
+  },
+  {
+    "loss": 1.0354,
+    "grad_norm": 3.6707351207733154,
+    "learning_rate": 6.388741433625746e-05,
+    "epoch": 0.6423173803526449,
+    "step": 1020
+  },
+  {
+    "eval_loss": 1.0599287748336792,
+    "eval_runtime": 18.7825,
+    "eval_samples_per_second": 27.632,
+    "eval_steps_per_second": 6.921,
+    "epoch": 0.6467254408060453,
+    "step": 1027
+  },
+  {
+    "loss": 1.0219,
+    "grad_norm": 3.590726613998413,
+    "learning_rate": 6.192679924510752e-05,
+    "epoch": 0.6486146095717884,
+    "step": 1030
+  },
+  {
+    "loss": 0.9563,
+    "grad_norm": 2.9363389015197754,
+    "learning_rate": 5.998315475169942e-05,
+    "epoch": 0.654911838790932,
+    "step": 1040
+  },
+  {
+    "loss": 1.0477,
+    "grad_norm": 3.508352279663086,
+    "learning_rate": 5.8057347208433266e-05,
+    "epoch": 0.6612090680100756,
+    "step": 1050
+  },
+  {
+    "loss": 0.9836,
+    "grad_norm": 3.6980814933776855,
+    "learning_rate": 5.615023501713752e-05,
+    "epoch": 0.6675062972292192,
+    "step": 1060
+  },
+  {
+    "loss": 1.1597,
+    "grad_norm": 4.591259002685547,
+    "learning_rate": 5.4262668246448475e-05,
+    "epoch": 0.6738035264483627,
+    "step": 1070
+  },
+  {
+    "loss": 1.0229,
+    "grad_norm": 4.845359802246094,
+    "learning_rate": 5.239548825290419e-05,
+    "epoch": 0.6801007556675063,
+    "step": 1080
+  },
+  {
+    "loss": 0.9992,
+    "grad_norm": 3.658125400543213,
+    "learning_rate": 5.0549527305921265e-05,
+    "epoch": 0.6863979848866498,
+    "step": 1090
+  },
+  {
+    "loss": 1.0752,
+    "grad_norm": 4.482686996459961,
+    "learning_rate": 4.872560821682256e-05,
+    "epoch": 0.6926952141057935,
+    "step": 1100
+  },
+  {
+    "eval_loss": 1.054300308227539,
+    "eval_runtime": 18.6514,
+    "eval_samples_per_second": 27.826,
+    "eval_steps_per_second": 6.97,
+    "epoch": 0.6964735516372796,
+    "step": 1106
+  },
+  {
+    "loss": 1.1193,
+    "grad_norm": 4.97657585144043,
+    "learning_rate": 4.692454397208048e-05,
+    "epoch": 0.698992443324937,
+    "step": 1110
+  },
+  {
+    "loss": 1.0533,
+    "grad_norm": 3.9969918727874756,
+    "learning_rate": 4.5147137370939595e-05,
+    "epoch": 0.7052896725440806,
+    "step": 1120
+  },
+  {
+    "loss": 0.9617,
+    "grad_norm": 4.142766952514648,
+    "learning_rate": 4.339418066758008e-05,
+    "epoch": 0.7115869017632241,
+    "step": 1130
+  },
+  {
+    "loss": 1.094,
+    "grad_norm": 3.6601150035858154,
+    "learning_rate": 4.166645521798157e-05,
+    "epoch": 0.7178841309823678,
+    "step": 1140
+  },
+  {
+    "loss": 0.9629,
+    "grad_norm": 3.6469886302948,
+    "learning_rate": 3.99647311316445e-05,
+    "epoch": 0.7241813602015114,
+    "step": 1150
+  },
+  {
+    "loss": 1.0169,
+    "grad_norm": 3.6334595680236816,
+    "learning_rate": 3.828976692832458e-05,
+    "epoch": 0.7304785894206549,
+    "step": 1160
+  },
+  {
+    "loss": 0.9622,
+    "grad_norm": 3.6162374019622803,
+    "learning_rate": 3.6642309199933275e-05,
+    "epoch": 0.7367758186397985,
+    "step": 1170
+  },
+  {
+    "loss": 1.0783,
+    "grad_norm": 3.4192535877227783,
+    "learning_rate": 3.502309227775482e-05,
+    "epoch": 0.743073047858942,
+    "step": 1180
+  },
+  {
+    "eval_loss": 1.046454906463623,
+    "eval_runtime": 18.8296,
+    "eval_samples_per_second": 27.563,
+    "eval_steps_per_second": 6.904,
+    "epoch": 0.7462216624685138,
+    "step": 1185
+  },
+  {
+    "loss": 0.9546,
+    "grad_norm": 3.3051228523254395,
+    "learning_rate": 3.343283790512829e-05,
+    "epoch": 0.7493702770780857,
+    "step": 1190
+  },
+  {
+    "loss": 0.9628,
+    "grad_norm": 3.579636812210083,
+    "learning_rate": 3.1872254915740686e-05,
+    "epoch": 0.7556675062972292,
+    "step": 1200
+  },
+  {
+    "loss": 1.1041,
+    "grad_norm": 4.326976776123047,
+    "learning_rate": 3.0342038917674064e-05,
+    "epoch": 0.7619647355163728,
+    "step": 1210
+  },
+  {
+    "loss": 1.1097,
+    "grad_norm": 4.363333225250244,
+    "learning_rate": 2.8842871983347998e-05,
+    "epoch": 0.7682619647355163,
+    "step": 1220
+  },
+  {
+    "loss": 0.9976,
+    "grad_norm": 3.557300567626953,
+    "learning_rate": 2.7375422345495448e-05,
+    "epoch": 0.77455919395466,
+    "step": 1230
+  },
+  {
+    "loss": 1.0998,
+    "grad_norm": 4.066524982452393,
+    "learning_rate": 2.5940344099307146e-05,
+    "epoch": 0.7808564231738035,
+    "step": 1240
+  },
+  {
+    "loss": 1.0098,
+    "grad_norm": 4.808539390563965,
+    "learning_rate": 2.45382769108779e-05,
+    "epoch": 0.7871536523929471,
+    "step": 1250
+  },
+  {
+    "loss": 1.0634,
+    "grad_norm": 3.593735456466675,
+    "learning_rate": 2.3169845732084338e-05,
+    "epoch": 0.7934508816120907,
+    "step": 1260
+  },
+  {
+    "eval_loss": 1.032251238822937,
+    "eval_runtime": 18.7335,
+    "eval_samples_per_second": 27.704,
+    "eval_steps_per_second": 6.939,
+    "epoch": 0.7959697732997482,
+    "step": 1264
+  },
+  {
+    "loss": 1.0001,
+    "grad_norm": 3.325232744216919,
+    "learning_rate": 2.1835660522021417e-05,
+    "epoch": 0.7997481108312342,
+    "step": 1270
+  },
+  {
+    "loss": 0.9349,
+    "grad_norm": 4.100666046142578,
+    "learning_rate": 2.0536315975121544e-05,
+    "epoch": 0.8060453400503779,
+    "step": 1280
+  },
+  {
+    "loss": 0.989,
+    "grad_norm": 3.3437387943267822,
+    "learning_rate": 1.927239125607788e-05,
+    "epoch": 0.8123425692695214,
+    "step": 1290
+  },
+  {
+    "loss": 1.0185,
+    "grad_norm": 3.979337453842163,
+    "learning_rate": 1.8044449741689794e-05,
+    "epoch": 0.818639798488665,
+    "step": 1300
+  },
+  {
+    "loss": 1.0439,
+    "grad_norm": 6.503814697265625,
+    "learning_rate": 1.6853038769745467e-05,
+    "epoch": 0.8249370277078085,
+    "step": 1310
+  },
+  {
+    "loss": 0.9196,
+    "grad_norm": 3.8204808235168457,
+    "learning_rate": 1.5698689395053835e-05,
+    "epoch": 0.8312342569269522,
+    "step": 1320
+  },
+  {
+    "loss": 1.1313,
+    "grad_norm": 4.812267303466797,
+    "learning_rate": 1.4581916152734132e-05,
+    "epoch": 0.8375314861460957,
+    "step": 1330
+  },
+  {
+    "loss": 0.98,
+    "grad_norm": 3.7348341941833496,
+    "learning_rate": 1.3503216828869192e-05,
+    "epoch": 0.8438287153652393,
+    "step": 1340
+  },
+  {
+    "eval_loss": 1.0260143280029297,
+    "eval_runtime": 18.6543,
+    "eval_samples_per_second": 27.822,
+    "eval_steps_per_second": 6.969,
+    "epoch": 0.8457178841309824,
+    "step": 1343
+  },
+  {
+    "loss": 1.1145,
+    "grad_norm": 5.438048362731934,
+    "learning_rate": 1.2463072238624185e-05,
+    "epoch": 0.8501259445843828,
+    "step": 1350
+  },
+  {
+    "loss": 0.9843,
+    "grad_norm": 3.407062292098999,
+    "learning_rate": 1.146194601192998e-05,
+    "epoch": 0.8564231738035264,
+    "step": 1360
+  },
+  {
+    "loss": 1.0344,
+    "grad_norm": 4.0413713455200195,
+    "learning_rate": 1.0500284386826597e-05,
+    "epoch": 0.8627204030226701,
+    "step": 1370
+  },
+  {
+    "loss": 1.0159,
+    "grad_norm": 4.10891580581665,
+    "learning_rate": 9.578516010558847e-06,
+    "epoch": 0.8690176322418136,
+    "step": 1380
+  },
+  {
+    "loss": 1.0693,
+    "grad_norm": 3.644531011581421,
+    "learning_rate": 8.697051748512708e-06,
+    "epoch": 0.8753148614609572,
+    "step": 1390
+  },
+  {
+    "loss": 1.0402,
+    "grad_norm": 3.6608030796051025,
+    "learning_rate": 7.856284501077926e-06,
+    "epoch": 0.8816120906801007,
+    "step": 1400
+  },
+  {
+    "loss": 0.9965,
+    "grad_norm": 3.7804226875305176,
+    "learning_rate": 7.056589028518068e-06,
+    "epoch": 0.8879093198992444,
+    "step": 1410
+  },
+  {
+    "loss": 1.0403,
+    "grad_norm": 4.699734687805176,
+    "learning_rate": 6.2983217839264756e-06,
+    "epoch": 0.8942065491183879,
+    "step": 1420
+  },
+  {
+    "eval_loss": 1.0198695659637451,
+    "eval_runtime": 18.7497,
+    "eval_samples_per_second": 27.68,
+    "eval_steps_per_second": 6.933,
+    "epoch": 0.8954659949622166,
+    "step": 1422
+  },
+  {
+    "loss": 0.997,
+    "grad_norm": 4.407861232757568,
+    "learning_rate": 5.581820754342137e-06,
+    "epoch": 0.9005037783375315,
+    "step": 1430
+  },
+  {
+    "loss": 0.9397,
+    "grad_norm": 3.5445852279663086,
+    "learning_rate": 4.907405310096769e-06,
+    "epoch": 0.906801007556675,
+    "step": 1440
+  },
+  {
+    "loss": 0.9893,
+    "grad_norm": 4.443542003631592,
+    "learning_rate": 4.275376062459946e-06,
+    "epoch": 0.9130982367758187,
+    "step": 1450
+  },
+  {
+    "loss": 0.9178,
+    "grad_norm": 3.4546210765838623,
+    "learning_rate": 3.6860147296457816e-06,
+    "epoch": 0.9193954659949622,
+    "step": 1460
+  },
+  {
+    "loss": 0.9573,
+    "grad_norm": 4.19729471206665,
+    "learning_rate": 3.1395840112410034e-06,
+    "epoch": 0.9256926952141058,
+    "step": 1470
+  },
+  {
+    "loss": 1.0305,
+    "grad_norm": 3.5583102703094482,
+    "learning_rate": 2.636327471110289e-06,
+    "epoch": 0.9319899244332494,
+    "step": 1480
+  },
+  {
+    "loss": 0.9903,
+    "grad_norm": 4.989317417144775,
+    "learning_rate": 2.1764694288310184e-06,
+    "epoch": 0.9382871536523929,
+    "step": 1490
+  },
+  {
+    "loss": 0.9876,
+    "grad_norm": 2.8516058921813965,
+    "learning_rate": 1.7602148597060108e-06,
+    "epoch": 0.9445843828715366,
+    "step": 1500
+  },
+  {
+    "eval_loss": 1.019551396369934,
+    "eval_runtime": 18.5676,
+    "eval_samples_per_second": 27.952,
+    "eval_steps_per_second": 7.001,
+    "epoch": 0.9452141057934509,
+    "step": 1501
+  },
+  {
+    "loss": 0.9417,
+    "grad_norm": 3.9421305656433105,
+    "learning_rate": 1.3877493033986223e-06,
+    "epoch": 0.9508816120906801,
+    "step": 1510
+  },
+  {
+    "loss": 0.9706,
+    "grad_norm": 4.05004358291626,
+    "learning_rate": 1.0592387812310311e-06,
+    "epoch": 0.9571788413098237,
+    "step": 1520
+  },
+  {
+    "loss": 0.9875,
+    "grad_norm": 3.352207660675049,
+    "learning_rate": 7.748297221825573e-07,
+    "epoch": 0.9634760705289672,
+    "step": 1530
+  },
+  {
+    "loss": 0.9976,
+    "grad_norm": 3.546070098876953,
+    "learning_rate": 5.346488976209596e-07,
+    "epoch": 0.9697732997481109,
+    "step": 1540
+  },
+  {
+    "loss": 0.9848,
+    "grad_norm": 4.3622822761535645,
+    "learning_rate": 3.3880336479590325e-07,
+    "epoch": 0.9760705289672544,
+    "step": 1550
+  },
+  {
+    "loss": 1.0212,
+    "grad_norm": 3.776240110397339,
+    "learning_rate": 1.8738041911969818e-07,
+    "epoch": 0.982367758186398,
+    "step": 1560
+  },
+  {
+    "loss": 1.0542,
+    "grad_norm": 4.287234306335449,
+    "learning_rate": 8.044755525652425e-08,
+    "epoch": 0.9886649874055415,
+    "step": 1570
+  },
+  {
+    "loss": 0.988,
+    "grad_norm": 3.729553461074829,
+    "learning_rate": 1.8052437037707758e-08,
+    "epoch": 0.9949622166246851,
+    "step": 1580
+  },
+  {
+    "eval_loss": 1.0195945501327515,
+    "eval_runtime": 18.5885,
+    "eval_samples_per_second": 27.921,
+    "eval_steps_per_second": 6.994,
+    "epoch": 0.9949622166246851,
+    "step": 1580
+  },
+  {
+    "train_runtime": 2468.3919,
+    "train_samples_per_second": 10.293,
+    "train_steps_per_second": 0.643,
+    "total_flos": 7.792418530588262e+16,
+    "train_loss": 1.1395688813639528,
+    "epoch": 1.0,
+    "step": 1588
+  }
+]

train/training_loss.png ADDED Viewed

train/validation_loss.png ADDED Viewed