Instructions to use RocketData/Qwen3.5-4B-rocketdata with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RocketData/Qwen3.5-4B-rocketdata with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="RocketData/Qwen3.5-4B-rocketdata")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("RocketData/Qwen3.5-4B-rocketdata")
model = AutoModelForMultimodalLM.from_pretrained("RocketData/Qwen3.5-4B-rocketdata", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use RocketData/Qwen3.5-4B-rocketdata with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf RocketData/Qwen3.5-4B-rocketdata:BF16
# Run inference directly in the terminal:
llama cli -hf RocketData/Qwen3.5-4B-rocketdata:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf RocketData/Qwen3.5-4B-rocketdata:BF16
# Run inference directly in the terminal:
llama cli -hf RocketData/Qwen3.5-4B-rocketdata:BF16

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf RocketData/Qwen3.5-4B-rocketdata:BF16
# Run inference directly in the terminal:
./build/bin/llama cli -hf RocketData/Qwen3.5-4B-rocketdata:BF16

Use Docker

docker model run hf.co/RocketData/Qwen3.5-4B-rocketdata:BF16

Llama new
LM Studio
Jan

vLLM

How to use RocketData/Qwen3.5-4B-rocketdata with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RocketData/Qwen3.5-4B-rocketdata"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RocketData/Qwen3.5-4B-rocketdata",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/RocketData/Qwen3.5-4B-rocketdata:BF16

SGLang

How to use RocketData/Qwen3.5-4B-rocketdata with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "RocketData/Qwen3.5-4B-rocketdata" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RocketData/Qwen3.5-4B-rocketdata",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "RocketData/Qwen3.5-4B-rocketdata" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RocketData/Qwen3.5-4B-rocketdata",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Ollama
How to use RocketData/Qwen3.5-4B-rocketdata with Ollama:
```
ollama run hf.co/RocketData/Qwen3.5-4B-rocketdata:BF16
```
Unsloth Desktop

How to use RocketData/Qwen3.5-4B-rocketdata with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf RocketData/Qwen3.5-4B-rocketdata:BF16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "RocketData/Qwen3.5-4B-rocketdata:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use RocketData/Qwen3.5-4B-rocketdata with Docker Model Runner:
```
docker model run hf.co/RocketData/Qwen3.5-4B-rocketdata:BF16
```

Lemonade

How to use RocketData/Qwen3.5-4B-rocketdata with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull RocketData/Qwen3.5-4B-rocketdata:BF16

Run and chat with the model

lemonade run user.Qwen3.5-4B-rocketdata-BF16

List all available models

lemonade list

Hermes Agent

How to use RocketData/Qwen3.5-4B-rocketdata with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf RocketData/Qwen3.5-4B-rocketdata:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default RocketData/Qwen3.5-4B-rocketdata:BF16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use RocketData/Qwen3.5-4B-rocketdata with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf RocketData/Qwen3.5-4B-rocketdata:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "RocketData/Qwen3.5-4B-rocketdata:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.5-4B-rocketdata / train_log.json

adirhn01

Add files using upload-large-folder tool

703601c verified 9 days ago

Raw History Blame Contribute Delete

11.1 kB

{
 "args": {
  "model": "Qwen/Qwen3.5-4B",
  "out": "C:\\ft\\qwen35-rocket-cap60-s42",
  "train": "C:\\ft\\data_qwen\\sft_v2_cap60.jsonl",
  "test": "new_results/sft_test_v2.jsonl",
  "epochs": 3,
  "lr": 0.0002,
  "rank": 16,
  "alpha": 16,
  "batch": 1,
  "accum": 8,
  "max_len": 6400,
  "seed": 42,
  "keep_thought": true,
  "smoke": 0,
  "no_export": false,
  "export_only": false,
  "gguf": "q4_k_m",
  "name": null
 },
 "lowest_test_loss_checkpoint": "C:\\ft\\qwen35-rocket-cap60-s42\\checkpoints\\checkpoint-57",
 "lowest_test_loss": 0.4498031437397003,
 "log_history": [
  {
   "loss": 1.1940197944641113,
   "grad_norm": 0.5686907768249512,
   "learning_rate": 0.0,
   "epoch": 0.052980132450331126,
   "step": 1
  },
  {
   "loss": 1.1503667831420898,
   "grad_norm": 0.5759090781211853,
   "learning_rate": 4e-05,
   "epoch": 0.10596026490066225,
   "step": 2
  },
  {
   "loss": 1.097617745399475,
   "grad_norm": 0.4405198097229004,
   "learning_rate": 8e-05,
   "epoch": 0.15894039735099338,
   "step": 3
  },
  {
   "loss": 1.2405256032943726,
   "grad_norm": 0.5403232574462891,
   "learning_rate": 0.00012,
   "epoch": 0.2119205298013245,
   "step": 4
  },
  {
   "loss": 1.0477312803268433,
   "grad_norm": 0.422463059425354,
   "learning_rate": 0.00016,
   "epoch": 0.26490066225165565,
   "step": 5
  },
  {
   "loss": 1.0621228218078613,
   "grad_norm": 0.42640793323516846,
   "learning_rate": 0.0002,
   "epoch": 0.31788079470198677,
   "step": 6
  },
  {
   "loss": 0.8830090761184692,
   "grad_norm": 0.3740926682949066,
   "learning_rate": 0.00019615384615384615,
   "epoch": 0.3708609271523179,
   "step": 7
  },
  {
   "loss": 0.8500662446022034,
   "grad_norm": 0.6125431060791016,
   "learning_rate": 0.00019230769230769233,
   "epoch": 0.423841059602649,
   "step": 8
  },
  {
   "loss": 0.7644960284233093,
   "grad_norm": 1.4327044486999512,
   "learning_rate": 0.00018846153846153847,
   "epoch": 0.4768211920529801,
   "step": 9
  },
  {
   "loss": 0.6838762164115906,
   "grad_norm": 0.5240151882171631,
   "learning_rate": 0.00018461538461538463,
   "epoch": 0.5298013245033113,
   "step": 10
  },
  {
   "loss": 0.631365954875946,
   "grad_norm": 0.548250675201416,
   "learning_rate": 0.00018076923076923077,
   "epoch": 0.5827814569536424,
   "step": 11
  },
  {
   "loss": 0.544463574886322,
   "grad_norm": 0.4309953451156616,
   "learning_rate": 0.00017692307692307693,
   "epoch": 0.6357615894039735,
   "step": 12
  },
  {
   "loss": 0.5192594528198242,
   "grad_norm": 0.5269620418548584,
   "learning_rate": 0.0001730769230769231,
   "epoch": 0.6887417218543046,
   "step": 13
  },
  {
   "loss": 0.5038818120956421,
   "grad_norm": 0.4541371464729309,
   "learning_rate": 0.00016923076923076923,
   "epoch": 0.7417218543046358,
   "step": 14
  },
  {
   "loss": 0.4713430404663086,
   "grad_norm": 0.2787449061870575,
   "learning_rate": 0.0001653846153846154,
   "epoch": 0.7947019867549668,
   "step": 15
  },
  {
   "loss": 0.45822674036026,
   "grad_norm": 0.42266541719436646,
   "learning_rate": 0.00016153846153846155,
   "epoch": 0.847682119205298,
   "step": 16
  },
  {
   "loss": 0.42965811491012573,
   "grad_norm": 0.3698597252368927,
   "learning_rate": 0.0001576923076923077,
   "epoch": 0.9006622516556292,
   "step": 17
  },
  {
   "loss": 0.4353315532207489,
   "grad_norm": 0.27575576305389404,
   "learning_rate": 0.00015384615384615385,
   "epoch": 0.9536423841059603,
   "step": 18
  },
  {
   "loss": 0.5523375868797302,
   "grad_norm": 0.4252977967262268,
   "learning_rate": 0.00015000000000000001,
   "epoch": 1.0,
   "step": 19
  },
  {
   "eval_loss": 0.49851980805397034,
   "eval_runtime": 5.9076,
   "eval_samples_per_second": 6.432,
   "eval_steps_per_second": 6.432,
   "epoch": 1.0,
   "step": 19
  },
  {
   "loss": 0.38191357254981995,
   "grad_norm": 0.2684159278869629,
   "learning_rate": 0.00014615384615384615,
   "epoch": 1.0529801324503312,
   "step": 20
  },
  {
   "loss": 0.46314921975135803,
   "grad_norm": 0.27592435479164124,
   "learning_rate": 0.0001423076923076923,
   "epoch": 1.1059602649006623,
   "step": 21
  },
  {
   "loss": 0.36137470602989197,
   "grad_norm": 0.25687986612319946,
   "learning_rate": 0.00013846153846153847,
   "epoch": 1.1589403973509933,
   "step": 22
  },
  {
   "loss": 0.4026918113231659,
   "grad_norm": 0.25701817870140076,
   "learning_rate": 0.00013461538461538464,
   "epoch": 1.2119205298013245,
   "step": 23
  },
  {
   "loss": 0.3442859351634979,
   "grad_norm": 0.32386720180511475,
   "learning_rate": 0.00013076923076923077,
   "epoch": 1.2649006622516556,
   "step": 24
  },
  {
   "loss": 0.33980318903923035,
   "grad_norm": 0.23817896842956543,
   "learning_rate": 0.00012692307692307693,
   "epoch": 1.3178807947019868,
   "step": 25
  },
  {
   "loss": 0.3924781084060669,
   "grad_norm": 0.2661411762237549,
   "learning_rate": 0.0001230769230769231,
   "epoch": 1.370860927152318,
   "step": 26
  },
  {
   "loss": 0.3287935256958008,
   "grad_norm": 0.2644951343536377,
   "learning_rate": 0.00011923076923076923,
   "epoch": 1.423841059602649,
   "step": 27
  },
  {
   "loss": 0.38360974192619324,
   "grad_norm": 0.2365025132894516,
   "learning_rate": 0.00011538461538461538,
   "epoch": 1.4768211920529801,
   "step": 28
  },
  {
   "loss": 0.34624168276786804,
   "grad_norm": 0.22081725299358368,
   "learning_rate": 0.00011153846153846154,
   "epoch": 1.5298013245033113,
   "step": 29
  },
  {
   "loss": 0.3713926374912262,
   "grad_norm": 0.30055052042007446,
   "learning_rate": 0.0001076923076923077,
   "epoch": 1.5827814569536423,
   "step": 30
  },
  {
   "loss": 0.341941773891449,
   "grad_norm": 0.2464168816804886,
   "learning_rate": 0.00010384615384615386,
   "epoch": 1.6357615894039736,
   "step": 31
  },
  {
   "loss": 0.3046576678752899,
   "grad_norm": 0.26711592078208923,
   "learning_rate": 0.0001,
   "epoch": 1.6887417218543046,
   "step": 32
  },
  {
   "loss": 0.3337293267250061,
   "grad_norm": 0.2371147722005844,
   "learning_rate": 9.615384615384617e-05,
   "epoch": 1.7417218543046358,
   "step": 33
  },
  {
   "loss": 0.3232137858867645,
   "grad_norm": 0.28805726766586304,
   "learning_rate": 9.230769230769232e-05,
   "epoch": 1.794701986754967,
   "step": 34
  },
  {
   "loss": 0.3332085609436035,
   "grad_norm": 0.2506534159183502,
   "learning_rate": 8.846153846153847e-05,
   "epoch": 1.847682119205298,
   "step": 35
  },
  {
   "loss": 0.32374894618988037,
   "grad_norm": 0.2647243142127991,
   "learning_rate": 8.461538461538461e-05,
   "epoch": 1.9006622516556293,
   "step": 36
  },
  {
   "loss": 0.3772493898868561,
   "grad_norm": 0.23724476993083954,
   "learning_rate": 8.076923076923078e-05,
   "epoch": 1.9536423841059603,
   "step": 37
  },
  {
   "loss": 0.32642272114753723,
   "grad_norm": 0.25431814789772034,
   "learning_rate": 7.692307692307693e-05,
   "epoch": 2.0,
   "step": 38
  },
  {
   "eval_loss": 0.4524935185909271,
   "eval_runtime": 4.617,
   "eval_samples_per_second": 8.23,
   "eval_steps_per_second": 8.23,
   "epoch": 2.0,
   "step": 38
  },
  {
   "loss": 0.32388836145401,
   "grad_norm": 0.242446631193161,
   "learning_rate": 7.307692307692307e-05,
   "epoch": 2.052980132450331,
   "step": 39
  },
  {
   "loss": 0.3339792490005493,
   "grad_norm": 0.24037185311317444,
   "learning_rate": 6.923076923076924e-05,
   "epoch": 2.1059602649006623,
   "step": 40
  },
  {
   "loss": 0.2732575237751007,
   "grad_norm": 0.29293331503868103,
   "learning_rate": 6.538461538461539e-05,
   "epoch": 2.1589403973509933,
   "step": 41
  },
  {
   "loss": 0.3503388464450836,
   "grad_norm": 0.2509494423866272,
   "learning_rate": 6.153846153846155e-05,
   "epoch": 2.2119205298013247,
   "step": 42
  },
  {
   "loss": 0.32519522309303284,
   "grad_norm": 0.2634280323982239,
   "learning_rate": 5.769230769230769e-05,
   "epoch": 2.2649006622516556,
   "step": 43
  },
  {
   "loss": 0.25674334168434143,
   "grad_norm": 0.2411653846502304,
   "learning_rate": 5.384615384615385e-05,
   "epoch": 2.3178807947019866,
   "step": 44
  },
  {
   "loss": 0.3119312524795532,
   "grad_norm": 0.21579930186271667,
   "learning_rate": 5e-05,
   "epoch": 2.370860927152318,
   "step": 45
  },
  {
   "loss": 0.294747918844223,
   "grad_norm": 0.25472205877304077,
   "learning_rate": 4.615384615384616e-05,
   "epoch": 2.423841059602649,
   "step": 46
  },
  {
   "loss": 0.2929841876029968,
   "grad_norm": 0.2879176139831543,
   "learning_rate": 4.230769230769231e-05,
   "epoch": 2.47682119205298,
   "step": 47
  },
  {
   "loss": 0.30371516942977905,
   "grad_norm": 0.28370824456214905,
   "learning_rate": 3.846153846153846e-05,
   "epoch": 2.5298013245033113,
   "step": 48
  },
  {
   "loss": 0.28118428587913513,
   "grad_norm": 0.2666170597076416,
   "learning_rate": 3.461538461538462e-05,
   "epoch": 2.5827814569536423,
   "step": 49
  },
  {
   "loss": 0.26528528332710266,
   "grad_norm": 0.281615287065506,
   "learning_rate": 3.0769230769230774e-05,
   "epoch": 2.6357615894039736,
   "step": 50
  },
  {
   "loss": 0.32126152515411377,
   "grad_norm": 0.24899466335773468,
   "learning_rate": 2.6923076923076923e-05,
   "epoch": 2.6887417218543046,
   "step": 51
  },
  {
   "loss": 0.2957591414451599,
   "grad_norm": 0.25963354110717773,
   "learning_rate": 2.307692307692308e-05,
   "epoch": 2.741721854304636,
   "step": 52
  },
  {
   "loss": 0.2610109746456146,
   "grad_norm": 0.3147188723087311,
   "learning_rate": 1.923076923076923e-05,
   "epoch": 2.794701986754967,
   "step": 53
  },
  {
   "loss": 0.3080472946166992,
   "grad_norm": 0.22268587350845337,
   "learning_rate": 1.5384615384615387e-05,
   "epoch": 2.847682119205298,
   "step": 54
  },
  {
   "loss": 0.291984498500824,
   "grad_norm": 0.28291916847229004,
   "learning_rate": 1.153846153846154e-05,
   "epoch": 2.9006622516556293,
   "step": 55
  },
  {
   "loss": 0.29318422079086304,
   "grad_norm": 0.22744245827198029,
   "learning_rate": 7.692307692307694e-06,
   "epoch": 2.9536423841059603,
   "step": 56
  },
  {
   "loss": 0.26727962493896484,
   "grad_norm": 0.2928221821784973,
   "learning_rate": 3.846153846153847e-06,
   "epoch": 3.0,
   "step": 57
  },
  {
   "eval_loss": 0.4498031437397003,
   "eval_runtime": 4.2761,
   "eval_samples_per_second": 8.887,
   "eval_steps_per_second": 8.887,
   "epoch": 3.0,
   "step": 57
  },
  {
   "train_runtime": 225.952,
   "train_samples_per_second": 2.005,
   "train_steps_per_second": 0.252,
   "total_flos": 1.1110113536504832e+16,
   "train_loss": 0.47283129211057695,
   "epoch": 3.0,
   "step": 57
  }
 ]
}

	{
	"args": {
	"model": "Qwen/Qwen3.5-4B",
	"out": "C:\\ft\\qwen35-rocket-cap60-s42",
	"train": "C:\\ft\\data_qwen\\sft_v2_cap60.jsonl",
	"test": "new_results/sft_test_v2.jsonl",
	"epochs": 3,
	"lr": 0.0002,
	"rank": 16,
	"alpha": 16,
	"batch": 1,
	"accum": 8,
	"max_len": 6400,
	"seed": 42,
	"keep_thought": true,
	"smoke": 0,
	"no_export": false,
	"export_only": false,
	"gguf": "q4_k_m",
	"name": null
	},
	"lowest_test_loss_checkpoint": "C:\\ft\\qwen35-rocket-cap60-s42\\checkpoints\\checkpoint-57",
	"lowest_test_loss": 0.4498031437397003,
	"log_history": [
	{
	"loss": 1.1940197944641113,
	"grad_norm": 0.5686907768249512,
	"learning_rate": 0.0,
	"epoch": 0.052980132450331126,
	"step": 1
	},
	{
	"loss": 1.1503667831420898,
	"grad_norm": 0.5759090781211853,
	"learning_rate": 4e-05,
	"epoch": 0.10596026490066225,
	"step": 2
	},
	{
	"loss": 1.097617745399475,
	"grad_norm": 0.4405198097229004,
	"learning_rate": 8e-05,
	"epoch": 0.15894039735099338,
	"step": 3
	},
	{
	"loss": 1.2405256032943726,
	"grad_norm": 0.5403232574462891,
	"learning_rate": 0.00012,
	"epoch": 0.2119205298013245,
	"step": 4
	},
	{
	"loss": 1.0477312803268433,
	"grad_norm": 0.422463059425354,
	"learning_rate": 0.00016,
	"epoch": 0.26490066225165565,
	"step": 5
	},
	{
	"loss": 1.0621228218078613,
	"grad_norm": 0.42640793323516846,
	"learning_rate": 0.0002,
	"epoch": 0.31788079470198677,
	"step": 6
	},
	{
	"loss": 0.8830090761184692,
	"grad_norm": 0.3740926682949066,
	"learning_rate": 0.00019615384615384615,
	"epoch": 0.3708609271523179,
	"step": 7
	},
	{
	"loss": 0.8500662446022034,
	"grad_norm": 0.6125431060791016,
	"learning_rate": 0.00019230769230769233,
	"epoch": 0.423841059602649,
	"step": 8
	},
	{
	"loss": 0.7644960284233093,
	"grad_norm": 1.4327044486999512,
	"learning_rate": 0.00018846153846153847,
	"epoch": 0.4768211920529801,
	"step": 9
	},
	{
	"loss": 0.6838762164115906,
	"grad_norm": 0.5240151882171631,
	"learning_rate": 0.00018461538461538463,
	"epoch": 0.5298013245033113,
	"step": 10
	},
	{
	"loss": 0.631365954875946,
	"grad_norm": 0.548250675201416,
	"learning_rate": 0.00018076923076923077,
	"epoch": 0.5827814569536424,
	"step": 11
	},
	{
	"loss": 0.544463574886322,
	"grad_norm": 0.4309953451156616,
	"learning_rate": 0.00017692307692307693,
	"epoch": 0.6357615894039735,
	"step": 12
	},
	{
	"loss": 0.5192594528198242,
	"grad_norm": 0.5269620418548584,
	"learning_rate": 0.0001730769230769231,
	"epoch": 0.6887417218543046,
	"step": 13
	},
	{
	"loss": 0.5038818120956421,
	"grad_norm": 0.4541371464729309,
	"learning_rate": 0.00016923076923076923,
	"epoch": 0.7417218543046358,
	"step": 14
	},
	{
	"loss": 0.4713430404663086,
	"grad_norm": 0.2787449061870575,
	"learning_rate": 0.0001653846153846154,
	"epoch": 0.7947019867549668,
	"step": 15
	},
	{
	"loss": 0.45822674036026,
	"grad_norm": 0.42266541719436646,
	"learning_rate": 0.00016153846153846155,
	"epoch": 0.847682119205298,
	"step": 16
	},
	{
	"loss": 0.42965811491012573,
	"grad_norm": 0.3698597252368927,
	"learning_rate": 0.0001576923076923077,
	"epoch": 0.9006622516556292,
	"step": 17
	},
	{
	"loss": 0.4353315532207489,
	"grad_norm": 0.27575576305389404,
	"learning_rate": 0.00015384615384615385,
	"epoch": 0.9536423841059603,
	"step": 18
	},
	{
	"loss": 0.5523375868797302,
	"grad_norm": 0.4252977967262268,
	"learning_rate": 0.00015000000000000001,
	"epoch": 1.0,
	"step": 19
	},
	{
	"eval_loss": 0.49851980805397034,
	"eval_runtime": 5.9076,
	"eval_samples_per_second": 6.432,
	"eval_steps_per_second": 6.432,
	"epoch": 1.0,
	"step": 19
	},
	{
	"loss": 0.38191357254981995,
	"grad_norm": 0.2684159278869629,
	"learning_rate": 0.00014615384615384615,
	"epoch": 1.0529801324503312,
	"step": 20
	},
	{
	"loss": 0.46314921975135803,
	"grad_norm": 0.27592435479164124,
	"learning_rate": 0.0001423076923076923,
	"epoch": 1.1059602649006623,
	"step": 21
	},
	{
	"loss": 0.36137470602989197,
	"grad_norm": 0.25687986612319946,
	"learning_rate": 0.00013846153846153847,
	"epoch": 1.1589403973509933,
	"step": 22
	},
	{
	"loss": 0.4026918113231659,
	"grad_norm": 0.25701817870140076,
	"learning_rate": 0.00013461538461538464,
	"epoch": 1.2119205298013245,
	"step": 23
	},
	{
	"loss": 0.3442859351634979,
	"grad_norm": 0.32386720180511475,
	"learning_rate": 0.00013076923076923077,
	"epoch": 1.2649006622516556,
	"step": 24
	},
	{
	"loss": 0.33980318903923035,
	"grad_norm": 0.23817896842956543,
	"learning_rate": 0.00012692307692307693,
	"epoch": 1.3178807947019868,
	"step": 25
	},
	{
	"loss": 0.3924781084060669,
	"grad_norm": 0.2661411762237549,
	"learning_rate": 0.0001230769230769231,
	"epoch": 1.370860927152318,
	"step": 26
	},
	{
	"loss": 0.3287935256958008,
	"grad_norm": 0.2644951343536377,
	"learning_rate": 0.00011923076923076923,
	"epoch": 1.423841059602649,
	"step": 27
	},
	{
	"loss": 0.38360974192619324,
	"grad_norm": 0.2365025132894516,
	"learning_rate": 0.00011538461538461538,
	"epoch": 1.4768211920529801,
	"step": 28
	},
	{
	"loss": 0.34624168276786804,
	"grad_norm": 0.22081725299358368,
	"learning_rate": 0.00011153846153846154,
	"epoch": 1.5298013245033113,
	"step": 29
	},
	{
	"loss": 0.3713926374912262,
	"grad_norm": 0.30055052042007446,
	"learning_rate": 0.0001076923076923077,
	"epoch": 1.5827814569536423,
	"step": 30
	},
	{
	"loss": 0.341941773891449,
	"grad_norm": 0.2464168816804886,
	"learning_rate": 0.00010384615384615386,
	"epoch": 1.6357615894039736,
	"step": 31
	},
	{
	"loss": 0.3046576678752899,
	"grad_norm": 0.26711592078208923,
	"learning_rate": 0.0001,
	"epoch": 1.6887417218543046,
	"step": 32
	},
	{
	"loss": 0.3337293267250061,
	"grad_norm": 0.2371147722005844,
	"learning_rate": 9.615384615384617e-05,
	"epoch": 1.7417218543046358,
	"step": 33
	},
	{
	"loss": 0.3232137858867645,
	"grad_norm": 0.28805726766586304,
	"learning_rate": 9.230769230769232e-05,
	"epoch": 1.794701986754967,
	"step": 34
	},
	{
	"loss": 0.3332085609436035,
	"grad_norm": 0.2506534159183502,
	"learning_rate": 8.846153846153847e-05,
	"epoch": 1.847682119205298,
	"step": 35
	},
	{
	"loss": 0.32374894618988037,
	"grad_norm": 0.2647243142127991,
	"learning_rate": 8.461538461538461e-05,
	"epoch": 1.9006622516556293,
	"step": 36
	},
	{
	"loss": 0.3772493898868561,
	"grad_norm": 0.23724476993083954,
	"learning_rate": 8.076923076923078e-05,
	"epoch": 1.9536423841059603,
	"step": 37
	},
	{
	"loss": 0.32642272114753723,
	"grad_norm": 0.25431814789772034,
	"learning_rate": 7.692307692307693e-05,
	"epoch": 2.0,
	"step": 38
	},
	{
	"eval_loss": 0.4524935185909271,
	"eval_runtime": 4.617,
	"eval_samples_per_second": 8.23,
	"eval_steps_per_second": 8.23,
	"epoch": 2.0,
	"step": 38
	},
	{
	"loss": 0.32388836145401,
	"grad_norm": 0.242446631193161,
	"learning_rate": 7.307692307692307e-05,
	"epoch": 2.052980132450331,
	"step": 39
	},
	{
	"loss": 0.3339792490005493,
	"grad_norm": 0.24037185311317444,
	"learning_rate": 6.923076923076924e-05,
	"epoch": 2.1059602649006623,
	"step": 40
	},
	{
	"loss": 0.2732575237751007,
	"grad_norm": 0.29293331503868103,
	"learning_rate": 6.538461538461539e-05,
	"epoch": 2.1589403973509933,
	"step": 41
	},
	{
	"loss": 0.3503388464450836,
	"grad_norm": 0.2509494423866272,
	"learning_rate": 6.153846153846155e-05,
	"epoch": 2.2119205298013247,
	"step": 42
	},
	{
	"loss": 0.32519522309303284,
	"grad_norm": 0.2634280323982239,
	"learning_rate": 5.769230769230769e-05,
	"epoch": 2.2649006622516556,
	"step": 43
	},
	{
	"loss": 0.25674334168434143,
	"grad_norm": 0.2411653846502304,
	"learning_rate": 5.384615384615385e-05,
	"epoch": 2.3178807947019866,
	"step": 44
	},
	{
	"loss": 0.3119312524795532,
	"grad_norm": 0.21579930186271667,
	"learning_rate": 5e-05,
	"epoch": 2.370860927152318,
	"step": 45
	},
	{
	"loss": 0.294747918844223,
	"grad_norm": 0.25472205877304077,
	"learning_rate": 4.615384615384616e-05,
	"epoch": 2.423841059602649,
	"step": 46
	},
	{
	"loss": 0.2929841876029968,
	"grad_norm": 0.2879176139831543,
	"learning_rate": 4.230769230769231e-05,
	"epoch": 2.47682119205298,
	"step": 47
	},
	{
	"loss": 0.30371516942977905,
	"grad_norm": 0.28370824456214905,
	"learning_rate": 3.846153846153846e-05,
	"epoch": 2.5298013245033113,
	"step": 48
	},
	{
	"loss": 0.28118428587913513,
	"grad_norm": 0.2666170597076416,
	"learning_rate": 3.461538461538462e-05,
	"epoch": 2.5827814569536423,
	"step": 49
	},
	{
	"loss": 0.26528528332710266,
	"grad_norm": 0.281615287065506,
	"learning_rate": 3.0769230769230774e-05,
	"epoch": 2.6357615894039736,
	"step": 50
	},
	{
	"loss": 0.32126152515411377,
	"grad_norm": 0.24899466335773468,
	"learning_rate": 2.6923076923076923e-05,
	"epoch": 2.6887417218543046,
	"step": 51
	},
	{
	"loss": 0.2957591414451599,
	"grad_norm": 0.25963354110717773,
	"learning_rate": 2.307692307692308e-05,
	"epoch": 2.741721854304636,
	"step": 52
	},
	{
	"loss": 0.2610109746456146,
	"grad_norm": 0.3147188723087311,
	"learning_rate": 1.923076923076923e-05,
	"epoch": 2.794701986754967,
	"step": 53
	},
	{
	"loss": 0.3080472946166992,
	"grad_norm": 0.22268587350845337,
	"learning_rate": 1.5384615384615387e-05,
	"epoch": 2.847682119205298,
	"step": 54
	},
	{
	"loss": 0.291984498500824,
	"grad_norm": 0.28291916847229004,
	"learning_rate": 1.153846153846154e-05,
	"epoch": 2.9006622516556293,
	"step": 55
	},
	{
	"loss": 0.29318422079086304,
	"grad_norm": 0.22744245827198029,
	"learning_rate": 7.692307692307694e-06,
	"epoch": 2.9536423841059603,
	"step": 56
	},
	{
	"loss": 0.26727962493896484,
	"grad_norm": 0.2928221821784973,
	"learning_rate": 3.846153846153847e-06,
	"epoch": 3.0,
	"step": 57
	},
	{
	"eval_loss": 0.4498031437397003,
	"eval_runtime": 4.2761,
	"eval_samples_per_second": 8.887,
	"eval_steps_per_second": 8.887,
	"epoch": 3.0,
	"step": 57
	},
	{
	"train_runtime": 225.952,
	"train_samples_per_second": 2.005,
	"train_steps_per_second": 0.252,
	"total_flos": 1.1110113536504832e+16,
	"train_loss": 0.47283129211057695,
	"epoch": 3.0,
	"step": 57
	}
	]
	}