Instructions to use YahanYu/sft-box_cot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use YahanYu/sft-box_cot with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="YahanYu/sft-box_cot")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("YahanYu/sft-box_cot")
model = AutoModelForMultimodalLM.from_pretrained("YahanYu/sft-box_cot")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use YahanYu/sft-box_cot with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "YahanYu/sft-box_cot"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "YahanYu/sft-box_cot",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/YahanYu/sft-box_cot

SGLang

How to use YahanYu/sft-box_cot with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "YahanYu/sft-box_cot" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "YahanYu/sft-box_cot",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "YahanYu/sft-box_cot" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "YahanYu/sft-box_cot",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use YahanYu/sft-box_cot with Docker Model Runner:
```
docker model run hf.co/YahanYu/sft-box_cot
```

sft-box_cot / trainer_log.jsonl

YahanYu

Upload folder using huggingface_hub

e4b2723 verified 12 months ago

Raw

History Blame Contribute Delete

8.63 kB

	{"current_steps": 10, "total_steps": 450, "loss": 1.0988, "lr": 2.0000000000000003e-06, "epoch": 0.06666666666666667, "percentage": 2.22, "elapsed_time": "0:01:09", "remaining_time": "0:51:10"}
	{"current_steps": 20, "total_steps": 450, "loss": 0.8794, "lr": 4.222222222222223e-06, "epoch": 0.13333333333333333, "percentage": 4.44, "elapsed_time": "0:02:10", "remaining_time": "0:46:53"}
	{"current_steps": 30, "total_steps": 450, "loss": 0.7456, "lr": 6.444444444444445e-06, "epoch": 0.2, "percentage": 6.67, "elapsed_time": "0:03:11", "remaining_time": "0:44:46"}
	{"current_steps": 40, "total_steps": 450, "loss": 0.7296, "lr": 8.666666666666668e-06, "epoch": 0.26666666666666666, "percentage": 8.89, "elapsed_time": "0:04:13", "remaining_time": "0:43:22"}
	{"current_steps": 50, "total_steps": 450, "loss": 0.7487, "lr": 9.997593339404757e-06, "epoch": 0.3333333333333333, "percentage": 11.11, "elapsed_time": "0:05:15", "remaining_time": "0:42:04"}
	{"current_steps": 60, "total_steps": 450, "loss": 0.7059, "lr": 9.970545007734807e-06, "epoch": 0.4, "percentage": 13.33, "elapsed_time": "0:06:16", "remaining_time": "0:40:49"}
	{"current_steps": 70, "total_steps": 450, "loss": 0.738, "lr": 9.913603233532067e-06, "epoch": 0.4666666666666667, "percentage": 15.56, "elapsed_time": "0:07:18", "remaining_time": "0:39:37"}
	{"current_steps": 80, "total_steps": 450, "loss": 0.7691, "lr": 9.827110471326612e-06, "epoch": 0.5333333333333333, "percentage": 17.78, "elapsed_time": "0:08:19", "remaining_time": "0:38:29"}
	{"current_steps": 90, "total_steps": 450, "loss": 0.7704, "lr": 9.711586898767462e-06, "epoch": 0.6, "percentage": 20.0, "elapsed_time": "0:09:20", "remaining_time": "0:37:22"}
	{"current_steps": 100, "total_steps": 450, "loss": 0.7406, "lr": 9.567727288213005e-06, "epoch": 0.6666666666666666, "percentage": 22.22, "elapsed_time": "0:10:22", "remaining_time": "0:36:17"}
	{"current_steps": 110, "total_steps": 450, "loss": 0.7235, "lr": 9.396396828288272e-06, "epoch": 0.7333333333333333, "percentage": 24.44, "elapsed_time": "0:11:23", "remaining_time": "0:35:13"}
	{"current_steps": 120, "total_steps": 450, "loss": 0.6883, "lr": 9.19862592053875e-06, "epoch": 0.8, "percentage": 26.67, "elapsed_time": "0:12:24", "remaining_time": "0:34:08"}
	{"current_steps": 130, "total_steps": 450, "loss": 0.7847, "lr": 8.97560398247424e-06, "epoch": 0.8666666666666667, "percentage": 28.89, "elapsed_time": "0:13:26", "remaining_time": "0:33:04"}
	{"current_steps": 140, "total_steps": 450, "loss": 0.6963, "lr": 8.728672294272009e-06, "epoch": 0.9333333333333333, "percentage": 31.11, "elapsed_time": "0:14:27", "remaining_time": "0:32:01"}
	{"current_steps": 150, "total_steps": 450, "loss": 0.7373, "lr": 8.45931593215998e-06, "epoch": 1.0, "percentage": 33.33, "elapsed_time": "0:15:28", "remaining_time": "0:30:57"}
	{"current_steps": 160, "total_steps": 450, "loss": 0.512, "lr": 8.16915483699355e-06, "epoch": 1.0666666666666667, "percentage": 35.56, "elapsed_time": "0:16:31", "remaining_time": "0:29:56"}
	{"current_steps": 170, "total_steps": 450, "loss": 0.4927, "lr": 7.859934071740693e-06, "epoch": 1.1333333333333333, "percentage": 37.78, "elapsed_time": "0:17:32", "remaining_time": "0:28:52"}
	{"current_steps": 180, "total_steps": 450, "loss": 0.5186, "lr": 7.533513326467911e-06, "epoch": 1.2, "percentage": 40.0, "elapsed_time": "0:18:33", "remaining_time": "0:27:50"}
	{"current_steps": 190, "total_steps": 450, "loss": 0.4908, "lr": 7.191855733945388e-06, "epoch": 1.2666666666666666, "percentage": 42.22, "elapsed_time": "0:19:34", "remaining_time": "0:26:47"}
	{"current_steps": 200, "total_steps": 450, "loss": 0.4654, "lr": 6.837016063135491e-06, "epoch": 1.3333333333333333, "percentage": 44.44, "elapsed_time": "0:20:36", "remaining_time": "0:25:45"}
	{"current_steps": 210, "total_steps": 450, "loss": 0.4729, "lr": 6.4711283615704755e-06, "epoch": 1.4, "percentage": 46.67, "elapsed_time": "0:21:37", "remaining_time": "0:24:42"}
	{"current_steps": 220, "total_steps": 450, "loss": 0.5102, "lr": 6.0963931209395165e-06, "epoch": 1.4666666666666668, "percentage": 48.89, "elapsed_time": "0:22:38", "remaining_time": "0:23:40"}
	{"current_steps": 230, "total_steps": 450, "loss": 0.4718, "lr": 5.715064043072771e-06, "epoch": 1.5333333333333332, "percentage": 51.11, "elapsed_time": "0:23:40", "remaining_time": "0:22:38"}
	{"current_steps": 240, "total_steps": 450, "loss": 0.44, "lr": 5.329434485913393e-06, "epoch": 1.6, "percentage": 53.33, "elapsed_time": "0:24:41", "remaining_time": "0:21:36"}
	{"current_steps": 250, "total_steps": 450, "loss": 0.4782, "lr": 4.941823670993016e-06, "epoch": 1.6666666666666665, "percentage": 55.56, "elapsed_time": "0:25:43", "remaining_time": "0:20:34"}
	{"current_steps": 260, "total_steps": 450, "loss": 0.4601, "lr": 4.5545627353605705e-06, "epoch": 1.7333333333333334, "percentage": 57.78, "elapsed_time": "0:26:45", "remaining_time": "0:19:32"}
	{"current_steps": 270, "total_steps": 450, "loss": 0.487, "lr": 4.1699807118497815e-06, "epoch": 1.8, "percentage": 60.0, "elapsed_time": "0:27:46", "remaining_time": "0:18:30"}
	{"current_steps": 280, "total_steps": 450, "loss": 0.5098, "lr": 3.790390522001662e-06, "epoch": 1.8666666666666667, "percentage": 62.22, "elapsed_time": "0:28:47", "remaining_time": "0:17:28"}
	{"current_steps": 290, "total_steps": 450, "loss": 0.4917, "lr": 3.418075065882217e-06, "epoch": 1.9333333333333333, "percentage": 64.44, "elapsed_time": "0:29:49", "remaining_time": "0:16:27"}
	{"current_steps": 300, "total_steps": 450, "loss": 0.4296, "lr": 3.0552734924528304e-06, "epoch": 2.0, "percentage": 66.67, "elapsed_time": "0:30:51", "remaining_time": "0:15:25"}
	{"current_steps": 310, "total_steps": 450, "loss": 0.2569, "lr": 2.7041677330649408e-06, "epoch": 2.066666666666667, "percentage": 68.89, "elapsed_time": "0:31:52", "remaining_time": "0:14:23"}
	{"current_steps": 320, "total_steps": 450, "loss": 0.2292, "lr": 2.3668693790681634e-06, "epoch": 2.1333333333333333, "percentage": 71.11, "elapsed_time": "0:32:53", "remaining_time": "0:13:21"}
	{"current_steps": 330, "total_steps": 450, "loss": 0.239, "lr": 2.0454069824514445e-06, "epoch": 2.2, "percentage": 73.33, "elapsed_time": "0:33:55", "remaining_time": "0:12:20"}
	{"current_steps": 340, "total_steps": 450, "loss": 0.2125, "lr": 1.7417138558927244e-06, "epoch": 2.2666666666666666, "percentage": 75.56, "elapsed_time": "0:34:56", "remaining_time": "0:11:18"}
	{"current_steps": 350, "total_steps": 450, "loss": 0.2144, "lr": 1.4576164455890014e-06, "epoch": 2.3333333333333335, "percentage": 77.78, "elapsed_time": "0:35:57", "remaining_time": "0:10:16"}
	{"current_steps": 360, "total_steps": 450, "loss": 0.2302, "lr": 1.1948233467939978e-06, "epoch": 2.4, "percentage": 80.0, "elapsed_time": "0:36:59", "remaining_time": "0:09:14"}
	{"current_steps": 370, "total_steps": 450, "loss": 0.2005, "lr": 9.549150281252633e-07, "epoch": 2.466666666666667, "percentage": 82.22, "elapsed_time": "0:38:00", "remaining_time": "0:08:13"}
	{"current_steps": 380, "total_steps": 450, "loss": 0.2277, "lr": 7.393343264399439e-07, "epoch": 2.533333333333333, "percentage": 84.44, "elapsed_time": "0:39:01", "remaining_time": "0:07:11"}
	{"current_steps": 390, "total_steps": 450, "loss": 0.194, "lr": 5.493777694441521e-07, "epoch": 2.6, "percentage": 86.67, "elapsed_time": "0:40:03", "remaining_time": "0:06:09"}
	{"current_steps": 400, "total_steps": 450, "loss": 0.2366, "lr": 3.8618777822278854e-07, "epoch": 2.6666666666666665, "percentage": 88.89, "elapsed_time": "0:41:04", "remaining_time": "0:05:08"}
	{"current_steps": 410, "total_steps": 450, "loss": 0.2157, "lr": 2.5074579658471266e-07, "epoch": 2.7333333333333334, "percentage": 91.11, "elapsed_time": "0:42:05", "remaining_time": "0:04:06"}
	{"current_steps": 420, "total_steps": 450, "loss": 0.2204, "lr": 1.438663885441982e-07, "epoch": 2.8, "percentage": 93.33, "elapsed_time": "0:43:07", "remaining_time": "0:03:04"}
	{"current_steps": 430, "total_steps": 450, "loss": 0.1931, "lr": 6.61923394371039e-08, "epoch": 2.8666666666666667, "percentage": 95.56, "elapsed_time": "0:44:08", "remaining_time": "0:02:03"}
	{"current_steps": 440, "total_steps": 450, "loss": 0.2086, "lr": 1.8190790134231528e-08, "epoch": 2.9333333333333336, "percentage": 97.78, "elapsed_time": "0:45:10", "remaining_time": "0:01:01"}
	{"current_steps": 450, "total_steps": 450, "loss": 0.2162, "lr": 1.504276011621286e-10, "epoch": 3.0, "percentage": 100.0, "elapsed_time": "0:46:11", "remaining_time": "0:00:00"}
	{"current_steps": 450, "total_steps": 450, "epoch": 3.0, "percentage": 100.0, "elapsed_time": "0:50:35", "remaining_time": "0:00:00"}