Instructions to use 17Lab/qwen7b-full-sft-s13 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 17Lab/qwen7b-full-sft-s13 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="17Lab/qwen7b-full-sft-s13")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("17Lab/qwen7b-full-sft-s13")
model = AutoModelForCausalLM.from_pretrained("17Lab/qwen7b-full-sft-s13", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 17Lab/qwen7b-full-sft-s13 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "17Lab/qwen7b-full-sft-s13"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "17Lab/qwen7b-full-sft-s13",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/17Lab/qwen7b-full-sft-s13

SGLang

How to use 17Lab/qwen7b-full-sft-s13 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "17Lab/qwen7b-full-sft-s13" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "17Lab/qwen7b-full-sft-s13",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "17Lab/qwen7b-full-sft-s13" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "17Lab/qwen7b-full-sft-s13",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 17Lab/qwen7b-full-sft-s13 with Docker Model Runner:
```
docker model run hf.co/17Lab/qwen7b-full-sft-s13
```

qwen7b-full-sft-s13 / trainer_log.jsonl

Yiqi-Liu

Upload folder using huggingface_hub

2079f76 verified about 1 month ago

Raw

History Blame Contribute Delete

10.4 kB

	{"current_steps": 5, "total_steps": 225, "loss": 3.3403587341308594, "lr": 6.666666666666667e-06, "epoch": 0.06711409395973154, "percentage": 2.22, "elapsed_time": "0:00:08", "remaining_time": "0:06:08"}
	{"current_steps": 10, "total_steps": 225, "loss": 1.2873397827148438, "lr": 1.5000000000000002e-05, "epoch": 0.1342281879194631, "percentage": 4.44, "elapsed_time": "0:00:13", "remaining_time": "0:04:53"}
	{"current_steps": 15, "total_steps": 225, "loss": 0.29829452037811277, "lr": 1.9995649501315172e-05, "epoch": 0.20134228187919462, "percentage": 6.67, "elapsed_time": "0:00:18", "remaining_time": "0:04:25"}
	{"current_steps": 20, "total_steps": 225, "loss": 0.1349006175994873, "lr": 1.9946749853292233e-05, "epoch": 0.2684563758389262, "percentage": 8.89, "elapsed_time": "0:00:24", "remaining_time": "0:04:10"}
	{"current_steps": 25, "total_steps": 225, "loss": 0.07990359663963317, "lr": 1.9843779142227258e-05, "epoch": 0.33557046979865773, "percentage": 11.11, "elapsed_time": "0:00:29", "remaining_time": "0:03:57"}
	{"current_steps": 30, "total_steps": 225, "loss": 0.03623477220535278, "lr": 1.9687297122400952e-05, "epoch": 0.40268456375838924, "percentage": 13.33, "elapsed_time": "0:00:34", "remaining_time": "0:03:46"}
	{"current_steps": 35, "total_steps": 225, "loss": 0.0021600183099508286, "lr": 1.947815443842424e-05, "epoch": 0.4697986577181208, "percentage": 15.56, "elapsed_time": "0:00:40", "remaining_time": "0:03:38"}
	{"current_steps": 40, "total_steps": 225, "loss": 0.00010623512789607049, "lr": 1.9217488001088784e-05, "epoch": 0.5369127516778524, "percentage": 17.78, "elapsed_time": "0:00:45", "remaining_time": "0:03:30"}
	{"current_steps": 45, "total_steps": 225, "loss": 2.885170979425311e-05, "lr": 1.8906714807059218e-05, "epoch": 0.6040268456375839, "percentage": 20.0, "elapsed_time": "0:00:50", "remaining_time": "0:03:22"}
	{"current_steps": 50, "total_steps": 225, "loss": 1.2394384248182177e-05, "lr": 1.8547524236003675e-05, "epoch": 0.6711409395973155, "percentage": 22.22, "elapsed_time": "0:00:55", "remaining_time": "0:03:15"}
	{"current_steps": 50, "total_steps": 225, "eval_loss": 9.160361514659598e-06, "epoch": 0.6711409395973155, "percentage": 22.22, "elapsed_time": "0:00:58", "remaining_time": "0:03:23"}
	{"current_steps": 55, "total_steps": 225, "loss": 7.670311606489123e-06, "lr": 1.8141868867035745e-05, "epoch": 0.738255033557047, "percentage": 24.44, "elapsed_time": "0:04:52", "remaining_time": "0:15:03"}
	{"current_steps": 60, "total_steps": 225, "loss": 5.718640022678301e-06, "lr": 1.7691953864390208e-05, "epoch": 0.8053691275167785, "percentage": 26.67, "elapsed_time": "0:04:57", "remaining_time": "0:13:38"}
	{"current_steps": 65, "total_steps": 225, "loss": 4.781161987921223e-06, "lr": 1.7200224990032577e-05, "epoch": 0.87248322147651, "percentage": 28.89, "elapsed_time": "0:05:02", "remaining_time": "0:12:25"}
	{"current_steps": 70, "total_steps": 225, "loss": 4.1388742829440165e-06, "lr": 1.666935530836651e-05, "epoch": 0.9395973154362416, "percentage": 31.11, "elapsed_time": "0:05:07", "remaining_time": "0:11:21"}
	{"current_steps": 75, "total_steps": 225, "loss": 3.85485400329344e-06, "lr": 1.6102230655313076e-05, "epoch": 1.0, "percentage": 33.33, "elapsed_time": "0:05:12", "remaining_time": "0:10:25"}
	{"current_steps": 80, "total_steps": 225, "loss": 3.514360651024617e-06, "lr": 1.5501933950752655e-05, "epoch": 1.0671140939597314, "percentage": 35.56, "elapsed_time": "0:05:17", "remaining_time": "0:09:36"}
	{"current_steps": 85, "total_steps": 225, "loss": 3.3707532566040753e-06, "lr": 1.4871728439607967e-05, "epoch": 1.1342281879194631, "percentage": 37.78, "elapsed_time": "0:05:23", "remaining_time": "0:08:52"}
	{"current_steps": 90, "total_steps": 225, "loss": 3.3559525036253033e-06, "lr": 1.4215039952670482e-05, "epoch": 1.2013422818791946, "percentage": 40.0, "elapsed_time": "0:05:28", "remaining_time": "0:08:12"}
	{"current_steps": 95, "total_steps": 225, "loss": 3.262608515797183e-06, "lr": 1.3535438283601437e-05, "epoch": 1.2684563758389262, "percentage": 42.22, "elapsed_time": "0:05:33", "remaining_time": "0:07:36"}
	{"current_steps": 100, "total_steps": 225, "loss": 3.128412936348468e-06, "lr": 1.2836617783342968e-05, "epoch": 1.3355704697986577, "percentage": 44.44, "elapsed_time": "0:05:38", "remaining_time": "0:07:03"}
	{"current_steps": 100, "total_steps": 225, "eval_loss": 3.1148604193731444e-06, "epoch": 1.3355704697986577, "percentage": 44.44, "elapsed_time": "0:05:40", "remaining_time": "0:07:06"}
	{"current_steps": 105, "total_steps": 225, "loss": 3.0415885703405365e-06, "lr": 1.2122377277429231e-05, "epoch": 1.4026845637583891, "percentage": 46.67, "elapsed_time": "0:09:37", "remaining_time": "0:10:59"}
	{"current_steps": 110, "total_steps": 225, "loss": 2.98672603094019e-06, "lr": 1.1396599415368062e-05, "epoch": 1.4697986577181208, "percentage": 48.89, "elapsed_time": "0:09:42", "remaining_time": "0:10:08"}
	{"current_steps": 115, "total_steps": 225, "loss": 2.9487675419659355e-06, "lr": 1.066322956435104e-05, "epoch": 1.5369127516778525, "percentage": 51.11, "elapsed_time": "0:09:47", "remaining_time": "0:09:22"}
	{"current_steps": 120, "total_steps": 225, "loss": 2.942025457741693e-06, "lr": 9.926254362026875e-06, "epoch": 1.604026845637584, "percentage": 53.33, "elapsed_time": "0:09:52", "remaining_time": "0:08:38"}
	{"current_steps": 125, "total_steps": 225, "loss": 2.926953311543912e-06, "lr": 9.189680044926402e-06, "epoch": 1.6711409395973154, "percentage": 55.56, "elapsed_time": "0:09:58", "remaining_time": "0:07:58"}
	{"current_steps": 130, "total_steps": 225, "loss": 2.9204267775639893e-06, "lr": 8.457510670346976e-06, "epoch": 1.738255033557047, "percentage": 57.78, "elapsed_time": "0:10:03", "remaining_time": "0:07:21"}
	{"current_steps": 135, "total_steps": 225, "loss": 2.9483762773452327e-06, "lr": 7.733726350083331e-06, "epoch": 1.8053691275167785, "percentage": 60.0, "elapsed_time": "0:10:08", "remaining_time": "0:06:45"}
	{"current_steps": 140, "total_steps": 225, "loss": 5.818682257086038e-06, "lr": 7.022261614327448e-06, "epoch": 1.87248322147651, "percentage": 62.22, "elapsed_time": "0:10:13", "remaining_time": "0:06:12"}
	{"current_steps": 145, "total_steps": 225, "loss": 2.8704187570838256e-06, "lr": 6.326984023352435e-06, "epoch": 1.9395973154362416, "percentage": 64.44, "elapsed_time": "0:10:19", "remaining_time": "0:05:41"}
	{"current_steps": 150, "total_steps": 225, "loss": 2.8186999770696274e-06, "lr": 5.651673143248509e-06, "epoch": 2.0, "percentage": 66.67, "elapsed_time": "0:10:23", "remaining_time": "0:05:11"}
	{"current_steps": 150, "total_steps": 225, "eval_loss": 2.8178355933050625e-06, "epoch": 2.0, "percentage": 66.67, "elapsed_time": "0:10:26", "remaining_time": "0:05:13"}
	{"current_steps": 155, "total_steps": 225, "loss": 2.7831285478896463e-06, "lr": 5.000000000000003e-06, "epoch": 2.0671140939597317, "percentage": 68.89, "elapsed_time": "0:14:23", "remaining_time": "0:06:29"}
	{"current_steps": 160, "total_steps": 225, "loss": 2.7615027647698297e-06, "lr": 4.375507123592194e-06, "epoch": 2.134228187919463, "percentage": 71.11, "elapsed_time": "0:14:28", "remaining_time": "0:05:52"}
	{"current_steps": 165, "total_steps": 225, "loss": 2.7448764740256593e-06, "lr": 3.781589290629266e-06, "epoch": 2.2013422818791946, "percentage": 73.33, "elapsed_time": "0:14:33", "remaining_time": "0:05:17"}
	{"current_steps": 170, "total_steps": 225, "loss": 2.7252637664787473e-06, "lr": 3.2214750701474875e-06, "epoch": 2.2684563758389262, "percentage": 75.56, "elapsed_time": "0:14:38", "remaining_time": "0:04:44"}
	{"current_steps": 175, "total_steps": 225, "loss": 2.7195979782845826e-06, "lr": 2.698209272941659e-06, "epoch": 2.335570469798658, "percentage": 77.78, "elapsed_time": "0:14:43", "remaining_time": "0:04:12"}
	{"current_steps": 180, "total_steps": 225, "loss": 2.7082156520918944e-06, "lr": 2.2146363998111077e-06, "epoch": 2.402684563758389, "percentage": 80.0, "elapsed_time": "0:14:49", "remaining_time": "0:03:42"}
	{"current_steps": 185, "total_steps": 225, "loss": 2.6918616640614347e-06, "lr": 1.773385178701561e-06, "epoch": 2.469798657718121, "percentage": 82.22, "elapsed_time": "0:14:54", "remaining_time": "0:03:13"}
	{"current_steps": 190, "total_steps": 225, "loss": 2.6803711079992356e-06, "lr": 1.3768542747997215e-06, "epoch": 2.5369127516778525, "percentage": 84.44, "elapsed_time": "0:14:59", "remaining_time": "0:02:45"}
	{"current_steps": 195, "total_steps": 225, "loss": 2.687972482817713e-06, "lr": 1.0271992512613239e-06, "epoch": 2.604026845637584, "percentage": 86.67, "elapsed_time": "0:15:04", "remaining_time": "0:02:19"}
	{"current_steps": 200, "total_steps": 225, "loss": 2.6694764528656378e-06, "lr": 7.263208514547548e-07, "epoch": 2.6711409395973154, "percentage": 88.89, "elapsed_time": "0:15:10", "remaining_time": "0:01:53"}
	{"current_steps": 200, "total_steps": 225, "eval_loss": 2.682836338863126e-06, "epoch": 2.6711409395973154, "percentage": 88.89, "elapsed_time": "0:15:12", "remaining_time": "0:01:54"}
	{"current_steps": 205, "total_steps": 225, "loss": 2.6877429263549858e-06, "lr": 4.7585466641868696e-07, "epoch": 2.738255033557047, "percentage": 91.11, "elapsed_time": "0:19:03", "remaining_time": "0:01:51"}
	{"current_steps": 210, "total_steps": 225, "loss": 2.6864749088417738e-06, "lr": 2.7716224370188836e-07, "epoch": 2.8053691275167782, "percentage": 93.33, "elapsed_time": "0:19:09", "remaining_time": "0:01:22"}
	{"current_steps": 215, "total_steps": 225, "loss": 2.678953023860231e-06, "lr": 1.3132368591809553e-07, "epoch": 2.87248322147651, "percentage": 95.56, "elapsed_time": "0:19:14", "remaining_time": "0:00:53"}
	{"current_steps": 220, "total_steps": 225, "loss": 2.6803716536960563e-06, "lr": 3.913177925055189e-08, "epoch": 2.9395973154362416, "percentage": 97.78, "elapsed_time": "0:19:19", "remaining_time": "0:00:26"}
	{"current_steps": 225, "total_steps": 225, "loss": 2.679282988538034e-06, "lr": 1.0876838240125865e-09, "epoch": 3.0, "percentage": 100.0, "elapsed_time": "0:19:24", "remaining_time": "0:00:00"}
	{"current_steps": 225, "total_steps": 225, "eval_loss": 2.67330960923573e-06, "epoch": 3.0, "percentage": 100.0, "elapsed_time": "0:19:26", "remaining_time": "0:00:00"}
	{"current_steps": 225, "total_steps": 225, "epoch": 3.0, "percentage": 100.0, "elapsed_time": "0:23:14", "remaining_time": "0:00:00"}