Instructions to use 17Lab/qwen7b-lora-sft-s13 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 17Lab/qwen7b-lora-sft-s13 with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B")
model = PeftModel.from_pretrained(base_model, "17Lab/qwen7b-lora-sft-s13")

Transformers

How to use 17Lab/qwen7b-lora-sft-s13 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="17Lab/qwen7b-lora-sft-s13")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("17Lab/qwen7b-lora-sft-s13", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 17Lab/qwen7b-lora-sft-s13 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "17Lab/qwen7b-lora-sft-s13"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "17Lab/qwen7b-lora-sft-s13",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/17Lab/qwen7b-lora-sft-s13

SGLang

How to use 17Lab/qwen7b-lora-sft-s13 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "17Lab/qwen7b-lora-sft-s13" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "17Lab/qwen7b-lora-sft-s13",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "17Lab/qwen7b-lora-sft-s13" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "17Lab/qwen7b-lora-sft-s13",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 17Lab/qwen7b-lora-sft-s13 with Docker Model Runner:
```
docker model run hf.co/17Lab/qwen7b-lora-sft-s13
```

qwen7b-lora-sft-s13 / trainer_log.jsonl

Yiqi-Liu

Upload folder using huggingface_hub

bfcb463 verified about 1 month ago

Raw

History Blame Contribute Delete

10.1 kB

	{"current_steps": 5, "total_steps": 225, "loss": 3.553193283081055, "lr": 3.3333333333333335e-05, "epoch": 0.06666666666666667, "percentage": 2.22, "elapsed_time": "0:00:04", "remaining_time": "0:03:14"}
	{"current_steps": 10, "total_steps": 225, "loss": 2.196733856201172, "lr": 7.500000000000001e-05, "epoch": 0.13333333333333333, "percentage": 4.44, "elapsed_time": "0:00:07", "remaining_time": "0:02:35"}
	{"current_steps": 15, "total_steps": 225, "loss": 0.5981613159179687, "lr": 9.997824750657585e-05, "epoch": 0.2, "percentage": 6.67, "elapsed_time": "0:00:10", "remaining_time": "0:02:21"}
	{"current_steps": 20, "total_steps": 225, "loss": 0.25100862979888916, "lr": 9.973374926646116e-05, "epoch": 0.26666666666666666, "percentage": 8.89, "elapsed_time": "0:00:12", "remaining_time": "0:02:12"}
	{"current_steps": 25, "total_steps": 225, "loss": 0.21380953788757323, "lr": 9.921889571113628e-05, "epoch": 0.3333333333333333, "percentage": 11.11, "elapsed_time": "0:00:15", "remaining_time": "0:02:06"}
	{"current_steps": 30, "total_steps": 225, "loss": 0.19729268550872803, "lr": 9.843648561200475e-05, "epoch": 0.4, "percentage": 13.33, "elapsed_time": "0:00:18", "remaining_time": "0:02:00"}
	{"current_steps": 35, "total_steps": 225, "loss": 0.19206844568252562, "lr": 9.73907721921212e-05, "epoch": 0.4666666666666667, "percentage": 15.56, "elapsed_time": "0:00:21", "remaining_time": "0:01:56"}
	{"current_steps": 40, "total_steps": 225, "loss": 0.1841310977935791, "lr": 9.608744000544392e-05, "epoch": 0.5333333333333333, "percentage": 17.78, "elapsed_time": "0:00:24", "remaining_time": "0:01:52"}
	{"current_steps": 45, "total_steps": 225, "loss": 0.16237072944641112, "lr": 9.453357403529609e-05, "epoch": 0.6, "percentage": 20.0, "elapsed_time": "0:00:27", "remaining_time": "0:01:48"}
	{"current_steps": 50, "total_steps": 225, "loss": 0.1566811203956604, "lr": 9.273762118001837e-05, "epoch": 0.6666666666666666, "percentage": 22.22, "elapsed_time": "0:00:29", "remaining_time": "0:01:44"}
	{"current_steps": 50, "total_steps": 225, "eval_loss": 0.15606221556663513, "epoch": 0.6666666666666666, "percentage": 22.22, "elapsed_time": "0:00:30", "remaining_time": "0:01:47"}
	{"current_steps": 55, "total_steps": 225, "loss": 0.15580005645751954, "lr": 9.070934433517873e-05, "epoch": 0.7333333333333333, "percentage": 24.44, "elapsed_time": "0:00:39", "remaining_time": "0:02:02"}
	{"current_steps": 60, "total_steps": 225, "loss": 0.1553624153137207, "lr": 8.845976932195103e-05, "epoch": 0.8, "percentage": 26.67, "elapsed_time": "0:00:42", "remaining_time": "0:01:56"}
	{"current_steps": 65, "total_steps": 225, "loss": 0.15475208759307862, "lr": 8.600112495016288e-05, "epoch": 0.8666666666666667, "percentage": 28.89, "elapsed_time": "0:00:45", "remaining_time": "0:01:51"}
	{"current_steps": 70, "total_steps": 225, "loss": 0.15437995195388793, "lr": 8.334677654183254e-05, "epoch": 0.9333333333333333, "percentage": 31.11, "elapsed_time": "0:00:48", "remaining_time": "0:01:46"}
	{"current_steps": 75, "total_steps": 225, "loss": 0.1542748212814331, "lr": 8.051115327656538e-05, "epoch": 1.0, "percentage": 33.33, "elapsed_time": "0:00:50", "remaining_time": "0:01:41"}
	{"current_steps": 80, "total_steps": 225, "loss": 0.15384273529052733, "lr": 7.750966975376328e-05, "epoch": 1.0666666666666667, "percentage": 35.56, "elapsed_time": "0:00:53", "remaining_time": "0:01:37"}
	{"current_steps": 85, "total_steps": 225, "loss": 0.153655743598938, "lr": 7.435864219803983e-05, "epoch": 1.1333333333333333, "percentage": 37.78, "elapsed_time": "0:00:56", "remaining_time": "0:01:33"}
	{"current_steps": 90, "total_steps": 225, "loss": 0.15383241176605225, "lr": 7.10751997633524e-05, "epoch": 1.2, "percentage": 40.0, "elapsed_time": "0:00:59", "remaining_time": "0:01:29"}
	{"current_steps": 95, "total_steps": 225, "loss": 0.1533586263656616, "lr": 6.767719141800717e-05, "epoch": 1.2666666666666666, "percentage": 42.22, "elapsed_time": "0:01:02", "remaining_time": "0:01:25"}
	{"current_steps": 100, "total_steps": 225, "loss": 0.1532520055770874, "lr": 6.418308891671484e-05, "epoch": 1.3333333333333333, "percentage": 44.44, "elapsed_time": "0:01:05", "remaining_time": "0:01:21"}
	{"current_steps": 100, "total_steps": 225, "eval_loss": 0.15319058299064636, "epoch": 1.3333333333333333, "percentage": 44.44, "elapsed_time": "0:01:05", "remaining_time": "0:01:22"}
	{"current_steps": 105, "total_steps": 225, "loss": 0.15315526723861694, "lr": 6.0611886387146156e-05, "epoch": 1.4, "percentage": 46.67, "elapsed_time": "0:01:15", "remaining_time": "0:01:26"}
	{"current_steps": 110, "total_steps": 225, "loss": 0.15307013988494872, "lr": 5.6982997076840306e-05, "epoch": 1.4666666666666668, "percentage": 48.89, "elapsed_time": "0:01:18", "remaining_time": "0:01:22"}
	{"current_steps": 115, "total_steps": 225, "loss": 0.15306075811386108, "lr": 5.3316147821755205e-05, "epoch": 1.5333333333333332, "percentage": 51.11, "elapsed_time": "0:01:21", "remaining_time": "0:01:18"}
	{"current_steps": 120, "total_steps": 225, "loss": 0.15295226573944093, "lr": 4.963127181013438e-05, "epoch": 1.6, "percentage": 53.33, "elapsed_time": "0:01:24", "remaining_time": "0:01:13"}
	{"current_steps": 125, "total_steps": 225, "loss": 0.15291054248809816, "lr": 4.5948400224632e-05, "epoch": 1.6666666666666665, "percentage": 55.56, "elapsed_time": "0:01:27", "remaining_time": "0:01:09"}
	{"current_steps": 130, "total_steps": 225, "loss": 0.15286383628845215, "lr": 4.228755335173488e-05, "epoch": 1.7333333333333334, "percentage": 57.78, "elapsed_time": "0:01:30", "remaining_time": "0:01:05"}
	{"current_steps": 135, "total_steps": 225, "loss": 0.15282795429229737, "lr": 3.866863175041666e-05, "epoch": 1.8, "percentage": 60.0, "elapsed_time": "0:01:32", "remaining_time": "0:01:01"}
	{"current_steps": 140, "total_steps": 225, "loss": 0.1527980923652649, "lr": 3.511130807163724e-05, "epoch": 1.8666666666666667, "percentage": 62.22, "elapsed_time": "0:01:35", "remaining_time": "0:00:58"}
	{"current_steps": 145, "total_steps": 225, "loss": 0.15277599096298217, "lr": 3.1634920116762176e-05, "epoch": 1.9333333333333333, "percentage": 64.44, "elapsed_time": "0:01:38", "remaining_time": "0:00:54"}
	{"current_steps": 150, "total_steps": 225, "loss": 0.15276691913604737, "lr": 2.825836571624254e-05, "epoch": 2.0, "percentage": 66.67, "elapsed_time": "0:01:41", "remaining_time": "0:00:50"}
	{"current_steps": 150, "total_steps": 225, "eval_loss": 0.15274152159690857, "epoch": 2.0, "percentage": 66.67, "elapsed_time": "0:01:42", "remaining_time": "0:00:51"}
	{"current_steps": 155, "total_steps": 225, "loss": 0.1527329206466675, "lr": 2.500000000000001e-05, "epoch": 2.066666666666667, "percentage": 68.89, "elapsed_time": "0:01:51", "remaining_time": "0:00:50"}
	{"current_steps": 160, "total_steps": 225, "loss": 0.15271611213684083, "lr": 2.187753561796097e-05, "epoch": 2.1333333333333333, "percentage": 71.11, "elapsed_time": "0:01:53", "remaining_time": "0:00:46"}
	{"current_steps": 165, "total_steps": 225, "loss": 0.152712082862854, "lr": 1.890794645314633e-05, "epoch": 2.2, "percentage": 73.33, "elapsed_time": "0:01:56", "remaining_time": "0:00:42"}
	{"current_steps": 170, "total_steps": 225, "loss": 0.1527017116546631, "lr": 1.6107375350737437e-05, "epoch": 2.2666666666666666, "percentage": 75.56, "elapsed_time": "0:01:59", "remaining_time": "0:00:38"}
	{"current_steps": 175, "total_steps": 225, "loss": 0.15268385410308838, "lr": 1.3491046364708293e-05, "epoch": 2.3333333333333335, "percentage": 77.78, "elapsed_time": "0:02:02", "remaining_time": "0:00:34"}
	{"current_steps": 180, "total_steps": 225, "loss": 0.15267916917800903, "lr": 1.1073181999055538e-05, "epoch": 2.4, "percentage": 80.0, "elapsed_time": "0:02:05", "remaining_time": "0:00:31"}
	{"current_steps": 185, "total_steps": 225, "loss": 0.152678918838501, "lr": 8.866925893507805e-06, "epoch": 2.466666666666667, "percentage": 82.22, "elapsed_time": "0:02:08", "remaining_time": "0:00:27"}
	{"current_steps": 190, "total_steps": 225, "loss": 0.15267302989959716, "lr": 6.8842713739986075e-06, "epoch": 2.533333333333333, "percentage": 84.44, "elapsed_time": "0:02:10", "remaining_time": "0:00:24"}
	{"current_steps": 195, "total_steps": 225, "loss": 0.15266704559326172, "lr": 5.135996256306619e-06, "epoch": 2.6, "percentage": 86.67, "elapsed_time": "0:02:13", "remaining_time": "0:00:20"}
	{"current_steps": 200, "total_steps": 225, "loss": 0.15267045497894288, "lr": 3.631604257273774e-06, "epoch": 2.6666666666666665, "percentage": 88.89, "elapsed_time": "0:02:16", "remaining_time": "0:00:17"}
	{"current_steps": 200, "total_steps": 225, "eval_loss": 0.15265707671642303, "epoch": 2.6666666666666665, "percentage": 88.89, "elapsed_time": "0:02:17", "remaining_time": "0:00:17"}
	{"current_steps": 205, "total_steps": 225, "loss": 0.15267107486724854, "lr": 2.3792733320934346e-06, "epoch": 2.7333333333333334, "percentage": 91.11, "elapsed_time": "0:02:26", "remaining_time": "0:00:14"}
	{"current_steps": 210, "total_steps": 225, "loss": 0.1526672959327698, "lr": 1.3858112185094418e-06, "epoch": 2.8, "percentage": 93.33, "elapsed_time": "0:02:29", "remaining_time": "0:00:10"}
	{"current_steps": 215, "total_steps": 225, "loss": 0.1526732087135315, "lr": 6.566184295904776e-07, "epoch": 2.8666666666666667, "percentage": 95.56, "elapsed_time": "0:02:31", "remaining_time": "0:00:07"}
	{"current_steps": 220, "total_steps": 225, "loss": 0.15266685485839843, "lr": 1.9565889625275946e-07, "epoch": 2.9333333333333336, "percentage": 97.78, "elapsed_time": "0:02:34", "remaining_time": "0:00:03"}
	{"current_steps": 225, "total_steps": 225, "loss": 0.15266673564910888, "lr": 5.438419120062932e-09, "epoch": 3.0, "percentage": 100.0, "elapsed_time": "0:02:37", "remaining_time": "0:00:00"}
	{"current_steps": 225, "total_steps": 225, "eval_loss": 0.1526603251695633, "epoch": 3.0, "percentage": 100.0, "elapsed_time": "0:02:38", "remaining_time": "0:00:00"}
	{"current_steps": 225, "total_steps": 225, "epoch": 3.0, "percentage": 100.0, "elapsed_time": "0:02:44", "remaining_time": "0:00:00"}