Instructions to use PARZ2344/web_llama_sft_random with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PARZ2344/web_llama_sft_random with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="PARZ2344/web_llama_sft_random")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("PARZ2344/web_llama_sft_random")
model = AutoModelForCausalLM.from_pretrained("PARZ2344/web_llama_sft_random")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use PARZ2344/web_llama_sft_random with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PARZ2344/web_llama_sft_random"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PARZ2344/web_llama_sft_random",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/PARZ2344/web_llama_sft_random

SGLang

How to use PARZ2344/web_llama_sft_random with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "PARZ2344/web_llama_sft_random" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PARZ2344/web_llama_sft_random",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "PARZ2344/web_llama_sft_random" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PARZ2344/web_llama_sft_random",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use PARZ2344/web_llama_sft_random with Docker Model Runner:
```
docker model run hf.co/PARZ2344/web_llama_sft_random
```

web_llama_sft_random / trainer_log.jsonl

PARZ2344

Duplicate from zizi-0123/web_llama_sft_random

249504b about 2 months ago

raw

history blame contribute delete

6.14 kB

	{"current_steps": 610, "total_steps": 915, "loss": 0.8327, "lr": 3.0403920675946826e-06, "epoch": 2.0032813781788352, "percentage": 66.67, "elapsed_time": "0:02:00", "remaining_time": "0:01:00"}
	{"current_steps": 620, "total_steps": 915, "loss": 0.7782, "lr": 2.8662692339278387e-06, "epoch": 2.036095159967186, "percentage": 67.76, "elapsed_time": "0:03:54", "remaining_time": "0:01:51"}
	{"current_steps": 630, "total_steps": 915, "loss": 0.7875, "lr": 2.6952551585633947e-06, "epoch": 2.0689089417555375, "percentage": 68.85, "elapsed_time": "0:05:30", "remaining_time": "0:02:29"}
	{"current_steps": 640, "total_steps": 915, "loss": 0.7708, "lr": 2.52759900200232e-06, "epoch": 2.1017227235438884, "percentage": 69.95, "elapsed_time": "0:07:31", "remaining_time": "0:03:14"}
	{"current_steps": 650, "total_steps": 915, "loss": 0.7927, "lr": 2.3635450323954773e-06, "epoch": 2.1345365053322394, "percentage": 71.04, "elapsed_time": "0:09:09", "remaining_time": "0:03:43"}
	{"current_steps": 660, "total_steps": 915, "loss": 0.7885, "lr": 2.2033322696549197e-06, "epoch": 2.1673502871205907, "percentage": 72.13, "elapsed_time": "0:10:50", "remaining_time": "0:04:11"}
	{"current_steps": 670, "total_steps": 915, "loss": 0.7626, "lr": 2.0471941372116793e-06, "epoch": 2.2001640689089417, "percentage": 73.22, "elapsed_time": "0:12:42", "remaining_time": "0:04:38"}
	{"current_steps": 680, "total_steps": 915, "loss": 0.7754, "lr": 1.8953581219273987e-06, "epoch": 2.232977850697293, "percentage": 74.32, "elapsed_time": "0:14:33", "remaining_time": "0:05:01"}
	{"current_steps": 690, "total_steps": 915, "loss": 0.7783, "lr": 1.7480454426552773e-06, "epoch": 2.265791632485644, "percentage": 75.41, "elapsed_time": "0:16:17", "remaining_time": "0:05:18"}
	{"current_steps": 700, "total_steps": 915, "loss": 0.7705, "lr": 1.6054707279332865e-06, "epoch": 2.298605414273995, "percentage": 76.5, "elapsed_time": "0:18:15", "remaining_time": "0:05:36"}
	{"current_steps": 710, "total_steps": 915, "loss": 0.7699, "lr": 1.4678417032791653e-06, "epoch": 2.3314191960623463, "percentage": 77.6, "elapsed_time": "0:19:52", "remaining_time": "0:05:44"}
	{"current_steps": 720, "total_steps": 915, "loss": 0.7526, "lr": 1.335358888542862e-06, "epoch": 2.364232977850697, "percentage": 78.69, "elapsed_time": "0:21:39", "remaining_time": "0:05:51"}
	{"current_steps": 730, "total_steps": 915, "loss": 0.7918, "lr": 1.20821530575733e-06, "epoch": 2.3970467596390486, "percentage": 79.78, "elapsed_time": "0:23:35", "remaining_time": "0:05:58"}
	{"current_steps": 740, "total_steps": 915, "loss": 0.7815, "lr": 1.0865961979133245e-06, "epoch": 2.4298605414273995, "percentage": 80.87, "elapsed_time": "0:25:10", "remaining_time": "0:05:57"}
	{"current_steps": 750, "total_steps": 915, "loss": 0.7731, "lr": 9.706787590679685e-07, "epoch": 2.462674323215751, "percentage": 81.97, "elapsed_time": "0:26:39", "remaining_time": "0:05:51"}
	{"current_steps": 760, "total_steps": 915, "loss": 0.7666, "lr": 8.606318761802584e-07, "epoch": 2.495488105004102, "percentage": 83.06, "elapsed_time": "0:28:16", "remaining_time": "0:05:45"}
	{"current_steps": 770, "total_steps": 915, "loss": 0.7657, "lr": 7.566158830496917e-07, "epoch": 2.5283018867924527, "percentage": 84.15, "elapsed_time": "0:29:58", "remaining_time": "0:05:38"}
	{"current_steps": 780, "total_steps": 915, "loss": 0.7798, "lr": 6.587823267164911e-07, "epoch": 2.561115668580804, "percentage": 85.25, "elapsed_time": "0:31:27", "remaining_time": "0:05:26"}
	{"current_steps": 790, "total_steps": 915, "loss": 0.7816, "lr": 5.672737466637701e-07, "epoch": 2.593929450369155, "percentage": 86.34, "elapsed_time": "0:33:28", "remaining_time": "0:05:17"}
	{"current_steps": 800, "total_steps": 915, "loss": 0.7837, "lr": 4.822234671433552e-07, "epoch": 2.626743232157506, "percentage": 87.43, "elapsed_time": "0:35:41", "remaining_time": "0:05:07"}
	{"current_steps": 810, "total_steps": 915, "loss": 0.7747, "lr": 4.03755402927804e-07, "epoch": 2.6595570139458573, "percentage": 88.52, "elapsed_time": "0:39:29", "remaining_time": "0:05:07"}
	{"current_steps": 820, "total_steps": 915, "loss": 0.7793, "lr": 3.319838787716634e-07, "epoch": 2.6923707957342082, "percentage": 89.62, "elapsed_time": "0:41:31", "remaining_time": "0:04:48"}
	{"current_steps": 830, "total_steps": 915, "loss": 0.7542, "lr": 2.6701346284499e-07, "epoch": 2.7251845775225596, "percentage": 90.71, "elapsed_time": "0:43:29", "remaining_time": "0:04:27"}
	{"current_steps": 840, "total_steps": 915, "loss": 0.7844, "lr": 2.0893881438180275e-07, "epoch": 2.7579983593109105, "percentage": 91.8, "elapsed_time": "0:45:26", "remaining_time": "0:04:03"}
	{"current_steps": 850, "total_steps": 915, "loss": 0.7643, "lr": 1.578445457654637e-07, "epoch": 2.790812141099262, "percentage": 92.9, "elapsed_time": "0:47:15", "remaining_time": "0:03:36"}
	{"current_steps": 860, "total_steps": 915, "loss": 0.7673, "lr": 1.1380509925189853e-07, "epoch": 2.823625922887613, "percentage": 93.99, "elapsed_time": "0:49:01", "remaining_time": "0:03:08"}
	{"current_steps": 870, "total_steps": 915, "loss": 0.769, "lr": 7.688463851028227e-08, "epoch": 2.8564397046759638, "percentage": 95.08, "elapsed_time": "0:51:01", "remaining_time": "0:02:38"}
	{"current_steps": 880, "total_steps": 915, "loss": 0.7799, "lr": 4.713695513920147e-08, "epoch": 2.889253486464315, "percentage": 96.17, "elapsed_time": "0:52:45", "remaining_time": "0:02:05"}
	{"current_steps": 890, "total_steps": 915, "loss": 0.785, "lr": 2.4605390294497043e-08, "epoch": 2.922067268252666, "percentage": 97.27, "elapsed_time": "0:54:45", "remaining_time": "0:01:32"}
	{"current_steps": 900, "total_steps": 915, "loss": 0.7753, "lr": 9.322771542978892e-09, "epoch": 2.954881050041017, "percentage": 98.36, "elapsed_time": "0:56:44", "remaining_time": "0:00:56"}
	{"current_steps": 910, "total_steps": 915, "loss": 0.7911, "lr": 1.3113650340046413e-09, "epoch": 2.9876948318293683, "percentage": 99.45, "elapsed_time": "0:58:10", "remaining_time": "0:00:19"}
	{"current_steps": 914, "total_steps": 915, "epoch": 3.0, "percentage": 99.89, "elapsed_time": "0:58:47", "remaining_time": "0:00:03"}