Instructions to use SystemAdmin123/tiny-dummy-qwen2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SystemAdmin123/tiny-dummy-qwen2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SystemAdmin123/tiny-dummy-qwen2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SystemAdmin123/tiny-dummy-qwen2")
model = AutoModelForCausalLM.from_pretrained("SystemAdmin123/tiny-dummy-qwen2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SystemAdmin123/tiny-dummy-qwen2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SystemAdmin123/tiny-dummy-qwen2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SystemAdmin123/tiny-dummy-qwen2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SystemAdmin123/tiny-dummy-qwen2

SGLang

How to use SystemAdmin123/tiny-dummy-qwen2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SystemAdmin123/tiny-dummy-qwen2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SystemAdmin123/tiny-dummy-qwen2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SystemAdmin123/tiny-dummy-qwen2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SystemAdmin123/tiny-dummy-qwen2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use SystemAdmin123/tiny-dummy-qwen2 with Docker Model Runner:
```
docker model run hf.co/SystemAdmin123/tiny-dummy-qwen2
```

See axolotl config

axolotl version: 0.6.0

base_model: peft-internal-testing/tiny-dummy-qwen2
batch_size: 32
bf16: true
chat_template: tokenizer_default_fallback_alpaca
datasets:
- format: custom
  path: argilla/databricks-dolly-15k-curated-en
  type:
    field_input: original-instruction
    field_instruction: original-instruction
    field_output: original-response
    format: '{instruction} {input}'
    no_input_format: '{instruction}'
    system_format: '{system}'
    system_prompt: ''
device_map: auto
eval_sample_packing: false
eval_steps: 200
flash_attention: true
gpu_memory_limit: 80GiB
group_by_length: true
hub_model_id: SystemAdmin123/tiny-dummy-qwen2
hub_strategy: checkpoint
learning_rate: 0.0002
logging_steps: 10
lr_scheduler: cosine
max_steps: 2500
micro_batch_size: 4
model_type: AutoModelForCausalLM
num_epochs: 100
optimizer: adamw_bnb_8bit
output_dir: /root/.sn56/axolotl/outputs/tiny-dummy-qwen2
pad_to_sequence_len: true
resize_token_embeddings_to_32x: false
sample_packing: false
save_steps: 400
save_total_limit: 1
sequence_len: 2048
tokenizer_type: Qwen2TokenizerFast
torch_dtype: bf16
trust_remote_code: true
val_set_size: 0.1
wandb_entity: ''
wandb_mode: online
wandb_name: peft-internal-testing/tiny-dummy-qwen2-argilla/databricks-dolly-15k-curated-en
wandb_project: Gradients-On-Demand
wandb_run: your_name
wandb_runid: default
warmup_ratio: 0.05

tiny-dummy-qwen2

This model is a fine-tuned version of peft-internal-testing/tiny-dummy-qwen2 on the argilla/databricks-dolly-15k-curated-en dataset. It achieves the following results on the evaluation set:

Loss: 10.5858

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 4
eval_batch_size: 4
seed: 42
optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 125
training_steps: 2500

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.0003	1	11.9292
11.78	0.0592	200	11.7501
10.974	0.1184	400	11.0010
10.6928	0.1776	600	10.6858
10.9148	0.2368	800	10.6098
10.6606	0.2959	1000	10.5931
10.5748	0.3551	1200	10.5911
10.6436	0.4143	1400	10.5852
10.5774	0.4735	1600	10.5880
10.707	0.5327	1800	10.5812
10.5304	0.5919	2000	10.5866
10.6148	0.6511	2200	10.5842
10.4931	0.7103	2400	10.5858

Framework versions

Transformers 4.48.1
Pytorch 2.4.1+cu124
Datasets 3.2.0
Tokenizers 0.21.0

Downloads last month: 3

Safetensors

Model size

1.22M params

Tensor type

BF16

Model tree for SystemAdmin123/tiny-dummy-qwen2

Base model

peft-internal-testing/tiny-dummy-qwen2

Finetuned

(1)

this model

SystemAdmin123
/

tiny-dummy-qwen2

tiny-dummy-qwen2

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Model tree for SystemAdmin123/tiny-dummy-qwen2

Dataset used to train SystemAdmin123/tiny-dummy-qwen2