Instructions to use artificialguybr/QWEN-2.5-0.5B-Synthia-II with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use artificialguybr/QWEN-2.5-0.5B-Synthia-II with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="artificialguybr/QWEN-2.5-0.5B-Synthia-II")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("artificialguybr/QWEN-2.5-0.5B-Synthia-II")
model = AutoModelForCausalLM.from_pretrained("artificialguybr/QWEN-2.5-0.5B-Synthia-II")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use artificialguybr/QWEN-2.5-0.5B-Synthia-II with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "artificialguybr/QWEN-2.5-0.5B-Synthia-II"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "artificialguybr/QWEN-2.5-0.5B-Synthia-II",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/artificialguybr/QWEN-2.5-0.5B-Synthia-II

SGLang

How to use artificialguybr/QWEN-2.5-0.5B-Synthia-II with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "artificialguybr/QWEN-2.5-0.5B-Synthia-II" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "artificialguybr/QWEN-2.5-0.5B-Synthia-II",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "artificialguybr/QWEN-2.5-0.5B-Synthia-II" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "artificialguybr/QWEN-2.5-0.5B-Synthia-II",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use artificialguybr/QWEN-2.5-0.5B-Synthia-II with Docker Model Runner:
```
docker model run hf.co/artificialguybr/QWEN-2.5-0.5B-Synthia-II
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen2.5-0.5B Synthia Fine-tuned Model

This is a fine-tuned version of Qwen/Qwen2.5-0.5B on the Synthia-v1.5-II dataset, optimized for conversational AI and instruction following.

Model Description

This model builds upon the powerful Qwen2.5-0.5B base model, which features:

490M parameters (360M non-embedding parameters)
24 transformer layers
14 attention heads for queries and 2 for key/values (GQA architecture)

🌐 Website

You can find more of my models, projects, and information on my official website:

artificialguy.com

🚀 Prompt Hub

Need high-quality prompts for image models and LLMs? Explore findgoodprompt.com.

💖 Support My Work

If you find this model useful, please consider supporting my work. It helps me cover server costs and dedicate more time to new open-source projects.

Patreon: Support on Patreon
Ko-fi: Buy me a Ko-fi
Buy Me a Coffee: Buy me a Coffee
Support for 32,768 context length
Advanced features like RoPE positional embeddings, SwiGLU activations, and RMSNorm

The model has been fine-tuned on the Synthia-v1.5-II dataset, which is designed to enhance instruction following and conversational abilities. The training process used careful hyperparameter tuning to maintain the base model's capabilities while optimizing for natural dialogue and instruction following.

Intended Uses & Limitations

This model is intended for:

Conversational AI applications
Instruction following tasks
Text generation with strong coherence
Multi-turn dialogue systems

Limitations:

The model inherits the 32K token context window from the base model
As a 0.5B parameter model, it may not match larger models in complex reasoning tasks
Performance in non-English languages may be limited
Users should be aware of potential biases present in the training data

Training and Evaluation Data

The model was fine-tuned on the Synthia-v1.5-II dataset, which is specifically designed for instruction-following and conversational AI. The training process used:

95% of data for training
5% for validation
Instruction format: "[INST] {instruction} [/INST]"

Training Procedure

Training Hyperparameters

Key hyperparameters:

Learning rate: 1e-05
Batch size: 40 (5 micro-batch × 8 gradient accumulation steps)
Training epochs: 3
Optimizer: AdamW (β1=0.9, β2=0.999, ε=1e-8)
Learning rate scheduler: Cosine with 100 warmup steps
Sequence length: 4096
Sample packing: Enabled
Mixed precision: BF16

Training Results

The model was trained for 672 steps over 3 epochs, showing consistent improvement throughout the training process.

Framework Versions

Transformers 4.46.0
PyTorch 2.3.1+cu121
Datasets 3.0.1
Tokenizers 0.20.1

Citation

If you use this model, please cite both the original Qwen2.5 work and this fine-tuned version:

Downloads last month: 1,772

Model tree for artificialguybr/QWEN-2.5-0.5B-Synthia-II

Base model

Qwen/Qwen2.5-0.5B

Finetuned

(628)

this model

Quantizations

3 models

artificialguybr
/

QWEN-2.5-0.5B-Synthia-II