Instructions to use jalasoft/qwen-3-4B-it-ft-typ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jalasoft/qwen-3-4B-it-ft-typ with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
model = PeftModel.from_pretrained(base_model, "jalasoft/qwen-3-4B-it-ft-typ")

Transformers

How to use jalasoft/qwen-3-4B-it-ft-typ with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jalasoft/qwen-3-4B-it-ft-typ")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("jalasoft/qwen-3-4B-it-ft-typ")
model = AutoModelForCausalLM.from_pretrained("jalasoft/qwen-3-4B-it-ft-typ", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jalasoft/qwen-3-4B-it-ft-typ with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jalasoft/qwen-3-4B-it-ft-typ"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jalasoft/qwen-3-4B-it-ft-typ",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jalasoft/qwen-3-4B-it-ft-typ

SGLang

How to use jalasoft/qwen-3-4B-it-ft-typ with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jalasoft/qwen-3-4B-it-ft-typ" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jalasoft/qwen-3-4B-it-ft-typ",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jalasoft/qwen-3-4B-it-ft-typ" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jalasoft/qwen-3-4B-it-ft-typ",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use jalasoft/qwen-3-4B-it-ft-typ with Docker Model Runner:
```
docker model run hf.co/jalasoft/qwen-3-4B-it-ft-typ
```

See axolotl config

axolotl version: 0.13.0.dev0

base_model: Qwen/Qwen3-4B-Instruct-2507
# optionally might have model_type or tokenizer_type
# Works for Qwen
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer

hub_model_id: jalasoft/qwen-3-4B-it-ft-typ

load_in_8bit: false
load_in_4bit: true

chat_template: qwen3
eot_tokens:
  - <|im_end|>
datasets:
  - path: jalasoft/typst-instruct
    type:
      system_prompt: 'You are an expert in Typst markup language. Generate clean, well-formatted Typst code based on user instructions:'
      field_instruction: prompt
      field_output: completion

val_set_size: 0.1
output_dir: /workspace-data/output

adapter: qlora
# Average 1k - 1.5 k dataset
lora_r: 32
lora_alpha: 64
lora_dropout: 0.05

# Explicit targeting for Qwen architecture
lora_target_modules:
  - q_proj
  - k_proj
  - v_proj
  - o_proj

# Support long academic papers (up to ~15-20k tokens = 16384)
# Support medium articles (up to ~5-10k tokens = 8192)
sequence_len: 4096
# Ensures consistent memory usage
pad_to_sequence_len: true
# Disable packing during evaluation for accuracy
eval_sample_packing: false
# Pack multiple samples for efficiency
sample_packing: true
# Better batching with packing enabled
multipack_real_batches: true

wandb_project: qwen-3-4B-it-ft-typ
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:

# Keep at 1 for log sequences (16k tokens)
micro_batch_size: 6
# Accumulate gradients for stable training 32 (Memory issues with 16)
gradient_accumulation_steps: 4
# Evaluation batch size
eval_batch_size: 4
# More epochs for better convergence
num_epochs: 5

# Fastest optimizer for CUDA
optimizer: adamw_torch_fused
# Cosine annealing for smooth learning rate decay
lr_scheduler: cosine
learning_rate: 8e-5

bf16: auto
tf32: true

# Note, we should define the best values for the next params in order to avoid memory leaks
gradient_checkpointing: true
gradient_checkpointing_kwargs:
  use_reentrant: false
resume_from_checkpoint:
logging_steps: 2
flash_attention: true

warmup_ratio: 0.1
weight_decay: 0.03
max_grad_norm: 1.0

evals_per_epoch: 4
saves_per_epoch: 2
special_tokens:

qwen-3-4B-it-ft-typ

This model is a fine-tuned version of Qwen/Qwen3-4B-Instruct-2507 on the jalasoft/typst-instruct dataset. It achieves the following results on the evaluation set:

Loss: 0.9541
Memory/max Active (gib): 26.06
Memory/max Allocated (gib): 26.06
Memory/device Reserved (gib): 56.71

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 8e-05
train_batch_size: 6
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 24
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 5
training_steps: 55

Training results

Training Loss	Epoch	Step	Validation Loss	Active (gib)	Allocated (gib)	Reserved (gib)
No log	0	0	1.3220	25.87	25.87	26.11
2.3749	0.2727	3	1.3128	49.04	49.04	56.71
2.2347	0.5455	6	1.2219	26.06	26.06	56.71
2.0072	0.8182	9	1.1436	49.04	49.04	56.71
1.5705	1.0909	12	1.1029	26.06	26.06	56.71
1.3271	1.3636	15	1.0747	49.04	49.04	56.71
1.2225	1.6364	18	1.0486	26.06	26.06	56.71
1.1502	1.9091	21	1.0271	49.04	49.04	56.71
1.0304	2.1818	24	1.0123	26.06	26.06	56.71
1.0389	2.4545	27	1.0006	49.04	49.04	56.71
0.9697	2.7273	30	0.9901	26.06	26.06	56.71
0.9739	3.0	33	0.9812	49.04	49.04	56.71
0.9763	3.2727	36	0.9735	26.06	26.06	56.71
0.9101	3.5455	39	0.9666	49.04	49.04	56.71
0.9482	3.8182	42	0.9612	26.06	26.06	56.71
0.8832	4.0909	45	0.9577	49.04	49.04	56.71
0.9486	4.3636	48	0.9553	26.06	26.06	56.71
0.9133	4.6364	51	0.9543	49.04	49.04	56.71
0.9085	4.9091	54	0.9541	26.06	26.06	56.71

Framework versions

PEFT 0.17.1
Transformers 4.57.1
Pytorch 2.8.0+cu128
Datasets 4.3.0
Tokenizers 0.22.1

Downloads last month: 4

Model tree for jalasoft/qwen-3-4B-it-ft-typ

Base model

Qwen/Qwen3-4B-Instruct-2507

Adapter

(5709)

this model

jalasoft
/

qwen-3-4B-it-ft-typ

qwen-3-4B-it-ft-typ

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Model tree for jalasoft/qwen-3-4B-it-ft-typ

Dataset used to train jalasoft/qwen-3-4B-it-ft-typ