Instructions to use aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft")
model = AutoModelForCausalLM.from_pretrained("aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft

SGLang

How to use aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft with Docker Model Runner:
```
docker model run hf.co/aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft
```

aadityabuilds commited on 5 days ago

Commit

c6c3ee2

verified ·

1 Parent(s): 72b6a20

Training in progress, step 50

Browse files

Files changed (9) hide show

.gitattributes +1 -0
chat_template.jinja +54 -0
config.json +61 -0
generation_config.json +13 -0
model.safetensors +3 -0
run_metadata.json +152 -0
tokenizer.json +3 -0
tokenizer_config.json +30 -0
training_args.bin +3 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

chat_template.jinja ADDED Viewed

	@@ -0,0 +1,54 @@

+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

config.json ADDED Viewed

	@@ -0,0 +1,61 @@

+{
+  "architectures": [
+    "Qwen2ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": null,
+  "dtype": "bfloat16",
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 3584,
+  "initializer_range": 0.02,
+  "intermediate_size": 18944,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "max_window_layers": 28,
+  "model_type": "qwen2",
+  "num_attention_heads": 28,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 4,
+  "pad_token_id": 151643,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "rope_theta": 1000000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.9.0",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "vocab_size": 152064
+}

generation_config.json ADDED Viewed

	@@ -0,0 +1,13 @@

+{
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "repetition_penalty": 1.1,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "5.9.0"
+}

model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:45fd13b8984124bba7d90ffcc306cb03256f8f859e1f3b2a6289f196d330c6f2
+size 15231272152

run_metadata.json ADDED Viewed

	@@ -0,0 +1,152 @@

+{
+  "cli_args": {
+    "attn_implementation": "sdpa",
+    "auto_resume": true,
+    "bf16": true,
+    "cache_dir": "/cache",
+    "data_dir": "/workspace/data/kernelbook",
+    "deepspeed": "/workspace/configs/deepspeed/zero3_bf16.json",
+    "distillation_alpha": 0.5,
+    "distillation_topk": 100,
+    "distillation_weight": 1.0,
+    "dry_run": false,
+    "effective_batch_size": 8,
+    "eval_steps": 100,
+    "fsdp_transformer_layer_cls": "Qwen2DecoderLayer",
+    "fsdp_use_orig_params": false,
+    "generate_from_teacher": true,
+    "generation_batch_size": null,
+    "gradient_accumulation_steps": 2,
+    "gradient_checkpointing": true,
+    "hub_model_id": "aadityabuilds/qwen2-5-coder-7b-kernelbook-sdft",
+    "learning_rate": 5e-06,
+    "logging_steps": 5,
+    "max_completion_length": 4096,
+    "max_eval_samples": 256,
+    "max_grad_norm": 1.0,
+    "max_prompt_length": 4096,
+    "max_steps": -1,
+    "max_train_samples": null,
+    "model": "/cache/local-models/sdft/qwen2-5-coder-7b-instruct",
+    "num_generations": 1,
+    "num_generations_eval": null,
+    "num_loss_tokens_to_skip": 0,
+    "num_train_epochs": 1.0,
+    "output_dir": "/__modal/volumes/vo-qWxmkR9prkx4LKrjcfqOmD/modal-sdft-qwen2-5-coder-7b-kernelbook-final",
+    "output_root": "/outputs",
+    "parallel_backend": "deepspeed",
+    "per_device_eval_batch_size": 1,
+    "per_device_train_batch_size": 1,
+    "push_to_hub": true,
+    "ref_model_mixup_alpha": 0.01,
+    "ref_model_sync_steps": 128,
+    "repetition_penalty": 1.0,
+    "report_to": "wandb",
+    "resume_from_checkpoint": null,
+    "run_name": "modal-sdft-qwen2-5-coder-7b-kernelbook-final",
+    "save_steps": 50,
+    "save_total_limit": 10,
+    "seed": 42,
+    "steps_per_generation": null,
+    "sync_ref_model": false,
+    "target_global_batch_size": 8,
+    "temperature": 0.7,
+    "top_k": 0,
+    "top_p": 0.95,
+    "wandb_entity": null,
+    "wandb_mode": "online",
+    "wandb_project": "triton-sdft",
+    "warmup_ratio": 0.03,
+    "weight_decay": 0.01,
+    "world_size": 4
+  },
+  "data_dir": "/workspace/data/kernelbook",
+  "effective_batch_size": 8,
+  "manifest": {
+    "config": {
+      "created_at": "2026-05-27T05:16:47.175016+00:00",
+      "dataset_id": "GPUMODE/KernelBook",
+      "max_output_tokens": 4096,
+      "max_seq_length": 8192,
+      "model": "Qwen/Qwen2.5-Coder-7B-Instruct",
+      "output_dir": "data/kernelbook",
+      "seed": 42,
+      "test_ratio": 0.1,
+      "train_ratio": 0.8,
+      "val_ratio": 0.1
+    },
+    "counts": {
+      "after_dedup": 15203,
+      "after_empty_filter": 18162,
+      "after_output_length_filter": 13267,
+      "loaded": 18162,
+      "test": 1360,
+      "train": 10578,
+      "validation": 1329
+    },
+    "sdft_trainer": {
+      "eval_dataset": "data/kernelbook/text/sdft/validation",
+      "sdft_config_hints": {
+        "generate_from_teacher": true,
+        "max_completion_length": 4096,
+        "max_prompt_length": 4096
+      },
+      "test_dataset": "data/kernelbook/text/sdft/test",
+      "train_dataset": "data/kernelbook/text/sdft/train"
+    },
+    "sft_trainer": {
+      "eval_dataset": "data/kernelbook/tokenized/Qwen2.5-Coder-7B-Instruct/validation",
+      "eval_packing": false,
+      "packing": true,
+      "requires_columns": [
+        "input_ids",
+        "completion_mask"
+      ],
+      "sft_config": {
+        "completion_only_loss": true,
+        "eval_packing": false,
+        "max_length": 8192,
+        "packing": true
+      },
+      "test_dataset": "data/kernelbook/tokenized/Qwen2.5-Coder-7B-Instruct/test",
+      "train_dataset": "data/kernelbook/tokenized/Qwen2.5-Coder-7B-Instruct/train"
+    },
+    "token_stats": {
+      "test": {
+        "count": 1360.0,
+        "max": 6072.0,
+        "min": 519.0,
+        "p50": 1742.5,
+        "p90": 3393.1000000000013,
+        "p95": 4133.1,
+        "p99": 4980.400000000003,
+        "truncated_fraction": 0.0
+      },
+      "train": {
+        "count": 10578.0,
+        "max": 7026.0,
+        "min": 517.0,
+        "p50": 1781.5,
+        "p90": 3559.0,
+        "p95": 4168.299999999999,
+        "p99": 4932.459999999999,
+        "truncated_fraction": 0.0
+      },
+      "validation": {
+        "count": 1329.0,
+        "max": 7012.0,
+        "min": 519.0,
+        "p50": 1787.0,
+        "p90": 3371.2,
+        "p95": 3914.3999999999996,
+        "p99": 4647.0,
+        "truncated_fraction": 0.0
+      }
+    }
+  },
+  "method": "sdft",
+  "model": "/cache/local-models/sdft/qwen2-5-coder-7b-instruct",
+  "output_dir": "/__modal/volumes/vo-qWxmkR9prkx4LKrjcfqOmD/modal-sdft-qwen2-5-coder-7b-kernelbook-final",
+  "run_name": "modal-sdft-qwen2-5-coder-7b-kernelbook-final",
+  "world_size": 4
+}

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,30 @@

+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:66c0ded08b47fd10e2f13d0c792f48eec7e1ce46dd22560ac64771efe1f05d75
+size 6993