Instructions to use DedeProGames/Wisp-15M-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DedeProGames/Wisp-15M-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DedeProGames/Wisp-15M-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DedeProGames/Wisp-15M-Instruct")
model = AutoModelForCausalLM.from_pretrained("DedeProGames/Wisp-15M-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DedeProGames/Wisp-15M-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DedeProGames/Wisp-15M-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DedeProGames/Wisp-15M-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DedeProGames/Wisp-15M-Instruct

SGLang

How to use DedeProGames/Wisp-15M-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DedeProGames/Wisp-15M-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DedeProGames/Wisp-15M-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DedeProGames/Wisp-15M-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DedeProGames/Wisp-15M-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DedeProGames/Wisp-15M-Instruct with Docker Model Runner:
```
docker model run hf.co/DedeProGames/Wisp-15M-Instruct
```

DedeProGames commited on 12 days ago

Commit

dd3c59f

verified ·

1 Parent(s): 9f22815

Card wording: GPQA sampling temperature; LiveBench rounding

Browse files

Files changed (1) hide show

README.md +21 -21

README.md CHANGED Viewed

@@ -4,7 +4,7 @@ language:
 - en
 library_name: transformers
 pipeline_tag: text-generation
-base_model: DedeProGames/Wisp-15M
 datasets:
 - HuggingFaceTB/smol-smoltalk
 tags:
@@ -16,21 +16,21 @@ tags:
 - llama
 ---
-# Wisp-15M-Instruct
-An instruction-tuned version of [Wisp-15M](https://huggingface.co/DedeProGames/Wisp-15M), a 15.5M-parameter LLaMA-style model pretrained from scratch. It was fully fine-tuned for two epochs on [smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk) with the loss computed only on assistant tokens (end-of-turn tokens weighted 5×, so the model learns to stop).
-At 15.5M parameters this is a small research model. It follows the chat format and answers simple requests, but factual knowledge, reasoning and arithmetic are weak. See the limitations below.
 ## Quickstart
-Use the recommended sampling settings (stored in `generation_config.json`). With near-greedy decoding this model can fall into repetition loops.
 ```python
 import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
-repo = "DedeProGames/Wisp-15M-Instruct"
 tokenizer = AutoTokenizer.from_pretrained(repo)
 model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.float32).eval()
@@ -57,41 +57,41 @@ The Wisp tokenizer already has `<|im_start|>` and `<|im_end|>`, so the model use
 ## Architecture
-Same as Wisp-15M: `LlamaForCausalLM`, 12 layers, hidden size 320, 10 heads with 2 KV heads (head dim 32), SwiGLU (intermediate 864), RMSNorm, RoPE (θ = 10,000), tied embeddings, 8,192-token byte-level BPE vocabulary, 2,048-token context. 15,531,840 parameters. Weights in BF16 (`model.safetensors`, 31 MB).
 ## Training
 | | |
 |---|---|
-| Base | DedeProGames/Wisp-15M |
-| Data | smol-smoltalk train split, conversations of up to 1,024 tokens (232,008 with this tokenizer; 71.0M supervised tokens) |
 | Epochs | 2 (14,502 steps, batch 32 = 16 × 2 gradient accumulation) |
 | Optimizer | AdamW (β = 0.9, 0.95), weight decay 0.1 on 2D matrices, grad clip 1.0 |
 | Schedule | lr 2e-4, 300 warmup steps, cosine to 2e-5 |
 | Loss | assistant tokens; `<|im_end|>` weighted 5× |
 | Precision | bf16 autocast, fp32 master weights |
-| Hardware | 1× NVIDIA RTX 3060 12 GB, 96 minutes |
-Held-out loss on assistant tokens (smol-smoltalk test split): 1.486 at the end of training.
 ## Evaluation
-Benchmarks follow the protocol of the [Kiyo-230M-Instruct](https://huggingface.co/DedeProGames/Kiyo-230M-Instruct) card: FP16, the model's own chat template, no system prompt, new-token limits IFEval 1,024, GPQA 384, LiveBench 512, and the model's full 2,048-token context. The main setting uses temperature 0.1, top-k 50 and repetition penalty 1.05, as on the card. The second setting is the recommended one shipped in `generation_config.json`.
-| Benchmark | Metric | Wisp-15M-Instruct, protocol | Wisp-15M-Instruct, recommended sampling |
 |---|---|---:|---:|
-| GPQA Diamond | pass@1 (198 × 5 samples, T = 0.6) | 12.32 | not run |
-| IFEval | prompt-level strict | 18.30 | 15.90 |
-| LiveBench instruction following | strict, 200 questions | 17.55 | not run |
 | CaseReportBench | token F1 | not run (not on the Hub) | not run |
-| EOS rate (IFEval) | share of answers that stop | 0.89 | 0.99 |
-For reference, the Kiyo card reports (not re-run here): Kiyo-230M-Instruct GPQA 21.11, IFEval 17.38, LiveBench 18.77; SmolLM2-135M-Instruct GPQA 16.57, IFEval 20.89, LiveBench 24.23; BananaMind-2-Pro GPQA 15.05, IFEval 20.15, LiveBench 19.75; Supra2-100M-Instruct GPQA 5.56, IFEval 9.06, LiveBench 11.28.
 Notes:
-- With the protocol setting, about 11% of IFEval answers never stop and loop. Under the recommended sampling this drops to about 1%, at a cost of about 2 IFEval points.
-- GPQA: among answered questions, accuracy is 22.1%, close to the 25% chance level. The model cannot solve these questions.
 - The GPQA prompt was written for this run, not taken from the Kiyo card, so the number is not strictly comparable.
 - Single run, single seed. Differences of a point or two are within noise.
@@ -99,7 +99,7 @@ Per-sample outputs and summaries are in [`eval/`](eval/).
 ## Limitations
-- It is small (15.5M parameters). Expect factual errors, invented details, arithmetic mistakes, weak multi-step reasoning, and repetitive text in long answers.
 - Under near-greedy decoding it can loop. Keep the recommended sampling settings.
 - Context is 2,048 tokens, but it was fine-tuned on conversations of up to 1,024 tokens.
 - English only.

 - en
 library_name: transformers
 pipeline_tag: text-generation
+base_model: DedeProGames/Wisp-5M
 datasets:
 - HuggingFaceTB/smol-smoltalk
 tags:
 - llama
 ---
+# Wisp-5M-Instruct
+An instruction-tuned version of [Wisp-5M](https://huggingface.co/DedeProGames/Wisp-5M), a 5.1M-parameter LLaMA-style model pretrained from scratch. It was fully fine-tuned for two epochs on [smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk) with the loss computed only on assistant tokens (end-of-turn tokens weighted 5×, so the model learns to stop).
+At 5M parameters this is a tiny research model. It follows the chat format and answers simple requests, but factual knowledge, reasoning and arithmetic are very weak. See the limitations below.
 ## Quickstart
+Use the recommended sampling settings (they are stored in `generation_config.json`). With near-greedy decoding this model often falls into repetition loops.
 ```python
 import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
+repo = "DedeProGames/Wisp-5M-Instruct"
 tokenizer = AutoTokenizer.from_pretrained(repo)
 model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.float32).eval()
 ## Architecture
+Same as Wisp-5M: `LlamaForCausalLM`, 9 layers, hidden size 192, 6 heads with 2 KV heads, SwiGLU (intermediate 512), RMSNorm, RoPE (θ = 10,000), tied embeddings, 8,192-token byte-level BPE vocabulary, 2,048-token context. Weights in BF16 (`model.safetensors`, 20 MB).
 ## Training
 | | |
 |---|---|
+| Base | DedeProGames/Wisp-5M |
+| Data | smol-smoltalk train split, conversations of up to 1,024 tokens (with this tokenizer) |
 | Epochs | 2 (14,502 steps, batch 32 = 16 × 2 gradient accumulation) |
 | Optimizer | AdamW (β = 0.9, 0.95), weight decay 0.1 on 2D matrices, grad clip 1.0 |
 | Schedule | lr 2e-4, 300 warmup steps, cosine to 2e-5 |
 | Loss | assistant tokens; `<|im_end|>` weighted 5× |
 | Precision | bf16 autocast, fp32 master weights |
+| Hardware | 1× NVIDIA RTX 3060 12 GB, 47 minutes |
+Held-out loss on assistant tokens (smol-smoltalk test split): 1.791 at the end of training.
 ## Evaluation
+Benchmarks follow the protocol of the [Kiyo-230M-Instruct](https://huggingface.co/DedeProGames/Kiyo-230M-Instruct) card: FP16, the model's own chat template, no system prompt, new-token limits IFEval 1,024, GPQA 384, LiveBench 512, and the model's full 2,048-token context. The main setting uses temperature 0.1, top-k 50 and repetition penalty 1.05, as on the card (GPQA is sampled at T = 0.6, as on the card). The second setting is the recommended one shipped in `generation_config.json`.
+| Benchmark | Metric | Wisp-5M-Instruct, protocol | Wisp-5M-Instruct, recommended sampling |
 |---|---|---:|---:|
+| GPQA Diamond | pass@1 (198 × 5 samples, T = 0.6) | 9.80 | not run |
+| IFEval | prompt-level strict | 12.38 | 11.65 |
+| LiveBench instruction following | strict, 200 questions | 16.53 | not run |
 | CaseReportBench | token F1 | not run (not on the Hub) | not run |
+| EOS rate (IFEval) | share of answers that stop | 0.82 | 0.98 |
+For reference, the Kiyo card reports (not re-run here): Supra2-100M-Instruct GPQA 5.56, IFEval 9.06, LiveBench 11.28; SmolLM2-135M-Instruct GPQA 16.57, IFEval 20.89, LiveBench 24.23.
 Notes:
+- With the protocol setting, about 18% of IFEval answers never stop and loop on a sentence. Under the recommended sampling this drops to about 2%.
+- GPQA: among answered questions, accuracy is 19.5%, which is below the 25% chance level. The model cannot solve these questions.
 - The GPQA prompt was written for this run, not taken from the Kiyo card, so the number is not strictly comparable.
 - Single run, single seed. Differences of a point or two are within noise.
 ## Limitations
+- It is tiny (5.1M parameters). Expect factual errors, invented details, arithmetic mistakes, weak multi-step reasoning, and repetitive text in long answers.
 - Under near-greedy decoding it can loop. Keep the recommended sampling settings.
 - Context is 2,048 tokens, but it was fine-tuned on conversations of up to 1,024 tokens.
 - English only.