tiny-Decision-2.0-Kai-0.6B-for-testing
This is a modified, damaged copy of vllm-sr/Decision-2.0-Kai-0.6B, made only to test software. Its output is gibberish. Do not use it for anything else.
It keeps the package layout (Qwen3 backbone in backbone/, decision head, decision_config.json, score_bias.json), tokenizer and chat template of Decision-2.0-Kai-0.6B, so it runs the same code paths as the real model in llama.cpp (vllm-decision2 decision type: one prompt per question, the head reads the last token of each option span and of the question span), but it is small enough for CI.
GGUF version: ggml-org-dev/tiny-Decision-2.0-Kai-0.6B-for-testing-GGUF
Changes from the original model
Sliced from the safetensors with slice_decision2.py, which follows the slice-model skill of llama.cpp:
- Backbone: 2 of 28 layers (0, 1), hidden size 1024 -> 32, heads 16 -> 2, KV heads 8 -> 1, FFN 3072 -> 64 (head_dim 128 kept),
backbone/config.jsonpatched to match - Decision head: input dim 1024 -> 32 (first channels of the norms and projections), head_dim 256 kept
decision_config.jsonandscore_bias.jsonare unchanged (sotext_parameter_countis the one of the original model)- The Python code,
MODEL_MANIFEST.jsonand the assets of the original repo are not included
License
Derived from Decision-2.0-Kai-0.6B by vLLM Semantic Router, under the Apache License 2.0.
- Downloads last month
- 14