tiny-Decision-2.0-Kai-0.6B-for-testing

This is a modified, damaged copy of vllm-sr/Decision-2.0-Kai-0.6B, made only to test software. Its output is gibberish. Do not use it for anything else.

It keeps the package layout (Qwen3 backbone in backbone/, decision head, decision_config.json, score_bias.json), tokenizer and chat template of Decision-2.0-Kai-0.6B, so it runs the same code paths as the real model in llama.cpp (vllm-decision2 decision type: one prompt per question, the head reads the last token of each option span and of the question span), but it is small enough for CI.

GGUF version: ggml-org-dev/tiny-Decision-2.0-Kai-0.6B-for-testing-GGUF

Changes from the original model

Sliced from the safetensors with slice_decision2.py, which follows the slice-model skill of llama.cpp:

  • Backbone: 2 of 28 layers (0, 1), hidden size 1024 -> 32, heads 16 -> 2, KV heads 8 -> 1, FFN 3072 -> 64 (head_dim 128 kept), backbone/config.json patched to match
  • Decision head: input dim 1024 -> 32 (first channels of the norms and projections), head_dim 256 kept
  • decision_config.json and score_bias.json are unchanged (so text_parameter_count is the one of the original model)
  • The Python code, MODEL_MANIFEST.json and the assets of the original repo are not included

License

Derived from Decision-2.0-Kai-0.6B by vLLM Semantic Router, under the Apache License 2.0.

Downloads last month
14
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for ggml-org-dev/tiny-Decision-2.0-Kai-0.6B-for-testing

Finetuned
(1)
this model
Quantizations
1 model