tiny-granite-switch-4.1-8b-for-testing
This is a modified, damaged copy of ibm-granite/granite-switch-4.1-8b, made only to test software. Its output is gibberish. Do not use it for anything else.
It keeps the architecture (dense Granite with embedded LoRA adapters selected per token by control tokens), the two adapters (guardian-core and the decision aLoRA), the tokenizer, chat template and adapter configs of granite-switch-4.1-8b, so it runs the same code paths as the real model in llama.cpp (granite-switch decision type, and text generation), but it is small enough for CI.
GGUF version: ggml-org-dev/tiny-granite-switch-4.1-8b-for-testing-GGUF
Changes from the original model
Sliced from the safetensors with slice_granite_switch.py, which follows the slice-model skill of llama.cpp:
- 2 of 40 weight layers (0, 1),
num_hidden_layers42 -> 4 (it also counts the 2 weightless switch layers) - Hidden size 4096 -> 256, heads 32 -> 2, KV heads 8 -> 1 (head_dim 128 kept), FFN 12800 -> 128
- Embedded LoRA adapters: both kept, in/out dims narrowed with the trunk, max rank 64 -> 16 (first 16 rank components,
adapter_rankscapped to 16) model.switch.*buffers kept as is- No Python code was in the original repo;
compose_report.json,BUILD.mdandmodel.sigare not included
License
Derived from granite-switch-4.1-8b by IBM, under the Apache License 2.0.
- Downloads last month
- -