tiny-granite-switch-4.1-8b-for-testing

This is a modified, damaged copy of ibm-granite/granite-switch-4.1-8b, made only to test software. Its output is gibberish. Do not use it for anything else.

It keeps the architecture (dense Granite with embedded LoRA adapters selected per token by control tokens), the two adapters (guardian-core and the decision aLoRA), the tokenizer, chat template and adapter configs of granite-switch-4.1-8b, so it runs the same code paths as the real model in llama.cpp (granite-switch decision type, and text generation), but it is small enough for CI.

GGUF version: ggml-org-dev/tiny-granite-switch-4.1-8b-for-testing-GGUF

Changes from the original model

Sliced from the safetensors with slice_granite_switch.py, which follows the slice-model skill of llama.cpp:

  • 2 of 40 weight layers (0, 1), num_hidden_layers 42 -> 4 (it also counts the 2 weightless switch layers)
  • Hidden size 4096 -> 256, heads 32 -> 2, KV heads 8 -> 1 (head_dim 128 kept), FFN 12800 -> 128
  • Embedded LoRA adapters: both kept, in/out dims narrowed with the trunk, max rank 64 -> 16 (first 16 rank components, adapter_ranks capped to 16)
  • model.switch.* buffers kept as is
  • No Python code was in the original repo; compose_report.json, BUILD.md and model.sig are not included

License

Derived from granite-switch-4.1-8b by IBM, under the Apache License 2.0.

Downloads last month
-
Safetensors
Model size
26.7M params
Tensor type
I64
路
F32
路
BF16
路
Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support

Model tree for ggml-org-dev/tiny-granite-switch-4.1-8b-for-testing

Finetuned
(1)
this model
Quantizations
1 model