--- license: apache-2.0 base_model: h2oai/h2o-lightning-4b tags: - testing --- # tiny-h2o-lightning-4b-for-testing > [!WARNING] > This is a **modified, damaged copy** of [h2oai/h2o-lightning-4b](https://huggingface.co/h2oai/h2o-lightning-4b), made only to test software. Its output is gibberish. Do not use it for anything else. It keeps the architecture, tokenizer, chat template, `serve_config.json` (prompt, labels, temperatures) and MTP layer of h2o-lightning-4b, so it runs the same code paths as the real model in llama.cpp (`h2o-lightning` decision type: labels read at the token after the `Answer:` prefill, labels and temperatures from `serve_config.json`, image input), but it is small enough for CI. GGUF version: [ggml-org-dev/tiny-h2o-lightning-4b-for-testing-GGUF](https://huggingface.co/ggml-org-dev/tiny-h2o-lightning-4b-for-testing-GGUF) ## Changes from the original model Sliced from the safetensors with [slice_qwen35_decision.py](slice_qwen35_decision.py) (`python3 slice_qwen35_decision.py h2oai/h2o-lightning-4b `), which follows the `slice-model` skill of llama.cpp: - Text model: 4 of 32 layers (0-3: 3 linear attention + 1 full attention), hidden size 2560 -> 32, FFN 9216 -> 64, heads 16 -> 2, KV heads 4 -> 1 (head_dim 256 kept), linear attention key heads 16 -> 2, value heads 32 -> 4 (head dims 128 kept). The MTP layer is narrowed the same way - Vision encoder: 2 of 24 blocks, hidden size 1024 -> 128, heads 16 -> 2 (head_dim 64 kept), FFN 4096 -> 256, output 2560 -> 32 - `h2o_lightning_shim.py` is kept unchanged only because `convert_hf_to_gguf.py` detects the checkpoint by it - The other Python code, tests, `label_rows.lightning` (two-way mode), `tokenizer/` copy, `vllm_patches/`, `engine/` and assets of the original repo are not included ## License Derived from h2o-lightning-4b by H2O.ai (itself built on Qwen/Qwen3.5-4B), under the [Apache License 2.0](LICENSE).