Falcon-H1-0.5B Rust Distilled
A Rust/CUDA-compatible, sequence-level distilled version of Falcon-H1-0.5B-Instruct.
The model uses the original Falcon-H1 hybrid architecture, combining grouped-query attention with Mamba-2 state-space layers.
The implementation performs Falcon-H1 inference using Rust, Candle and CUDA, including fused Mamba execution.
Model provenance
Original pretrained student
tiiuae/Falcon-H1-0.5B-Instruct
Teacher
tiiuae/Falcon-H1-1.5B-Deep-Instruct
Dataset
C4 English (allenai/c4, subset en).
Rust implementation
https://github.com/AngshulMajumdar/Falcon-H1-Rust-Transformer-API
Distillation method
The Python teacher generates text continuations from C4 prompts.
Because teacher and student tokenizers differ, teacher-generated continuations are retokenized using the student tokenizer.
The Rust/CUDA student learns from these continuations using masked autoregressive cross-entropy.
The pretrained Falcon-H1 backbone is frozen.
A rank-16 trainable correction is applied to the output head. The correction is trained using Rust, Candle and CUDA.
After training, the correction is merged into the original pretrained output-head weights.
Thus the published checkpoint retains the original Falcon-H1-0.5B architecture and parameter shapes.
This is output-head adaptation using sequence-level knowledge distillation, not full-backbone fine-tuning.
Training
| Setting | Value |
|---|---|
| Teacher | Falcon-H1-1.5B-Deep-Instruct |
| Student | Falcon-H1-0.5B-Instruct |
| Dataset | C4 English |
| Student framework | Rust / Candle / CUDA |
| Training GPU | NVIDIA L4 |
| Trainable component | Rank-16 output-head correction |
| Training target tokens in final run | 97,706 |
| Held-out target tokens | 4,161 |
| Selected V3 epoch | 5 |
| Training batch size | 128 |
The final run continued from the previously selected V2 adapter. The original pretrained backbone was retained.
Held-out C4 evaluation
Evaluation uses masked next-token prediction on 4,161 held-out teacher-continuation tokens.
| Model | Perplexity |
|---|---|
| Original Falcon-H1-0.5B | 3.3695 |
| Distilled V2 | 3.2312 |
| Distilled V3 | 3.2182 |
The V3 adapter was selected using the lowest validation cross-entropy, at epoch 5.
These results measure agreement with teacher-generated continuations on the held-out C4 set.
They do not establish improvements in general reasoning, instruction following, or independent downstream benchmarks.
Download
The primary model file is:
falcon-h1-c4-distilled-rust-v3.f32pack
This is a custom FH1W float32 checkpoint, not a standard
Hugging Face Transformers safetensors checkpoint.
Download it with:
hf download Angshul/Falcon-H1-0.5B-Rust-Distilled \
falcon-h1-c4-distilled-rust-v3.f32pack \
--local-dir ./model
Rust CUDA inference
Clone the Rust implementation:
git clone https://github.com/AngshulMajumdar/Falcon-H1-Rust-Transformer-API
cd Falcon-H1-Rust-Transformer-API
Compile and run the API with an NVIDIA CUDA GPU:
cargo run --release --features cuda --bin falcon-h1-api -- \
--checkpoint ./model/falcon-h1-c4-distilled-rust-v3.f32pack \
--device cuda \
--bind 127.0.0.1:8080
The API accepts token IDs and supports generation through its HTTP endpoints.
Use the tokenizer associated with the original Falcon-H1-0.5B-Instruct student.
Limitations
- The model is specialized using C4 English text continuations.
- Only the output-head correction was trained.
- The backbone is unchanged from the pretrained student.
- The released
.f32packrequires the compatible Rust loader. - The reported perplexity is measured on teacher-generated continuation targets, not a standard language-model benchmark.
- General downstream improvements have not been established.
License and attribution
The original Falcon-H1 model weights were developed by the Technology Innovation Institute (TII).
The underlying model weights and derived checkpoint are subject to the Falcon-LLM License.
The separate Rust implementation is available under its own repository license.
Original Falcon-H1 model:
https://huggingface.co/tiiuae/Falcon-H1-0.5B-Instruct
References
Falcon-H1 technical report: https://arxiv.org/abs/2507.22448
Rust implementation: https://github.com/AngshulMajumdar/Falcon-H1-Rust-Transformer-API
Model tree for Angshul/Falcon-H1-0.5B-Rust-Distilled
Base model
tiiuae/Falcon-H1-0.5B-Base