README.md

---
library_name: transformers
tags:
- falcon-h1
- edge
license: other
license_name: falcon-llm-license
license_link: https://falconllm.tii.ae/falcon-terms-and-conditions.html
---

<img src="https://cdn-uploads.huggingface.co/production/uploads/62441d1d9fdefb55a0b7d12c/l1du02RjuAZJcksI5tQ-F.png" alt="drawing" width="800"/>

#  Table of Contents

0. [TL;DR](#TL;DR)
1. [Model Details](#model-details)
2. [Training Details](#training-details)
3. [Usage](#usage)
4. [Evaluation](#evaluation)
5. [Citation](#citation)

# TL;DR

# Model Details

## Model Description

- **Developed by:** [https://www.tii.ae](https://www.tii.ae)
- **Model type:** Causal decoder-only
- **Architecture:** Hybrid Transformers + Mamba architecture
- **Language(s) (NLP):** English
- **Number of Parameters:** 90M
- **License:** Falcon-LLM License

# Training details

For more details about the training protocol of this model, please refer to the [Falcon-H1-Tiny technical blogpost](https://huggingface.co/spaces/tiiuae/tiny-h1-blogpost).

# Usage

Currently to use this model you can either rely on Hugging Face `transformers`, `vLLM`, `sglang`, `llama.cpp`, `ollama` or `mlx` library.

## Inference

### 🤗 transformers

Refer to the snippet below to run H1 models using 🤗 transformers:

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon-H1-Tiny-R-90M"

model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype=torch.bfloat16,
  device_map="auto"
)

# Perform text generation
```

or

```bash
transformers serve tiiuae/Falcon-H1-Tiny-R-90M
```

### `llama.cpp`

You can find all GGUF files compatible with `llama.cpp` under [our official collection]() - an example setup could be:

```bash
brew install llama.cpp 
pip install huggingface_hub 
hf download tiiuae/Falcon-H1-Tiny-R-90M-GGUF Falcon-H1-Tiny-R-90M-Q8_0.gguf --local-dir ./ 
llama-cli ./Falcon-H1-Tiny-R-90M-Q8_0.gguf -cnv 
```

### `ollama`

```bash
ollama run hf.co/tiiuae/Falcon-H1-Tiny-R-90M:Q8_0 
```

### Apple `mlx` 

```bash
mlx_lm.chat --model tiiuae/Falcon-H1-Tiny-R-90M 
```

### vLLM

For vLLM, simply start a server by executing the command below:

```bash
# pip install vllm>=0.9.0
vllm serve tiiuae/Falcon-H1-Tiny-R-90M --tensor-parallel-size 2 --data-parallel-size 1
```

### sglang

```bash
python -m sglang.launch_server \
  --model ttiiuae/Falcon-H1-Tiny-R-90M \
  --tensor-parallel-size 1 
```

# Evaluation

For detailed evaluation of Falcon-H1-Tiny series, please refer to our [technical blogpost](https://huggingface.co/spaces/tiiuae/tiny-h1-blogpost)

# Useful links

- View [our release blogpost](https://huggingface.co/spaces/tiiuae/tiny-h1-blogpost).
- Feel free to join [our discord server](https://discord.gg/trwMYP9PYm) if you have any questions or to interact with our researchers and developers.

# Citation

If the Falcon-H1-Tiny family of models were helpful to your work, feel free to give us a cite.

```
@misc{falcon_h1_tiny,
  title={Falcon-H1-Tiny: A series of extremely small, yet powerful language models redefining capabilities at small scale},
  author={Falcon-LLM Team},
  year={2026}, 
}
```