---
license: mit
tags:
  - tensalang
  - llm-inference
  - mlir
  - safetensors
language:
  - en
---

# TensaLang Example Models

The example models' weights for [TensaLang](https://github.com/BenChaliah/Tensa-Lang), a programming language for LLM inference.

## What is TensaLang?

A programming language for LLM inference. Implement new models with ease and compile through MLIR to CUDA, CPU-SIMD, MLX, or ROCm. The runtime is the program. 

```
fn attention_f16(q: Tensor<f32, [D]>,
                 key_cache: Tensor<f16, [L, SeqLen, KvDim]>,
                 value_cache: Tensor<f16, [L, SeqLen, KvDim]>,
                 layer: i32, pos: i32, H: i32, scale: f32) -> Tensor<f32, [D]>
    with tile=[8, 64], parallel=[h, t] {

  var att: Tensor<f32, [H, SeqLen]> = zeros([H, SeqLen])

  # Compute attention scores
  att[h, t] = if t > pos { -inf } else {
    sum(i) q[h * Dh + i] * (key_cache[layer, t, h * Dh + i] as f32) * scale
  }

  var weights: Tensor<f32, [H, SeqLen]> = softmax(att)
  # ... weighted sum over values
}
```

From the creator of [Datarus-R1-14B](https://huggingface.co/Datarus/Datarus-R1-14B).

## Example models

| Model | Parameters | Format | Description |
|-------|------------|--------|-------------|
| `llama2_7b_f16.safetensors` | 7B | FP16 | Llama2-7B |
| `qwen2.5_coder_0.5b_bf16.safetensors` | 0.5B | BF16 | Qwen2.5-Coder-0.5B-Instruct |

## Usage

```bash
# Clone TensaLang
git clone https://github.com/BenChaliah/Tensa-Lang.git
cd Tensa-Lang && ./build.sh

# Download models
huggingface-cli download BenChaliah/TensaLang-models --local-dir ./models

# Or download a specific model
huggingface-cli download BenChaliah/TensaLang-models llama2_7b_f16.safetensors --local-dir ./Llama2-assets
```

### Run Llama2

```bash
./bin/tensalang-run examples/llama2_manual_tiling_fp16.tl \
  --model Llama2-assets/llama2_7b_f16.safetensors \
  --tokenizer Llama2-assets/tokenizer.json \
  --prompt "Once upon a time" \
  --target cuda \
  --steps 128 \
  --fused-attention 2 \
  --cuda-arch sm_89
```

### Run Qwen2.5-Coder

```bash
./bin/tensalang-run examples/qwen25_coder_bf16.tl \
  --model Qwen25-assets/qwen2.5_coder_0.5b_bf16.safetensors \
  --tokenizer Qwen25-assets/tokenizer.json \
  --prompt "def quicksort(arr):" \
  --target cuda \
  --steps 64 \
  --cuda-arch sm_89
```

## Source

Weights converted from:
- [meta-llama/Llama-2-7b](https://huggingface.co/meta-llama/Llama-2-7b)
- [Qwen/Qwen2.5-Coder-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct)

## License

Model weights retain their original licenses. TensaLang compiler is MIT licensed.

## Links

- [TensaLang GitHub](https://github.com/BenChaliah/Tensa-Lang)
- [Documentation](https://tensa-lang.org/docs.html)
- [Website](https://tensa-lang.org/)
- [Datarus-R1-14B](https://huggingface.co/Datarus/Datarus-R1-14B)