README.md

---
title: Exllamav2 Patch
sdk: static
license: mit
short_description: Patch for ExLlamaV2 to support Unigram-based tokenizers
---

# Unigram Tokenizer Patch for ExLlamaV2

This repository provides a patch to support **Unigram-based tokenizers** with `ExLlamaV2`.
By default, using a Unigram tokenizer may result in an error during inference. This patch resolves that issue.

## Files

- `hf.py.patch`
  A patch file that modifies `hf.py` in the ExLlamaV2 repository to support Unigram tokenizers.

- `apply_patch.sh`
  A shell script to apply the patch.

## Usage

You can apply the patch with the following command:

```bash
$ ./apply_patch.sh EXLLAMAV2_DIR
```

Replace EXLLAMAV2_DIR with the path to your local clone of ExLlamaV2.

Example:
```bash
$ ./apply_patch.sh ~/repos/exllamav2
```

## Purpose

The patch is intended to:
- Prevent runtime errors when using tokenizers.models.Unigram.
- Add fallback handling for missing unk_token attributes.
- Improve compatibility with models trained using SentencePiece + Unigram tokenizers.

## Notes

- This patch is intended to be applied to the hf.py file in the ExLlamaV2 repository.
- Tested Versions: **v0.2.6** through **v0.2.8**
- If the patch fails, please ensure your copy of hf.py matches the original before applying.