Spaces:
Running
Running
File size: 1,279 Bytes
8b3b702 a0f7b9d 715ff6a a0f7b9d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
---
title: Exllamav2 Patch
sdk: static
license: mit
short_description: Patch for ExLlamaV2 to support Unigram-based tokenizers
---
# Unigram Tokenizer Patch for ExLlamaV2
This repository provides a patch to support **Unigram-based tokenizers** with `ExLlamaV2`.
By default, using a Unigram tokenizer may result in an error during inference. This patch resolves that issue.
## Files
- `hf.py.patch`
A patch file that modifies `hf.py` in the ExLlamaV2 repository to support Unigram tokenizers.
- `apply_patch.sh`
A shell script to apply the patch.
## Usage
You can apply the patch with the following command:
```bash
$ ./apply_patch.sh EXLLAMAV2_DIR
```
Replace EXLLAMAV2_DIR with the path to your local clone of ExLlamaV2.
Example:
```bash
$ ./apply_patch.sh ~/repos/exllamav2
```
## Purpose
The patch is intended to:
- Prevent runtime errors when using tokenizers.models.Unigram.
- Add fallback handling for missing unk_token attributes.
- Improve compatibility with models trained using SentencePiece + Unigram tokenizers.
## Notes
- This patch is intended to be applied to the hf.py file in the ExLlamaV2 repository.
- Tested Versions: **v0.2.6** through **v0.2.8**
- If the patch fails, please ensure your copy of hf.py matches the original before applying.
|