Spaces:
Running
Running
Upload model_tools.md
Browse files- model_tools.md +1 -1
model_tools.md
CHANGED
|
@@ -42,7 +42,7 @@ Tools to enhance LLM quantizations and merging
|
|
| 42 |
# [tokensurgeon.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/tokensurgeon.py)
|
| 43 |
- Uses adaptive VRAM from Grim Jim's `measure.py` like `graph_v18` to prevent OOM. Use recommended [batch file](https://huggingface.co/spaces/Naphula/model_tools/blob/main/fix_tokenizers.bat) here or modify sh. This supposedly avoids 'cardboard town' fake patches.
|
| 44 |
|
| 45 |
-
# [tokeninspector.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/
|
| 46 |
- Audit your tokensurgeon results.
|
| 47 |
|
| 48 |
# [eos_scanner.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/eos_scanner.py)
|
|
|
|
| 42 |
# [tokensurgeon.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/tokensurgeon.py)
|
| 43 |
- Uses adaptive VRAM from Grim Jim's `measure.py` like `graph_v18` to prevent OOM. Use recommended [batch file](https://huggingface.co/spaces/Naphula/model_tools/blob/main/fix_tokenizers.bat) here or modify sh. This supposedly avoids 'cardboard town' fake patches.
|
| 44 |
|
| 45 |
+
# [tokeninspector.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/tokeninspector.py)
|
| 46 |
- Audit your tokensurgeon results.
|
| 47 |
|
| 48 |
# [eos_scanner.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/eos_scanner.py)
|