Spaces:
Running
Running
Update model_tools.md
Browse files- model_tools.md +14 -7
model_tools.md
CHANGED
|
@@ -14,12 +14,12 @@ Tools to enhance LLM quantizations and merging
|
|
| 14 |
- Merge models in minutes instead of hours on low VRAM. For a 3060/3060 Ti user: This script enables functionality that is otherwise impossible (merging 70B models or large 7B merges with `--cuda`) without OOM. [More details here](https://huggingface.co/spaces/Naphula/model_tools/blob/main/mergekit_low-VRAM-graph_patch.md)
|
| 15 |
- Update: v18 is much faster than v4 and replaces the trial-and-error loop with an adaptive math-based calculator (using GrimJim's measure.py logic)
|
| 16 |
|
| 17 |
-
# [metadata_audit.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/metadata_audit.py)
|
| 18 |
-
- Checks multiple models within subdirectories for vocab or rope mismatch (useful for large merges). Calibrated for Mistral Nemo 12B by default.
|
| 19 |
-
|
| 20 |
# config.py
|
| 21 |
- Simply replace line 13 | BEFORE `ScalarOrGradient: TypeAlias = Union[float, List[float]]` → AFTER `ScalarOrGradient: TypeAlias = Union[float, List[float], str, bool]` | to allow for custom filepath strings within parameter settings.
|
| 22 |
|
|
|
|
|
|
|
|
|
|
| 23 |
# [fp32_to_fp16.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/fp32_to_fp16.py)
|
| 24 |
- Converts FP32 to FP16 safetensors
|
| 25 |
|
|
@@ -41,9 +41,6 @@ Tools to enhance LLM quantizations and merging
|
|
| 41 |
# [GGUF Repo Suite](https://huggingface.co/spaces/Naphula/gguf-repo-suite)
|
| 42 |
- Create and quantize Hugging Face models
|
| 43 |
|
| 44 |
-
# [Failed Experiment gguf_to_safetensors_v2.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/gguf_to_safetensors_v2.py)
|
| 45 |
-
- Unsuccessful attempt by Gemini to patch the gguf_to_safetensors script. Missing json files are hard to reconstruct. Also see [safetensors_meta_ripper_v1.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/safetensors_meta_ripper_v1.py) and [tokenizer_ripper_v1.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/tokenizer_ripper_v1.py)
|
| 46 |
-
|
| 47 |
# [Markdown Viewer](https://huggingface.co/spaces/Naphula/Portable_Offline_Markdown_Viewer)
|
| 48 |
- Portable Offline Markdown Viewer
|
| 49 |
|
|
@@ -54,4 +51,14 @@ Tools to enhance LLM quantizations and merging
|
|
| 54 |
- A tool which allows you to recreate UD quants such as Q8_K_XL. Examples: [Mistral 24B](https://huggingface.co/spaces/Naphula/model_tools/raw/main/Mistral-Small-3.2-24B-Instruct-2506-UD-Q8_K_XL_UD.txt), [Mistral 7B](https://huggingface.co/spaces/Naphula/model_tools/raw/main/Warlock-7B-v2-Q8_K_XL.txt)
|
| 55 |
|
| 56 |
# [Text Analysis Suite v1.5](https://huggingface.co/spaces/Naphula/TAS_1.5)
|
| 57 |
-
- Analyze text files with advanced metrics
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
- Merge models in minutes instead of hours on low VRAM. For a 3060/3060 Ti user: This script enables functionality that is otherwise impossible (merging 70B models or large 7B merges with `--cuda`) without OOM. [More details here](https://huggingface.co/spaces/Naphula/model_tools/blob/main/mergekit_low-VRAM-graph_patch.md)
|
| 15 |
- Update: v18 is much faster than v4 and replaces the trial-and-error loop with an adaptive math-based calculator (using GrimJim's measure.py logic)
|
| 16 |
|
|
|
|
|
|
|
|
|
|
| 17 |
# config.py
|
| 18 |
- Simply replace line 13 | BEFORE `ScalarOrGradient: TypeAlias = Union[float, List[float]]` → AFTER `ScalarOrGradient: TypeAlias = Union[float, List[float], str, bool]` | to allow for custom filepath strings within parameter settings.
|
| 19 |
|
| 20 |
+
# [metadata_audit.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/metadata_audit.py)
|
| 21 |
+
- Checks multiple models within subdirectories for vocab or rope mismatch (useful for large merges). Calibrated for Mistral Nemo 12B by default.
|
| 22 |
+
|
| 23 |
# [fp32_to_fp16.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/fp32_to_fp16.py)
|
| 24 |
- Converts FP32 to FP16 safetensors
|
| 25 |
|
|
|
|
| 41 |
# [GGUF Repo Suite](https://huggingface.co/spaces/Naphula/gguf-repo-suite)
|
| 42 |
- Create and quantize Hugging Face models
|
| 43 |
|
|
|
|
|
|
|
|
|
|
| 44 |
# [Markdown Viewer](https://huggingface.co/spaces/Naphula/Portable_Offline_Markdown_Viewer)
|
| 45 |
- Portable Offline Markdown Viewer
|
| 46 |
|
|
|
|
| 51 |
- A tool which allows you to recreate UD quants such as Q8_K_XL. Examples: [Mistral 24B](https://huggingface.co/spaces/Naphula/model_tools/raw/main/Mistral-Small-3.2-24B-Instruct-2506-UD-Q8_K_XL_UD.txt), [Mistral 7B](https://huggingface.co/spaces/Naphula/model_tools/raw/main/Warlock-7B-v2-Q8_K_XL.txt)
|
| 52 |
|
| 53 |
# [Text Analysis Suite v1.5](https://huggingface.co/spaces/Naphula/TAS_1.5)
|
| 54 |
+
- Analyze text files with advanced metrics
|
| 55 |
+
|
| 56 |
+
---
|
| 57 |
+
|
| 58 |
+
# Not Functional
|
| 59 |
+
|
| 60 |
+
# [Failed Experiment gguf_to_safetensors_v2.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/gguf_to_safetensors_v2.py)
|
| 61 |
+
- Unsuccessful attempt by Gemini to patch the gguf_to_safetensors script. Missing json files are hard to reconstruct. Also see [safetensors_meta_ripper_v1.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/safetensors_meta_ripper_v1.py) and [tokenizer_ripper_v1.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/tokenizer_ripper_v1.py)
|
| 62 |
+
|
| 63 |
+
# [IQ5_NL.md](https://huggingface.co/spaces/Naphula/model_tools/blob/main/IQ5_NL.md)
|
| 64 |
+
- Note: Not functional yet. Includes the code needed to quantize IQ5_NL GGUFs using block size 32.
|