Naphula commited on
Commit
1c113e0
·
verified ·
1 Parent(s): 9da2f3d

Update model_tools.md

Browse files
Files changed (1) hide show
  1. model_tools.md +14 -7
model_tools.md CHANGED
@@ -14,12 +14,12 @@ Tools to enhance LLM quantizations and merging
14
  - Merge models in minutes instead of hours on low VRAM. For a 3060/3060 Ti user: This script enables functionality that is otherwise impossible (merging 70B models or large 7B merges with `--cuda`) without OOM. [More details here](https://huggingface.co/spaces/Naphula/model_tools/blob/main/mergekit_low-VRAM-graph_patch.md)
15
  - Update: v18 is much faster than v4 and replaces the trial-and-error loop with an adaptive math-based calculator (using GrimJim's measure.py logic)
16
 
17
- # [metadata_audit.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/metadata_audit.py)
18
- - Checks multiple models within subdirectories for vocab or rope mismatch (useful for large merges). Calibrated for Mistral Nemo 12B by default.
19
-
20
  # config.py
21
  - Simply replace line 13 | BEFORE `ScalarOrGradient: TypeAlias = Union[float, List[float]]` → AFTER `ScalarOrGradient: TypeAlias = Union[float, List[float], str, bool]` | to allow for custom filepath strings within parameter settings.
22
 
 
 
 
23
  # [fp32_to_fp16.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/fp32_to_fp16.py)
24
  - Converts FP32 to FP16 safetensors
25
 
@@ -41,9 +41,6 @@ Tools to enhance LLM quantizations and merging
41
  # [GGUF Repo Suite](https://huggingface.co/spaces/Naphula/gguf-repo-suite)
42
  - Create and quantize Hugging Face models
43
 
44
- # [Failed Experiment gguf_to_safetensors_v2.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/gguf_to_safetensors_v2.py)
45
- - Unsuccessful attempt by Gemini to patch the gguf_to_safetensors script. Missing json files are hard to reconstruct. Also see [safetensors_meta_ripper_v1.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/safetensors_meta_ripper_v1.py) and [tokenizer_ripper_v1.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/tokenizer_ripper_v1.py)
46
-
47
  # [Markdown Viewer](https://huggingface.co/spaces/Naphula/Portable_Offline_Markdown_Viewer)
48
  - Portable Offline Markdown Viewer
49
 
@@ -54,4 +51,14 @@ Tools to enhance LLM quantizations and merging
54
  - A tool which allows you to recreate UD quants such as Q8_K_XL. Examples: [Mistral 24B](https://huggingface.co/spaces/Naphula/model_tools/raw/main/Mistral-Small-3.2-24B-Instruct-2506-UD-Q8_K_XL_UD.txt), [Mistral 7B](https://huggingface.co/spaces/Naphula/model_tools/raw/main/Warlock-7B-v2-Q8_K_XL.txt)
55
 
56
  # [Text Analysis Suite v1.5](https://huggingface.co/spaces/Naphula/TAS_1.5)
57
- - Analyze text files with advanced metrics
 
 
 
 
 
 
 
 
 
 
 
14
  - Merge models in minutes instead of hours on low VRAM. For a 3060/3060 Ti user: This script enables functionality that is otherwise impossible (merging 70B models or large 7B merges with `--cuda`) without OOM. [More details here](https://huggingface.co/spaces/Naphula/model_tools/blob/main/mergekit_low-VRAM-graph_patch.md)
15
  - Update: v18 is much faster than v4 and replaces the trial-and-error loop with an adaptive math-based calculator (using GrimJim's measure.py logic)
16
 
 
 
 
17
  # config.py
18
  - Simply replace line 13 | BEFORE `ScalarOrGradient: TypeAlias = Union[float, List[float]]` → AFTER `ScalarOrGradient: TypeAlias = Union[float, List[float], str, bool]` | to allow for custom filepath strings within parameter settings.
19
 
20
+ # [metadata_audit.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/metadata_audit.py)
21
+ - Checks multiple models within subdirectories for vocab or rope mismatch (useful for large merges). Calibrated for Mistral Nemo 12B by default.
22
+
23
  # [fp32_to_fp16.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/fp32_to_fp16.py)
24
  - Converts FP32 to FP16 safetensors
25
 
 
41
  # [GGUF Repo Suite](https://huggingface.co/spaces/Naphula/gguf-repo-suite)
42
  - Create and quantize Hugging Face models
43
 
 
 
 
44
  # [Markdown Viewer](https://huggingface.co/spaces/Naphula/Portable_Offline_Markdown_Viewer)
45
  - Portable Offline Markdown Viewer
46
 
 
51
  - A tool which allows you to recreate UD quants such as Q8_K_XL. Examples: [Mistral 24B](https://huggingface.co/spaces/Naphula/model_tools/raw/main/Mistral-Small-3.2-24B-Instruct-2506-UD-Q8_K_XL_UD.txt), [Mistral 7B](https://huggingface.co/spaces/Naphula/model_tools/raw/main/Warlock-7B-v2-Q8_K_XL.txt)
52
 
53
  # [Text Analysis Suite v1.5](https://huggingface.co/spaces/Naphula/TAS_1.5)
54
+ - Analyze text files with advanced metrics
55
+
56
+ ---
57
+
58
+ # Not Functional
59
+
60
+ # [Failed Experiment gguf_to_safetensors_v2.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/gguf_to_safetensors_v2.py)
61
+ - Unsuccessful attempt by Gemini to patch the gguf_to_safetensors script. Missing json files are hard to reconstruct. Also see [safetensors_meta_ripper_v1.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/safetensors_meta_ripper_v1.py) and [tokenizer_ripper_v1.py](https://huggingface.co/spaces/Naphula/model_tools/blob/main/tokenizer_ripper_v1.py)
62
+
63
+ # [IQ5_NL.md](https://huggingface.co/spaces/Naphula/model_tools/blob/main/IQ5_NL.md)
64
+ - Note: Not functional yet. Includes the code needed to quantize IQ5_NL GGUFs using block size 32.