catplusplus commited on
Commit
7996513
·
verified ·
1 Parent(s): 64e4e9b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -11,5 +11,6 @@ them from NVIDIA model. The resulting model here seems to work fine for chat, wi
11
  I have included extras directory in model files with the following tools:
12
  - Script used to delete experts from nvidia model based on structure of saricles model, should be reusable for other REAP transfer between quants
13
  - Modified chat template that allows turning off reasoning through the same mechanism as Qwen 3.5, enable_thinking: false in kwargs
 
14
  - cpp tool to aggressively clear memory and swap out inactive processes before running model in order to max out context length, I get ~120K tokens with FP8 kv cache
15
- - Example script to run vLLM with optimized parameters for speed and memory efficiency, for example limited CUDA graph captures.
 
11
  I have included extras directory in model files with the following tools:
12
  - Script used to delete experts from nvidia model based on structure of saricles model, should be reusable for other REAP transfer between quants
13
  - Modified chat template that allows turning off reasoning through the same mechanism as Qwen 3.5, enable_thinking: false in kwargs
14
+ - Modified reasoning parser to support new template
15
  - cpp tool to aggressively clear memory and swap out inactive processes before running model in order to max out context length, I get ~120K tokens with FP8 kv cache
16
+ - Example script to run vLLM with optimized parameters for speed and memory efficiency, for example limited CUDA graph captures, as well as custom chat template/reasoning parser.