TheBloke
/

starcoderplus-GPTQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions

TheBloke commited on Jun 9, 2023

Commit

1aa0491

·

1 Parent(s): 32c27c3

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -174,8 +174,8 @@ It was created without group_size to lower VRAM requirements, and with --act-ord
 * `gptq_model-4bit--1g.safetensors`
   * Works with AutoGPTQ in CUDA or Triton modes.
-  * Works with GPTQ-for-LLaMa in CUDA mode.  May have issues with GPTQ-for-LLaMa Triton mode.
   * Works with text-generation-webui, including one-click-installers.
   * Parameters: Groupsize = -1. Act Order / desc_act = True.
 <!-- footer start -->

 * `gptq_model-4bit--1g.safetensors`
   * Works with AutoGPTQ in CUDA or Triton modes.
   * Works with text-generation-webui, including one-click-installers.
+  * Works with GPTQ-for-LLaMa in CUDA mode.  May have issues with GPTQ-for-LLaMa Triton mode.
   * Parameters: Groupsize = -1. Act Order / desc_act = True.
 <!-- footer start -->