Intel
/

phi-2-int4-inc

Text Generation

text-generation-inference

4-bit precision

intel/auto-round

Model card Files Files and versions

yintongl commited on Jul 4, 2024

Commit

4be85d7

·

verified ·

1 Parent(s): f7c0764

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -104,8 +104,8 @@ python3 main.py \
 --bits 4 \
 --iters 1000 \
 --sym \
 --deployment_device 'gpu' \
---disable_low_gpu_mem_usage \
 --output_dir "./tmp_autoround" \
 ```

 --bits 4 \
 --iters 1000 \
 --sym \
+--nsamples \
 --deployment_device 'gpu' \
 --output_dir "./tmp_autoround" \
 ```