--max-model-len 32768 ?
#1
by
pathosethoslogos
- opened
The original model doesn't have this parameter in the model card, but your AWQ verion's model card does. Is the context limit as little as 32k?
it supports up to 200k tokens per context