What's the context length for this model?
#6
by
DesertCookie - opened
The table on the original model says 128k for the vanilla model and 8k for quantized models. I thus assume, if I go for anything but the FP16 I do get the limited 8k, correct?