Runtime Error
#28
by prudhvi95 - opened
I am getting the following error when running jais on a large input text. Below is the code. What is the maximum context length for Jais-13b?
Error: The size of tensor a (2048) must match the size of tensor b (2049) at non-singleton dimension 3/
generate_ids = model.generate(
inputs,
top_p=0.9,
temperature=0.3,
max_new_tokens=1024,
min_length=input_len + 4,
repetition_penalty=1.2,
do_sample=True,
)
for JAIS-13B and JAIS-30B-v1 it is 2048. for JAIS-30B-v3 it is 8192. it says so in the whitepaper