Error running with most recent version of transformers
#9
by Minus0 - opened
Hello! Running this with the current version of Transformers gives me the following error:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-5-5a5008a62dec> in <cell line: 0>()
1 model_inputs = tokenizer(["A list of colors: red, blue"], return_tensors="pt")
----> 2 generated_ids = model.generate(**model_inputs)
3 tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
3 frames
~/.cache/huggingface/modules/transformers_modules/facebook/MobileLLM-125M/e09efb976ae4a767cf27a4bf277b7aade91dbd3a/modeling_mobilellm.py in prepare_inputs_for_generation(self, input_ids, past_key_values, attention_mask, inputs_embeds, cache_position, use_cache, **kwargs)
1277 max_cache_length = (
1278 torch.tensor(past_key_values.get_max_length(), device=input_ids.device)
-> 1279 if past_key_values.get_max_length() is not None
1280 else None
1281 )
AttributeError: 'DynamicCache' object has no attribute 'get_max_length'
It seems like get_max_length was removed in the latest version of transformers and replaced with cache.get_max_cache_shape() instead.