facebook

/

MobileLLM-125M

like 128

Follow

AI at Meta 10.4k

Hello! Running this with the current version of Transformers gives me the following error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-5-5a5008a62dec> in <cell line: 0>()
      1 model_inputs = tokenizer(["A list of colors: red, blue"], return_tensors="pt")
----> 2 generated_ids = model.generate(**model_inputs)
      3 tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

3 frames
~/.cache/huggingface/modules/transformers_modules/facebook/MobileLLM-125M/e09efb976ae4a767cf27a4bf277b7aade91dbd3a/modeling_mobilellm.py in prepare_inputs_for_generation(self, input_ids, past_key_values, attention_mask, inputs_embeds, cache_position, use_cache, **kwargs)
   1277             max_cache_length = (
   1278                 torch.tensor(past_key_values.get_max_length(), device=input_ids.device)
-> 1279                 if past_key_values.get_max_length() is not None
   1280                 else None
   1281             )

AttributeError: 'DynamicCache' object has no attribute 'get_max_length'

It seems like get_max_length was removed in the latest version of transformers and replaced with cache.get_max_cache_shape() instead.