vikhyatk

/

moondream2

like 1.41k

For the first iteration where we we are using input_embeds to start the generation, the conditional statement fails as the past_key_values now does not return none but returns Dynamic_Cache() according to the latest transformers version. Using input_ids as a means to check if it's the first iteration serves a good proxy. The current fix works both on 4.42+ and other versions of transformers as well.