Ashish Talreja

I really appreciate your effort to explaining this so well. Just one doubt I have, what exactly is being cached?

1. The QK^t dot product results and the Value vectors of the already generated tokens
   or
2. The just the key vectors and the value vectors of already generated tokens?

Also, is this done for each transformer block in an LLM?

Hey Yi, could you please share the training data details, like what were the datasets this model was finetuned on?