transformers.js/pr_1665/en/api/generation/cache.md

generation/cache

Cache classes used by generate() to preserve and reuse decoder past key/value tensors across calls. Pass a DynamicCache through generation options when you need to persist KV state between turns.

Type Definitions

DynamicCache

Mutable cache of decoder past key/value tensors used by generate().

Pass a DynamicCache through generation options when you need to preserve cache tensors across calls or inspect them from return_dict_in_generate.

Type: Record<string, Tensor>