input context -2k
#2
by
LittleTaro
- opened
I have one question about the context-length. why 2k? When encountering long texts exceeding 2k tokens, is it necessary to split them before translation?
Hi
@LittleTaro
Yes, you absolutely must split texts longer than 2k tokens and the safest way is to split your long text is into chunks of paragraphs .
2k limit is a deliberate design choice as translateGemma is designed to run locally on consumer hardware so a smaller context window significantly reduces memory usage (KV cache) and latency.
Thanks