Update handler.py

The shim adds back the single missing method (get_max_length) that transformers ≥ 4.38 removed, returning None which is safe — the internal code that calls it only uses the value for optional length checks, not critical logic.

Without cache (old): token 1 → recompute all attention for N tokens
token 2 → recompute all attention for N+1 tokens
token K → recompute all attention for N+K tokens ← O(n²)

With cache (new): token 1 → compute, store KV in DynamicCache
token 2 → only compute for new token, reuse cache ← O(n)

handler.py

@@ -6,7 +6,12 @@ Fix: DynamicCache.get_max_length() removed in transformers>=4.38.
      Solution: use_cache=False bypasses DynamicCache entirely.
      Slightly slower but fully compatible without any version pinning.
 """
+# ── DynamicCache compatibility shim (transformers >= 4.38) ──────────────────
+from transformers import DynamicCache
 
+if not hasattr(DynamicCache, "get_max_length"):
+    DynamicCache.get_max_length = lambda self: None
+# ───────────────────────────────────────────────────────────────────────────
 from transformers import AutoTokenizer, AutoModelForCausalLM
 import torch
 
@@ -66,7 +71,7 @@ class EndpointHandler:
             # use_cache=False bypasses DynamicCache entirely —
             # avoids get_max_length() removal in transformers>=4.38
             # without requiring any version pinning.
-            use_cache=False,
+            # use_cache=False,
         )
 
         if do_sample: