Nalin Gupta Claude Sonnet 4.5 commited on
Commit
f98e3ae
·
1 Parent(s): ca19e57

Fix Phi-3 DynamicCache compatibility issue

Browse files

- Add use_cache=False to phi_model.generate() to prevent AttributeError
- Pin transformers version to 4.40.0-4.45.x for Phi-3 compatibility
- Resolves: DynamicCache object has no attribute 'seen_tokens'

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Files changed (2) hide show
  1. app.py +2 -1
  2. requirements.txt +1 -1
app.py CHANGED
@@ -141,7 +141,8 @@ Output:<|end|>
141
  max_new_tokens=150,
142
  temperature=0.1,
143
  do_sample=True,
144
- pad_token_id=phi_tokenizer.eos_token_id
 
145
  )
146
 
147
  response = phi_tokenizer.decode(outputs[0], skip_special_tokens=True)
 
141
  max_new_tokens=150,
142
  temperature=0.1,
143
  do_sample=True,
144
+ pad_token_id=phi_tokenizer.eos_token_id,
145
+ use_cache=False # Fix for DynamicCache compatibility issue
146
  )
147
 
148
  response = phi_tokenizer.decode(outputs[0], skip_special_tokens=True)
requirements.txt CHANGED
@@ -3,7 +3,7 @@
3
 
4
  # Core
5
  torch>=2.0.0
6
- transformers>=4.35.0
7
  gradio>=4.0.0
8
 
9
  # Model adapters
 
3
 
4
  # Core
5
  torch>=2.0.0
6
+ transformers>=4.40.0,<4.46.0 # Pin compatible version for Phi-3
7
  gradio>=4.0.0
8
 
9
  # Model adapters