alex4cip Claude commited on
Commit
e6dc16b
·
1 Parent(s): c6d4144

fix: Add safetensors support and improve model loading

Browse files

**Model Loading Improvements:**
- Add safetensors>=0.4.0 to requirements for modern model format support
- Add accelerate>=0.20.0 for optimized model loading
- Enable trust_remote_code for tokenizer and model loading
- Add low_cpu_mem_usage=True to reduce memory footprint
- Fix torch_dtype deprecation warning (use dtype instead)

**Technical Changes:**
- Support both safetensors and pytorch_model.bin formats
- Better memory management for large models
- Enable remote code execution for special tokenizers
- Improved compatibility with HuggingFace Hub

This fixes the model loading error on Hugging Face Spaces:
"Can't load the model for 'microsoft/DialoGPT-small'"

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Files changed (2) hide show
  1. app.py +6 -3
  2. requirements.txt +2 -0
app.py CHANGED
@@ -73,18 +73,21 @@ def load_model(model_name):
73
  tokenizer = AutoTokenizer.from_pretrained(
74
  model_name,
75
  token=HF_TOKEN,
76
- padding_side='left'
 
77
  )
78
 
79
  # Add pad token if missing
80
  if tokenizer.pad_token is None:
81
  tokenizer.pad_token = tokenizer.eos_token
82
 
83
- # Load model
84
  model = AutoModelForCausalLM.from_pretrained(
85
  model_name,
86
  token=HF_TOKEN,
87
- torch_dtype=torch.float32,
 
 
88
  )
89
  model.to(device)
90
  model.eval()
 
73
  tokenizer = AutoTokenizer.from_pretrained(
74
  model_name,
75
  token=HF_TOKEN,
76
+ padding_side='left',
77
+ trust_remote_code=True
78
  )
79
 
80
  # Add pad token if missing
81
  if tokenizer.pad_token is None:
82
  tokenizer.pad_token = tokenizer.eos_token
83
 
84
+ # Load model with safetensors support
85
  model = AutoModelForCausalLM.from_pretrained(
86
  model_name,
87
  token=HF_TOKEN,
88
+ dtype=torch.float32,
89
+ low_cpu_mem_usage=True,
90
+ trust_remote_code=True
91
  )
92
  model.to(device)
93
  model.eval()
requirements.txt CHANGED
@@ -1,3 +1,5 @@
1
  gradio>=5.0.0
2
  transformers>=4.30.0
3
  torch>=2.0.0
 
 
 
1
  gradio>=5.0.0
2
  transformers>=4.30.0
3
  torch>=2.0.0
4
+ safetensors>=0.4.0
5
+ accelerate>=0.20.0