Update model.py

model_url = configs["model"]["generation_model"] # Fixed "genration_model" -> "generation_model"

# 3. Add authentication to model loading
self.model = AutoModelForCausalLM.from_pretrained(
model_url,
token=self.hf_token, # Added authentication
torch_dtype=torch.float16,
low_cpu_mem_usage=True,
attn_implementation="sdpa",
device_map="auto" # Better device handling


Low CPU memory changed from false to true

model.py

@@ -33,7 +33,7 @@ class RAGModel:
             model_url,
             token=self.hf_token,  # Added authentication
             torch_dtype=torch.float16,
-            low_cpu_mem_usage=False,
+            low_cpu_mem_usage=True,
             attn_implementation="sdpa",
             device_map="auto"  # Better device handling
         )