Spaces:

hypeconqueror1
/

BudgetBuddyPDFChat

Sleeping

hypeconqueror1 commited on Mar 1, 2024

Commit

2eabd29

verified ·

1 Parent(s): fb3e99e

Update LoadLLM.py

Files changed (1) hide show

LoadLLM.py CHANGED Viewed

@@ -15,7 +15,7 @@ class Loadllm:
             model_path=model_path,
             n_gpu_layers=40,
             n_batch=512,
-            n_ctx=512,
             f16_kv=True,  # MUST set to True, otherwise you will run into problem after a couple of calls
             callback_manager=callback_manager,
             verbose=True,

             model_path=model_path,
             n_gpu_layers=40,
             n_batch=512,
+            n_ctx=1024,
             f16_kv=True,  # MUST set to True, otherwise you will run into problem after a couple of calls
             callback_manager=callback_manager,
             verbose=True,