TeleChars-AI-API3

Running

MarcosFRGames commited on Dec 2, 2025

Commit

0893fac

verified ·

1 Parent(s): 24de8e1

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -62,7 +62,7 @@ class LLMManager:
                 llm_instance = Llama(
                     model_path=temp_path,
                     n_ctx=MAX_CONTEXT_TOKENS,
-                    n_batch=512,
                     n_threads=2,
                     n_threads_batch=2,
                     use_mlock=True,

                 llm_instance = Llama(
                     model_path=temp_path,
                     n_ctx=MAX_CONTEXT_TOKENS,
+                    n_batch=384,
                     n_threads=2,
                     n_threads_batch=2,
                     use_mlock=True,