chatbotai_rag / src /core /local_llm_engine.py

Commit History

perf: Downgrade model to 1.5B for faster CPU inference
9e9ace3
Running

quoctrong commited on

perf: Optimize Latency (Remove query translation, async cache, tune LLM threads, limit tokens)
9d3288b

quoctrong commited on

feat: Phase 4 Upgrade - RAG Architecture Stability (mMARCO, pdfplumber, true recursive chunk, 3B LLM)
7a67d93

quoctrong commited on

feat: Implement Phase 3 - Optimize inference engine with llama.cpp
03f3ef0

quoctrong commited on

feat: production-hardening, upload validation, admin auth, threadpool, logging & HF dataset repo integration
0d6af6d

quoctrong commited on

Expand LLM context window max length to 6500 tokens (~5000 words)
59cac60

quoctrong commited on

Use native Transformers GGUF loader to fix C++ compile delay in Docker build
8db2a53

quoctrong commited on

Upgrade Local LLM to GGUF (llama.cpp) format for 5x-7x faster CPU inference
9087f4e

quoctrong commited on

Add accelerate dependency and fix Local LLM loading error
d22721f

quoctrong commited on

DocBrain RAG official release for HF Space
81e471d

quoctrong commited on