Refactor the C++ LLM manager into modular components, moves Python modules under python/, and keeps the current control-plane behavior intact. The C++ server now has clearer separation for config, model lifecycle, runtime services, request parsing, HTTP helpers, and server routing, while Docker build/runtime paths were updated to compile multiple C++ files and load Python code from the new package folder

add auth, token policy, queue scheduler, and cancel flow, etc

add new endpoint to cancel all processing prompts

add new build profile

fix encoding

fix model config

fix proxied response in llm manager

fix routing in llm manager

add cpp server

change llm model

change llm model

change model to Qwen2.5-Math-7B-Instruct-GGUF

change llm model to qwen2 math

change llm model to mistral

fix dockerfile

change compilation flags

change compilation flags

reduce context and batch

fix repo name of model

fix repo of model

fix cmd in dockerfile

fix dockerfile

switch to qwen model via cpp server

Log elapsed time and token rate when the response arrives.

a “slow request” logging, log when a request exceeds the budgeted prompt and gets compacted

add simple compacting

fix context window size

fix payload processing

fix logger

fix app to handle exceptions

add requests logging

fix dockerfile

fix dockerfile

fix gitignore, app and logger, etc

add readme

add readme

Add automatic API documentation and in-memory model caching

Force Docker rebuild for web search dependencies

fix app, dockerfile, pyproject.toml to add web search

fix app

fix app

fix dockerfile

fix dockerfile

fix compiling flags in dockerfile

fix compiling flags in dockerfile

fix dockerfile and app module

fix dockerfile

fix dockerfile

fix dockerfile

fix dockerfile