add mmproj vision support

update custom llama cpp support

increase batch size

bring verbosity back

some experimental

increase batch size

bring back flash attention

experimental flash attention

fix verbose

fix balanced profile

fix balanced profile

update batch size

fix flash attention

max optimisation

optimised cpu

minor fixes

experimental boosting changes

remove parallel support for faster inference

Fix llama-server startup and enable LangSearch via shell tool

Optimize inference performance: add continuous-batching, NO_WARMUP support, improve thread/batch recommendations

Update start.sh

Add LangSearch web search tool for small model knowledge augmentation

Add REASONING env toggle for llama-server reasoning mode

Use temp download dir and robust rmtree handler to avoid permission errors on cleanup

Save models using repo filename, set MODEL_PATH from marker, ensure HF xet log dirs writable

Allow MODEL_FILE override; improve diagnostics when no .gguf files present

Auto-tune ctx-size, batch and cache types for low-memory (<=16GB) Spaces; export CACHE_TYPE_{K,V}

Add QUANT_PREFERENCE and PERF_PROFILE (low_latency/balanced/throughput) with server defaults

Add startup phase logs and avoid recursive permission setup

Copy llama.cpp shared libraries into runtime image

Add model list and download progress logging

Speed up model download by selecting one GGUF

Speed up build (ccache) and fix /data permissions: create appuser and run downloader/server as appuser

Support CONTEXT_LENGTH env var (default 4096) for ctx-size

Use Python venv in runtime to avoid PEP 668 pip install error

Fix llama.cpp math compiler flags

Fix llama.cpp Debian slim build deps

Add optimized llama.cpp CPU inference backend

initial commit