Commit History

Fix: lazy model load, GGUF integrity check, re-download corrupt files
9a8d0cd

vykanand commited on

Fix: lazy model load on first request, add error handling
d240987

vykanand commited on

Fix: use background process with signal trap instead of exec
0ab33b3

vykanand commited on

Fix: replace llama_cpp.server with custom FastAPI server
386a1cf

vykanand commited on

Fix: replace llama_cpp.server with custom FastAPI server (no mlock, no health check loop)
a7f7a81

vykanand commited on

Upload folder using huggingface_hub
9364470
verified

vykanand commited on

Upload folder using huggingface_hub
9c9456a
verified

vykanand commited on

Fix: --api_key (underscore) not --api-key (dash) for llama_cpp.server
fdd6703

vykanand commited on

Fix: always set api-key (default: not-set) so health check auth works
130da7d

vykanand commited on

Fix: correct CLI flags for llama_cpp.server 0.3.32 (use_mlock, no cont_batching)
d5e1d77

vykanand commited on

Fix: add uvicorn+fastapi deps (llama_cpp.server requires them)
c856bcf

vykanand commited on

Fix: correct GGUF magic bytes check (47475546 not 46474755)
09f4207

vykanand commited on

Fix: use hf_hub_download instead of python -m huggingface_hub
ed1dfc2

vykanand commited on

Fix: install git (required by HF Spaces build system)
dc7c627

vykanand commited on

Pre-built CPU wheel: pip install in 30s instead of 10min source compile
af3cfcb

vykanand commited on

CPU perf: OpenBLAS build, auto-threads, mlock, batched inference
6d7a7ac

vykanand commited on

Pin all deps: python:3.10-slim-bookworm, exact pip versions in requirements.txt
ebd79f8

vykanand commited on

Fix: add build-essential cmake for llama-cpp-python source build
c6c37a2

vykanand commited on

Switch from vLLM to llama-cpp-python (CPU, free HF tier, GGUF Q3_K_M)
90385c2

vykanand commited on

Pin to v0.7.3 tag (immutable, cacheable on HF)
d3ca23c

vykanand commited on

Switch to latest tag for faster HF cache hit
f3d57ef

vykanand commited on

Production vLLM deploy: SHA-pinned Docker, pre-flight checks, ai.js client, usage guide
76fdc26

vykanand commited on

Create README.md
9586539
verified

vykanand commited on

Create Dockerfile
be6d600
verified

vykanand commited on

initial commit
1b3836d
verified

vykanand commited on