inference / README.md

Commit History

some experimental
57426fb

0xarchit commited on

fix flash attention
3a9ad3f

0xarchit commited on

max optimisation
0c43530

0xarchit commited on

optimised cpu
d83933c

0xarchit commited on

minor fixes
edf2ef0

0xarchit commited on

experimental boosting changes
434da24

0xarchit commited on

remove parallel support for faster inference
e22f356

0xarchit commited on

Fix llama-server startup and enable LangSearch via shell tool
1420bd1

0xarchit commited on

Optimize inference performance: add continuous-batching, NO_WARMUP support, improve thread/batch recommendations
392efee

0xarchit commited on

Add LangSearch web search tool for small model knowledge augmentation
c459436

0xarchit commited on

Add REASONING env toggle for llama-server reasoning mode
a6b5235

0xarchit commited on

Add optimized llama.cpp CPU inference backend
8ebf335

0xarchit commited on

initial commit
957406c
verified

zrxarchit commited on