InferRoute / inferroute /cache.py

Commit History

Fix SQLite, OpenAI mock, circuit breaker fallback for pinned models, cache isolation, prefix cache answers, docs relative paths, and integration tests
0a1d5dd

Ypeng12 commited on

feat: implement streaming request dedup, cache-affinity trie routing, speculative cascade, vegas concurrency limiter, web UI playground, and multi-tenant wallet billing
80db329

Ypeng12 commited on

feat: Kubernetes-ready multi-provider LLM gateway v0.2.0
8a841b2

Ypeng12 commited on

first commit
6e8d20e

Ypeng12 commited on