Spaces:
Running
Running
File size: 611 Bytes
67ee11f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | ---
title: Semantic Cache for LLMs
emoji: ⚡
colorFrom: yellow
colorTo: red
sdk: docker
app_port: 7860
pinned: false
short_description: Embedding-keyed LLM response cache
---
# Semantic Cache for LLM Responses
Caches LLM answers by prompt **embedding**: a semantically similar prompt returns
the cached response instantly, skipping the LLM call (saving latency and token
cost). Hybrid exact-hash + cosine-similarity lookup with a tunable threshold,
LRU eviction, and live hit-rate / savings metrics. Runs on free OpenRouter models.
Source: https://github.com/saiteja007-mv/semantic-cache
|