README.md

---
title: Semantic Cache for LLMs
emoji: ⚡
colorFrom: yellow
colorTo: red
sdk: docker
app_port: 7860
pinned: false
short_description: Embedding-keyed LLM response cache
---

# Semantic Cache for LLM Responses

Caches LLM answers by prompt **embedding**: a semantically similar prompt returns
the cached response instantly, skipping the LLM call (saving latency and token
cost). Hybrid exact-hash + cosine-similarity lookup with a tunable threshold,
LRU eviction, and live hit-rate / savings metrics. Runs on free OpenRouter models.

Source: https://github.com/saiteja007-mv/semantic-cache