README.md

---
title: LLM API Cost Optimizer
emoji: 💰
colorFrom: yellow
colorTo: blue
sdk: docker
pinned: false
license: mit
---

# LLM API Cost Optimizer

## Question

How do token volume, model price, caching, batching, and routing change monthly LLM cost?

## System Boundary

This Streamlit Space is a deterministic cost model. It is not a billing system; it is a planning tool for model-serving decisions.

## Method

The user enters request volume, input/output token counts, model prices, cache hit rate, batching gain, and routable traffic share. The app computes baseline and optimized monthly costs.

## Technique

This is systems modeling for LLM inference. It separates the variables that drive cost: request count, input tokens, output tokens, price per token, cache hit rate, batching efficiency, and model routing.

The calculations are deterministic so the economic assumptions are visible.

## Output

The app returns cost metrics, a strategy comparison table, and a bar chart of monthly cost by optimization strategy.

## Why It Matters

LLM engineering includes economics. A model that works technically may still fail if prompt length, routing, or caching is ignored.

## What To Notice

The biggest savings often come from reducing repeated work: semantic caching, shorter prompts, and routing easy requests to smaller models.

## Effect In Practice

Cost modeling informs whether to use hosted APIs, Hugging Face Inference Endpoints, local inference, batching, caching, or a router.

## Hugging Face Extension

The Space can be extended with real endpoint pricing, model latency measurements, and quality/cost Pareto charts for open models.

## Limitations

Real costs depend on provider pricing, latency constraints, cache implementation, batch windows, retries, and quality tradeoffs. Treat this as a planning estimate.

## Run Locally

```bash
pip install -r requirements.txt
streamlit run app.py
```