Spaces:
Running
Running
| # TCG Case Study Template | |
| Use after a beta user agrees to be referenced (anonymized or named). Keeps ROI claims defensible for marketing and investors. | |
| **File naming:** `artifacts/case-studies/YYYY-MM-<slug>.md` | |
| --- | |
| ## Metadata | |
| | Field | Value | | |
| |-------|-------| | |
| | Customer (public name) | | | |
| | Segment | indie / YC startup / SMB product team | | |
| | Stack | LangChain / AutoGen / custom / other | | |
| | Agent topology | # agents, avg steps per task | | |
| | TCG plan | Free / Pro | | |
| | Period analyzed | YYYY-MM-DD β YYYY-MM-DD | | |
| | Author | | | |
| --- | |
| ## 1. Baseline (before TCG) | |
| - **Monthly LLM spend (USD):** | |
| - **Primary models:** | |
| - **Main workload:** e.g. coding agent, support bot, RAG pipeline | |
| - **Pain:** e.g. bill spike, no attribution, unknown cache usage | |
| - **Tools used before TCG:** LangSmith / none / spreadsheet / other | |
| --- | |
| ## 2. Integration | |
| - **Endpoint:** `POST /api/v1/optimize` (+ feedback if used) | |
| - **Frequency:** per trace / nightly batch / other | |
| - **Data sent:** full logs / truncated / PII redacted by customer | |
| - **Applied recommendations?** yes / partial / no (critical for honesty) | |
| | Intervention | Applied? | How | | |
| |--------------|----------|-----| | |
| | Compression / HCA summary | | | | |
| | Routing tier change | | | | |
| | Prompt structure / cache blocks (v1) | | | | |
| | HERMES / policy | | | | |
| --- | |
| ## 3. Metrics (quantitative) | |
| | Metric | Before | After | Ξ | Source | | |
| |--------|--------|-------|---|--------| | |
| | Est. monthly tokens (input) | | | | TCG / provider bill | | |
| | Est. monthly cost (USD) | | | | | | |
| | Avg tokens per successful task | | | | | | |
| | % calls on premium model | | | | | | |
| | Cached input tokens (if provider reports) | | | | | | |
| | p95 latency (optional) | | | | | | |
| **TCG-reported fields (from API):** | |
| - `_guard.baseline` / `_guard.saved` (note: may be **estimated** from logs, not provider invoice) | |
| - `_vmm.saving_ratio` if compression applied | |
| - `_routing.targetTier` if used | |
| **Claimed savings %:** ___% | |
| **Method:** `(baseline_cost - after_cost) / baseline` from TCG **or** invoice comparison β **state which**. | |
| --- | |
| ## 4. Qualitative | |
| - **What surprised the team?** | |
| - **What they would not give up (quality guardrails)?** | |
| - **Would they pay $12/mo? Why / why not?** | |
| --- | |
| ## 5. Limitations (required) | |
| - [ ] Savings include model routing, not only prompt optimization | |
| - [ ] Sample size: N traces / M days | |
| - [ ] Seasonality or product change during period | |
| - [ ] TCG did not proxy LLM traffic (beta) | |
| --- | |
| ## 6. Publishable quote (optional) | |
| > ββ¦β β Role, Company | |
| --- | |
| ## 7. Internal only | |
| - `tenant_id` / trace ids (do not publish) | |
| - Support issues / blockers for product | |