---
sidebar_position: 6
title: "Context Compression & Prompt Caching"
description: "How Hermes compresses long conversations and applies provider-side prompt caching"
---

# Context Compression & Prompt Caching

Hermes manages long conversations with two complementary mechanisms:

- prompt caching
- context compression

Primary files:

- `agent/prompt_caching.py`
- `agent/context_compressor.py`
- `run_agent.py`

## Prompt caching

For Anthropic/native and Claude-via-OpenRouter flows, Hermes applies Anthropic-style cache markers.

Current strategy:

- cache the system prompt
- cache the last 3 non-system messages
- default TTL is 5 minutes unless explicitly extended

This is implemented in `agent/prompt_caching.py`.

## Why prompt stability matters

Prompt caching only helps when the stable prefix remains stable. That is why Hermes avoids rebuilding or mutating the core system prompt mid-session unless it has to.

## Compression trigger

Hermes can compress context when conversations become large. Configuration defaults live in `config.yaml`, and the compressor also has runtime checks based on actual prompt token counts.

## Compression algorithm

The compressor protects:

- the first N turns
- the last N turns

and summarizes the middle section.

It also cleans up structural issues such as orphaned tool-call/result pairs so the API never receives invalid conversation structure after compression.

## Pre-compression memory flush

Before compression, Hermes can give the model one last chance to persist memory so facts are not lost when middle turns are summarized away.

## Session lineage after compression

Compression can split the session into a new session ID while preserving parent lineage in the state DB.

This lets Hermes continue operating with a smaller active context while retaining a searchable ancestry chain.

## Re-injected state after compression

After compression, Hermes may re-inject compact operational state such as:

- todo snapshot
- prior-read-files summary

## Related docs

- [Prompt Assembly](./prompt-assembly.md)
- [Session Storage](./session-storage.md)
- [Agent Loop Internals](./agent-loop.md)