Deepen InferScale simulation research workflow

README.md

@@ -36,6 +36,7 @@ The research direction has continued rapidly. Recent work includes GPU-free serv
 - log-normal prompt/output-length distributions
 - exact **CSV/JSON trace replay** using supplied arrival times and token lengths
 - shared seeds for controlled A/B experiments
+- stateful multi-turn agent sessions with ordered turn dependencies and explicit tool-call gaps
 
 Generated load is open-loop: arrivals are scheduled independently of response completion so queueing delay remains visible under overload. Trace replay accepts:
 
@@ -70,6 +71,7 @@ arrival_time,prompt_tokens,output_tokens
 - prefix-cache hit rate / saved prefill work
 - P/D transfer latency / transfer volume / role utilization
 - heuristic bottleneck diagnoses with explicit simulator provenance
+- cross-turn KV hit rate, recomputed history tokens, routing locality, HBM GB-seconds, and TTL/pressure evictions for stateful sessions
 
 ## Interactive experiments
 
@@ -103,6 +105,18 @@ Run a bounded live sweep across scheduler, batch size, prefix caching, and P/D w
 - **performance:** maximize goodput while minimizing p95 TTFT
 - **efficiency:** maximize goodput per accelerator while minimizing p95 TTFT
 
+### Agent Sessions
+
+Model stateful multi-turn programs rather than isolated requests. Each program has ordered LLM turns separated by tool-call gaps. The simulator tracks cross-turn KV residency on multiple replicas, routing locality, TTL expiry, pressure eviction, recomputation after cache loss, and memory-time residency.
+
+Three live experiments are included:
+
+1. **Session run:** inspect one retention/routing policy on a generated program trace.
+2. **Session Policy Arena:** replay the exact same program trace through stateless least-load, retained least-load, TTL+affinity, and retained+affinity policies.
+3. **TTL frontier:** sweep KV retention time under common random numbers and expose the non-dominated trade-off between p95 turn TTFT and mean resident KV.
+
+Agent mode intentionally uses a serial service station per replica. The existing Serving Lab models batching; Agent Sessions isolates program dependencies, cache locality, tool gaps, and state-management decisions so those effects are interpretable rather than conflated.
+
 ### Research Studies
 
 This is the statistical/research layer rather than another configuration dashboard.
@@ -152,7 +166,7 @@ This makes profile uncertainty visible rather than allowing one uncalibrated pro
 |                  |                                          |     |
 |                  +-----------------------+------------------+     |
 |                                          v                        |
-|                         metrics / search / research                |
+|                  metrics / search / research / agent sessions       |
 +------------------------------------------------------------------+
 ```
 
@@ -194,6 +208,20 @@ The link is intentionally simple and explicit. It does not claim to reproduce NC
 
 The research stress test can multiplicatively perturb prefill/decode/transfer timing to expose conclusions that are sensitive to analytical-model error.
 
+## Stateful agent-session model
+
+Agent-session mode follows a program-level event trace:
+
+```text
+session arrival -> turn 1 -> tool gap -> turn 2 -> ... -> completion
+```
+
+When a turn becomes ready, it is routed either to the least-loaded replica or to a replica already holding that session's KV state. A cache hit prefills only newly appended tokens; a miss recomputes the accumulated history. KV can be evicted immediately, retained until the session ends, or retained under a TTL. Per-replica capacity uses the same analytical KV-byte model as the request simulator and falls back to LRU-style pressure eviction when the retained working set exceeds the configured budget.
+
+The public app reports **HBM GB-seconds** in addition to peak/mean KV occupancy. This makes the cost of keeping state resident during tool gaps visible rather than treating cache hits as free.
+
+This module is inspired by recent work on agentic serving, including AGENTSERVESIM's program/tool-gap/session-routing abstraction and CacheTTL's observation that retention across variable tool gaps is a latency-vs-memory decision. It is intentionally smaller and does not claim their measured-system fidelity.
+
 ## Empirical validation hook
 
 The public project does **not** ship invented "measured" GPU results. Instead, the repository includes an explicit validation path for future real measurements:

app.js

@@ -10,6 +10,9 @@ let lastTopologyRows = [];
 let lastDesignRows = [];
 let lastPairedStudy = null;
 let lastRobustStudy = null;
+let lastAgentRun = null;
+let lastAgentCompare = null;
+let lastAgentTtl = null;
 let traceRequests = [];
 
 const COLORS = {
@@ -32,7 +35,7 @@ worker.addEventListener("message", (event) => {
   if (data.type === "ready") {
     runtimePill.classList.add("ready");
     runtimeText.textContent = "Python runtime ready";
-    ["runBtn", "arenaBtn", "capacityBtn", "topologyBtn", "designBtn", "pairedStudyBtn", "robustStudyBtn"].forEach((id) => { $(id).disabled = false; });
+    ["runBtn", "arenaBtn", "capacityBtn", "topologyBtn", "designBtn", "pairedStudyBtn", "robustStudyBtn", "agentRunBtn", "agentCompareBtn", "agentTtlBtn"].forEach((id) => { $(id).disabled = false; });
     syncConditionalControls();
     window.setTimeout(() => document.body.classList.add("runtime-ready"), 650);
     return;
@@ -215,8 +218,12 @@ function destroyChart(name) {
 }
 
 function chartFileName(card) {
-  const cfg = configFromUI();
   const chartName = card.dataset.chartName || "figure";
+  if (chartName.startsWith("agent-")) {
+    const cfg = agentConfigFromUI();
+    return `inferscale_${slug(chartName)}_${slug(cfg.model)}_${slug(cfg.accelerator)}_${slug(cfg.retention_policy)}_${slug(cfg.routing_policy)}_${stamp()}.png`;
+  }
+  const cfg = configFromUI();
   const topology = cfg.topology === "disaggregated_pd"
     ? `pd-${cfg.prefill_workers}p-${cfg.decode_workers}d-${cfg.prefill_accelerator}-${cfg.decode_accelerator}`
     : `colocated-${cfg.accelerator}`;
@@ -734,13 +741,157 @@ $("robustStudyBtn").addEventListener("click", async () => {
     alert(`Robustness study failed: ${error.message}`);
   } finally {
     button.disabled = false;
-    button.textContent = "Stress-test conclusion";
+    button.textContent = "Stress-test selected hypothesis";
   }
 });
 $("robustCopyBtn").addEventListener("click", () => { if (lastRobustStudy) copyText(tableText(robustHeaders, robustTableRows(lastRobustStudy)), "Robustness table copied"); });
 $("robustCsvBtn").addEventListener("click", () => { if (lastRobustStudy) downloadCsv(`inferscale_robustness_${slug(lastRobustStudy.study)}_${stamp()}.csv`, robustHeaders, robustTableRows(lastRobustStudy)); });
 $("robustJsonBtn").addEventListener("click", () => { if (lastRobustStudy) downloadNamedJson(`inferscale_robustness_${slug(lastRobustStudy.study)}`, lastRobustStudy); });
 
+
+function agentConfigFromUI(overrides = {}) {
+  return {
+    model: $("agentModel").value,
+    accelerator: $("agentAccelerator").value,
+    quantization: $("agentQuantization").value,
+    replicas: num("agentReplicas"),
+    session_rate_rps: num("agentRate"),
+    duration_s: num("agentDuration"),
+    turns_mean: num("agentTurns"),
+    turns_cv: num("agentTurnsCv"),
+    initial_prompt_tokens_mean: num("agentInitialPrompt"),
+    append_tokens_mean: num("agentAppend"),
+    output_tokens_mean: num("agentOutput"),
+    token_cv: num("agentTokenCv"),
+    output_tokens_cv: num("agentTokenCv"),
+    tool_gap_mean_s: num("agentToolGap"),
+    tool_gap_cv: num("agentToolGapCv"),
+    seed: num("agentSeed"),
+    retention_policy: $("agentRetention").value,
+    routing_policy: $("agentRouting").value,
+    kv_ttl_s: num("agentTtl"),
+    kv_memory_fraction: num("agentKvFraction"),
+    slo_turn_ttft_ms: num("agentTtftSlo"),
+    slo_session_e2e_ms: num("agentSessionSlo"),
+    ...overrides,
+  };
+}
+
+function renderAgentRun(result) {
+  lastAgentRun = result;
+  $("agentRunEmpty").classList.add("hidden");
+  $("agentRunContent").classList.remove("hidden");
+  const l = result.latency;
+  const r = result.resource;
+  const sm = result.summary;
+  $("agentTtft").textContent = `${fmt(l.turn_ttft_ms.p95)} ms`;
+  $("agentSessionE2e").textContent = `${fmt(l.session_e2e_ms.p95)} ms`;
+  $("agentCacheHit").textContent = pct(r.cross_turn_cache_hit_rate);
+  $("agentRecompute").textContent = `${fmt(r.recomputed_history_tokens, 0)} tok`;
+  $("agentPeakKv").textContent = `${fmt(r.peak_kv_gb, 3)} GB`;
+  $("agentMeanKv").textContent = `${fmt(r.mean_kv_gb, 3)} GB`;
+  const state = $("agentRunState");
+  state.textContent = `${sm.sessions_completed}/${sm.sessions_generated} sessions`;
+  state.className = `tag ${sm.sessions_completed === sm.sessions_generated && sm.turns_failed === 0 ? "good" : "bad"}`;
+  $("agentRunSummary").innerHTML = `<strong>${escapeHtml(result.config.retention_policy)} retention + ${escapeHtml(result.config.routing_policy.replaceAll("_", " "))} routing.</strong> ${sm.turns_completed} turns completed across ${sm.sessions_completed} sessions. Cross-turn reuse avoided ${fmt(r.recomputed_history_tokens, 0)} history tokens of recomputation only when state was resident on the selected replica; the reported HBM residency is a simulator-side memory-time accounting metric.`;
+
+  const timeline = result.timeline || [];
+  destroyChart("agentTimeline");
+  charts.agentTimeline = new Chart($("agentTimelineChart"), {
+    type: "line",
+    data: {
+      datasets: [
+        { label: "KV GB", data: timeline.map((x) => ({ x: x.time_s, y: x.kv_used_gb })), borderColor: COLORS.green, backgroundColor: COLORS.green, pointRadius: 0, tension: .08, yAxisID: "yKv" },
+        { label: "Busy replicas", data: timeline.map((x) => ({ x: x.time_s, y: x.busy_replicas })), borderColor: COLORS.blue, backgroundColor: COLORS.blue, pointRadius: 0, tension: .08, yAxisID: "yCount" },
+        { label: "Queued turns", data: timeline.map((x) => ({ x: x.time_s, y: x.queued_turns })), borderColor: COLORS.amber, backgroundColor: COLORS.amber, pointRadius: 0, tension: .08, yAxisID: "yCount" },
+      ],
+    },
+    options: { ...commonChartOptions(), parsing: false, plugins: { legend: lineLegend() }, scales: { x: { type: "linear", title: { display: true, text: "Virtual time (s)" } }, yCount: { position: "left", beginAtZero: true, title: { display: true, text: "Replicas / queued turns" } }, yKv: { position: "right", beginAtZero: true, grid: { drawOnChartArea: false }, title: { display: true, text: "KV GB" } } } },
+  });
+
+  destroyChart("agentTurn");
+  charts.agentTurn = new Chart($("agentTurnChart"), {
+    type: "scatter",
+    data: { datasets: [{ label: "Turn", data: (result.turns || []).map((x) => ({ x: x.turn_index, y: x.ttft_ms })), backgroundColor: COLORS.blue, pointRadius: 3, pointHoverRadius: 5 }] },
+    options: { ...commonChartOptions(), plugins: { legend: { display: false } }, scales: { x: { type: "linear", ticks: { precision: 0 }, title: { display: true, text: "Turn index" } }, y: { beginAtZero: true, title: { display: true, text: "TTFT (ms)" } } } },
+  });
+}
+
+$("agentRunBtn").addEventListener("click", async () => {
+  const button = $("agentRunBtn");
+  button.disabled = true;
+  button.textContent = "Running stateful simulation...";
+  $("agentRunState").textContent = "Running...";
+  $("agentRunState").className = "tag neutral";
+  try { renderAgentRun(await callPython("agent_simulate", { config: agentConfigFromUI() })); }
+  catch (error) { $("agentRunState").textContent = "Error"; $("agentRunState").className = "tag bad"; alert(`Agent session simulation failed: ${error.message}`); }
+  finally { button.disabled = false; button.textContent = "Run stateful session simulation"; }
+});
+
+const agentCompareHeaders = ["Policy", "p95 turn TTFT", "p95 session E2E", "Cache hit", "Route locality", "Recomputed history", "Mean KV", "Peak KV", "HBM GB-s", "Evictions"];
+function agentCompareTableRows(result) {
+  return result.rows.map((r) => [r.label, `${fmt(r.p95_turn_ttft_ms)} ms`, `${fmt(r.p95_session_e2e_ms)} ms`, pct(r.cache_hit_rate), pct(r.routing_locality_rate), `${fmt(r.recomputed_history_tokens, 0)} tok`, `${fmt(r.mean_kv_gb, 3)} GB`, `${fmt(r.peak_kv_gb, 3)} GB`, fmt(r.hbm_gb_seconds, 3), r.pressure_evictions + r.ttl_evictions]);
+}
+function renderAgentCompare(result) {
+  lastAgentCompare = result;
+  $("agentCompareEmpty").classList.add("hidden");
+  $("agentCompareContent").classList.remove("hidden");
+  ["agentCompareCopy", "agentCompareCsv"].forEach((id) => { $(id).disabled = false; });
+  $("agentCompareRows").innerHTML = result.rows.map((r) => `<tr><td>${escapeHtml(r.label)}</td><td>${fmt(r.p95_turn_ttft_ms)} ms</td><td>${fmt(r.p95_session_e2e_ms)} ms</td><td>${pct(r.cache_hit_rate)}</td><td>${pct(r.routing_locality_rate)}</td><td>${fmt(r.recomputed_history_tokens, 0)} tok</td><td>${fmt(r.mean_kv_gb, 3)} GB</td><td>${fmt(r.peak_kv_gb, 3)} GB</td><td>${fmt(r.hbm_gb_seconds, 3)}</td><td>${r.pressure_evictions + r.ttl_evictions}</td></tr>`).join("");
+  destroyChart("agentCompare");
+  charts.agentCompare = new Chart($("agentCompareChart"), {
+    type: "scatter",
+    data: { datasets: result.rows.map((r, i) => ({ label: r.label, data: [{ x: r.mean_kv_gb, y: r.p95_turn_ttft_ms }], backgroundColor: [COLORS.gray, COLORS.amber, COLORS.green, COLORS.blue][i], borderColor: [COLORS.gray, COLORS.amber, COLORS.green, COLORS.blue][i], pointRadius: 7, pointHoverRadius: 9 })) },
+    options: { ...commonChartOptions(), plugins: { legend: pointLegend() }, scales: { x: { beginAtZero: true, title: { display: true, text: "Mean resident KV (GB)" } }, y: { beginAtZero: true, title: { display: true, text: "p95 turn TTFT (ms)" } } } },
+  });
+}
+$("agentCompareBtn").addEventListener("click", async () => {
+  const button = $("agentCompareBtn"); button.disabled = true; button.textContent = "Comparing...";
+  try { renderAgentCompare(await callPython("agent_compare", { config: agentConfigFromUI() })); }
+  catch (error) { alert(`Agent policy comparison failed: ${error.message}`); }
+  finally { button.disabled = false; button.textContent = "Compare 4 policies"; }
+});
+$("agentCompareCopy").addEventListener("click", () => { if (lastAgentCompare) copyText(tableText(agentCompareHeaders, agentCompareTableRows(lastAgentCompare)), "Agent policy table copied"); });
+$("agentCompareCsv").addEventListener("click", () => { if (lastAgentCompare) downloadCsv(`inferscale_agent-policy-comparison_${stamp()}.csv`, agentCompareHeaders, agentCompareTableRows(lastAgentCompare)); });
+
+const agentTtlHeaders = ["TTL", "Pareto", "p95 turn TTFT", "p95 session E2E", "Cache hit", "Recomputed history", "Mean KV", "HBM GB-s", "TTL evictions", "Pressure evictions"];
+function agentTtlTableRows(result) {
+  return result.rows.map((r) => [`${fmt(r.ttl_s, 2)} s`, r.pareto ? "YES" : "NO", `${fmt(r.p95_turn_ttft_ms)} ms`, `${fmt(r.p95_session_e2e_ms)} ms`, pct(r.cache_hit_rate), `${fmt(r.recomputed_history_tokens, 0)} tok`, `${fmt(r.mean_kv_gb, 3)} GB`, fmt(r.hbm_gb_seconds, 3), r.ttl_evictions, r.pressure_evictions]);
+}
+function renderAgentTtl(result) {
+  lastAgentTtl = result;
+  $("agentTtlEmpty").classList.add("hidden");
+  $("agentTtlContent").classList.remove("hidden");
+  ["agentTtlCopy", "agentTtlCsv"].forEach((id) => { $(id).disabled = false; });
+  $("agentTtlRows").innerHTML = result.rows.map((r) => `<tr><td>${fmt(r.ttl_s, 2)} s</td><td class="${r.pareto ? "pass" : ""}">${r.pareto ? "YES" : "NO"}</td><td>${fmt(r.p95_turn_ttft_ms)} ms</td><td>${fmt(r.p95_session_e2e_ms)} ms</td><td>${pct(r.cache_hit_rate)}</td><td>${fmt(r.recomputed_history_tokens, 0)} tok</td><td>${fmt(r.mean_kv_gb, 3)} GB</td><td>${fmt(r.hbm_gb_seconds, 3)}</td><td>${r.ttl_evictions}</td><td>${r.pressure_evictions}</td></tr>`).join("");
+  const pareto = result.rows.filter((r) => r.pareto).sort((a, b) => a.mean_kv_gb - b.mean_kv_gb);
+  destroyChart("agentTtl");
+  charts.agentTtl = new Chart($("agentTtlChart"), {
+    type: "scatter",
+    data: { datasets: [
+      { label: "TTL candidates", data: result.rows.map((r) => ({ x: r.mean_kv_gb, y: r.p95_turn_ttft_ms, ttl: r.ttl_s })), backgroundColor: COLORS.blue, pointRadius: 5, pointHoverRadius: 7 },
+      { type: "line", label: "Retention frontier", data: pareto.map((r) => ({ x: r.mean_kv_gb, y: r.p95_turn_ttft_ms })), borderColor: COLORS.amber, backgroundColor: COLORS.amber, pointRadius: 5, tension: 0, fill: false },
+    ] },
+    options: { ...commonChartOptions(), plugins: { legend: lineLegend(), tooltip: { callbacks: { afterLabel: (ctx) => ctx.datasetIndex === 0 ? `TTL ${fmt(ctx.raw.ttl, 2)} s` : "" } } }, scales: { x: { beginAtZero: true, title: { display: true, text: "Mean resident KV (GB)" } }, y: { beginAtZero: true, title: { display: true, text: "p95 turn TTFT (ms)" } } } },
+  });
+}
+$("agentTtlBtn").addEventListener("click", async () => {
+  const button = $("agentTtlBtn"); button.disabled = true; button.textContent = "Sweeping TTL...";
+  try { renderAgentTtl(await callPython("agent_ttl_sweep", { config: agentConfigFromUI() })); }
+  catch (error) { alert(`TTL sweep failed: ${error.message}`); }
+  finally { button.disabled = false; button.textContent = "Run TTL sweep"; }
+});
+$("agentTtlCopy").addEventListener("click", () => { if (lastAgentTtl) copyText(tableText(agentTtlHeaders, agentTtlTableRows(lastAgentTtl)), "TTL sweep copied"); });
+$("agentTtlCsv").addEventListener("click", () => { if (lastAgentTtl) downloadCsv(`inferscale_agent-ttl-frontier_${stamp()}.csv`, agentTtlHeaders, agentTtlTableRows(lastAgentTtl)); });
+
+function syncAgentControls() {
+  const ttl = $("agentRetention").value === "ttl";
+  $("agentTtl").disabled = !ttl;
+  $("agentTtlLabel").classList.toggle("field-disabled", !ttl);
+}
+$("agentRetention").addEventListener("change", syncAgentControls);
+syncAgentControls();
+
 function normalizeTraceRows(rows) {
   if (!Array.isArray(rows)) throw new Error("Trace JSON must be an array or contain a requests array");
   if (rows.length > 10000) throw new Error("Trace replay is limited to 10,000 requests in the browser");

docs/architecture.md

@@ -13,8 +13,9 @@ InferScale-Sim separates **serving-system logic**, **analytical latency estimati
 7. `diagnostics.py` converts simulated telemetry into explicit heuristic bottleneck labels.
 8. `optimizer.py` implements capacity search, scheduler comparison, topology/cache comparison, and Pareto sweeps.
 9. `research.py` implements paired common-seed A/B studies, bootstrap intervals, and analytical-model sensitivity analysis.
-10. `validation.py` compares predictions against externally supplied measured cases.
-11. `api.py` exposes JSON-like actions to local Python and Pyodide.
+10. `agentic.py` models multi-turn programs, tool gaps, session routing, KV retention/TTL eviction, and retention frontiers.
+11. `validation.py` compares predictions against externally supplied measured cases.
+12. `api.py` exposes JSON-like actions to local Python and Pyodide.
 
 ## Colocated path
 
@@ -36,6 +37,24 @@ arrival
 
 The P/D path is a genuine discrete-event loop: prefill workers, decode workers, and the transfer link can overlap in virtual time.
 
+
+## Stateful agent-session path
+
+```text
+session arrival
+  -> turn ready
+  -> route to replica
+  -> [KV hit: append prefill | KV miss: full-history prefill]
+  -> decode
+  -> retain / TTL / evict KV
+  -> tool gap
+  -> next turn ready
+  -> ...
+  -> session complete
+```
+
+Each replica is intentionally a serial service station in this mode. This isolates state residency, routing locality, and tool-gap effects from the dynamic-batching questions already covered by the request-level simulators.
+
 ## Research path
 
 ```text

docs/methodology.md

@@ -129,6 +129,22 @@ The robustness study draws shared multiplicative prefill/decode/transfer scales
 
 The perturbation distribution is deliberately described as a **sensitivity analysis**, not a calibrated posterior over real hardware. Its purpose is to identify conclusions that reverse under modest model error.
 
+
+## Stateful agent sessions
+
+Agent mode is a separate program-level discrete-event model. Session arrivals are open-loop. Each session receives a deterministic number of turns, token increments, outputs, and tool gaps from a seeded trace. A later turn cannot become ready until the prior turn completes and its tool gap elapses.
+
+Each simulated replica is a serial analytical service station in this mode. This is deliberate: dynamic batching remains covered by Serving Lab, while Agent Sessions isolates four stateful effects:
+
+1. **cross-turn reuse:** a resident KV entry means the next turn prefills only appended tokens;
+2. **tool-gap residency:** retained KV occupies memory while the program waits outside the model;
+3. **routing locality:** session-affinity can preserve reuse but may trade against load balance;
+4. **eviction:** TTL expiry and LRU-style memory-pressure eviction can force full-history recomputation.
+
+The cache working set is tracked independently on each replica. The simulator integrates occupancy over virtual time and reports `HBM GB-seconds`, peak KV, mean KV, cache-hit rate, recomputed history tokens, routing-locality rate, and eviction counts.
+
+The TTL sweep replays one identical agent-program trace for every TTL and reports the non-dominated frontier minimizing both p95 turn TTFT and mean KV residency. It is a controlled what-if study, not an optimizer over a measured production system.
+
 ## Empirical validation
 
 `validation.py` accepts externally measured cases and compares them against simulator predictions. Supported observations include p95 TTFT, p95 E2E, goodput, request throughput, and SLO attainment. The report contains per-case residuals, MAPE, median APE, and maximum APE.

docs/research.md

@@ -57,7 +57,7 @@ Source: https://arxiv.org/abs/2605.21312
 
 AgentServeSim extends serving simulation to multi-turn programs with tool-induced gaps, session-aware routing, cache locality, and KV residency. It reports reproducing real-system behavior within 6% across its evaluated metrics while executing on CPUs.
 
-InferScale uses this as a roadmap boundary: exact trace replay exists today, while session identity, tool gaps, and cross-turn residency are explicit future work rather than being approximated silently.
+InferScale now implements a smaller program-level analogue: session identity, ordered turns, explicit tool gaps, cross-turn KV residency, session-affinity routing, and TTL/pressure eviction. Its per-replica service station is intentionally simpler than AGENTSERVESIM and remains analytically profiled rather than empirically validated.
 
 Source: https://arxiv.org/abs/2606.09613
 
@@ -83,3 +83,26 @@ Source: https://arxiv.org/abs/2608.03741
 ## Scope boundary
 
 InferScale-Sim is not intended to compete with these research systems on fidelity, hardware scale, or runtime compatibility. Its aim is an inspectable Python implementation with a zero-backend interactive interface, explicit uncertainty, controlled experiments, and a path to external empirical validation.
+
+
+## CacheTTL (2026)
+
+CacheTTL studies multi-turn agent scheduling under variable tool-call gaps and treats KV retention as a time-to-live decision: holding state improves future reuse but occupies accelerator memory while the program is waiting on tools.
+
+InferScale inspiration: explicit TTL retention, expiry events, memory-time accounting, and a TTL frontier that makes latency versus retained KV visible on a common program trace.
+
+Source: https://arxiv.org/abs/2511.02230
+
+## AgentSysBench (August 2026)
+
+AgentSysBench characterizes agentic applications as long-running, stateful workflows with heterogeneous components, shifting bottlenecks, and idle state that can persist between active steps. The paper reports that non-LLM components can dominate latency in several applications and that state/offloading choices materially affect resource use.
+
+InferScale inspiration: model tool gaps and whole-session latency rather than treating every LLM call as an independent request.
+
+Source: https://arxiv.org/abs/2608.15127
+
+## CacheScout / agent-aware KV management (2026)
+
+Recent agent-aware KV work argues that future reuse is governed by program execution semantics rather than recency alone and uses predicted transitions to guide retention/prefetch. InferScale does not implement learned prediction, but its session-affinity and TTL experiments provide an inspectable baseline for studying why agent identity changes cache decisions.
+
+Source: https://arxiv.org/abs/2608.14624

docs/validation.md

@@ -24,6 +24,11 @@ Release checks prevent software, deployment, and provenance mistakes; they do no
 - bootstrap paired-effect intervals
 - analytical-profile perturbation study
 - external-measurement validation report generation
+- deterministic stateful agent-session trace generation
+- zero cross-turn hits under immediate eviction
+- KV reuse and reduced recomputation under retention + affinity
+- common-trace four-policy agent comparison
+- TTL latency/memory Pareto frontier
 - ASCII-only public UI labels
 - chart export controls present
 - explicit planner worst-repetition and target columns
@@ -67,6 +72,7 @@ The report provides residuals and aggregate percentage errors. Calibration shoul
 - radix-tree prefix-cache eviction/scheduling
 - speculative decoding
 - Attention-FFN disaggregation
-- multi-turn/agentic session fidelity
+- production-fidelity agentic serving or dynamic batching inside Agent Sessions
+- learned workflow prediction, proactive prefetch, host/CXL KV tiers, or real tool execution
 
 These are explicit scope boundaries, not hidden assumptions.

index.html

@@ -44,6 +44,7 @@
       <button class="tab" data-tab="modern">Modern Serving</button>
       <button class="tab" data-tab="design">Design Explorer</button>
       <button class="tab" data-tab="research">Research Studies</button>
+      <button class="tab" data-tab="agent">Agent Sessions</button>
       <button class="tab" data-tab="method">Methodology</button>
     </nav>
 
@@ -280,6 +281,7 @@
             <option value="chunked_vs_fcfs">Scheduling: FCFS vs chunked prefill</option>
             <option value="slo_vs_fcfs">Scheduling: FCFS vs least-slack</option>
           </select></label>
+          <p class="control-help">The selected hypothesis controls both the paired study and the model-uncertainty stress test below.</p>
           <hr />
           <div class="section-kicker">Paired Monte Carlo</div>
           <div class="field-grid two">
@@ -289,11 +291,12 @@
           <button id="pairedStudyBtn" class="primary" disabled>Run paired study</button>
           <hr />
           <div class="section-kicker">Model uncertainty stress test</div>
+          <p class="control-help">Uses the Hypothesis selected above. For P/D robustness, choose Topology: colocated vs P/D, set the perturbation controls here, then click the button below.</p>
           <div class="field-grid two">
             <label>Samples<input id="robustSamples" type="number" min="4" max="96" value="32" /></label>
             <label>Latency uncertainty<input id="robustUncertainty" type="number" min="0" max="0.75" step="0.05" value="0.20" /><span class="unit">fraction</span></label>
           </div>
-          <button id="robustStudyBtn" class="secondary research-secondary" disabled>Stress-test conclusion</button>
+          <button id="robustStudyBtn" class="secondary research-secondary" disabled>Stress-test selected hypothesis</button>
         </aside>
 
         <div class="research-results">
@@ -330,11 +333,110 @@
       </div>
     </section>
 
+
+    <section id="agent" class="tab-panel">
+      <div class="agent-layout">
+        <aside class="panel agent-controls">
+          <div class="panel-title-row"><div><div class="section-kicker">Stateful workload</div><h2>Agent session protocol</h2></div><span class="tag">Common program trace</span></div>
+          <p class="muted">Model multi-turn programs separated by tool calls. Cross-turn KV can be evicted, retained, or held under a TTL, while routing can prioritize load balance or session locality.</p>
+          <div class="field-grid two">
+            <label>Model<select id="agentModel"><option>Qwen2.5-3B</option><option>Llama-3.1-8B</option><option>Mistral-7B-v0.3</option></select></label>
+            <label>Accelerator<select id="agentAccelerator"><option value="L4">NVIDIA L4</option><option value="A10G">NVIDIA A10G</option><option value="A100-40GB">NVIDIA A100 40GB</option></select></label>
+            <label>Weight precision<select id="agentQuantization"><option value="fp16">FP16</option><option value="int8" selected>INT8 scenario</option><option value="int4">INT4 scenario</option></select></label>
+            <label>Replicas<input id="agentReplicas" type="number" min="1" max="8" value="2" /></label>
+          </div>
+          <hr />
+          <div class="section-kicker">Program arrivals</div>
+          <div class="field-grid two">
+            <label>Session rate<input id="agentRate" type="number" min="0.01" step="0.05" value="0.20" /><span class="unit">sessions/s</span></label>
+            <label>Arrival horizon<input id="agentDuration" type="number" min="5" step="5" value="30" /><span class="unit">s</span></label>
+            <label>Mean turns<input id="agentTurns" type="number" min="2" step="0.5" value="4" /></label>
+            <label>Turns CV<input id="agentTurnsCv" type="number" min="0" max="1.5" step="0.05" value="0.25" /></label>
+            <label>Initial prompt<input id="agentInitialPrompt" type="number" min="32" step="32" value="640" /><span class="unit">tokens</span></label>
+            <label>Per-turn append<input id="agentAppend" type="number" min="1" step="16" value="180" /><span class="unit">tokens</span></label>
+            <label>Mean output<input id="agentOutput" type="number" min="1" step="8" value="72" /><span class="unit">tokens</span></label>
+            <label>Token CV<input id="agentTokenCv" type="number" min="0" max="1.5" step="0.05" value="0.35" /></label>
+            <label>Tool gap mean<input id="agentToolGap" type="number" min="0" step="0.25" value="1.5" /><span class="unit">s</span></label>
+            <label>Tool gap CV<input id="agentToolGapCv" type="number" min="0" max="2" step="0.05" value="0.75" /></label>
+            <label>Seed<input id="agentSeed" type="number" step="1" value="7" /></label>
+          </div>
+          <hr />
+          <div class="section-kicker">State policy</div>
+          <div class="field-grid two">
+            <label>KV retention<select id="agentRetention"><option value="evict">Evict after every turn</option><option value="ttl" selected>TTL retention</option><option value="retain">Retain until session ends</option></select></label>
+            <label>Routing<select id="agentRouting"><option value="least_load">Least-load</option><option value="session_affinity" selected>Session affinity</option></select></label>
+            <label id="agentTtlLabel">KV TTL<input id="agentTtl" type="number" min="0" step="0.25" value="3" /><span class="unit">s</span></label>
+            <label>KV memory fraction<input id="agentKvFraction" type="number" min="0.05" max="0.98" step="0.05" value="0.85" /><span class="unit">fraction</span></label>
+          </div>
+          <div class="field-grid two">
+            <label>Turn TTFT SLO<input id="agentTtftSlo" type="number" min="1" value="500" /><span class="unit">ms</span></label>
+            <label>Session E2E SLO<input id="agentSessionSlo" type="number" min="1000" step="1000" value="30000" /><span class="unit">ms</span></label>
+          </div>
+          <button id="agentRunBtn" class="primary" disabled>Run stateful session simulation</button>
+        </aside>
+
+        <div class="agent-results">
+          <section class="panel research-panel">
+            <div class="panel-title-row"><div><div class="section-kicker">Program execution</div><h2>Session run</h2></div><span id="agentRunState" class="tag neutral">Waiting</span></div>
+            <div id="agentRunEmpty" class="empty-state small"><h3>No stateful run yet</h3><p>Each session has ordered LLM turns separated by sampled tool gaps. A replica processes one turn at a time in this mode so cache/routing effects remain isolated from the batching experiments in Serving Lab.</p></div>
+            <div id="agentRunContent" class="hidden">
+              <div class="metric-grid six-agent">
+                <div class="metric emphasis"><span>p95 turn TTFT</span><strong id="agentTtft">N/A</strong></div>
+                <div class="metric"><span>p95 session E2E</span><strong id="agentSessionE2e">N/A</strong></div>
+                <div class="metric"><span>Cross-turn cache hit</span><strong id="agentCacheHit">N/A</strong></div>
+                <div class="metric"><span>Recomputed history</span><strong id="agentRecompute">N/A</strong></div>
+                <div class="metric"><span>Peak KV</span><strong id="agentPeakKv">N/A</strong></div>
+                <div class="metric"><span>Mean resident KV</span><strong id="agentMeanKv">N/A</strong></div>
+              </div>
+              <div class="study-summary" id="agentRunSummary"></div>
+              <div class="chart-grid">
+                <div class="chart-card" data-chart-card data-chart-name="agent-session-kv-residency-timeline">
+                  <div class="chart-head"><div class="chart-title">KV residency and replica activity</div><div class="chart-actions"><button class="chart-download" type="button">Download PNG</button><button class="chart-expand" type="button">Expand</button></div></div>
+                  <div class="chart-body large"><canvas id="agentTimelineChart"></canvas></div>
+                </div>
+                <div class="chart-card" data-chart-card data-chart-name="agent-turn-ttft-by-turn-index">
+                  <div class="chart-head"><div class="chart-title">Turn TTFT across session depth</div><div class="chart-actions"><button class="chart-download" type="button">Download PNG</button><button class="chart-expand" type="button">Expand</button></div></div>
+                  <div class="chart-body large"><canvas id="agentTurnChart"></canvas></div>
+                </div>
+              </div>
+            </div>
+          </section>
+
+          <section class="panel research-panel">
+            <div class="panel-title-row"><div><div class="section-kicker">Controlled policy comparison</div><h2>Session Policy Arena</h2><p class="muted">All candidates replay the identical generated program trace.</p></div><button id="agentCompareBtn" class="primary compact" disabled>Compare 4 policies</button></div>
+            <div id="agentCompareEmpty" class="empty-state small"><h3>No policy comparison yet</h3><p>Compare stateless load balancing, retention without affinity, TTL with affinity, and full retention with affinity.</p></div>
+            <div id="agentCompareContent" class="hidden">
+              <div class="chart-card full" data-chart-card data-chart-name="agent-policy-latency-memory-tradeoff">
+                <div class="chart-head"><div class="chart-title">p95 turn TTFT vs mean resident KV</div><div class="chart-actions"><button class="chart-download" type="button">Download PNG</button><button class="chart-expand" type="button">Expand</button></div></div>
+                <div class="chart-body research-chart"><canvas id="agentCompareChart"></canvas></div>
+              </div>
+              <div class="table-toolbar"><span>Common-trace policy results</span><div><button id="agentCompareCopy" class="mini-button" disabled>Copy table</button><button id="agentCompareCsv" class="mini-button" disabled>Download CSV</button></div></div>
+              <div class="table-wrap"><table><thead><tr><th>Policy</th><th>p95 turn TTFT</th><th>p95 session E2E</th><th>Cache hit</th><th>Route locality</th><th>Recomputed history</th><th>Mean KV</th><th>Peak KV</th><th>HBM GB-s</th><th>Evictions</th></tr></thead><tbody id="agentCompareRows"></tbody></table></div>
+            </div>
+          </section>
+
+          <section class="panel research-panel">
+            <div class="panel-title-row"><div><div class="section-kicker">Retention trade-off</div><h2>TTL frontier</h2><p class="muted">Sweep the KV retention horizon on the same program trace and expose the latency vs memory-residency frontier.</p></div><button id="agentTtlBtn" class="primary compact" disabled>Run TTL sweep</button></div>
+            <div id="agentTtlEmpty" class="empty-state small"><h3>No TTL sweep yet</h3><p>The sweep varies retention from immediate eviction through long-lived state while keeping session arrivals, turns, tool gaps, and token lengths fixed.</p></div>
+            <div id="agentTtlContent" class="hidden">
+              <div class="chart-card full" data-chart-card data-chart-name="agent-ttl-retention-frontier">
+                <div class="chart-head"><div class="chart-title">Mean resident KV vs p95 turn TTFT</div><div class="chart-actions"><button class="chart-download" type="button">Download PNG</button><button class="chart-expand" type="button">Expand</button></div></div>
+                <div class="chart-body research-chart"><canvas id="agentTtlChart"></canvas></div>
+              </div>
+              <div class="table-toolbar"><span>TTL sweep</span><div><button id="agentTtlCopy" class="mini-button" disabled>Copy table</button><button id="agentTtlCsv" class="mini-button" disabled>Download CSV</button></div></div>
+              <div class="table-wrap"><table><thead><tr><th>TTL</th><th>Pareto</th><th>p95 turn TTFT</th><th>p95 session E2E</th><th>Cache hit</th><th>Recomputed history</th><th>Mean KV</th><th>HBM GB-s</th><th>TTL evictions</th><th>Pressure evictions</th></tr></thead><tbody id="agentTtlRows"></tbody></table></div>
+            </div>
+          </section>
+        </div>
+      </div>
+    </section>
+
     <section id="method" class="tab-panel">
       <div class="method-grid">
         <article class="panel prose"><div class="section-kicker">Simulation core</div><h2>What is actually simulated?</h2><p>Requests are generated from deterministic workload distributions and advanced through virtual time. Colocated runs model admission, prefill, paged KV allocation, dynamic decode batches, and completion. P/D runs use separate prefill and decode worker pools plus an explicit serialized KV-transfer link.</p><div class="formula">request -> queue -> prefill -> [KV transfer] -> decode -> completion</div></article>
         <article class="panel prose"><div class="section-kicker">Prefix reuse</div><h2>Cache without pretending to implement a radix tree</h2><p>The simulator models a single shared prompt prefix with configurable length and reuse fraction. Cache hits avoid redundant prefill work and share one persistent KV allocation. It is deliberately a controlled what-if abstraction, not a claim to reproduce SGLang's full RadixAttention policy.</p></article>
         <article class="panel prose"><div class="section-kicker">P/D disaggregation</div><h2>Role-specific resources and transfer cost</h2><p>Prefill and decode have separate accelerator profiles and worker counts. Prompt KV state crosses a modeled interconnect before decode admission. The simulator reports role utilization, transfer latency, and transfer pressure so the benefit of isolation can be weighed against data-movement overhead.</p></article>
+        <article class="panel prose"><div class="section-kicker">Stateful agent sessions</div><h2>Tool gaps turn KV into a residency decision</h2><p>Agent Sessions preserves program identity and turn order, materializes tool-induced gaps, and tracks whether cross-turn KV remains resident, expires under a TTL, or is recomputed after eviction. Session-affinity routing can trade load balance for cache locality. The per-replica service model is intentionally serial in this mode so those state-management effects are not confounded with batching.</p></article>
         <article class="panel prose"><div class="section-kicker">Design search</div><h2>Pareto, not one magic configuration</h2><p>The Design Explorer reports both a raw-performance frontier and a resource-normalized frontier using goodput per accelerator. That prevents a multi-GPU P/D layout from looking unconditionally better merely because it uses more simulated hardware.</p></article>
         <article class="panel prose"><div class="section-kicker">Trace methodology</div><h2>Generated load or exact replay</h2><p>Poisson, constant, and bursty workloads are open-loop: arrivals are scheduled independently of response completion, so queueing delay is not hidden by client backpressure. Exact CSV/JSON traces can be replayed with their original arrival times and token lengths.</p></article>
         <article class="panel prose"><div class="section-kicker">Research protocol</div><h2>Paired conclusions, not one lucky seed</h2><p>Research Studies use common random numbers: baseline and treatment receive identical seeds, reducing workload variance in the paired difference. A bootstrap interval summarizes the repeated effect, while a separate sensitivity study perturbs analytical latency scales to reveal conclusions that depend too strongly on one reference profile.</p></article>

py/inferscale/__init__.py

@@ -1,3 +1,4 @@
+from .agentic import compare_agent_policies, run_agent_session_simulation, ttl_retention_sweep
 from .api import execute, metadata
 from .models import SimulationConfig
 from .optimizer import capacity_search, compare_schedulers, compare_topologies, design_space_search
@@ -7,6 +8,7 @@ from .validation import validate_cases
 
 __all__ = [
     "SimulationConfig",
+    "compare_agent_policies",
     "capacity_search",
     "compare_schedulers",
     "compare_topologies",
@@ -15,10 +17,12 @@ __all__ = [
     "metadata",
     "paired_study",
     "robustness_study",
+    "run_agent_session_simulation",
     "run_simulation",
+    "ttl_retention_sweep",
     "validate_cases",
 ]
 
 # Internal package metadata only; the public project intentionally avoids
 # release/version branding in the interface and documentation.
-__version__ = "0.4.0"
+__version__ = "0.5.0"

py/inferscale/agentic.py

@@ -0,0 +1,622 @@
+from __future__ import annotations
+
+import heapq
+import math
+import random
+from dataclasses import asdict, dataclass
+from statistics import mean, median
+from typing import Any
+
+from .latency import AnalyticalLatencyModel
+from .metrics import percentile
+from .profiles import get_accelerator, get_model
+
+
+ROUTING_POLICIES = {"least_load", "session_affinity"}
+RETENTION_POLICIES = {"evict", "retain", "ttl"}
+
+
+@dataclass
+class AgentSessionConfig:
+    model: str = "Qwen2.5-3B"
+    accelerator: str = "L4"
+    quantization: str = "int8"
+    seed: int = 7
+    duration_s: float = 30.0
+    session_rate_rps: float = 0.45
+    replicas: int = 2
+    turns_mean: float = 4.0
+    turns_cv: float = 0.25
+    initial_prompt_tokens_mean: int = 640
+    append_tokens_mean: int = 180
+    token_cv: float = 0.35
+    output_tokens_mean: int = 72
+    output_tokens_cv: float = 0.45
+    tool_gap_mean_s: float = 1.5
+    tool_gap_cv: float = 0.75
+    retention_policy: str = "ttl"
+    kv_ttl_s: float = 3.0
+    routing_policy: str = "session_affinity"
+    kv_memory_fraction: float = 0.85
+    slo_turn_ttft_ms: float = 500.0
+    slo_session_e2e_ms: float = 30000.0
+    timeline_points: int = 240
+
+    @classmethod
+    def from_dict(cls, data: dict[str, Any]) -> "AgentSessionConfig":
+        allowed = cls.__dataclass_fields__.keys()
+        cfg = cls(**{k: data[k] for k in allowed if k in data})
+        if cfg.routing_policy not in ROUTING_POLICIES:
+            raise ValueError(f"Unsupported agent routing policy: {cfg.routing_policy}")
+        if cfg.retention_policy not in RETENTION_POLICIES:
+            raise ValueError(f"Unsupported KV retention policy: {cfg.retention_policy}")
+        cfg.replicas = max(1, min(int(cfg.replicas), 8))
+        cfg.session_rate_rps = max(float(cfg.session_rate_rps), 0.01)
+        cfg.duration_s = max(float(cfg.duration_s), 1.0)
+        cfg.kv_ttl_s = max(float(cfg.kv_ttl_s), 0.0)
+        cfg.kv_memory_fraction = min(max(float(cfg.kv_memory_fraction), 0.05), 0.98)
+        return cfg
+
+    def to_dict(self) -> dict[str, Any]:
+        return asdict(self)
+
+
+@dataclass
+class TurnSpec:
+    session_id: int
+    turn_index: int
+    append_tokens: int
+    output_tokens: int
+    tool_gap_after_s: float
+
+
+@dataclass
+class SessionSpec:
+    session_id: int
+    arrival_time: float
+    initial_prompt_tokens: int
+    turns: list[TurnSpec]
+
+
+@dataclass
+class CacheEntry:
+    session_id: int
+    tokens: int
+    size_gb: float
+    last_access: float
+    expiry_time: float | None
+    generation: int
+
+
+@dataclass
+class ReplicaState:
+    replica_id: int
+    busy: bool = False
+    busy_until: float = 0.0
+    queue: list[tuple[int, TurnSpec, float]] | None = None
+    cache: dict[int, CacheEntry] | None = None
+    current_session: int | None = None
+
+    def __post_init__(self) -> None:
+        if self.queue is None:
+            self.queue = []
+        if self.cache is None:
+            self.cache = {}
+
+
+@dataclass
+class SessionRuntime:
+    spec: SessionSpec
+    context_tokens: int
+    completed_turns: int = 0
+    completion_time: float | None = None
+    last_replica: int | None = None
+
+
+def _sample_positive_lognormal(rng: random.Random, mean_value: float, cv: float, minimum: float) -> float:
+    mean_value = max(float(mean_value), minimum)
+    cv = max(float(cv), 0.0)
+    if mean_value <= 0.0:
+        return minimum
+    if cv <= 1e-12:
+        return mean_value
+    sigma2 = math.log1p(cv * cv)
+    sigma = math.sqrt(sigma2)
+    mu = math.log(mean_value) - sigma2 / 2.0
+    return max(minimum, rng.lognormvariate(mu, sigma))
+
+
+def _sample_int(rng: random.Random, mean_value: float, cv: float, minimum: int = 1) -> int:
+    return max(minimum, int(round(_sample_positive_lognormal(rng, mean_value, cv, minimum))))
+
+
+def generate_agent_sessions(cfg: AgentSessionConfig) -> list[SessionSpec]:
+    """Generate a deterministic multi-turn program trace.
+
+    All policies consume this same trace when the seed/config are unchanged. Tool
+    gaps are sampled up front so policy comparisons use common random numbers.
+    """
+    rng = random.Random(cfg.seed ^ 0xA63E17)
+    sessions: list[SessionSpec] = []
+    now = 0.0
+    sid = 0
+    while True:
+        now += rng.expovariate(cfg.session_rate_rps)
+        if now > cfg.duration_s:
+            break
+        turns = max(2, _sample_int(rng, cfg.turns_mean, cfg.turns_cv, 2))
+        initial = _sample_int(rng, cfg.initial_prompt_tokens_mean, cfg.token_cv, 32)
+        specs = []
+        for turn_idx in range(turns):
+            append = 0 if turn_idx == 0 else _sample_int(rng, cfg.append_tokens_mean, cfg.token_cv, 8)
+            output = _sample_int(rng, cfg.output_tokens_mean, cfg.output_tokens_cv, 1)
+            gap = 0.0 if turn_idx == turns - 1 else _sample_positive_lognormal(
+                rng, cfg.tool_gap_mean_s, cfg.tool_gap_cv, 0.0
+            )
+            specs.append(TurnSpec(sid, turn_idx, append, output, gap))
+        sessions.append(SessionSpec(sid, now, initial, specs))
+        sid += 1
+    return sessions
+
+
+class AgentSessionSimulator:
+    """Discrete-event simulator for stateful multi-turn serving.
+
+    The module deliberately isolates session locality / KV-retention effects from
+    dynamic batching. Each replica is a serial analytical service station; the
+    existing Serving Lab remains the place to study continuous batching. This
+    separation keeps the agent experiment interpretable while still modelling
+    session dependencies, tool gaps, routing, memory pressure, and cross-turn KV.
+    """
+
+    def __init__(self, cfg: AgentSessionConfig, sessions: list[SessionSpec] | None = None):
+        self.cfg = cfg
+        self.model = get_model(cfg.model)
+        self.accelerator = get_accelerator(cfg.accelerator)
+        self.latency = AnalyticalLatencyModel(self.model, self.accelerator, cfg.quantization)
+        remaining = max(0.0, self.accelerator.vram_gb - self.latency.model_weight_gb - 1.2)
+        self.kv_capacity_gb = remaining * cfg.kv_memory_fraction
+        self.replicas = [ReplicaState(i) for i in range(cfg.replicas)]
+        specs = sessions if sessions is not None else generate_agent_sessions(cfg)
+        self.sessions = {s.session_id: SessionRuntime(s, s.initial_prompt_tokens) for s in specs}
+        self.events: list[tuple[float, int, str, tuple[Any, ...]]] = []
+        self.event_seq = 0
+        self.now = 0.0
+        self.turn_rows: list[dict[str, Any]] = []
+        self.timeline: list[dict[str, Any]] = []
+        self.peak_kv_gb = 0.0
+        self.hbm_gb_seconds = 0.0
+        self.last_memory_time = 0.0
+        self.pressure_evictions = 0
+        self.ttl_evictions = 0
+        self.recompute_tokens = 0
+        self.cache_hits = 0
+        self.cache_eligible_turns = 0
+        self.affinity_routes = 0
+        self.route_opportunities = 0
+        self.failed_turns = 0
+        self.tool_gap_total_s = 0.0
+        for session in specs:
+            if session.turns:
+                self._push(session.arrival_time, "turn_ready", session.session_id, 0)
+
+    def _push(self, when: float, kind: str, *payload: Any) -> None:
+        self.event_seq += 1
+        heapq.heappush(self.events, (float(when), self.event_seq, kind, tuple(payload)))
+
+    def _used_gb(self) -> float:
+        return sum(entry.size_gb for replica in self.replicas for entry in replica.cache.values())
+
+    def _integrate_memory(self, new_time: float) -> None:
+        new_time = max(float(new_time), self.last_memory_time)
+        used = self._used_gb()
+        self.hbm_gb_seconds += used * (new_time - self.last_memory_time)
+        self.last_memory_time = new_time
+        self.peak_kv_gb = max(self.peak_kv_gb, used)
+
+    def _record_timeline(self) -> None:
+        max_points = max(int(self.cfg.timeline_points), 40)
+        if self.timeline and self.now - self.timeline[-1]["time_s"] < max(self.cfg.duration_s / max_points, 0.05):
+            return
+        self.timeline.append(
+            {
+                "time_s": self.now,
+                "queued_turns": sum(len(r.queue) for r in self.replicas),
+                "busy_replicas": sum(1 for r in self.replicas if r.busy),
+                "kv_used_gb": self._used_gb(),
+                "resident_sessions": sum(len(r.cache) for r in self.replicas),
+            }
+        )
+        if len(self.timeline) > max_points * 2:
+            self.timeline = self.timeline[::2]
+
+    def _entry(self, replica: ReplicaState, session_id: int) -> CacheEntry | None:
+        return replica.cache.get(session_id)
+
+    def _cache_replica(self, session_id: int) -> ReplicaState | None:
+        for replica in self.replicas:
+            if session_id in replica.cache:
+                return replica
+        return None
+
+    def _route(self, session_id: int) -> ReplicaState:
+        cached = self._cache_replica(session_id)
+        if cached is not None:
+            self.route_opportunities += 1
+        if self.cfg.routing_policy == "session_affinity" and cached is not None:
+            self.affinity_routes += 1
+            return cached
+        # Approximate least-finish-time routing using queued work count then busy horizon.
+        return min(self.replicas, key=lambda r: (len(r.queue), max(r.busy_until, self.now), r.replica_id))
+
+    def _remove_cache(self, replica: ReplicaState, session_id: int, reason: str) -> None:
+        if session_id not in replica.cache:
+            return
+        self._integrate_memory(self.now)
+        del replica.cache[session_id]
+        if reason == "pressure":
+            self.pressure_evictions += 1
+        elif reason == "ttl":
+            self.ttl_evictions += 1
+        self._record_timeline()
+
+    def _ensure_capacity(self, replica: ReplicaState, session_id: int, target_gb: float) -> bool:
+        current = replica.cache.get(session_id)
+        current_gb = current.size_gb if current else 0.0
+        additional = max(0.0, target_gb - current_gb)
+        if additional <= 1e-12:
+            return True
+        # Capacity is per replica; evict least-recently-used inactive session KV.
+        while sum(e.size_gb for e in replica.cache.values()) + additional > self.kv_capacity_gb + 1e-12:
+            victims = [e for sid, e in replica.cache.items() if sid != session_id]
+            if not victims:
+                return False
+            victim = min(victims, key=lambda e: (e.last_access, e.session_id))
+            self._remove_cache(replica, victim.session_id, "pressure")
+        return True
+
+    def _put_cache(self, replica: ReplicaState, session_id: int, tokens: int, keep: bool) -> bool:
+        size_gb = tokens * self.latency.kv_bytes_per_token() / 1e9
+        if size_gb > self.kv_capacity_gb + 1e-12:
+            return False
+        if not self._ensure_capacity(replica, session_id, size_gb):
+            return False
+        self._integrate_memory(self.now)
+        previous = replica.cache.get(session_id)
+        generation = (previous.generation + 1) if previous else 1
+        expiry: float | None = None
+        if keep and self.cfg.retention_policy == "ttl":
+            expiry = self.now + self.cfg.kv_ttl_s
+        replica.cache[session_id] = CacheEntry(session_id, tokens, size_gb, self.now, expiry, generation)
+        self.peak_kv_gb = max(self.peak_kv_gb, self._used_gb())
+        if expiry is not None:
+            self._push(expiry, "cache_expire", replica.replica_id, session_id, generation)
+        self._record_timeline()
+        return True
+
+    def _start_next(self, replica: ReplicaState) -> None:
+        if replica.busy or not replica.queue:
+            return
+        session_id, turn, ready_time = replica.queue.pop(0)
+        runtime = self.sessions[session_id]
+        cache = self._entry(replica, session_id)
+        hit = turn.turn_index > 0 and cache is not None
+        if turn.turn_index > 0:
+            self.cache_eligible_turns += 1
+        if hit:
+            self.cache_hits += 1
+            prefill_tokens = max(turn.append_tokens, 1)
+        else:
+            prefill_tokens = max(runtime.context_tokens + turn.append_tokens, 1)
+            if turn.turn_index > 0:
+                self.recompute_tokens += runtime.context_tokens
+
+        context_before_decode = runtime.context_tokens + turn.append_tokens
+        projected_tokens = context_before_decode + turn.output_tokens
+        projected_gb = projected_tokens * self.latency.kv_bytes_per_token() / 1e9
+        if not self._ensure_capacity(replica, session_id, projected_gb):
+            self.failed_turns += 1
+            self._push(self.now, "turn_failed", session_id, turn.turn_index, replica.replica_id, ready_time)
+            self._start_next(replica)
+            return
+
+        # Model the current turn's KV as resident while it executes, even when
+        # the selected policy will evict it immediately after the turn.
+        if not self._put_cache(replica, session_id, projected_tokens, keep=False):
+            self.failed_turns += 1
+            self._push(self.now, "turn_failed", session_id, turn.turn_index, replica.replica_id, ready_time)
+            self._start_next(replica)
+            return
+
+        prefill_s = self.latency.prefill_seconds([prefill_tokens])
+        first_decode_s = self.latency.decode_step_seconds([context_before_decode])
+        midpoint = context_before_decode + max(turn.output_tokens // 2, 1)
+        decode_step_s = self.latency.decode_step_seconds([midpoint])
+        service_s = prefill_s + turn.output_tokens * decode_step_s
+        ttft_s = (self.now - ready_time) + prefill_s + first_decode_s
+
+        replica.busy = True
+        replica.current_session = session_id
+        replica.busy_until = self.now + service_s
+        self._push(
+            replica.busy_until,
+            "turn_complete",
+            session_id,
+            turn.turn_index,
+            replica.replica_id,
+            ready_time,
+            hit,
+            prefill_tokens,
+            ttft_s,
+            service_s,
+            projected_tokens,
+        )
+
+    def _finish_turn(
+        self,
+        session_id: int,
+        turn_index: int,
+        replica_id: int,
+        ready_time: float,
+        cache_hit: bool,
+        prefill_tokens: int,
+        ttft_s: float,
+        service_s: float,
+        projected_tokens: int,
+    ) -> None:
+        replica = self.replicas[replica_id]
+        runtime = self.sessions[session_id]
+        turn = runtime.spec.turns[turn_index]
+        runtime.context_tokens = projected_tokens
+        runtime.completed_turns += 1
+        runtime.last_replica = replica_id
+        replica.busy = False
+        replica.current_session = None
+        replica.busy_until = self.now
+
+        is_final = turn_index == len(runtime.spec.turns) - 1
+        keep = (not is_final) and self.cfg.retention_policy != "evict"
+        if keep:
+            self._put_cache(replica, session_id, projected_tokens, keep=True)
+        else:
+            self._remove_cache(replica, session_id, "complete")
+
+        e2e_ms = (self.now - ready_time) * 1000.0
+        self.turn_rows.append(
+            {
+                "session_id": session_id,
+                "turn_index": turn_index + 1,
+                "replica": replica_id,
+                "ready_time": ready_time,
+                "completion_time": self.now,
+                "cache_hit": cache_hit,
+                "prefill_tokens": prefill_tokens,
+                "context_tokens_after": projected_tokens,
+                "output_tokens": turn.output_tokens,
+                "ttft_ms": ttft_s * 1000.0,
+                "e2e_ms": e2e_ms,
+                "queue_ms": max(0.0, e2e_ms - service_s * 1000.0),
+            }
+        )
+
+        if is_final:
+            runtime.completion_time = self.now
+        else:
+            self.tool_gap_total_s += turn.tool_gap_after_s
+            self._push(self.now + turn.tool_gap_after_s, "turn_ready", session_id, turn_index + 1)
+        self._start_next(replica)
+
+    def run(self) -> dict[str, Any]:
+        self._record_timeline()
+        while self.events:
+            event_time, _, kind, payload = heapq.heappop(self.events)
+            self._integrate_memory(event_time)
+            self.now = event_time
+            if kind == "turn_ready":
+                session_id, turn_index = int(payload[0]), int(payload[1])
+                turn = self.sessions[session_id].spec.turns[turn_index]
+                replica = self._route(session_id)
+                replica.queue.append((session_id, turn, self.now))
+                self._start_next(replica)
+            elif kind == "turn_complete":
+                self._finish_turn(
+                    int(payload[0]), int(payload[1]), int(payload[2]), float(payload[3]), bool(payload[4]),
+                    int(payload[5]), float(payload[6]), float(payload[7]), int(payload[8]),
+                )
+            elif kind == "cache_expire":
+                replica_id, session_id, generation = map(int, payload)
+                replica = self.replicas[replica_id]
+                entry = replica.cache.get(session_id)
+                if entry is not None and entry.generation == generation and entry.expiry_time is not None and entry.expiry_time <= self.now + 1e-12:
+                    self._remove_cache(replica, session_id, "ttl")
+            elif kind == "turn_failed":
+                session_id, turn_index, replica_id, ready_time = int(payload[0]), int(payload[1]), int(payload[2]), float(payload[3])
+                self.turn_rows.append({
+                    "session_id": session_id, "turn_index": turn_index + 1, "replica": replica_id,
+                    "ready_time": ready_time, "completion_time": self.now, "cache_hit": False,
+                    "prefill_tokens": 0, "context_tokens_after": self.sessions[session_id].context_tokens,
+                    "output_tokens": 0, "ttft_ms": 0.0, "e2e_ms": 0.0, "queue_ms": 0.0, "failed": True,
+                })
+            self._record_timeline()
+
+        self._integrate_memory(self.now)
+        self._record_timeline()
+        completed_sessions = [s for s in self.sessions.values() if s.completion_time is not None]
+        successful_turns = [row for row in self.turn_rows if not row.get("failed")]
+        ttfts = [float(row["ttft_ms"]) for row in successful_turns]
+        turn_e2e = [float(row["e2e_ms"]) for row in successful_turns]
+        session_e2e = [
+            (s.completion_time - s.spec.arrival_time) * 1000.0
+            for s in completed_sessions
+            if s.completion_time is not None
+        ]
+        session_slo = sum(1 for v in session_e2e if v <= self.cfg.slo_session_e2e_ms)
+        turn_slo = sum(1 for v in ttfts if v <= self.cfg.slo_turn_ttft_ms)
+        completion_horizon = max([s.completion_time or 0.0 for s in self.sessions.values()] + [self.cfg.duration_s, 1e-9])
+        mean_kv_gb = self.hbm_gb_seconds / completion_horizon
+        summary = {
+            "sessions_generated": len(self.sessions),
+            "sessions_completed": len(completed_sessions),
+            "turns_completed": len(successful_turns),
+            "turns_failed": self.failed_turns,
+            "session_throughput_rps": len(completed_sessions) / completion_horizon,
+            "turn_throughput_rps": len(successful_turns) / completion_horizon,
+            "turn_ttft_slo_attainment": turn_slo / len(successful_turns) if successful_turns else 0.0,
+            "session_slo_attainment": session_slo / len(completed_sessions) if completed_sessions else 0.0,
+            "simulated_makespan_s": completion_horizon,
+        }
+        latency = {
+            "turn_ttft_ms": {"p50": percentile(ttfts, 0.50), "p95": percentile(ttfts, 0.95), "p99": percentile(ttfts, 0.99)},
+            "turn_e2e_ms": {"p50": percentile(turn_e2e, 0.50), "p95": percentile(turn_e2e, 0.95), "p99": percentile(turn_e2e, 0.99)},
+            "session_e2e_ms": {"p50": percentile(session_e2e, 0.50), "p95": percentile(session_e2e, 0.95), "p99": percentile(session_e2e, 0.99)},
+        }
+        resource = {
+            "replicas": self.cfg.replicas,
+            "kv_capacity_gb_per_replica": self.kv_capacity_gb,
+            "peak_kv_gb": self.peak_kv_gb,
+            "mean_kv_gb": mean_kv_gb,
+            "hbm_gb_seconds": self.hbm_gb_seconds,
+            "cross_turn_cache_hits": self.cache_hits,
+            "cross_turn_cache_eligible": self.cache_eligible_turns,
+            "cross_turn_cache_hit_rate": self.cache_hits / self.cache_eligible_turns if self.cache_eligible_turns else 0.0,
+            "routing_locality_rate": self.affinity_routes / self.route_opportunities if self.route_opportunities else 0.0,
+            "recomputed_history_tokens": self.recompute_tokens,
+            "pressure_evictions": self.pressure_evictions,
+            "ttl_evictions": self.ttl_evictions,
+            "tool_gap_total_s": self.tool_gap_total_s,
+        }
+        return {
+            "config": self.cfg.to_dict(),
+            "provenance": {
+                "simulator": "InferScale-Sim",
+                "mode": "stateful-agent-session-simulation",
+                "latency_profile_type": "analytical-reference",
+                "agent_service_model": "serial-per-replica-reference",
+                "warning": "Agent-session mode isolates routing/KV-retention effects and does not model dynamic batching within each replica.",
+            },
+            "summary": summary,
+            "latency": latency,
+            "resource": resource,
+            "turns": successful_turns[:3000],
+            "sessions": [
+                {
+                    "session_id": s.spec.session_id,
+                    "arrival_time": s.spec.arrival_time,
+                    "turns": len(s.spec.turns),
+                    "completion_time": s.completion_time,
+                    "e2e_ms": (s.completion_time - s.spec.arrival_time) * 1000.0 if s.completion_time is not None else None,
+                }
+                for s in list(self.sessions.values())[:1000]
+            ],
+            "timeline": self.timeline,
+        }
+
+
+def run_agent_session_simulation(config: dict[str, Any], sessions: list[SessionSpec] | None = None) -> dict[str, Any]:
+    cfg = AgentSessionConfig.from_dict(config)
+    return AgentSessionSimulator(cfg, sessions=sessions).run()
+
+
+def _same_trace(cfg: AgentSessionConfig) -> list[SessionSpec]:
+    return generate_agent_sessions(cfg)
+
+
+def compare_agent_policies(config: dict[str, Any]) -> dict[str, Any]:
+    base = AgentSessionConfig.from_dict(config)
+    trace = _same_trace(base)
+    policies = [
+        ("Stateless / least-load", "evict", "least_load", 0.0),
+        ("Retain / least-load", "retain", "least_load", base.kv_ttl_s),
+        ("TTL / affinity", "ttl", "session_affinity", base.kv_ttl_s),
+        ("Retain / affinity", "retain", "session_affinity", base.kv_ttl_s),
+    ]
+    rows = []
+    results = []
+    for label, retention, routing, ttl in policies:
+        cfg = AgentSessionConfig.from_dict(base.to_dict())
+        cfg.retention_policy = retention
+        cfg.routing_policy = routing
+        cfg.kv_ttl_s = ttl
+        result = run_agent_session_simulation(cfg.to_dict(), trace)
+        results.append(result)
+        rows.append(
+            {
+                "label": label,
+                "retention": retention,
+                "routing": routing,
+                "p95_turn_ttft_ms": result["latency"]["turn_ttft_ms"]["p95"],
+                "p95_session_e2e_ms": result["latency"]["session_e2e_ms"]["p95"],
+                "cache_hit_rate": result["resource"]["cross_turn_cache_hit_rate"],
+                "routing_locality_rate": result["resource"]["routing_locality_rate"],
+                "recomputed_history_tokens": result["resource"]["recomputed_history_tokens"],
+                "peak_kv_gb": result["resource"]["peak_kv_gb"],
+                "mean_kv_gb": result["resource"]["mean_kv_gb"],
+                "hbm_gb_seconds": result["resource"]["hbm_gb_seconds"],
+                "pressure_evictions": result["resource"]["pressure_evictions"],
+                "ttl_evictions": result["resource"]["ttl_evictions"],
+                "sessions_completed": result["summary"]["sessions_completed"],
+            }
+        )
+    return {"protocol": "common-agent-program-trace", "candidate_count": len(rows), "rows": rows, "results": results}
+
+
+def _pareto(rows: list[dict[str, Any]], x: str, y: str) -> set[int]:
+    # Both x and y are minimized.
+    front: set[int] = set()
+    for idx, row in enumerate(rows):
+        dominated = False
+        for jdx, other in enumerate(rows):
+            if idx == jdx:
+                continue
+            if other[x] <= row[x] and other[y] <= row[y] and (other[x] < row[x] or other[y] < row[y]):
+                dominated = True
+                break
+        if not dominated:
+            front.add(idx)
+    return front
+
+
+def ttl_retention_sweep(config: dict[str, Any], ttl_values: list[float] | None = None) -> dict[str, Any]:
+    base = AgentSessionConfig.from_dict(config)
+    base.retention_policy = "ttl"
+    base.routing_policy = "session_affinity"
+    trace = _same_trace(base)
+    values = ttl_values or [0.0, 0.25, 0.5, 1.0, 2.0, 3.0, 5.0, 8.0, 13.0]
+    cleaned = sorted({max(0.0, min(float(v), 120.0)) for v in values})
+    rows = []
+    for ttl in cleaned:
+        cfg = AgentSessionConfig.from_dict(base.to_dict())
+        cfg.kv_ttl_s = ttl
+        result = run_agent_session_simulation(cfg.to_dict(), trace)
+        rows.append(
+            {
+                "ttl_s": ttl,
+                "p95_turn_ttft_ms": result["latency"]["turn_ttft_ms"]["p95"],
+                "p95_session_e2e_ms": result["latency"]["session_e2e_ms"]["p95"],
+                "cache_hit_rate": result["resource"]["cross_turn_cache_hit_rate"],
+                "recomputed_history_tokens": result["resource"]["recomputed_history_tokens"],
+                "mean_kv_gb": result["resource"]["mean_kv_gb"],
+                "hbm_gb_seconds": result["resource"]["hbm_gb_seconds"],
+                "pressure_evictions": result["resource"]["pressure_evictions"],
+                "ttl_evictions": result["resource"]["ttl_evictions"],
+            }
+        )
+    front = _pareto(rows, "p95_turn_ttft_ms", "mean_kv_gb")
+    # Collapse numerically equivalent frontier points to the shortest TTL. A
+    # longer retention horizon with indistinguishable latency/residency is not a
+    # distinct engineering trade-off.
+    unique_front: set[int] = set()
+    seen: set[tuple[float, float]] = set()
+    for idx in sorted(front, key=lambda i: rows[i]["ttl_s"]):
+        key = (round(rows[idx]["p95_turn_ttft_ms"], 6), round(rows[idx]["mean_kv_gb"], 6))
+        if key not in seen:
+            unique_front.add(idx)
+            seen.add(key)
+    for idx, row in enumerate(rows):
+        row["pareto"] = idx in unique_front
+    return {
+        "protocol": "common-agent-program-trace",
+        "objective": "minimize-p95-turn-ttft-and-mean-kv-residency",
+        "rows": rows,
+        "pareto_count": len(unique_front),
+    }

py/inferscale/api.py

@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+from .agentic import compare_agent_policies, run_agent_session_simulation, ttl_retention_sweep
 from .optimizer import capacity_search, compare_schedulers, compare_topologies, design_space_search
 from .profiles import ACCELERATORS, MODELS
 from .research import STUDIES, paired_study, robustness_study
@@ -14,6 +15,7 @@ def metadata() -> dict:
         "topologies": ["colocated", "disaggregated_pd"],
         "research_studies": STUDIES,
         "profile_type": "analytical-reference",
+        "agentic_modes": ["session_simulation", "policy_compare", "ttl_sweep"],
     }
 
 
@@ -47,6 +49,12 @@ def execute(action: str, payload: dict) -> dict:
             repetitions=int(payload.get("repetitions", 12)),
             bootstrap_samples=int(payload.get("bootstrap_samples", 500)),
         )
+    if action == "agent_simulate":
+        return run_agent_session_simulation(payload.get("config", payload))
+    if action == "agent_compare":
+        return compare_agent_policies(payload.get("config", payload))
+    if action == "agent_ttl_sweep":
+        return ttl_retention_sweep(payload.get("config", payload), payload.get("ttl_values"))
     if action == "robustness_study":
         config = payload.get("config", payload)
         return robustness_study(

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "inferscale-sim"
-version = "0.4.0"
+version = "0.5.0"
 description = "Interactive LLM serving simulator and SLO capacity planner"
 readme = "README.md"
 requires-python = ">=3.10"

scripts/release_check.py

@@ -13,8 +13,11 @@ sys.path.insert(0, str(SRC))
 inferscale = importlib.import_module("inferscale")
 internal_version = inferscale.__version__
 design_space_search = inferscale.design_space_search
+compare_agent_policies = inferscale.compare_agent_policies
 paired_study = inferscale.paired_study
 robustness_study = inferscale.robustness_study
+run_agent_session_simulation = inferscale.run_agent_session_simulation
+ttl_retention_sweep = inferscale.ttl_retention_sweep
 run_simulation = inferscale.run_simulation
 validate_cases = inferscale.validate_cases
 
@@ -31,8 +34,8 @@ else:
 
 if "sdk: static" not in README:
     errors.append("README metadata must use sdk: static")
-if internal_version != "0.4.0":
-    errors.append(f"internal package version is {internal_version}; expected 0.4.0")
+if internal_version != "0.5.0":
+    errors.append(f"internal package version is {internal_version}; expected 0.5.0")
 
 # Public-facing release/version branding is intentionally absent. Model names
 # such as Mistral-7B-v0.3 are allowed; project headings/badges are not.
@@ -71,7 +74,7 @@ if "Download PNG" not in index or ".chart-download" not in app:
     errors.append("chart PNG export controls are missing")
 if "Worst repetition" not in index or "Target" not in index:
     errors.append("capacity evidence columns are missing")
-for expected in ["Trace replay", "Research Studies", "Run paired study", "Stress-test conclusion"]:
+for expected in ["Trace replay", "Research Studies", "Run paired study", "Stress-test selected hypothesis", "Agent Sessions", "Session Policy Arena", "TTL frontier"]:
     if expected not in index:
         errors.append(f"UI is missing research/trace feature: {expected}")
 
@@ -164,6 +167,27 @@ try:
 except Exception as exc:  # pragma: no cover
     errors.append(f"robustness study smoke test raised: {exc}")
 
+
+try:
+    agent_cfg = {
+        "model": "Qwen2.5-3B", "accelerator": "L4", "quantization": "int8",
+        "duration_s": 12, "session_rate_rps": 0.25, "replicas": 2, "seed": 7,
+        "retention_policy": "ttl", "routing_policy": "session_affinity", "kv_ttl_s": 3,
+    }
+    agent = run_agent_session_simulation(agent_cfg)
+    if agent["provenance"].get("mode") != "stateful-agent-session-simulation":
+        errors.append("agent-session provenance guard is missing")
+    if agent["summary"].get("turns_completed", 0) <= 0:
+        errors.append("agent-session smoke test completed zero turns")
+    agent_compare = compare_agent_policies(agent_cfg)
+    if agent_compare.get("candidate_count") != 4:
+        errors.append("agent policy arena did not return four candidates")
+    agent_ttl = ttl_retention_sweep(agent_cfg, [0, 1, 3])
+    if len(agent_ttl.get("rows", [])) != 3 or agent_ttl.get("pareto_count", 0) < 1:
+        errors.append("agent TTL frontier smoke test is incomplete")
+except Exception as exc:  # pragma: no cover
+    errors.append(f"agent-session smoke test raised: {exc}")
+
 try:
     validation = validate_cases([{
         "name": "release-fixture",
@@ -190,5 +214,8 @@ print(f"P/D transfer p95: {pd['resource']['p95_transfer_ms']:.3f} ms")
 print(f"Design candidates: {design['candidate_count']}")
 print(f"Paired-study metrics: {len(paired['metrics'])}")
 print(f"Robustness perturbations: {len(robust['rows'])}")
+print(f"Agent turns: {agent['summary']['turns_completed']}")
+print(f"Agent policy candidates: {agent_compare['candidate_count']}")
+print(f"Agent TTL candidates: {len(agent_ttl['rows'])}")
 print(f"Validation observations: {validation['observation_count']}")
 print(f"Profile provenance: {smoke['provenance']['latency_profile_type']}")

src/inferscale/__init__.py

@@ -1,3 +1,4 @@
+from .agentic import compare_agent_policies, run_agent_session_simulation, ttl_retention_sweep
 from .api import execute, metadata
 from .models import SimulationConfig
 from .optimizer import capacity_search, compare_schedulers, compare_topologies, design_space_search
@@ -7,6 +8,7 @@ from .validation import validate_cases
 
 __all__ = [
     "SimulationConfig",
+    "compare_agent_policies",
     "capacity_search",
     "compare_schedulers",
     "compare_topologies",
@@ -15,10 +17,12 @@ __all__ = [
     "metadata",
     "paired_study",
     "robustness_study",
+    "run_agent_session_simulation",
     "run_simulation",
+    "ttl_retention_sweep",
     "validate_cases",
 ]
 
 # Internal package metadata only; the public project intentionally avoids
 # release/version branding in the interface and documentation.
-__version__ = "0.4.0"
+__version__ = "0.5.0"

src/inferscale/agentic.py

@@ -0,0 +1,622 @@
+from __future__ import annotations
+
+import heapq
+import math
+import random
+from dataclasses import asdict, dataclass
+from statistics import mean, median
+from typing import Any
+
+from .latency import AnalyticalLatencyModel
+from .metrics import percentile
+from .profiles import get_accelerator, get_model
+
+
+ROUTING_POLICIES = {"least_load", "session_affinity"}
+RETENTION_POLICIES = {"evict", "retain", "ttl"}
+
+
+@dataclass
+class AgentSessionConfig:
+    model: str = "Qwen2.5-3B"
+    accelerator: str = "L4"
+    quantization: str = "int8"
+    seed: int = 7
+    duration_s: float = 30.0
+    session_rate_rps: float = 0.45
+    replicas: int = 2
+    turns_mean: float = 4.0
+    turns_cv: float = 0.25
+    initial_prompt_tokens_mean: int = 640
+    append_tokens_mean: int = 180
+    token_cv: float = 0.35
+    output_tokens_mean: int = 72
+    output_tokens_cv: float = 0.45
+    tool_gap_mean_s: float = 1.5
+    tool_gap_cv: float = 0.75
+    retention_policy: str = "ttl"
+    kv_ttl_s: float = 3.0
+    routing_policy: str = "session_affinity"
+    kv_memory_fraction: float = 0.85
+    slo_turn_ttft_ms: float = 500.0
+    slo_session_e2e_ms: float = 30000.0
+    timeline_points: int = 240
+
+    @classmethod
+    def from_dict(cls, data: dict[str, Any]) -> "AgentSessionConfig":
+        allowed = cls.__dataclass_fields__.keys()
+        cfg = cls(**{k: data[k] for k in allowed if k in data})
+        if cfg.routing_policy not in ROUTING_POLICIES:
+            raise ValueError(f"Unsupported agent routing policy: {cfg.routing_policy}")
+        if cfg.retention_policy not in RETENTION_POLICIES:
+            raise ValueError(f"Unsupported KV retention policy: {cfg.retention_policy}")
+        cfg.replicas = max(1, min(int(cfg.replicas), 8))
+        cfg.session_rate_rps = max(float(cfg.session_rate_rps), 0.01)
+        cfg.duration_s = max(float(cfg.duration_s), 1.0)
+        cfg.kv_ttl_s = max(float(cfg.kv_ttl_s), 0.0)
+        cfg.kv_memory_fraction = min(max(float(cfg.kv_memory_fraction), 0.05), 0.98)
+        return cfg
+
+    def to_dict(self) -> dict[str, Any]:
+        return asdict(self)
+
+
+@dataclass
+class TurnSpec:
+    session_id: int
+    turn_index: int
+    append_tokens: int
+    output_tokens: int
+    tool_gap_after_s: float
+
+
+@dataclass
+class SessionSpec:
+    session_id: int
+    arrival_time: float
+    initial_prompt_tokens: int
+    turns: list[TurnSpec]
+
+
+@dataclass
+class CacheEntry:
+    session_id: int
+    tokens: int
+    size_gb: float
+    last_access: float
+    expiry_time: float | None
+    generation: int
+
+
+@dataclass
+class ReplicaState:
+    replica_id: int
+    busy: bool = False
+    busy_until: float = 0.0
+    queue: list[tuple[int, TurnSpec, float]] | None = None
+    cache: dict[int, CacheEntry] | None = None
+    current_session: int | None = None
+
+    def __post_init__(self) -> None:
+        if self.queue is None:
+            self.queue = []
+        if self.cache is None:
+            self.cache = {}
+
+
+@dataclass
+class SessionRuntime:
+    spec: SessionSpec
+    context_tokens: int
+    completed_turns: int = 0
+    completion_time: float | None = None
+    last_replica: int | None = None
+
+
+def _sample_positive_lognormal(rng: random.Random, mean_value: float, cv: float, minimum: float) -> float:
+    mean_value = max(float(mean_value), minimum)
+    cv = max(float(cv), 0.0)
+    if mean_value <= 0.0:
+        return minimum
+    if cv <= 1e-12:
+        return mean_value
+    sigma2 = math.log1p(cv * cv)
+    sigma = math.sqrt(sigma2)
+    mu = math.log(mean_value) - sigma2 / 2.0
+    return max(minimum, rng.lognormvariate(mu, sigma))
+
+
+def _sample_int(rng: random.Random, mean_value: float, cv: float, minimum: int = 1) -> int:
+    return max(minimum, int(round(_sample_positive_lognormal(rng, mean_value, cv, minimum))))
+
+
+def generate_agent_sessions(cfg: AgentSessionConfig) -> list[SessionSpec]:
+    """Generate a deterministic multi-turn program trace.
+
+    All policies consume this same trace when the seed/config are unchanged. Tool
+    gaps are sampled up front so policy comparisons use common random numbers.
+    """
+    rng = random.Random(cfg.seed ^ 0xA63E17)
+    sessions: list[SessionSpec] = []
+    now = 0.0
+    sid = 0
+    while True:
+        now += rng.expovariate(cfg.session_rate_rps)
+        if now > cfg.duration_s:
+            break
+        turns = max(2, _sample_int(rng, cfg.turns_mean, cfg.turns_cv, 2))
+        initial = _sample_int(rng, cfg.initial_prompt_tokens_mean, cfg.token_cv, 32)
+        specs = []
+        for turn_idx in range(turns):
+            append = 0 if turn_idx == 0 else _sample_int(rng, cfg.append_tokens_mean, cfg.token_cv, 8)
+            output = _sample_int(rng, cfg.output_tokens_mean, cfg.output_tokens_cv, 1)
+            gap = 0.0 if turn_idx == turns - 1 else _sample_positive_lognormal(
+                rng, cfg.tool_gap_mean_s, cfg.tool_gap_cv, 0.0
+            )
+            specs.append(TurnSpec(sid, turn_idx, append, output, gap))
+        sessions.append(SessionSpec(sid, now, initial, specs))
+        sid += 1
+    return sessions
+
+
+class AgentSessionSimulator:
+    """Discrete-event simulator for stateful multi-turn serving.
+
+    The module deliberately isolates session locality / KV-retention effects from
+    dynamic batching. Each replica is a serial analytical service station; the
+    existing Serving Lab remains the place to study continuous batching. This
+    separation keeps the agent experiment interpretable while still modelling
+    session dependencies, tool gaps, routing, memory pressure, and cross-turn KV.
+    """
+
+    def __init__(self, cfg: AgentSessionConfig, sessions: list[SessionSpec] | None = None):
+        self.cfg = cfg
+        self.model = get_model(cfg.model)
+        self.accelerator = get_accelerator(cfg.accelerator)
+        self.latency = AnalyticalLatencyModel(self.model, self.accelerator, cfg.quantization)
+        remaining = max(0.0, self.accelerator.vram_gb - self.latency.model_weight_gb - 1.2)
+        self.kv_capacity_gb = remaining * cfg.kv_memory_fraction
+        self.replicas = [ReplicaState(i) for i in range(cfg.replicas)]
+        specs = sessions if sessions is not None else generate_agent_sessions(cfg)
+        self.sessions = {s.session_id: SessionRuntime(s, s.initial_prompt_tokens) for s in specs}
+        self.events: list[tuple[float, int, str, tuple[Any, ...]]] = []
+        self.event_seq = 0
+        self.now = 0.0
+        self.turn_rows: list[dict[str, Any]] = []
+        self.timeline: list[dict[str, Any]] = []
+        self.peak_kv_gb = 0.0
+        self.hbm_gb_seconds = 0.0
+        self.last_memory_time = 0.0
+        self.pressure_evictions = 0
+        self.ttl_evictions = 0
+        self.recompute_tokens = 0
+        self.cache_hits = 0
+        self.cache_eligible_turns = 0
+        self.affinity_routes = 0
+        self.route_opportunities = 0
+        self.failed_turns = 0
+        self.tool_gap_total_s = 0.0
+        for session in specs:
+            if session.turns:
+                self._push(session.arrival_time, "turn_ready", session.session_id, 0)
+
+    def _push(self, when: float, kind: str, *payload: Any) -> None:
+        self.event_seq += 1
+        heapq.heappush(self.events, (float(when), self.event_seq, kind, tuple(payload)))
+
+    def _used_gb(self) -> float:
+        return sum(entry.size_gb for replica in self.replicas for entry in replica.cache.values())
+
+    def _integrate_memory(self, new_time: float) -> None:
+        new_time = max(float(new_time), self.last_memory_time)
+        used = self._used_gb()
+        self.hbm_gb_seconds += used * (new_time - self.last_memory_time)
+        self.last_memory_time = new_time
+        self.peak_kv_gb = max(self.peak_kv_gb, used)
+
+    def _record_timeline(self) -> None:
+        max_points = max(int(self.cfg.timeline_points), 40)
+        if self.timeline and self.now - self.timeline[-1]["time_s"] < max(self.cfg.duration_s / max_points, 0.05):
+            return
+        self.timeline.append(
+            {
+                "time_s": self.now,
+                "queued_turns": sum(len(r.queue) for r in self.replicas),
+                "busy_replicas": sum(1 for r in self.replicas if r.busy),
+                "kv_used_gb": self._used_gb(),
+                "resident_sessions": sum(len(r.cache) for r in self.replicas),
+            }
+        )
+        if len(self.timeline) > max_points * 2:
+            self.timeline = self.timeline[::2]
+
+    def _entry(self, replica: ReplicaState, session_id: int) -> CacheEntry | None:
+        return replica.cache.get(session_id)
+
+    def _cache_replica(self, session_id: int) -> ReplicaState | None:
+        for replica in self.replicas:
+            if session_id in replica.cache:
+                return replica
+        return None
+
+    def _route(self, session_id: int) -> ReplicaState:
+        cached = self._cache_replica(session_id)
+        if cached is not None:
+            self.route_opportunities += 1
+        if self.cfg.routing_policy == "session_affinity" and cached is not None:
+            self.affinity_routes += 1
+            return cached
+        # Approximate least-finish-time routing using queued work count then busy horizon.
+        return min(self.replicas, key=lambda r: (len(r.queue), max(r.busy_until, self.now), r.replica_id))
+
+    def _remove_cache(self, replica: ReplicaState, session_id: int, reason: str) -> None:
+        if session_id not in replica.cache:
+            return
+        self._integrate_memory(self.now)
+        del replica.cache[session_id]
+        if reason == "pressure":
+            self.pressure_evictions += 1
+        elif reason == "ttl":
+            self.ttl_evictions += 1
+        self._record_timeline()
+
+    def _ensure_capacity(self, replica: ReplicaState, session_id: int, target_gb: float) -> bool:
+        current = replica.cache.get(session_id)
+        current_gb = current.size_gb if current else 0.0
+        additional = max(0.0, target_gb - current_gb)
+        if additional <= 1e-12:
+            return True
+        # Capacity is per replica; evict least-recently-used inactive session KV.
+        while sum(e.size_gb for e in replica.cache.values()) + additional > self.kv_capacity_gb + 1e-12:
+            victims = [e for sid, e in replica.cache.items() if sid != session_id]
+            if not victims:
+                return False
+            victim = min(victims, key=lambda e: (e.last_access, e.session_id))
+            self._remove_cache(replica, victim.session_id, "pressure")
+        return True
+
+    def _put_cache(self, replica: ReplicaState, session_id: int, tokens: int, keep: bool) -> bool:
+        size_gb = tokens * self.latency.kv_bytes_per_token() / 1e9
+        if size_gb > self.kv_capacity_gb + 1e-12:
+            return False
+        if not self._ensure_capacity(replica, session_id, size_gb):
+            return False
+        self._integrate_memory(self.now)
+        previous = replica.cache.get(session_id)
+        generation = (previous.generation + 1) if previous else 1
+        expiry: float | None = None
+        if keep and self.cfg.retention_policy == "ttl":
+            expiry = self.now + self.cfg.kv_ttl_s
+        replica.cache[session_id] = CacheEntry(session_id, tokens, size_gb, self.now, expiry, generation)
+        self.peak_kv_gb = max(self.peak_kv_gb, self._used_gb())
+        if expiry is not None:
+            self._push(expiry, "cache_expire", replica.replica_id, session_id, generation)
+        self._record_timeline()
+        return True
+
+    def _start_next(self, replica: ReplicaState) -> None:
+        if replica.busy or not replica.queue:
+            return
+        session_id, turn, ready_time = replica.queue.pop(0)
+        runtime = self.sessions[session_id]
+        cache = self._entry(replica, session_id)
+        hit = turn.turn_index > 0 and cache is not None
+        if turn.turn_index > 0:
+            self.cache_eligible_turns += 1
+        if hit:
+            self.cache_hits += 1
+            prefill_tokens = max(turn.append_tokens, 1)
+        else:
+            prefill_tokens = max(runtime.context_tokens + turn.append_tokens, 1)
+            if turn.turn_index > 0:
+                self.recompute_tokens += runtime.context_tokens
+
+        context_before_decode = runtime.context_tokens + turn.append_tokens
+        projected_tokens = context_before_decode + turn.output_tokens
+        projected_gb = projected_tokens * self.latency.kv_bytes_per_token() / 1e9
+        if not self._ensure_capacity(replica, session_id, projected_gb):
+            self.failed_turns += 1
+            self._push(self.now, "turn_failed", session_id, turn.turn_index, replica.replica_id, ready_time)
+            self._start_next(replica)
+            return
+
+        # Model the current turn's KV as resident while it executes, even when
+        # the selected policy will evict it immediately after the turn.
+        if not self._put_cache(replica, session_id, projected_tokens, keep=False):
+            self.failed_turns += 1
+            self._push(self.now, "turn_failed", session_id, turn.turn_index, replica.replica_id, ready_time)
+            self._start_next(replica)
+            return
+
+        prefill_s = self.latency.prefill_seconds([prefill_tokens])
+        first_decode_s = self.latency.decode_step_seconds([context_before_decode])
+        midpoint = context_before_decode + max(turn.output_tokens // 2, 1)
+        decode_step_s = self.latency.decode_step_seconds([midpoint])
+        service_s = prefill_s + turn.output_tokens * decode_step_s
+        ttft_s = (self.now - ready_time) + prefill_s + first_decode_s
+
+        replica.busy = True
+        replica.current_session = session_id
+        replica.busy_until = self.now + service_s
+        self._push(
+            replica.busy_until,
+            "turn_complete",
+            session_id,
+            turn.turn_index,
+            replica.replica_id,
+            ready_time,
+            hit,
+            prefill_tokens,
+            ttft_s,
+            service_s,
+            projected_tokens,
+        )
+
+    def _finish_turn(
+        self,
+        session_id: int,
+        turn_index: int,
+        replica_id: int,
+        ready_time: float,
+        cache_hit: bool,
+        prefill_tokens: int,
+        ttft_s: float,
+        service_s: float,
+        projected_tokens: int,
+    ) -> None:
+        replica = self.replicas[replica_id]
+        runtime = self.sessions[session_id]
+        turn = runtime.spec.turns[turn_index]
+        runtime.context_tokens = projected_tokens
+        runtime.completed_turns += 1
+        runtime.last_replica = replica_id
+        replica.busy = False
+        replica.current_session = None
+        replica.busy_until = self.now
+
+        is_final = turn_index == len(runtime.spec.turns) - 1
+        keep = (not is_final) and self.cfg.retention_policy != "evict"
+        if keep:
+            self._put_cache(replica, session_id, projected_tokens, keep=True)
+        else:
+            self._remove_cache(replica, session_id, "complete")
+
+        e2e_ms = (self.now - ready_time) * 1000.0
+        self.turn_rows.append(
+            {
+                "session_id": session_id,
+                "turn_index": turn_index + 1,
+                "replica": replica_id,
+                "ready_time": ready_time,
+                "completion_time": self.now,
+                "cache_hit": cache_hit,
+                "prefill_tokens": prefill_tokens,
+                "context_tokens_after": projected_tokens,
+                "output_tokens": turn.output_tokens,
+                "ttft_ms": ttft_s * 1000.0,
+                "e2e_ms": e2e_ms,
+                "queue_ms": max(0.0, e2e_ms - service_s * 1000.0),
+            }
+        )
+
+        if is_final:
+            runtime.completion_time = self.now
+        else:
+            self.tool_gap_total_s += turn.tool_gap_after_s
+            self._push(self.now + turn.tool_gap_after_s, "turn_ready", session_id, turn_index + 1)
+        self._start_next(replica)
+
+    def run(self) -> dict[str, Any]:
+        self._record_timeline()
+        while self.events:
+            event_time, _, kind, payload = heapq.heappop(self.events)
+            self._integrate_memory(event_time)
+            self.now = event_time
+            if kind == "turn_ready":
+                session_id, turn_index = int(payload[0]), int(payload[1])
+                turn = self.sessions[session_id].spec.turns[turn_index]
+                replica = self._route(session_id)
+                replica.queue.append((session_id, turn, self.now))
+                self._start_next(replica)
+            elif kind == "turn_complete":
+                self._finish_turn(
+                    int(payload[0]), int(payload[1]), int(payload[2]), float(payload[3]), bool(payload[4]),
+                    int(payload[5]), float(payload[6]), float(payload[7]), int(payload[8]),
+                )
+            elif kind == "cache_expire":
+                replica_id, session_id, generation = map(int, payload)
+                replica = self.replicas[replica_id]
+                entry = replica.cache.get(session_id)
+                if entry is not None and entry.generation == generation and entry.expiry_time is not None and entry.expiry_time <= self.now + 1e-12:
+                    self._remove_cache(replica, session_id, "ttl")
+            elif kind == "turn_failed":
+                session_id, turn_index, replica_id, ready_time = int(payload[0]), int(payload[1]), int(payload[2]), float(payload[3])
+                self.turn_rows.append({
+                    "session_id": session_id, "turn_index": turn_index + 1, "replica": replica_id,
+                    "ready_time": ready_time, "completion_time": self.now, "cache_hit": False,
+                    "prefill_tokens": 0, "context_tokens_after": self.sessions[session_id].context_tokens,
+                    "output_tokens": 0, "ttft_ms": 0.0, "e2e_ms": 0.0, "queue_ms": 0.0, "failed": True,
+                })
+            self._record_timeline()
+
+        self._integrate_memory(self.now)
+        self._record_timeline()
+        completed_sessions = [s for s in self.sessions.values() if s.completion_time is not None]
+        successful_turns = [row for row in self.turn_rows if not row.get("failed")]
+        ttfts = [float(row["ttft_ms"]) for row in successful_turns]
+        turn_e2e = [float(row["e2e_ms"]) for row in successful_turns]
+        session_e2e = [
+            (s.completion_time - s.spec.arrival_time) * 1000.0
+            for s in completed_sessions
+            if s.completion_time is not None
+        ]
+        session_slo = sum(1 for v in session_e2e if v <= self.cfg.slo_session_e2e_ms)
+        turn_slo = sum(1 for v in ttfts if v <= self.cfg.slo_turn_ttft_ms)
+        completion_horizon = max([s.completion_time or 0.0 for s in self.sessions.values()] + [self.cfg.duration_s, 1e-9])
+        mean_kv_gb = self.hbm_gb_seconds / completion_horizon
+        summary = {
+            "sessions_generated": len(self.sessions),
+            "sessions_completed": len(completed_sessions),
+            "turns_completed": len(successful_turns),
+            "turns_failed": self.failed_turns,
+            "session_throughput_rps": len(completed_sessions) / completion_horizon,
+            "turn_throughput_rps": len(successful_turns) / completion_horizon,
+            "turn_ttft_slo_attainment": turn_slo / len(successful_turns) if successful_turns else 0.0,
+            "session_slo_attainment": session_slo / len(completed_sessions) if completed_sessions else 0.0,
+            "simulated_makespan_s": completion_horizon,
+        }
+        latency = {
+            "turn_ttft_ms": {"p50": percentile(ttfts, 0.50), "p95": percentile(ttfts, 0.95), "p99": percentile(ttfts, 0.99)},
+            "turn_e2e_ms": {"p50": percentile(turn_e2e, 0.50), "p95": percentile(turn_e2e, 0.95), "p99": percentile(turn_e2e, 0.99)},
+            "session_e2e_ms": {"p50": percentile(session_e2e, 0.50), "p95": percentile(session_e2e, 0.95), "p99": percentile(session_e2e, 0.99)},
+        }
+        resource = {
+            "replicas": self.cfg.replicas,
+            "kv_capacity_gb_per_replica": self.kv_capacity_gb,
+            "peak_kv_gb": self.peak_kv_gb,
+            "mean_kv_gb": mean_kv_gb,
+            "hbm_gb_seconds": self.hbm_gb_seconds,
+            "cross_turn_cache_hits": self.cache_hits,
+            "cross_turn_cache_eligible": self.cache_eligible_turns,
+            "cross_turn_cache_hit_rate": self.cache_hits / self.cache_eligible_turns if self.cache_eligible_turns else 0.0,
+            "routing_locality_rate": self.affinity_routes / self.route_opportunities if self.route_opportunities else 0.0,
+            "recomputed_history_tokens": self.recompute_tokens,
+            "pressure_evictions": self.pressure_evictions,
+            "ttl_evictions": self.ttl_evictions,
+            "tool_gap_total_s": self.tool_gap_total_s,
+        }
+        return {
+            "config": self.cfg.to_dict(),
+            "provenance": {
+                "simulator": "InferScale-Sim",
+                "mode": "stateful-agent-session-simulation",
+                "latency_profile_type": "analytical-reference",
+                "agent_service_model": "serial-per-replica-reference",
+                "warning": "Agent-session mode isolates routing/KV-retention effects and does not model dynamic batching within each replica.",
+            },
+            "summary": summary,
+            "latency": latency,
+            "resource": resource,
+            "turns": successful_turns[:3000],
+            "sessions": [
+                {
+                    "session_id": s.spec.session_id,
+                    "arrival_time": s.spec.arrival_time,
+                    "turns": len(s.spec.turns),
+                    "completion_time": s.completion_time,
+                    "e2e_ms": (s.completion_time - s.spec.arrival_time) * 1000.0 if s.completion_time is not None else None,
+                }
+                for s in list(self.sessions.values())[:1000]
+            ],
+            "timeline": self.timeline,
+        }
+
+
+def run_agent_session_simulation(config: dict[str, Any], sessions: list[SessionSpec] | None = None) -> dict[str, Any]:
+    cfg = AgentSessionConfig.from_dict(config)
+    return AgentSessionSimulator(cfg, sessions=sessions).run()
+
+
+def _same_trace(cfg: AgentSessionConfig) -> list[SessionSpec]:
+    return generate_agent_sessions(cfg)
+
+
+def compare_agent_policies(config: dict[str, Any]) -> dict[str, Any]:
+    base = AgentSessionConfig.from_dict(config)
+    trace = _same_trace(base)
+    policies = [
+        ("Stateless / least-load", "evict", "least_load", 0.0),
+        ("Retain / least-load", "retain", "least_load", base.kv_ttl_s),
+        ("TTL / affinity", "ttl", "session_affinity", base.kv_ttl_s),
+        ("Retain / affinity", "retain", "session_affinity", base.kv_ttl_s),
+    ]
+    rows = []
+    results = []
+    for label, retention, routing, ttl in policies:
+        cfg = AgentSessionConfig.from_dict(base.to_dict())
+        cfg.retention_policy = retention
+        cfg.routing_policy = routing
+        cfg.kv_ttl_s = ttl
+        result = run_agent_session_simulation(cfg.to_dict(), trace)
+        results.append(result)
+        rows.append(
+            {
+                "label": label,
+                "retention": retention,
+                "routing": routing,
+                "p95_turn_ttft_ms": result["latency"]["turn_ttft_ms"]["p95"],
+                "p95_session_e2e_ms": result["latency"]["session_e2e_ms"]["p95"],
+                "cache_hit_rate": result["resource"]["cross_turn_cache_hit_rate"],
+                "routing_locality_rate": result["resource"]["routing_locality_rate"],
+                "recomputed_history_tokens": result["resource"]["recomputed_history_tokens"],
+                "peak_kv_gb": result["resource"]["peak_kv_gb"],
+                "mean_kv_gb": result["resource"]["mean_kv_gb"],
+                "hbm_gb_seconds": result["resource"]["hbm_gb_seconds"],
+                "pressure_evictions": result["resource"]["pressure_evictions"],
+                "ttl_evictions": result["resource"]["ttl_evictions"],
+                "sessions_completed": result["summary"]["sessions_completed"],
+            }
+        )
+    return {"protocol": "common-agent-program-trace", "candidate_count": len(rows), "rows": rows, "results": results}
+
+
+def _pareto(rows: list[dict[str, Any]], x: str, y: str) -> set[int]:
+    # Both x and y are minimized.
+    front: set[int] = set()
+    for idx, row in enumerate(rows):
+        dominated = False
+        for jdx, other in enumerate(rows):
+            if idx == jdx:
+                continue
+            if other[x] <= row[x] and other[y] <= row[y] and (other[x] < row[x] or other[y] < row[y]):
+                dominated = True
+                break
+        if not dominated:
+            front.add(idx)
+    return front
+
+
+def ttl_retention_sweep(config: dict[str, Any], ttl_values: list[float] | None = None) -> dict[str, Any]:
+    base = AgentSessionConfig.from_dict(config)
+    base.retention_policy = "ttl"
+    base.routing_policy = "session_affinity"
+    trace = _same_trace(base)
+    values = ttl_values or [0.0, 0.25, 0.5, 1.0, 2.0, 3.0, 5.0, 8.0, 13.0]
+    cleaned = sorted({max(0.0, min(float(v), 120.0)) for v in values})
+    rows = []
+    for ttl in cleaned:
+        cfg = AgentSessionConfig.from_dict(base.to_dict())
+        cfg.kv_ttl_s = ttl
+        result = run_agent_session_simulation(cfg.to_dict(), trace)
+        rows.append(
+            {
+                "ttl_s": ttl,
+                "p95_turn_ttft_ms": result["latency"]["turn_ttft_ms"]["p95"],
+                "p95_session_e2e_ms": result["latency"]["session_e2e_ms"]["p95"],
+                "cache_hit_rate": result["resource"]["cross_turn_cache_hit_rate"],
+                "recomputed_history_tokens": result["resource"]["recomputed_history_tokens"],
+                "mean_kv_gb": result["resource"]["mean_kv_gb"],
+                "hbm_gb_seconds": result["resource"]["hbm_gb_seconds"],
+                "pressure_evictions": result["resource"]["pressure_evictions"],
+                "ttl_evictions": result["resource"]["ttl_evictions"],
+            }
+        )
+    front = _pareto(rows, "p95_turn_ttft_ms", "mean_kv_gb")
+    # Collapse numerically equivalent frontier points to the shortest TTL. A
+    # longer retention horizon with indistinguishable latency/residency is not a
+    # distinct engineering trade-off.
+    unique_front: set[int] = set()
+    seen: set[tuple[float, float]] = set()
+    for idx in sorted(front, key=lambda i: rows[i]["ttl_s"]):
+        key = (round(rows[idx]["p95_turn_ttft_ms"], 6), round(rows[idx]["mean_kv_gb"], 6))
+        if key not in seen:
+            unique_front.add(idx)
+            seen.add(key)
+    for idx, row in enumerate(rows):
+        row["pareto"] = idx in unique_front
+    return {
+        "protocol": "common-agent-program-trace",
+        "objective": "minimize-p95-turn-ttft-and-mean-kv-residency",
+        "rows": rows,
+        "pareto_count": len(unique_front),
+    }

src/inferscale/api.py

@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+from .agentic import compare_agent_policies, run_agent_session_simulation, ttl_retention_sweep
 from .optimizer import capacity_search, compare_schedulers, compare_topologies, design_space_search
 from .profiles import ACCELERATORS, MODELS
 from .research import STUDIES, paired_study, robustness_study
@@ -14,6 +15,7 @@ def metadata() -> dict:
         "topologies": ["colocated", "disaggregated_pd"],
         "research_studies": STUDIES,
         "profile_type": "analytical-reference",
+        "agentic_modes": ["session_simulation", "policy_compare", "ttl_sweep"],
     }
 
 
@@ -47,6 +49,12 @@ def execute(action: str, payload: dict) -> dict:
             repetitions=int(payload.get("repetitions", 12)),
             bootstrap_samples=int(payload.get("bootstrap_samples", 500)),
         )
+    if action == "agent_simulate":
+        return run_agent_session_simulation(payload.get("config", payload))
+    if action == "agent_compare":
+        return compare_agent_policies(payload.get("config", payload))
+    if action == "agent_ttl_sweep":
+        return ttl_retention_sweep(payload.get("config", payload), payload.get("ttl_values"))
     if action == "robustness_study":
         config = payload.get("config", payload)
         return robustness_study(

styles.css

@@ -312,3 +312,13 @@ input[type="file"]::file-selector-button {
 .study-summary strong { color: #e1e7ee; }
 .research-chart { height: 310px; }
 @media (max-width: 1100px) { .research-grid { grid-template-columns: 1fr; } .research-controls { position: static; } }
+
+/* Stateful agent-session experiments */
+.agent-layout { display: grid; grid-template-columns: 420px minmax(0, 1fr); gap: 14px; align-items: start; }
+.agent-controls { padding: 20px; position: sticky; top: 16px; }
+.agent-results { display: grid; gap: 14px; min-width: 0; }
+.metric-grid.six-agent { grid-template-columns: repeat(6, minmax(0, 1fr)); }
+@media (max-width: 1250px) { .metric-grid.six-agent { grid-template-columns: repeat(3, 1fr); } }
+@media (max-width: 1100px) { .agent-layout { grid-template-columns: 1fr; } .agent-controls { position: static; } }
+@media (max-width: 720px) { .metric-grid.six-agent { grid-template-columns: 1fr 1fr; } }
+.field-disabled { opacity: .55; }

tests/test_agentic.py

@@ -0,0 +1,58 @@
+from inferscale.agentic import compare_agent_policies, run_agent_session_simulation, ttl_retention_sweep
+
+
+BASE = {
+    "model": "Qwen2.5-3B",
+    "accelerator": "L4",
+    "quantization": "int8",
+    "duration_s": 30,
+    "session_rate_rps": 0.20,
+    "replicas": 2,
+    "turns_mean": 4,
+    "tool_gap_mean_s": 1.5,
+    "seed": 7,
+}
+
+
+def test_agent_session_run_is_deterministic():
+    first = run_agent_session_simulation(BASE | {"retention_policy": "ttl", "routing_policy": "session_affinity", "kv_ttl_s": 3})
+    second = run_agent_session_simulation(BASE | {"retention_policy": "ttl", "routing_policy": "session_affinity", "kv_ttl_s": 3})
+    assert first["summary"] == second["summary"]
+    assert first["latency"] == second["latency"]
+    assert first["resource"] == second["resource"]
+    assert first["provenance"]["mode"] == "stateful-agent-session-simulation"
+
+
+def test_evict_policy_has_no_cross_turn_hits():
+    result = run_agent_session_simulation(BASE | {"retention_policy": "evict", "routing_policy": "least_load"})
+    assert result["resource"]["cross_turn_cache_hit_rate"] == 0
+    assert result["resource"]["recomputed_history_tokens"] > 0
+
+
+def test_retention_with_affinity_reuses_state():
+    baseline = run_agent_session_simulation(BASE | {"retention_policy": "evict", "routing_policy": "least_load"})
+    retained = run_agent_session_simulation(BASE | {"retention_policy": "retain", "routing_policy": "session_affinity"})
+    assert retained["resource"]["cross_turn_cache_hit_rate"] > baseline["resource"]["cross_turn_cache_hit_rate"]
+    assert retained["resource"]["recomputed_history_tokens"] < baseline["resource"]["recomputed_history_tokens"]
+    assert retained["resource"]["mean_kv_gb"] >= baseline["resource"]["mean_kv_gb"]
+
+
+def test_policy_compare_uses_four_common_trace_candidates():
+    result = compare_agent_policies(BASE | {"kv_ttl_s": 3})
+    assert result["protocol"] == "common-agent-program-trace"
+    assert result["candidate_count"] == 4
+    assert {row["label"] for row in result["rows"]} == {
+        "Stateless / least-load",
+        "Retain / least-load",
+        "TTL / affinity",
+        "Retain / affinity",
+    }
+
+
+def test_ttl_sweep_reports_latency_memory_frontier():
+    result = ttl_retention_sweep(BASE, [0, 1, 3, 8])
+    assert result["protocol"] == "common-agent-program-trace"
+    assert result["objective"] == "minimize-p95-turn-ttft-and-mean-kv-residency"
+    assert result["pareto_count"] >= 1
+    assert len(result["rows"]) == 4
+    assert any(row["pareto"] for row in result["rows"])

worker.mjs

@@ -2,6 +2,7 @@ import { loadPyodide } from "https://cdn.jsdelivr.net/pyodide/v314.0.5/full/pyod
 
 const MODULES = [
   "__init__.py",
+  "agentic.py",
   "api.py",
   "diagnostics.py",
   "disaggregated.py",