ArchitSharma commited on
Commit
20fb354
·
1 Parent(s): 0c6c82c

Release InferScale-Sim v0.2.0

Browse files
CHANGELOG.md ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Changelog
2
+
3
+ ## v0.2.0
4
+
5
+ - auto-hide runtime header after Pyodide becomes ready; hover/focus at the top edge reveals it
6
+ - redesigned the interface around a flatter, quieter engineering-dashboard visual system
7
+ - replaced decorative/non-ASCII ranking glyphs with explicit text labels
8
+ - added chart expansion with Escape-to-close support
9
+ - added copy-to-clipboard and CSV export for Scheduler Arena and Capacity Planner tables
10
+ - added JSON copy alongside download for individual simulations
11
+ - exposed burst multiplier and burst-period controls
12
+ - added component TTFT/E2E SLO-attainment accounting and failure breakdowns
13
+ - added heuristic simulator-side bottleneck diagnosis with evidence and next-step guidance
14
+ - made capacity feasibility conservative across repetitions: every repetition must meet the target and fully drain
15
+ - added minimum/mean/maximum repetition evidence to capacity traces
16
+ - changed the capacity plot to a true linear request-rate x-axis and added worst-repetition evidence
17
+ - strengthened release checks for browser-module synchronization, package version, provenance, and ASCII-safe UI labels
18
+
19
+ ## v0.1.0
20
+
21
+ Initial browser-native Python release with workload generation, static/continuous scheduling, chunked prefill, KV-cache accounting, analytical latency profiles, Scheduler Arena, and SLO capacity search.
README.md CHANGED
@@ -10,7 +10,7 @@ license: mit
10
  short_description: Interactive LLM serving simulator and SLO planner
11
  ---
12
 
13
- # InferScale-Sim v0.1.0
14
 
15
  **Interactive LLM serving simulator and SLO-aware capacity planner — written in Python, executed entirely in the browser.**
16
 
@@ -21,7 +21,7 @@ InferScale-Sim explores a practical systems question:
21
  The public Hugging Face Space uses **no server CPU, no GPU, no API key, and no paid inference provider**. Hugging Face serves static files; Pyodide executes the same Python simulator used by the local test suite inside a Web Worker on the visitor's ordinary CPU.
22
 
23
  > [!IMPORTANT]
24
- > v0.1 ships with **analytical reference latency profiles**, not measured GPU calibration data. The simulator is useful for studying serving-system dynamics and relative scheduler behavior, but its absolute latency predictions must not be presented as benchmark measurements. A future calibrated profile bundle can replace the analytical backend without changing the simulator.
25
 
26
  ## Why simulation?
27
 
@@ -31,7 +31,7 @@ The area has continued moving quickly. InferScale-Sim references the broader lin
31
 
32
  InferScale-Sim is intentionally narrower: a compact, inspectable Python implementation focused on workload dynamics, scheduling, KV-cache behavior, SLOs, and interactive configuration search.
33
 
34
- ## v0.1 capabilities
35
 
36
  - deterministic **constant, Poisson, and bursty** arrival processes
37
  - log-normal prompt/output-length distributions
@@ -47,8 +47,13 @@ InferScale-Sim is intentionally narrower: a compact, inspectable Python implemen
47
  - throughput, output-token throughput, **goodput**, and SLO attainment
48
  - live queue / decode / KV-cache timeline
49
  - scheduler arena using an identical deterministic workload
50
- - binary **capacity search** with configurable safety headroom
51
- - JSON experiment export
 
 
 
 
 
52
  - completely client-side Hugging Face deployment via **Pyodide Web Worker**
53
 
54
  ## Architecture
@@ -185,9 +190,9 @@ See [`docs/research.md`](docs/research.md) and [`docs/methodology.md`](docs/meth
185
 
186
  ## Roadmap
187
 
188
- **v0.1** focuses on a clean single-accelerator serving loop and explicit scheduler trade-offs.
189
 
190
- Potential v0.2 extensions, only after v0.1 validation:
191
 
192
  - empirical calibration profile import
193
  - prefill/decode disaggregation + KV transfer
 
10
  short_description: Interactive LLM serving simulator and SLO planner
11
  ---
12
 
13
+ # InferScale-Sim v0.2.0
14
 
15
  **Interactive LLM serving simulator and SLO-aware capacity planner — written in Python, executed entirely in the browser.**
16
 
 
21
  The public Hugging Face Space uses **no server CPU, no GPU, no API key, and no paid inference provider**. Hugging Face serves static files; Pyodide executes the same Python simulator used by the local test suite inside a Web Worker on the visitor's ordinary CPU.
22
 
23
  > [!IMPORTANT]
24
+ > v0.2 ships with **analytical reference latency profiles**, not measured GPU calibration data. The simulator is useful for studying serving-system dynamics and relative scheduler behavior, but its absolute latency predictions must not be presented as benchmark measurements. A future calibrated profile bundle can replace the analytical backend without changing the simulator.
25
 
26
  ## Why simulation?
27
 
 
31
 
32
  InferScale-Sim is intentionally narrower: a compact, inspectable Python implementation focused on workload dynamics, scheduling, KV-cache behavior, SLOs, and interactive configuration search.
33
 
34
+ ## v0.2 capabilities
35
 
36
  - deterministic **constant, Poisson, and bursty** arrival processes
37
  - log-normal prompt/output-length distributions
 
47
  - throughput, output-token throughput, **goodput**, and SLO attainment
48
  - live queue / decode / KV-cache timeline
49
  - scheduler arena using an identical deterministic workload
50
+ - conservative binary **capacity search** where every repetition must meet the target and drain
51
+ - repetition min/mean/max SLO evidence for searched rates
52
+ - simulator-side **bottleneck diagnosis** from queueing, SLO misses, busy fraction, unfinished work, and KV pressure
53
+ - exposed burst stress controls
54
+ - copy-to-clipboard and CSV export for Arena / Planner tables
55
+ - expandable full-screen charts
56
+ - JSON experiment copy/export
57
  - completely client-side Hugging Face deployment via **Pyodide Web Worker**
58
 
59
  ## Architecture
 
190
 
191
  ## Roadmap
192
 
193
+ **v0.2** keeps the single-accelerator serving loop while improving inspectability and planner robustness. It adds bottleneck explanations, repetition-aware capacity evidence, burst controls, copy/CSV tooling, expandable charts, and a restrained engineering-oriented interface.
194
 
195
+ Potential v0.3 extensions, only after v0.2 validation:
196
 
197
  - empirical calibration profile import
198
  - prefill/decode disaggregation + KV transfer
app.js CHANGED
@@ -1,19 +1,24 @@
1
  const $ = (id) => document.getElementById(id);
2
-
3
  const runtimePill = $("runtimePill");
4
  const runtimeText = $("runtimeText");
 
 
5
  let lastResult = null;
6
- let charts = {};
 
7
  let requestId = 0;
8
- const pending = new Map();
9
 
10
- const worker = new Worker("./worker.mjs", { type: "module" });
 
 
11
  worker.addEventListener("message", (event) => {
12
  const data = event.data || {};
13
  if (data.type === "ready") {
14
  runtimePill.classList.add("ready");
15
  runtimeText.textContent = "Python runtime ready";
16
- ["runBtn", "arenaBtn", "capacityBtn"].forEach((id) => $(id).disabled = false);
 
17
  return;
18
  }
19
  if (data.type === "fatal") {
@@ -23,193 +28,59 @@ worker.addEventListener("message", (event) => {
23
  return;
24
  }
25
  if (!pending.has(data.id)) return;
26
- const { resolve, reject } = pending.get(data.id);
27
  pending.delete(data.id);
28
  data.error ? reject(new Error(data.error)) : resolve(data.result);
29
  });
30
 
31
- function callPython(action, payload) {
32
- const id = ++requestId;
33
- return new Promise((resolve, reject) => {
34
- pending.set(id, { resolve, reject });
35
- worker.postMessage({ id, action, payload });
36
- });
37
- }
38
-
39
- function num(id) { return Number($(id).value); }
40
- function configFromUI() {
41
- return {
42
- model: $("model").value,
43
- accelerator: $("accelerator").value,
44
- scheduler: $("scheduler").value,
45
- quantization: $("quantization").value,
46
- arrival_process: $("arrival").value,
47
- request_rate_rps: num("rate"),
48
- duration_s: num("duration"),
49
- prompt_tokens_mean: num("promptMean"),
50
- prompt_tokens_cv: num("promptCv"),
51
- output_tokens_mean: num("outputMean"),
52
- output_tokens_cv: num("outputCv"),
53
- max_batch_size: num("maxBatch"),
54
- max_batch_tokens: num("maxBatchTokens"),
55
- chunk_size: num("chunkSize"),
56
- kv_block_tokens: num("kvBlock"),
57
- seed: num("seed"),
58
- slo_ttft_ms: num("sloTtft"),
59
- slo_e2e_ms: num("sloE2e"),
60
- slo_attainment_target: num("targetSlo"),
61
- };
62
- }
63
-
64
- function fmt(value, digits = 1) {
65
- if (!Number.isFinite(value)) return "—";
66
- return value.toLocaleString(undefined, { maximumFractionDigits: digits });
67
- }
68
- function pct(v, digits = 1) { return `${fmt(v * 100, digits)}%`; }
69
-
70
- function destroyChart(name) {
71
- if (charts[name]) { charts[name].destroy(); delete charts[name]; }
72
- }
73
-
74
- const chartDefaults = {
75
- color: "#8f9aab",
76
- borderColor: "rgba(150,160,180,.13)",
77
- };
78
- Chart.defaults.color = chartDefaults.color;
79
- Chart.defaults.borderColor = chartDefaults.borderColor;
80
- Chart.defaults.font.family = getComputedStyle(document.body).fontFamily;
81
-
82
- function renderSimulation(result) {
83
- lastResult = result;
84
- $("emptyState").classList.add("hidden");
85
- $("resultContent").classList.remove("hidden");
86
- $("exportBtn").disabled = false;
87
- const s = result.summary, l = result.latency, r = result.resource;
88
- $("mTtft").textContent = `${fmt(l.ttft_ms.p95)} ms`;
89
- $("mE2e").textContent = `${fmt(l.e2e_ms.p95)} ms`;
90
- $("mGoodput").textContent = `${fmt(s.goodput_rps, 2)} req/s`;
91
- $("mSlo").textContent = pct(s.slo_attainment);
92
- $("mReq").textContent = `${fmt(s.request_throughput_rps, 2)} req/s`;
93
- $("mKv").textContent = `${fmt(r.peak_kv_gb, 2)} GB`;
94
- const tag = $("runState");
95
- tag.textContent = `${s.requests_completed}/${s.requests_generated} completed`;
96
- tag.className = `tag ${s.slo_attainment >= 0.99 && s.requests_unfinished === 0 ? "good" : "bad"}`;
97
-
98
- destroyChart("latency");
99
- charts.latency = new Chart($("latencyChart"), {
100
- type: "bar",
101
- data: {
102
- labels: ["TTFT p50", "TTFT p95", "E2E p50", "E2E p95", "Queue p95"],
103
- datasets: [{ label: "milliseconds", data: [l.ttft_ms.p50,l.ttft_ms.p95,l.e2e_ms.p50,l.e2e_ms.p95,l.queue_ms.p95], backgroundColor: ["#766ef0","#8b7cff","#3b82f6","#55c2ff","#63d9a5"], borderRadius: 5 }]
104
- },
105
- options: { responsive:true, maintainAspectRatio:false, plugins:{legend:{display:false}}, scales:{y:{beginAtZero:true}} }
106
- });
107
-
108
- destroyChart("timeline");
109
- const timeline = result.timeline;
110
- charts.timeline = new Chart($("timelineChart"), {
111
- type: "line",
112
- data: {
113
- labels: timeline.map(x => Number(x.time_s.toFixed(2))),
114
- datasets: [
115
- {label:"Waiting",data:timeline.map(x=>x.waiting),borderColor:"#ffad66",pointRadius:0,tension:.15},
116
- {label:"Decoding",data:timeline.map(x=>x.decoding),borderColor:"#8b7cff",pointRadius:0,tension:.15},
117
- {label:"KV GB",data:timeline.map(x=>x.kv_used_gb),borderColor:"#63d9a5",pointRadius:0,tension:.15,yAxisID:"y1"}
118
- ]
119
- },
120
- options:{responsive:true,maintainAspectRatio:false,interaction:{mode:"index",intersect:false},scales:{x:{title:{display:true,text:"virtual time (s)"}},y:{beginAtZero:true,title:{display:true,text:"requests"}},y1:{beginAtZero:true,position:"right",grid:{drawOnChartArea:false},title:{display:true,text:"KV GB"}}}}
121
- });
122
-
123
- destroyChart("scatter");
124
- const sample = result.requests || [];
125
- charts.scatter = new Chart($("scatterChart"), {
126
- type:"scatter",
127
- data:{datasets:[{label:"requests",data:sample.map(x=>({x:x.prompt_tokens,y:x.ttft_ms})),backgroundColor:"rgba(85,194,255,.55)",pointRadius:2.5}]},
128
- options:{responsive:true,maintainAspectRatio:false,plugins:{legend:{display:false}},scales:{x:{title:{display:true,text:"prompt tokens"}},y:{title:{display:true,text:"TTFT (ms)"},beginAtZero:true}}}
129
- });
130
-
131
- const warnings = $("warnings");
132
- const allWarnings = [...(result.warnings || [])];
133
- if (result.provenance?.profile_warning) allWarnings.unshift(result.provenance.profile_warning);
134
- if (allWarnings.length) {
135
- warnings.innerHTML = allWarnings.map(w=>`<div>• ${escapeHtml(w)}</div>`).join("");
136
- warnings.classList.remove("hidden");
137
- } else warnings.classList.add("hidden");
138
- }
139
-
140
- function escapeHtml(value) {
141
- return String(value).replace(/[&<>'"]/g, c => ({"&":"&amp;","<":"&lt;",">":"&gt;","'":"&#39;",'"':"&quot;"}[c]));
142
- }
143
-
144
- function setBusy(button, stateEl, busy, label) {
145
- button.disabled = busy;
146
- if (stateEl) {
147
- stateEl.textContent = busy ? label : stateEl.textContent;
148
- if (busy) stateEl.className = "tag neutral";
149
- }
150
- }
151
-
152
- $("runBtn").addEventListener("click", async () => {
153
- const btn = $("runBtn"), state = $("runState");
154
- setBusy(btn,state,true,"Simulating…"); btn.textContent="Running Python simulation…";
155
- try {
156
- const result = await callPython("simulate", configFromUI());
157
- renderSimulation(result);
158
- } catch (err) {
159
- state.textContent="Error"; state.className="tag bad"; alert(`Simulation failed: ${err.message}`);
160
- } finally {
161
- btn.textContent="Run simulation"; btn.disabled=false;
162
- }
163
- });
164
-
165
- $("exportBtn").addEventListener("click", () => {
166
- if (!lastResult) return;
167
- const blob = new Blob([JSON.stringify(lastResult,null,2)], {type:"application/json"});
168
- const a = document.createElement("a"); a.href=URL.createObjectURL(blob); a.download=`inferscale-${Date.now()}.json`; a.click(); URL.revokeObjectURL(a.href);
169
- });
170
-
171
- $("arenaBtn").addEventListener("click", async () => {
172
- const btn=$("arenaBtn"); btn.disabled=true; btn.textContent="Comparing…";
173
- try {
174
- const result = await callPython("compare", {config:configFromUI()});
175
- renderArena(result.rows);
176
- } catch(err){ alert(`Scheduler comparison failed: ${err.message}`); }
177
- finally { btn.disabled=false; btn.textContent="Compare schedulers"; }
178
- });
179
-
180
- function schedulerLabel(s) { return ({static_fcfs:"Static FCFS",continuous_fcfs:"Continuous FCFS",continuous_sjf:"Continuous SJF",continuous_slo:"Continuous SLO",chunked_slo:"Chunked SLO"})[s] || s; }
181
- function renderArena(rows) {
182
- $("arenaEmpty").classList.add("hidden"); $("arenaContent").classList.remove("hidden");
183
- $("arenaRows").innerHTML = rows.map((r,i)=>`<tr><td>${i===0?"★ ":""}${schedulerLabel(r.scheduler)}</td><td>${fmt(r.goodput_rps,2)} req/s</td><td>${pct(r.slo_attainment)}</td><td>${fmt(r.p95_ttft_ms)} ms</td><td>${fmt(r.p95_e2e_ms)} ms</td><td>${pct(r.peak_kv_utilization)}</td><td>${fmt(r.unfinished,0)}</td></tr>`).join("");
184
- destroyChart("arena");
185
- charts.arena = new Chart($("arenaChart"),{type:"bar",data:{labels:rows.map(r=>schedulerLabel(r.scheduler)),datasets:[{label:"Goodput (req/s)",data:rows.map(r=>r.goodput_rps),backgroundColor:"#8b7cff",borderRadius:5},{label:"Raw throughput (req/s)",data:rows.map(r=>r.request_throughput_rps),backgroundColor:"#3f7ee8",borderRadius:5}]},options:{responsive:true,maintainAspectRatio:false,scales:{y:{beginAtZero:true}}}});
186
  }
187
 
188
- $("capacityBtn").addEventListener("click", async () => {
189
- const btn=$("capacityBtn"), state=$("plannerState");
190
- btn.disabled=true; btn.textContent="Searching…"; state.textContent="Running simulations…"; state.className="tag neutral";
191
- try {
192
- const config=configFromUI(); config.slo_attainment_target=num("targetSlo");
193
- const result=await callPython("capacity",{config,min_rate:num("minRate"),max_rate:num("maxRate"),iterations:num("searchIter"),repetitions:num("repetitions"),headroom:num("headroom")});
194
- renderCapacity(result);
195
- } catch(err){ state.textContent="Error";state.className="tag bad";alert(`Capacity search failed: ${err.message}`); }
196
- finally { btn.disabled=false; btn.textContent="Find sustainable capacity"; }
197
- });
198
-
199
- function renderCapacity(result) {
200
- $("plannerEmpty").classList.add("hidden"); $("plannerContent").classList.remove("hidden");
201
- $("pCapacity").textContent=`${fmt(result.capacity_rps,2)} req/s`; $("pRecommended").textContent=`${fmt(result.recommended_rps,2)} req/s`; $("pHeadroom").textContent=pct(result.headroom ?? num("headroom")); $("pStatus").textContent=result.status.replaceAll("_"," ");
202
- const state=$("plannerState"); state.textContent=result.status==="ok"?"Search complete":result.status.replaceAll("_"," "); state.className=`tag ${result.capacity_rps>0?"good":"bad"}`;
203
- const target=num("targetSlo");
204
- $("capacityRows").innerHTML=result.trace.map(r=>`<tr><td>${fmt(r.rate_rps,2)} req/s</td><td class="${r.passed?"pass":"fail"}">${r.passed?"PASS":"FAIL"}</td><td>${pct(r.slo_attainment)}</td><td>${fmt(r.goodput_rps,2)} req/s</td><td>${fmt(r.p95_ttft_ms)} ms</td><td>${fmt(r.p95_e2e_ms)} ms</td></tr>`).join("");
205
- destroyChart("capacity");
206
- charts.capacity=new Chart($("capacityChart"),{type:"line",data:{labels:result.trace.map(r=>r.rate_rps),datasets:[{label:"SLO attainment",data:result.trace.map(r=>r.slo_attainment),borderColor:"#8b7cff",backgroundColor:"rgba(139,124,255,.13)",fill:true,tension:.15,pointRadius:4},{label:"Target",data:result.trace.map(()=>target),borderColor:"#63d9a5",borderDash:[6,5],pointRadius:0}]},options:{responsive:true,maintainAspectRatio:false,scales:{x:{title:{display:true,text:"offered load (req/s)"}},y:{min:0,max:1,ticks:{callback:v=>`${Math.round(v*100)}%`}}}}});
207
- }
208
-
209
- for (const tab of document.querySelectorAll(".tab")) {
210
- tab.addEventListener("click", () => {
211
- document.querySelectorAll(".tab").forEach(t=>t.classList.remove("active"));
212
- document.querySelectorAll(".tab-panel").forEach(p=>p.classList.remove("active"));
213
- tab.classList.add("active"); $(tab.dataset.tab).classList.add("active");
214
- });
215
- }
 
1
  const $ = (id) => document.getElementById(id);
 
2
  const runtimePill = $("runtimePill");
3
  const runtimeText = $("runtimeText");
4
+ const charts = {};
5
+ const pending = new Map();
6
  let lastResult = null;
7
+ let lastArenaRows = [];
8
+ let lastCapacityTrace = [];
9
  let requestId = 0;
10
+ let toastTimer = null;
11
 
12
+ const COLORS = {blue:"#79a7ff", blue2:"#5d8fe9", steel:"#92a2b5", green:"#69c99a", amber:"#dfb966", grid:"rgba(140,155,175,.14)"};
13
+
14
+ const worker = new Worker("./worker.mjs", {type:"module"});
15
  worker.addEventListener("message", (event) => {
16
  const data = event.data || {};
17
  if (data.type === "ready") {
18
  runtimePill.classList.add("ready");
19
  runtimeText.textContent = "Python runtime ready";
20
+ ["runBtn","arenaBtn","capacityBtn"].forEach((id) => $(id).disabled = false);
21
+ window.setTimeout(() => document.body.classList.add("runtime-ready"), 650);
22
  return;
23
  }
24
  if (data.type === "fatal") {
 
28
  return;
29
  }
30
  if (!pending.has(data.id)) return;
31
+ const {resolve,reject} = pending.get(data.id);
32
  pending.delete(data.id);
33
  data.error ? reject(new Error(data.error)) : resolve(data.result);
34
  });
35
 
36
+ function callPython(action,payload){const id=++requestId;return new Promise((resolve,reject)=>{pending.set(id,{resolve,reject});worker.postMessage({id,action,payload});});}
37
+ function num(id){return Number($(id).value);}
38
+ function configFromUI(overrides={}){return {model:$("model").value,accelerator:$("accelerator").value,scheduler:$("scheduler").value,quantization:$("quantization").value,arrival_process:$("arrival").value,request_rate_rps:num("rate"),duration_s:num("duration"),prompt_tokens_mean:num("promptMean"),prompt_tokens_cv:num("promptCv"),output_tokens_mean:num("outputMean"),output_tokens_cv:num("outputCv"),max_batch_size:num("maxBatch"),max_batch_tokens:num("maxBatchTokens"),chunk_size:num("chunkSize"),kv_block_tokens:num("kvBlock"),burst_multiplier:num("burstMultiplier"),burst_period_s:num("burstPeriod"),seed:num("seed"),slo_ttft_ms:num("sloTtft"),slo_e2e_ms:num("sloE2e"),slo_attainment_target:.99,...overrides};}
39
+ function fmt(value,digits=1){if(!Number.isFinite(value))return "N/A";return value.toLocaleString(undefined,{maximumFractionDigits:digits});}
40
+ function pct(value,digits=1){return `${fmt(value*100,digits)}%`;}
41
+ function escapeHtml(value){return String(value).replace(/[&<>'"]/g,(c)=>({"&":"&amp;","<":"&lt;",">":"&gt;","'":"&#39;",'"':"&quot;"})[c]);}
42
+ function destroyChart(name){if(charts[name]){charts[name].destroy();delete charts[name];}}
43
+ function showToast(message){const toast=$("toast");toast.textContent=message;toast.classList.add("show");if(toastTimer)clearTimeout(toastTimer);toastTimer=setTimeout(()=>toast.classList.remove("show"),1800);}
44
+ async function copyText(text,message){try{await navigator.clipboard.writeText(text);showToast(message);}catch{const area=document.createElement("textarea");area.value=text;area.style.position="fixed";area.style.opacity="0";document.body.appendChild(area);area.select();document.execCommand("copy");area.remove();showToast(message);}}
45
+ function csvCell(value){const text=String(value??"");return /[",\n]/.test(text)?`"${text.replaceAll('"','""')}"`:text;}
46
+ function tableText(headers,rows,separator="\t"){return [headers,...rows].map((row)=>row.join(separator)).join("\n");}
47
+ function downloadCsv(filename,headers,rows){const csv=[headers,...rows].map((row)=>row.map(csvCell).join(",")).join("\n");const blob=new Blob([csv],{type:"text/csv;charset=utf-8"});const url=URL.createObjectURL(blob);const a=document.createElement("a");a.href=url;a.download=filename;a.click();URL.revokeObjectURL(url);}
48
+
49
+ Chart.defaults.color="#929dab";Chart.defaults.borderColor=COLORS.grid;Chart.defaults.font.family=getComputedStyle(document.body).fontFamily;Chart.defaults.animation.duration=180;
50
+ function commonChartOptions(){return {responsive:true,maintainAspectRatio:false,plugins:{legend:{labels:{boxWidth:12,boxHeight:8}}}};}
51
+
52
+ function renderSimulation(result){
53
+ lastResult=result;$("emptyState").classList.add("hidden");$("resultContent").classList.remove("hidden");$("exportBtn").disabled=false;$("copyResultBtn").disabled=false;
54
+ const s=result.summary,l=result.latency,r=result.resource,d=result.diagnostics||{};
55
+ $("mTtft").textContent=`${fmt(l.ttft_ms.p95)} ms`;$("mE2e").textContent=`${fmt(l.e2e_ms.p95)} ms`;$("mGoodput").textContent=`${fmt(s.goodput_rps,2)} req/s`;$("mSlo").textContent=pct(s.slo_attainment);$("mReq").textContent=`${fmt(s.request_throughput_rps,2)} req/s`;$("mKv").textContent=`${fmt(r.peak_kv_gb,2)} GB`;
56
+ const tag=$("runState");tag.textContent=`${s.requests_completed}/${s.requests_generated} completed`;tag.className=`tag ${s.slo_attainment>=.99&&s.requests_unfinished===0?"good":"bad"}`;
57
+ $("mBottleneck").textContent=d.label||"N/A";$("mDiagnosis").textContent=d.explanation||"No simulator diagnosis available.";$("mRecommendation").textContent=d.recommendation?`Next check: ${d.recommendation}`:"";
58
+ const e=d.evidence||{};$("mEvidence").innerHTML=[`busy ${pct(e.busy_fraction??s.busy_fraction)}`,`KV ${pct(e.peak_kv_utilization??r.peak_kv_utilization)}`,`TTFT pass ${pct(e.ttft_slo_attainment??s.ttft_slo_attainment)}`,`E2E pass ${pct(e.e2e_slo_attainment??s.e2e_slo_attainment)}`,`queue p95 ${fmt(e.queue_p95_ms??l.queue_ms.p95)} ms`,`target 99%`].map((x)=>`<span>${escapeHtml(x)}</span>`).join("");
59
+ destroyChart("latency");charts.latency=new Chart($("latencyChart"),{type:"bar",data:{labels:["TTFT p50","TTFT p95","E2E p50","E2E p95","Queue p95"],datasets:[{label:"Milliseconds",data:[l.ttft_ms.p50,l.ttft_ms.p95,l.e2e_ms.p50,l.e2e_ms.p95,l.queue_ms.p95],backgroundColor:[COLORS.steel,COLORS.blue,"#8196b1",COLORS.blue2,COLORS.amber]}]},options:{...commonChartOptions(),plugins:{legend:{display:false}},scales:{y:{beginAtZero:true}}}});
60
+ const timeline=result.timeline||[];destroyChart("timeline");charts.timeline=new Chart($("timelineChart"),{type:"line",data:{datasets:[{label:"Waiting",data:timeline.map((x)=>({x:x.time_s,y:x.waiting})),borderColor:COLORS.amber,pointRadius:0,tension:.08,yAxisID:"y"},{label:"Decoding",data:timeline.map((x)=>({x:x.time_s,y:x.decoding})),borderColor:COLORS.blue,pointRadius:0,tension:.08,yAxisID:"y"},{label:"KV GB",data:timeline.map((x)=>({x:x.time_s,y:x.kv_used_gb})),borderColor:COLORS.green,pointRadius:0,tension:.08,yAxisID:"y1"}]},options:{...commonChartOptions(),parsing:false,interaction:{mode:"nearest",intersect:false},scales:{x:{type:"linear",title:{display:true,text:"Virtual time (s)"}},y:{beginAtZero:true,title:{display:true,text:"Requests"}},y1:{beginAtZero:true,position:"right",grid:{drawOnChartArea:false},title:{display:true,text:"KV GB"}}}}});
61
+ const sample=result.requests||[];destroyChart("scatter");charts.scatter=new Chart($("scatterChart"),{type:"scatter",data:{datasets:[{label:"Requests",data:sample.map((x)=>({x:x.prompt_tokens,y:x.ttft_ms})),backgroundColor:"rgba(121,167,255,.52)",pointRadius:2.2}]},options:{...commonChartOptions(),plugins:{legend:{display:false}},scales:{x:{title:{display:true,text:"Prompt tokens"}},y:{title:{display:true,text:"TTFT (ms)"},beginAtZero:true}}}});
62
+ const warnings=$("warnings"),all=[...(result.warnings||[])];if(result.provenance?.profile_warning)all.unshift(result.provenance.profile_warning);if(all.length){warnings.innerHTML=all.map((w)=>`<div>${escapeHtml(w)}</div>`).join("");warnings.classList.remove("hidden");}else warnings.classList.add("hidden");
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
63
  }
64
 
65
+ $("runBtn").addEventListener("click",async()=>{const b=$("runBtn"),s=$("runState");b.disabled=true;b.textContent="Running simulation...";s.textContent="Simulating...";s.className="tag neutral";try{renderSimulation(await callPython("simulate",configFromUI()));}catch(e){s.textContent="Error";s.className="tag bad";alert(`Simulation failed: ${e.message}`);}finally{b.textContent="Run simulation";b.disabled=false;}});
66
+ $("copyResultBtn").addEventListener("click",()=>{if(lastResult)copyText(JSON.stringify(lastResult,null,2),"Result JSON copied");});
67
+ $("exportBtn").addEventListener("click",()=>{if(!lastResult)return;const blob=new Blob([JSON.stringify(lastResult,null,2)],{type:"application/json"});const url=URL.createObjectURL(blob);const a=document.createElement("a");a.href=url;a.download=`inferscale-run-${Date.now()}.json`;a.click();URL.revokeObjectURL(url);});
68
+
69
+ function schedulerLabel(s){return ({static_fcfs:"Static FCFS",continuous_fcfs:"Continuous FCFS",continuous_sjf:"Continuous SJF",continuous_slo:"Continuous SLO",chunked_slo:"Chunked SLO"})[s]||s;}
70
+ $("arenaBtn").addEventListener("click",async()=>{const b=$("arenaBtn");b.disabled=true;b.textContent="Comparing...";try{renderArena((await callPython("compare",{config:configFromUI()})).rows);}catch(e){alert(`Scheduler comparison failed: ${e.message}`);}finally{b.disabled=false;b.textContent="Compare schedulers";}});
71
+ function arenaTableRows(rows){return rows.map((r)=>[schedulerLabel(r.scheduler),`${fmt(r.goodput_rps,2)} req/s`,pct(r.slo_attainment),`${fmt(r.p95_ttft_ms)} ms`,`${fmt(r.p95_e2e_ms)} ms`,pct(r.peak_kv_utilization),fmt(r.unfinished,0),r.bottleneck||"N/A"]);}
72
+ function renderArena(rows){lastArenaRows=rows;$("arenaEmpty").classList.add("hidden");$("arenaContent").classList.remove("hidden");$("arenaCopyBtn").disabled=false;$("arenaCsvBtn").disabled=false;$("arenaRows").innerHTML=rows.map((r,i)=>`<tr><td>${escapeHtml(schedulerLabel(r.scheduler))}${i===0?'<span class="best-label">Best</span>':""}</td><td>${fmt(r.goodput_rps,2)} req/s</td><td>${pct(r.slo_attainment)}</td><td>${fmt(r.p95_ttft_ms)} ms</td><td>${fmt(r.p95_e2e_ms)} ms</td><td>${pct(r.peak_kv_utilization)}</td><td>${fmt(r.unfinished,0)}</td><td>${escapeHtml(r.bottleneck||"N/A")}</td></tr>`).join("");destroyChart("arena");charts.arena=new Chart($("arenaChart"),{type:"bar",data:{labels:rows.map((r)=>schedulerLabel(r.scheduler)),datasets:[{label:"Goodput (req/s)",data:rows.map((r)=>r.goodput_rps),backgroundColor:COLORS.blue},{label:"Raw throughput (req/s)",data:rows.map((r)=>r.request_throughput_rps),backgroundColor:"#53677f"}]},options:{...commonChartOptions(),scales:{y:{beginAtZero:true}}}});}
73
+ const arenaHeaders=["Scheduler","Goodput","SLO attainment","p95 TTFT","p95 E2E","KV peak","Unfinished","Diagnosis"];
74
+ $("arenaCopyBtn").addEventListener("click",()=>copyText(tableText(arenaHeaders,arenaTableRows(lastArenaRows)),"Scheduler table copied"));$("arenaCsvBtn").addEventListener("click",()=>downloadCsv("inferscale-scheduler-arena.csv",arenaHeaders,arenaTableRows(lastArenaRows)));
75
+
76
+ $("capacityBtn").addEventListener("click",async()=>{const b=$("capacityBtn"),s=$("plannerState");b.disabled=true;b.textContent="Searching...";s.textContent="Running simulations...";s.className="tag neutral";try{renderCapacity(await callPython("capacity",{config:configFromUI({slo_attainment_target:num("targetSlo")}),min_rate:num("minRate"),max_rate:num("maxRate"),iterations:num("searchIter"),repetitions:num("repetitions"),headroom:num("headroom")}));}catch(e){s.textContent="Error";s.className="tag bad";alert(`Capacity search failed: ${e.message}`);}finally{b.disabled=false;b.textContent="Find sustainable capacity";}});
77
+ function capacityTableRows(trace){return trace.map((r)=>[`${fmt(r.rate_rps,2)} req/s`,r.passed?"PASS":"FAIL",pct(r.slo_attainment),`${pct(r.slo_attainment_min)} - ${pct(r.slo_attainment_max)}`,`${fmt(r.goodput_rps,2)} req/s`,`${fmt(r.p95_ttft_ms)} ms`,`${fmt(r.p95_e2e_ms)} ms`]);}
78
+ function renderCapacity(result){lastCapacityTrace=result.trace||[];$("plannerEmpty").classList.add("hidden");$("plannerContent").classList.remove("hidden");$("capacityCopyBtn").disabled=false;$("capacityCsvBtn").disabled=false;$("pCapacity").textContent=`${fmt(result.capacity_rps,2)} req/s`;$("pRecommended").textContent=`${fmt(result.recommended_rps,2)} req/s`;$("pHeadroom").textContent=pct(result.headroom??num("headroom"));$("pStatus").textContent=result.status.replaceAll("_"," ");const state=$("plannerState");state.textContent=result.status==="ok"?"Search complete":result.status.replaceAll("_"," ");state.className=`tag ${result.capacity_rps>0?"good":"bad"}`;$("capacityRows").innerHTML=lastCapacityTrace.map((r)=>`<tr><td>${fmt(r.rate_rps,2)} req/s</td><td class="${r.passed?"pass":"fail"}">${r.passed?"PASS":"FAIL"}</td><td>${pct(r.slo_attainment)}</td><td>${pct(r.slo_attainment_min)} - ${pct(r.slo_attainment_max)}</td><td>${fmt(r.goodput_rps,2)} req/s</td><td>${fmt(r.p95_ttft_ms)} ms</td><td>${fmt(r.p95_e2e_ms)} ms</td></tr>`).join("");const target=num("targetSlo"),mean=lastCapacityTrace.map((r)=>({x:r.rate_rps,y:r.slo_attainment})),worst=lastCapacityTrace.map((r)=>({x:r.rate_rps,y:r.slo_attainment_min})),xs=lastCapacityTrace.map((r)=>r.rate_rps),minX=Math.min(...xs),maxX=Math.max(...xs);destroyChart("capacity");charts.capacity=new Chart($("capacityChart"),{type:"line",data:{datasets:[{label:"Mean SLO attainment",data:mean,borderColor:COLORS.blue,backgroundColor:"rgba(121,167,255,.10)",fill:true,tension:.08,pointRadius:3},{label:"Worst repetition",data:worst,borderColor:COLORS.amber,borderDash:[5,4],pointRadius:2,tension:.08},{label:"Target",data:[{x:minX,y:target},{x:maxX,y:target}],borderColor:COLORS.green,borderDash:[6,5],pointRadius:0}]},options:{...commonChartOptions(),parsing:false,scales:{x:{type:"linear",title:{display:true,text:"Offered load (req/s)"},ticks:{maxTicksLimit:8}},y:{min:0,max:1,ticks:{callback:(v)=>`${Math.round(v*100)}%`}}}}});}
79
+ const capacityHeaders=["Rate","Pass","Mean SLO","SLO range","Goodput","p95 TTFT","p95 E2E"];
80
+ $("capacityCopyBtn").addEventListener("click",()=>copyText(tableText(capacityHeaders,capacityTableRows(lastCapacityTrace)),"Capacity table copied"));$("capacityCsvBtn").addEventListener("click",()=>downloadCsv("inferscale-capacity-trace.csv",capacityHeaders,capacityTableRows(lastCapacityTrace)));
81
+
82
+ $("arrival").addEventListener("change",()=>$("burstControls").classList.toggle("hidden",$("arrival").value!=="bursty"));
83
+ for(const tab of document.querySelectorAll(".tab")){tab.addEventListener("click",()=>{document.querySelectorAll(".tab").forEach((t)=>t.classList.remove("active"));document.querySelectorAll(".tab-panel").forEach((p)=>p.classList.remove("active"));tab.classList.add("active");$(tab.dataset.tab).classList.add("active");setTimeout(()=>Object.values(charts).forEach((chart)=>chart.resize()),20);});}
84
+ function closeExpandedChart(){const card=document.querySelector(".chart-card.chart-expanded");if(!card)return;card.classList.remove("chart-expanded");const b=card.querySelector(".chart-expand");if(b)b.textContent="Expand";document.body.classList.remove("chart-open");setTimeout(()=>Chart.getChart(card.querySelector("canvas"))?.resize(),20);}
85
+ for(const b of document.querySelectorAll(".chart-expand")){b.addEventListener("click",()=>{const card=b.closest(".chart-card"),was=card.classList.contains("chart-expanded");closeExpandedChart();if(!was){card.classList.add("chart-expanded");b.textContent="Close";document.body.classList.add("chart-open");setTimeout(()=>Chart.getChart(card.querySelector("canvas"))?.resize(),20);}});}
86
+ document.addEventListener("keydown",(e)=>{if(e.key==="Escape")closeExpandedChart();});
 
 
 
 
 
 
docs/methodology.md CHANGED
@@ -1,6 +1,6 @@
1
  # Methodology and limitations
2
 
3
- ## What v0.1 simulates
4
 
5
  InferScale models request arrival, queueing, admission, prefill, autoregressive decode, dynamic batch membership, KV-cache memory and request completion. Metrics are computed from per-request virtual timestamps.
6
 
@@ -34,7 +34,7 @@ This produces useful qualitative dynamics but is **not empirically calibrated**.
34
 
35
  ## Quantization
36
 
37
- INT8 and INT4 alter model weight footprint and apply a conservative compute-overhead multiplier. v0.1 does not claim a specific kernel implementation, quantization scheme, or quality impact.
38
 
39
  ## KV-cache
40
 
@@ -48,13 +48,22 @@ for K and V with FP16 KV state. Paged allocation rounds live sequence lengths to
48
 
49
  ## Capacity search
50
 
51
- Capacity search evaluates an offered request rate over multiple deterministic seeds and marks the rate feasible when:
52
 
53
- 1. mean SLO attainment >= configured target; and
54
- 2. no requests remain unfinished at the end of the simulated drain.
55
 
56
- A bounded binary search then estimates the highest feasible offered rate. The recommended rate applies user-configured safety headroom.
57
 
58
  ## Validation status
59
 
60
- v0.1 validates software invariants and expected qualitative behavior through unit tests. It does **not** yet provide held-out GPU calibration error. Empirical profile calibration and validation are intentionally listed as v0.2 work rather than being fabricated for the initial release.
 
 
 
 
 
 
 
 
 
 
1
  # Methodology and limitations
2
 
3
+ ## What v0.2 simulates
4
 
5
  InferScale models request arrival, queueing, admission, prefill, autoregressive decode, dynamic batch membership, KV-cache memory and request completion. Metrics are computed from per-request virtual timestamps.
6
 
 
34
 
35
  ## Quantization
36
 
37
+ INT8 and INT4 alter model weight footprint and apply a conservative compute-overhead multiplier. v0.2 does not claim a specific kernel implementation, quantization scheme, or quality impact.
38
 
39
  ## KV-cache
40
 
 
48
 
49
  ## Capacity search
50
 
51
+ Capacity search evaluates an offered request rate over multiple deterministic seeds and marks the rate feasible only when **every repetition**:
52
 
53
+ 1. reaches the configured SLO-attainment target; and
54
+ 2. leaves no requests unfinished after the simulated drain.
55
 
56
+ A bounded binary search then estimates the highest feasible offered rate. Mean/min/max attainment are retained for evidence, and the recommended rate applies user-configured safety headroom.
57
 
58
  ## Validation status
59
 
60
+ v0.2 validates software invariants and expected qualitative behavior through unit tests. It does **not** yet provide held-out GPU calibration error. Empirical profile calibration and validation remain future work rather than being fabricated for this release.
61
+
62
+
63
+ ## v0.2 bottleneck diagnosis
64
+
65
+ InferScale derives a compact explanation from the simulator's own state: component SLO attainment, p95 queue latency, virtual device busy fraction, peak KV utilization, and unfinished requests. These are heuristic simulator diagnoses, not hardware-counter diagnoses of a real GPU.
66
+
67
+ ## v0.2 robust capacity criterion
68
+
69
+ For each searched load, the planner repeats the workload with deterministic seed offsets. A rate passes only when every repetition reaches the configured SLO target and fully drains. Mean attainment is plotted while minimum and maximum attainment remain visible in the trace.
docs/research.md CHANGED
@@ -30,8 +30,8 @@ Project inspiration: profile-based hardware abstraction and future disaggregatio
30
 
31
  Frontier models modern serving structures including co-location, prefill/decode disaggregation, Attention-FFN disaggregation, runtime optimizations and stateful workloads. Its May 2026 paper reports average throughput error below 4% on a 16-H800 testbed and large improvements in end-to-end latency error over baseline simulators.
32
 
33
- Project inspiration for v0.2: prefill/decode disaggregation, KV transfer and workload-state dependencies.
34
 
35
  ## Scope boundary
36
 
37
- InferScale-Sim v0.1 is not intended to compete with these research systems on fidelity or scale. Its contribution is an inspectable, dependency-light Python implementation and a zero-backend interactive interface for exploring the underlying serving dynamics.
 
30
 
31
  Frontier models modern serving structures including co-location, prefill/decode disaggregation, Attention-FFN disaggregation, runtime optimizations and stateful workloads. Its May 2026 paper reports average throughput error below 4% on a 16-H800 testbed and large improvements in end-to-end latency error over baseline simulators.
32
 
33
+ Project inspiration for future work: prefill/decode disaggregation, KV transfer and workload-state dependencies.
34
 
35
  ## Scope boundary
36
 
37
+ InferScale-Sim v0.2 is not intended to compete with these research systems on fidelity or scale. Its contribution is an inspectable, dependency-light Python implementation and a zero-backend interactive interface for exploring the underlying serving dynamics.
docs/validation.md CHANGED
@@ -1,4 +1,4 @@
1
- # v0.1 validation checklist
2
 
3
  The release check is designed to prevent deployment and provenance mistakes rather than pretend the analytical profile has empirical accuracy.
4
 
@@ -13,6 +13,10 @@ The release check is designed to prevent deployment and provenance mistakes rath
13
  - static vs continuous batching behavioral difference
14
  - capacity-search output sanity
15
  - scheduler-arena output sanity
 
 
 
 
16
  - Hugging Face `short_description` <= 60 characters
17
  - `sdk: static` metadata
18
  - canonical Python source == browser mirror
@@ -20,7 +24,7 @@ The release check is designed to prevent deployment and provenance mistakes rath
20
  - JavaScript syntax parse
21
  - Python compilation
22
 
23
- ## Not claimed in v0.1
24
 
25
  - empirical L4/A10G/A100 latency accuracy
26
  - exact vLLM/SGLang scheduler equivalence
 
1
+ # v0.2 validation checklist
2
 
3
  The release check is designed to prevent deployment and provenance mistakes rather than pretend the analytical profile has empirical accuracy.
4
 
 
13
  - static vs continuous batching behavioral difference
14
  - capacity-search output sanity
15
  - scheduler-arena output sanity
16
+ - component TTFT/E2E SLO-attainment accounting
17
+ - simulator bottleneck diagnosis provenance
18
+ - capacity repetition range / conservative pass criterion
19
+ - ASCII-only public UI labels for reliable rendering
20
  - Hugging Face `short_description` <= 60 characters
21
  - `sdk: static` metadata
22
  - canonical Python source == browser mirror
 
24
  - JavaScript syntax parse
25
  - Python compilation
26
 
27
+ ## Not claimed in v0.2
28
 
29
  - empirical L4/A10G/A100 latency accuracy
30
  - exact vLLM/SGLang scheduler equivalence
index.html CHANGED
@@ -10,198 +10,52 @@
10
  <script src="https://cdn.jsdelivr.net/npm/chart.js@4.5.1/dist/chart.umd.min.js"></script>
11
  </head>
12
  <body>
13
- <header class="topbar">
14
- <div class="brand-wrap">
15
- <div class="logo">IS</div>
16
- <div>
17
- <div class="brand">InferScale-Sim</div>
18
- <div class="subtitle">LLM serving systems laboratory</div>
19
- </div>
20
- </div>
21
- <div class="runtime-pill" id="runtimePill"><span class="dot"></span><span id="runtimeText">Loading Python runtime…</span></div>
22
  </header>
23
 
24
  <main class="shell">
25
- <section class="hero">
26
- <div>
27
- <div class="eyebrow">v0.1.0 · browser-native Python</div>
28
- <h1>Explore LLM serving dynamics without provisioning a GPU.</h1>
29
- <p>Generate workloads, compare schedulers, inspect queueing and KV pressure, then search for the maximum SLO-compliant request rate. The simulator runs locally in your browser through Pyodide.</p>
30
- </div>
31
- <div class="hero-stat-grid">
32
- <div class="hero-stat"><span>Backend</span><strong>None</strong></div>
33
- <div class="hero-stat"><span>Runtime</span><strong>Python / WASM</strong></div>
34
- <div class="hero-stat"><span>Data sent</span><strong>0 bytes</strong></div>
35
- <div class="hero-stat"><span>Profile type</span><strong>Analytical</strong></div>
36
- </div>
37
- </section>
38
-
39
- <div class="notice">
40
- <strong>Reference-profile mode.</strong> v0.1 uses analytical hardware/model profiles to study systems behavior. Absolute latency values are predictions, not measured GPU benchmarks.
41
- </div>
42
-
43
- <nav class="tabs" aria-label="InferScale sections">
44
- <button class="tab active" data-tab="lab">Serving Lab</button>
45
- <button class="tab" data-tab="arena">Scheduler Arena</button>
46
- <button class="tab" data-tab="planner">Capacity Planner</button>
47
- <button class="tab" data-tab="method">Methodology</button>
48
- </nav>
49
-
50
- <section id="lab" class="tab-panel active">
51
- <div class="workspace">
52
- <aside class="panel controls-panel">
53
- <div class="panel-title-row"><h2>Experiment</h2><span class="tag">Deterministic seed</span></div>
54
-
55
- <div class="field-grid two">
56
- <label>Model<select id="model"><option>Qwen2.5-3B</option><option>Llama-3.1-8B</option><option>Mistral-7B-v0.3</option></select></label>
57
- <label>Accelerator<select id="accelerator"><option value="L4">NVIDIA L4</option><option value="A10G">NVIDIA A10G</option><option value="A100-40GB">NVIDIA A100 40GB</option></select></label>
58
- </div>
59
-
60
- <div class="field-grid two">
61
- <label>Scheduler<select id="scheduler"><option value="continuous_fcfs">Continuous · FCFS</option><option value="continuous_sjf">Continuous · SJF</option><option value="continuous_slo">Continuous · SLO-aware</option><option value="chunked_slo">Chunked prefill · SLO-aware</option><option value="static_fcfs">Static batching · FCFS</option></select></label>
62
- <label>Weight precision<select id="quantization"><option value="fp16">FP16</option><option value="int8" selected>INT8 scenario</option><option value="int4">INT4 scenario</option></select></label>
63
- </div>
64
-
65
- <hr />
66
- <div class="section-kicker">Workload</div>
67
- <div class="field-grid two">
68
- <label>Arrival process<select id="arrival"><option value="poisson">Poisson</option><option value="constant">Constant</option><option value="bursty">Bursty</option></select></label>
69
- <label>Request rate<input id="rate" type="number" min="0.1" step="0.1" value="4" /><span class="unit">req/s</span></label>
70
- <label>Duration<input id="duration" type="number" min="2" step="1" value="30" /><span class="unit">simulated s</span></label>
71
- <label>Seed<input id="seed" type="number" step="1" value="7" /></label>
72
- </div>
73
- <div class="field-grid two">
74
- <label>Prompt mean<input id="promptMean" type="number" min="16" value="512" /><span class="unit">tokens</span></label>
75
- <label>Prompt CV<input id="promptCv" type="number" min="0" max="2" step="0.05" value="0.50" /></label>
76
- <label>Output mean<input id="outputMean" type="number" min="1" value="64" /><span class="unit">tokens</span></label>
77
- <label>Output CV<input id="outputCv" type="number" min="0" max="2" step="0.05" value="0.60" /></label>
78
- </div>
79
-
80
- <hr />
81
- <div class="section-kicker">Serving controls</div>
82
- <div class="field-grid two">
83
- <label>Max batch size<input id="maxBatch" type="number" min="1" max="128" value="16" /></label>
84
- <label>Max batch tokens<input id="maxBatchTokens" type="number" min="128" step="128" value="8192" /></label>
85
- <label>Prefill chunk<input id="chunkSize" type="number" min="64" step="64" value="512" /><span class="unit">tokens</span></label>
86
- <label>KV block<input id="kvBlock" type="number" min="1" value="16" /><span class="unit">tokens</span></label>
87
- </div>
88
-
89
- <hr />
90
- <div class="section-kicker">SLO</div>
91
- <div class="field-grid two">
92
- <label>TTFT limit<input id="sloTtft" type="number" min="1" value="500" /><span class="unit">ms</span></label>
93
- <label>E2E limit<input id="sloE2e" type="number" min="100" value="5000" /><span class="unit">ms</span></label>
94
- </div>
95
-
96
- <button id="runBtn" class="primary" disabled>Run simulation</button>
97
- <button id="exportBtn" class="secondary" disabled>Export last result</button>
98
- </aside>
99
-
100
- <div class="results-column">
101
- <section class="panel result-panel">
102
- <div class="panel-title-row"><h2>Run summary</h2><span id="runState" class="tag neutral">Waiting</span></div>
103
- <div id="emptyState" class="empty-state"><div class="empty-icon">↗</div><h3>Configure a workload and run it</h3><p>The Python simulator will execute in a Web Worker and return request-level virtual timestamps.</p></div>
104
- <div id="resultContent" class="hidden">
105
- <div class="metric-grid">
106
- <div class="metric"><span>p95 TTFT</span><strong id="mTtft">—</strong></div>
107
- <div class="metric"><span>p95 E2E</span><strong id="mE2e">—</strong></div>
108
- <div class="metric"><span>Goodput</span><strong id="mGoodput">—</strong></div>
109
- <div class="metric"><span>SLO attainment</span><strong id="mSlo">—</strong></div>
110
- <div class="metric"><span>Throughput</span><strong id="mReq">—</strong></div>
111
- <div class="metric"><span>Peak KV</span><strong id="mKv">—</strong></div>
112
- </div>
113
- <div class="chart-grid">
114
- <div class="chart-card"><div class="chart-title">Latency percentiles</div><canvas id="latencyChart"></canvas></div>
115
- <div class="chart-card"><div class="chart-title">Queue & decode timeline</div><canvas id="timelineChart"></canvas></div>
116
- </div>
117
- <div class="chart-card full"><div class="chart-title">Request TTFT vs prompt length</div><canvas id="scatterChart"></canvas></div>
118
- <div id="warnings" class="warnings hidden"></div>
119
- </div>
120
- </section>
121
- </div>
122
- </div>
123
- </section>
124
-
125
- <section id="arena" class="tab-panel">
126
- <div class="panel wide-panel">
127
- <div class="panel-title-row">
128
- <div><div class="section-kicker">Same workload · same seed</div><h2>Scheduler Arena</h2><p class="muted">Run every v0.1 scheduler against the current Serving Lab configuration and rank by SLO attainment then goodput.</p></div>
129
- <button id="arenaBtn" class="primary compact" disabled>Compare schedulers</button>
130
- </div>
131
- <div id="arenaEmpty" class="empty-state small"><h3>No comparison yet</h3><p>Your Serving Lab controls are reused automatically.</p></div>
132
- <div id="arenaContent" class="hidden">
133
- <div class="chart-card full"><canvas id="arenaChart"></canvas></div>
134
- <div class="table-wrap"><table><thead><tr><th>Scheduler</th><th>Goodput</th><th>SLO attainment</th><th>p95 TTFT</th><th>p95 E2E</th><th>KV peak</th><th>Unfinished</th></tr></thead><tbody id="arenaRows"></tbody></table></div>
135
- </div>
136
- </div>
137
- </section>
138
-
139
- <section id="planner" class="tab-panel">
140
- <div class="workspace planner-grid">
141
- <aside class="panel controls-panel">
142
- <div class="panel-title-row"><h2>Capacity search</h2><span class="tag">Binary search</span></div>
143
- <p class="muted">Find the highest offered load that reaches the target SLO attainment and drains all generated requests.</p>
144
- <label>Required SLO attainment<input id="targetSlo" type="number" min="0.5" max="1" step="0.001" value="0.99" /></label>
145
- <div class="field-grid two">
146
- <label>Minimum rate<input id="minRate" type="number" min="0.05" step="0.1" value="0.25" /><span class="unit">req/s</span></label>
147
- <label>Maximum rate<input id="maxRate" type="number" min="0.1" step="1" value="20" /><span class="unit">req/s</span></label>
148
- <label>Search iterations<input id="searchIter" type="number" min="2" max="12" value="7" /></label>
149
- <label>Repetitions / rate<input id="repetitions" type="number" min="1" max="5" value="2" /></label>
150
- </div>
151
- <label>Safety headroom<input id="headroom" type="number" min="0" max="0.8" step="0.05" value="0.20" /><span class="unit">fraction</span></label>
152
- <button id="capacityBtn" class="primary" disabled>Find sustainable capacity</button>
153
- </aside>
154
- <section class="panel result-panel">
155
- <div class="panel-title-row"><h2>Planner result</h2><span id="plannerState" class="tag neutral">Waiting</span></div>
156
- <div id="plannerEmpty" class="empty-state"><h3>No search yet</h3><p>The planner repeatedly runs the simulator at different offered loads.</p></div>
157
- <div id="plannerContent" class="hidden">
158
- <div class="metric-grid four">
159
- <div class="metric emphasis"><span>Estimated capacity</span><strong id="pCapacity">—</strong></div>
160
- <div class="metric"><span>Recommended load</span><strong id="pRecommended">—</strong></div>
161
- <div class="metric"><span>Safety headroom</span><strong id="pHeadroom">—</strong></div>
162
- <div class="metric"><span>Status</span><strong id="pStatus">—</strong></div>
163
- </div>
164
- <div class="chart-card full"><div class="chart-title">SLO attainment across searched rates</div><canvas id="capacityChart"></canvas></div>
165
- <div class="table-wrap"><table><thead><tr><th>Rate</th><th>Pass</th><th>SLO attainment</th><th>Goodput</th><th>p95 TTFT</th><th>p95 E2E</th></tr></thead><tbody id="capacityRows"></tbody></table></div>
166
- </div>
167
- </section>
168
- </div>
169
- </section>
170
-
171
- <section id="method" class="tab-panel">
172
- <div class="method-grid">
173
- <article class="panel prose">
174
- <div class="section-kicker">What is simulated?</div>
175
- <h2>A serving loop, not CUDA kernels.</h2>
176
- <p>InferScale advances virtual time through request arrivals, queueing, prefill, autoregressive decode, batch membership changes, KV allocation and completion. A predicted 40 ms decode step becomes <code>simulated_time += 0.040</code>; the browser never waits 40 ms.</p>
177
- <p>The latency backend is roofline-inspired and intentionally replaceable. It combines model size, attention shape, accelerator peak throughput and memory bandwidth with conservative efficiency factors. v0.1 uses it to expose systems interactions, not to claim measured L4/A100 latency.</p>
178
- </article>
179
- <article class="panel prose">
180
- <div class="section-kicker">Why goodput?</div>
181
- <h2>Throughput can reward overload.</h2>
182
- <p>InferScale defines goodput as the number of completed requests that satisfy both the TTFT and end-to-end latency SLOs divided by simulated makespan. A system can increase raw throughput while simultaneously becoming less useful to latency-sensitive applications.</p>
183
- <div class="formula">goodput = SLO-compliant completions / simulated time</div>
184
- </article>
185
- <article class="panel prose wide-method">
186
- <div class="section-kicker">Research lineage</div>
187
- <h2>From Vidur to modern disaggregated simulators.</h2>
188
- <div class="paper-grid">
189
- <div><strong>Vidur · 2024</strong><span>Predictive profiling + deployment search. Reported a LLaMA2-70B search in ~1 CPU-hour versus an estimated 42K GPU-hours (~$218K).</span></div>
190
- <div><strong>TokenSim · 2025</strong><span>Extensible hardware/software exploration with scheduler and memory-management modeling.</span></div>
191
- <div><strong>Revati · 2026</strong><span>GPU-free time-warp emulation that executes real serving control logic rather than reimplementing it.</span></div>
192
- <div><strong>LLMServingSim 2.0 · 2026</strong><span>Heterogeneous/disaggregated infrastructure with runtime-driven batching, routing, memory and power.</span></div>
193
- <div><strong>Frontier · 2026</strong><span>Modern disaggregation, runtime optimizations, stateful workloads and large-scale configuration exploration.</span></div>
194
- </div>
195
- </article>
196
- </div>
197
  </section>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
198
  </main>
199
-
200
- <footer>
201
- <span>InferScale-Sim v0.1.0</span>
202
- <span>Simulation executes locally in your browser. No experiment data is transmitted.</span>
203
- </footer>
204
-
205
  <script type="module" src="app.js"></script>
206
  </body>
207
  </html>
 
10
  <script src="https://cdn.jsdelivr.net/npm/chart.js@4.5.1/dist/chart.umd.min.js"></script>
11
  </head>
12
  <body>
13
+ <header class="topbar" id="topbar">
14
+ <div class="brand-wrap"><div class="logo">IS</div><div><div class="brand">InferScale-Sim</div><div class="subtitle">LLM serving systems laboratory</div></div></div>
15
+ <div class="runtime-pill" id="runtimePill"><span class="dot"></span><span id="runtimeText">Loading Python runtime...</span></div>
 
 
 
 
 
 
16
  </header>
17
 
18
  <main class="shell">
19
+ <section class="intro">
20
+ <div class="intro-copy"><div class="eyebrow">InferScale-Sim / v0.2.0</div><h1>Explore LLM serving dynamics without provisioning a GPU.</h1><p>Generate workloads, compare schedulers, inspect queueing and KV pressure, and search for the highest SLO-compliant request rate. The simulator is Python running locally in a Pyodide Web Worker.</p></div>
21
+ <dl class="project-facts"><div><dt>Backend</dt><dd>None</dd></div><div><dt>Runtime</dt><dd>Python / WASM</dd></div><div><dt>Network data</dt><dd>None after load</dd></div><div><dt>Profiles</dt><dd>Analytical reference</dd></div></dl>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
  </section>
23
+ <div class="reference-note"><strong>Reference-profile mode.</strong> Absolute L4, A10G, and A100 latency values are analytical predictions, not measured GPU benchmarks. Scheduler, queueing, KV-cache, and SLO behavior is simulated live.</div>
24
+
25
+ <nav class="tabs" aria-label="InferScale sections"><button class="tab active" data-tab="lab">Serving Lab</button><button class="tab" data-tab="arena">Scheduler Arena</button><button class="tab" data-tab="planner">Capacity Planner</button><button class="tab" data-tab="method">Methodology</button></nav>
26
+
27
+ <section id="lab" class="tab-panel active"><div class="workspace">
28
+ <aside class="panel controls-panel">
29
+ <div class="panel-title-row"><h2>Experiment</h2><span class="tag">Deterministic seed</span></div>
30
+ <div class="field-grid two"><label>Model<select id="model"><option>Qwen2.5-3B</option><option>Llama-3.1-8B</option><option>Mistral-7B-v0.3</option></select></label><label>Accelerator<select id="accelerator"><option value="L4">NVIDIA L4</option><option value="A10G">NVIDIA A10G</option><option value="A100-40GB">NVIDIA A100 40GB</option></select></label></div>
31
+ <div class="field-grid two"><label>Scheduler<select id="scheduler"><option value="continuous_fcfs">Continuous - FCFS</option><option value="continuous_sjf">Continuous - SJF</option><option value="continuous_slo">Continuous - SLO-aware</option><option value="chunked_slo">Chunked prefill - SLO-aware</option><option value="static_fcfs">Static batching - FCFS</option></select></label><label>Weight precision<select id="quantization"><option value="fp16">FP16</option><option value="int8" selected>INT8 scenario</option><option value="int4">INT4 scenario</option></select></label></div>
32
+ <hr /><div class="section-kicker">Workload</div>
33
+ <div class="field-grid two"><label>Arrival process<select id="arrival"><option value="poisson">Poisson</option><option value="constant">Constant</option><option value="bursty">Bursty</option></select></label><label>Request rate<input id="rate" type="number" min="0.1" step="0.1" value="4" /><span class="unit">req/s</span></label><label>Duration<input id="duration" type="number" min="2" step="1" value="30" /><span class="unit">sim s</span></label><label>Seed<input id="seed" type="number" step="1" value="7" /></label></div>
34
+ <div id="burstControls" class="field-grid two hidden"><label>Burst multiplier<input id="burstMultiplier" type="number" min="1" step="0.25" value="3" /><span class="unit">x</span></label><label>Burst period<input id="burstPeriod" type="number" min="0.5" step="0.5" value="10" /><span class="unit">s</span></label></div>
35
+ <div class="field-grid two"><label>Prompt mean<input id="promptMean" type="number" min="16" value="512" /><span class="unit">tokens</span></label><label>Prompt CV<input id="promptCv" type="number" min="0" max="2" step="0.05" value="0.50" /></label><label>Output mean<input id="outputMean" type="number" min="1" value="64" /><span class="unit">tokens</span></label><label>Output CV<input id="outputCv" type="number" min="0" max="2" step="0.05" value="0.60" /></label></div>
36
+ <hr /><div class="section-kicker">Serving controls</div>
37
+ <div class="field-grid two"><label>Max batch size<input id="maxBatch" type="number" min="1" max="128" value="16" /></label><label>Max batch tokens<input id="maxBatchTokens" type="number" min="128" step="128" value="8192" /></label><label>Prefill chunk<input id="chunkSize" type="number" min="64" step="64" value="512" /><span class="unit">tokens</span></label><label>KV block<input id="kvBlock" type="number" min="1" value="16" /><span class="unit">tokens</span></label></div>
38
+ <hr /><div class="section-kicker">SLO</div><div class="field-grid two"><label>TTFT limit<input id="sloTtft" type="number" min="1" value="500" /><span class="unit">ms</span></label><label>E2E limit<input id="sloE2e" type="number" min="100" value="5000" /><span class="unit">ms</span></label></div>
39
+ <button id="runBtn" class="primary" disabled>Run simulation</button><div class="button-row"><button id="copyResultBtn" class="secondary" disabled>Copy JSON</button><button id="exportBtn" class="secondary" disabled>Download JSON</button></div>
40
+ </aside>
41
+ <div class="results-column"><section class="panel result-panel"><div class="panel-title-row"><h2>Run summary</h2><span id="runState" class="tag neutral">Waiting</span></div><div id="emptyState" class="empty-state"><h3>Configure a workload and run it</h3><p>The Python simulator executes in a background Web Worker and returns request-level virtual timestamps.</p></div><div id="resultContent" class="hidden">
42
+ <div class="metric-grid"><div class="metric"><span>p95 TTFT</span><strong id="mTtft">N/A</strong></div><div class="metric"><span>p95 E2E</span><strong id="mE2e">N/A</strong></div><div class="metric"><span>Goodput</span><strong id="mGoodput">N/A</strong></div><div class="metric"><span>SLO attainment</span><strong id="mSlo">N/A</strong></div><div class="metric"><span>Throughput</span><strong id="mReq">N/A</strong></div><div class="metric"><span>Peak KV</span><strong id="mKv">N/A</strong></div></div>
43
+ <section class="diagnostic-card" id="diagnosticCard"><div class="diagnostic-head"><span>Simulator diagnosis</span><strong id="mBottleneck">N/A</strong></div><p id="mDiagnosis">Run a simulation to generate a bottleneck explanation.</p><p class="diagnostic-action" id="mRecommendation"></p><div class="evidence-row" id="mEvidence"></div></section>
44
+ <div class="chart-grid"><div class="chart-card" data-chart-card><div class="chart-head"><div class="chart-title">Latency percentiles</div><button class="chart-expand" type="button">Expand</button></div><div class="chart-body"><canvas id="latencyChart"></canvas></div></div><div class="chart-card" data-chart-card><div class="chart-head"><div class="chart-title">Queue, decode, and KV timeline</div><button class="chart-expand" type="button">Expand</button></div><div class="chart-body"><canvas id="timelineChart"></canvas></div></div></div>
45
+ <div class="chart-card full" data-chart-card><div class="chart-head"><div class="chart-title">Request TTFT vs prompt length</div><button class="chart-expand" type="button">Expand</button></div><div class="chart-body"><canvas id="scatterChart"></canvas></div></div><div id="warnings" class="warnings hidden"></div>
46
+ </div></section></div>
47
+ </div></section>
48
+
49
+ <section id="arena" class="tab-panel"><div class="panel wide-panel"><div class="panel-title-row arena-title-row"><div><div class="section-kicker">Same workload - same seed</div><h2>Scheduler Arena</h2><p class="muted">Run every scheduler against the current Serving Lab configuration and rank by SLO attainment, then goodput.</p></div><button id="arenaBtn" class="primary compact" disabled>Compare schedulers</button></div><div id="arenaEmpty" class="empty-state small"><h3>No comparison yet</h3><p>Your Serving Lab controls are reused automatically.</p></div><div id="arenaContent" class="hidden"><div class="chart-card full" data-chart-card><div class="chart-head"><div class="chart-title">Useful vs raw request throughput</div><button class="chart-expand" type="button">Expand</button></div><div class="chart-body large"><canvas id="arenaChart"></canvas></div></div><div class="table-toolbar"><span>Scheduler comparison</span><div><button id="arenaCopyBtn" class="mini-button" disabled>Copy table</button><button id="arenaCsvBtn" class="mini-button" disabled>Download CSV</button></div></div><div class="table-wrap"><table><thead><tr><th>Scheduler</th><th>Goodput</th><th>SLO attainment</th><th>p95 TTFT</th><th>p95 E2E</th><th>KV peak</th><th>Unfinished</th><th>Diagnosis</th></tr></thead><tbody id="arenaRows"></tbody></table></div></div></div></section>
50
+
51
+ <section id="planner" class="tab-panel"><div class="workspace planner-grid"><aside class="panel controls-panel"><div class="panel-title-row"><h2>Capacity search</h2><span class="tag">Robust binary search</span></div><p class="muted">Find the highest offered load where every repetition meets the SLO target and fully drains.</p><label>Required SLO attainment<input id="targetSlo" type="number" min="0.5" max="1" step="0.001" value="0.99" /></label><div class="field-grid two"><label>Minimum rate<input id="minRate" type="number" min="0.05" step="0.1" value="0.25" /><span class="unit">req/s</span></label><label>Maximum rate<input id="maxRate" type="number" min="0.1" step="1" value="20" /><span class="unit">req/s</span></label><label>Search iterations<input id="searchIter" type="number" min="2" max="12" value="7" /></label><label>Repetitions / rate<input id="repetitions" type="number" min="1" max="5" value="2" /></label></div><label>Safety headroom<input id="headroom" type="number" min="0" max="0.8" step="0.05" value="0.20" /><span class="unit">fraction</span></label><button id="capacityBtn" class="primary" disabled>Find sustainable capacity</button></aside>
52
+ <section class="panel result-panel"><div class="panel-title-row"><h2>Planner result</h2><span id="plannerState" class="tag neutral">Waiting</span></div><div id="plannerEmpty" class="empty-state"><h3>No search yet</h3><p>The planner repeatedly runs the simulator at different offered loads.</p></div><div id="plannerContent" class="hidden"><div class="metric-grid four"><div class="metric emphasis"><span>Estimated capacity</span><strong id="pCapacity">N/A</strong></div><div class="metric"><span>Recommended load</span><strong id="pRecommended">N/A</strong></div><div class="metric"><span>Safety headroom</span><strong id="pHeadroom">N/A</strong></div><div class="metric"><span>Status</span><strong id="pStatus">N/A</strong></div></div><div class="planner-note" id="plannerCriterion">A rate passes only if every repetition meets the target and drains all generated requests.</div><div class="chart-card full" data-chart-card><div class="chart-head"><div class="chart-title">SLO attainment across searched rates</div><button class="chart-expand" type="button">Expand</button></div><div class="chart-body large"><canvas id="capacityChart"></canvas></div></div><div class="table-toolbar"><span>Search trace</span><div><button id="capacityCopyBtn" class="mini-button" disabled>Copy table</button><button id="capacityCsvBtn" class="mini-button" disabled>Download CSV</button></div></div><div class="table-wrap"><table><thead><tr><th>Rate</th><th>Pass</th><th>Mean SLO</th><th>SLO range</th><th>Goodput</th><th>p95 TTFT</th><th>p95 E2E</th></tr></thead><tbody id="capacityRows"></tbody></table></div></div></section>
53
+ </div></section>
54
+
55
+ <section id="method" class="tab-panel"><div class="method-grid"><article class="panel prose"><div class="section-kicker">What is simulated?</div><h2>A serving loop, not CUDA kernels.</h2><p>InferScale advances virtual time through request arrivals, queueing, prefill, autoregressive decode, batch membership changes, KV allocation, and completion. A predicted 40 ms decode step becomes <code>simulated_time += 0.040</code>; the browser never waits 40 ms.</p><p>The latency backend is roofline-inspired and replaceable. v0.2 uses it to expose systems interactions, not to claim measured L4/A100 latency.</p></article><article class="panel prose"><div class="section-kicker">Why goodput?</div><h2>Throughput can reward overload.</h2><p>Goodput counts completed requests satisfying both TTFT and E2E SLOs per simulated second.</p><div class="formula">goodput = SLO-compliant completions / simulated time</div></article><article class="panel prose"><div class="section-kicker">v0.2 analysis</div><h2>Explain the limiting regime.</h2><p>Each run derives a simulator-side diagnosis from SLO failures, queue latency, busy fraction, unfinished work, and KV utilization. These labels explain simulated state, not hardware counters.</p><div class="formula">healthy | queue/prefill | decode/E2E | KV pressure | over-capacity</div></article><article class="panel prose"><div class="section-kicker">Capacity criterion</div><h2>Do not average away a bad repetition.</h2><p>A searched rate passes only when every configured repetition reaches the SLO target and fully drains. Mean, minimum, and maximum attainment remain visible in the trace.</p></article><article class="panel prose wide-method"><div class="section-kicker">Research lineage</div><h2>From Vidur to modern disaggregated simulators.</h2><div class="paper-grid"><div><strong>Vidur - 2024</strong><span>Predictive profiling and deployment search. Reported about 1 CPU-hour versus an estimated 42K GPU-hours (about $218K) for one LLaMA2-70B search.</span></div><div><strong>TokenSim - 2025</strong><span>Extensible hardware/software exploration with scheduler and memory-management modeling.</span></div><div><strong>Revati - 2026</strong><span>GPU-free time-warp emulation using serving control logic.</span></div><div><strong>LLMServingSim 2.0 - 2026</strong><span>Heterogeneous and disaggregated serving with runtime-driven batching, routing, memory, and power.</span></div><div><strong>Frontier - 2026</strong><span>Modern disaggregation, runtime optimizations, stateful workloads, and large-scale configuration exploration.</span></div></div></article></div></section>
56
  </main>
57
+ <footer><span>InferScale-Sim v0.2.0</span><span>Simulation executes locally in your browser. Experiment data is not transmitted.</span></footer>
58
+ <div id="toast" class="toast" role="status" aria-live="polite"></div>
 
 
 
 
59
  <script type="module" src="app.js"></script>
60
  </body>
61
  </html>
py/inferscale/__init__.py CHANGED
@@ -12,4 +12,4 @@ __all__ = [
12
  "run_simulation",
13
  ]
14
 
15
- __version__ = "0.1.0"
 
12
  "run_simulation",
13
  ]
14
 
15
+ __version__ = "0.2.0"
py/inferscale/api.py CHANGED
@@ -7,7 +7,7 @@ from .simulator import SCHEDULERS, run_simulation
7
 
8
  def metadata() -> dict:
9
  return {
10
- "version": "0.1.0",
11
  "models": list(MODELS.keys()),
12
  "accelerators": list(ACCELERATORS.keys()),
13
  "schedulers": sorted(SCHEDULERS),
 
7
 
8
  def metadata() -> dict:
9
  return {
10
+ "version": "0.2.0",
11
  "models": list(MODELS.keys()),
12
  "accelerators": list(ACCELERATORS.keys()),
13
  "schedulers": sorted(SCHEDULERS),
py/inferscale/diagnostics.py ADDED
@@ -0,0 +1,81 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ from typing import Any
4
+
5
+ from .models import SimulationConfig
6
+
7
+
8
+ def diagnose_run(summary: dict[str, Any], latency: dict[str, Any], resource: dict[str, Any], cfg: SimulationConfig) -> dict[str, Any]:
9
+ """Explain the simulator's limiting regime from its own state."""
10
+ target = cfg.slo_attainment_target
11
+ attainment = float(summary.get("slo_attainment", 0.0))
12
+ ttft_attainment = float(summary.get("ttft_slo_attainment", 0.0))
13
+ e2e_attainment = float(summary.get("e2e_slo_attainment", 0.0))
14
+ unfinished = int(summary.get("requests_unfinished", 0))
15
+ busy = float(summary.get("busy_fraction", 0.0))
16
+ kv_util = float(resource.get("peak_kv_utilization", 0.0))
17
+ queue_p95 = float(latency.get("queue_ms", {}).get("p95", 0.0))
18
+
19
+ code = "mixed_pressure"
20
+ label = "Mixed SLO pressure"
21
+ explanation = "Several constraints are active; inspect queueing, KV pressure, and tail latency together."
22
+ recommendation = "Compare schedulers and run the capacity planner to isolate the limiting regime."
23
+
24
+ if unfinished > 0 and kv_util >= 0.85:
25
+ code, label = "kv_pressure", "KV memory pressure"
26
+ explanation = "Requests remain unfinished while the simulated KV cache approaches its configured capacity."
27
+ recommendation = "Reduce sequence pressure, lower offered load, use a smaller KV block, or choose a larger-memory profile."
28
+ elif unfinished > 0 and busy >= 0.92:
29
+ code, label = "over_capacity", "Offered load above capacity"
30
+ explanation = "The virtual device remains busy while the workload fails to drain."
31
+ recommendation = "Reduce offered load or use the capacity planner to find a sustainable operating point."
32
+ elif unfinished > 0:
33
+ code, label = "admission_stall", "Admission stall"
34
+ explanation = "Some requests cannot be admitted or completed under the current serving constraints."
35
+ recommendation = "Inspect batch-token and KV-cache limits, then compare a less restrictive scheduler configuration."
36
+ elif attainment >= target:
37
+ if busy >= 0.90:
38
+ code, label = "healthy_near_saturation", "Healthy, little headroom"
39
+ explanation = "The workload meets the configured SLO target, but simulated device busy time is already high."
40
+ recommendation = "Run the capacity planner before increasing traffic; the current point is close to saturation."
41
+ else:
42
+ code, label = "healthy", "Healthy"
43
+ explanation = "The workload drains and meets the configured SLO-attainment target with visible headroom."
44
+ recommendation = "Use the capacity planner to quantify how much additional offered load the profile can absorb."
45
+ elif kv_util >= 0.90:
46
+ code, label = "kv_pressure", "KV memory pressure"
47
+ explanation = "Peak simulated KV utilization is high enough to constrain admission and batching flexibility."
48
+ recommendation = "Reduce sequence pressure, use smaller KV blocks, or select a larger-memory accelerator profile."
49
+ elif ttft_attainment < target and queue_p95 >= max(25.0, cfg.slo_ttft_ms * 0.20):
50
+ code, label = "queue_prefill_pressure", "Queue / prefill pressure"
51
+ explanation = "TTFT misses coincide with substantial queueing before prefill begins."
52
+ recommendation = "Lower offered load, increase safe batching capacity, or try chunked prefill / SLO-aware scheduling."
53
+ elif ttft_attainment < target:
54
+ code, label = "ttft_pressure", "TTFT pressure"
55
+ explanation = "First-token latency is the dominant SLO failure even though queueing is not the only contributor."
56
+ recommendation = "Compare prefill-oriented schedulers and reduce prompt or offered-load pressure."
57
+ elif e2e_attainment < target:
58
+ code, label = "decode_pressure", "Decode / E2E pressure"
59
+ explanation = "Most first tokens arrive within target, but end-to-end latency still misses the configured SLO."
60
+ recommendation = "Reduce output-length pressure or offered load; larger decode batches may help when memory allows."
61
+ elif busy >= 0.95:
62
+ code, label = "compute_saturation", "Compute saturation"
63
+ explanation = "The simulated device is effectively saturated even though no single SLO component dominates."
64
+ recommendation = "Reduce load or switch to a higher-throughput accelerator profile."
65
+
66
+ return {
67
+ "code": code,
68
+ "label": label,
69
+ "explanation": explanation,
70
+ "recommendation": recommendation,
71
+ "evidence": {
72
+ "slo_attainment": attainment,
73
+ "ttft_slo_attainment": ttft_attainment,
74
+ "e2e_slo_attainment": e2e_attainment,
75
+ "busy_fraction": busy,
76
+ "peak_kv_utilization": kv_util,
77
+ "queue_p95_ms": queue_p95,
78
+ "unfinished_requests": unfinished,
79
+ },
80
+ "provenance": "heuristic-simulator-diagnosis",
81
+ }
py/inferscale/latency.py CHANGED
@@ -29,7 +29,7 @@ class AnalyticalLatencyModel:
29
  return self.model.params_b * self.weight_bytes_per_param
30
 
31
  def kv_bytes_per_token(self) -> float:
32
- # K + V, all layers, KV heads only. KV state is assumed fp16 in v0.1.
33
  return (
34
  2
35
  * self.model.layers
 
29
  return self.model.params_b * self.weight_bytes_per_param
30
 
31
  def kv_bytes_per_token(self) -> float:
32
+ # K + V, all layers, KV heads only. KV state is assumed fp16 in v0.2.
33
  return (
34
  2
35
  * self.model.layers
py/inferscale/metrics.py CHANGED
@@ -57,14 +57,23 @@ def summarize(completed: list[Request], cfg: SimulationConfig, makespan_s: float
57
  queue = [m.queue_ms for m in metrics]
58
  total_output = sum(r.output_tokens for r in completed)
59
  met = sum(m.met_all_slos for m in metrics)
 
 
 
 
 
60
  duration = max(makespan_s, 1e-9)
 
61
 
62
  summary = {
63
  "requests_completed": len(completed),
64
  "request_throughput_rps": len(completed) / duration,
65
  "output_throughput_tps": total_output / duration,
66
  "goodput_rps": met / duration,
67
- "slo_attainment": met / len(metrics) if metrics else 0.0,
 
 
 
68
  "simulated_makespan_s": makespan_s,
69
  "busy_fraction": min(1.0, busy_time_s / duration),
70
  "mean_prompt_tokens": mean([r.prompt_tokens for r in completed]) if completed else 0.0,
 
57
  queue = [m.queue_ms for m in metrics]
58
  total_output = sum(r.output_tokens for r in completed)
59
  met = sum(m.met_all_slos for m in metrics)
60
+ met_ttft = sum(m.met_ttft_slo for m in metrics)
61
+ met_e2e = sum(m.met_e2e_slo for m in metrics)
62
+ ttft_only_fail = sum((not m.met_ttft_slo) and m.met_e2e_slo for m in metrics)
63
+ e2e_only_fail = sum(m.met_ttft_slo and (not m.met_e2e_slo) for m in metrics)
64
+ both_fail = sum((not m.met_ttft_slo) and (not m.met_e2e_slo) for m in metrics)
65
  duration = max(makespan_s, 1e-9)
66
+ count = len(metrics)
67
 
68
  summary = {
69
  "requests_completed": len(completed),
70
  "request_throughput_rps": len(completed) / duration,
71
  "output_throughput_tps": total_output / duration,
72
  "goodput_rps": met / duration,
73
+ "slo_attainment": met / count if count else 0.0,
74
+ "ttft_slo_attainment": met_ttft / count if count else 0.0,
75
+ "e2e_slo_attainment": met_e2e / count if count else 0.0,
76
+ "slo_failure_breakdown": {"ttft_only": ttft_only_fail, "e2e_only": e2e_only_fail, "both": both_fail},
77
  "simulated_makespan_s": makespan_s,
78
  "busy_fraction": min(1.0, busy_time_s / duration),
79
  "mean_prompt_tokens": mean([r.prompt_tokens for r in completed]) if completed else 0.0,
py/inferscale/models.py CHANGED
@@ -122,6 +122,7 @@ class SimulationResult:
122
  summary: dict[str, Any]
123
  latency: dict[str, Any]
124
  resource: dict[str, Any]
 
125
  requests: list[dict[str, Any]] = field(default_factory=list)
126
  timeline: list[dict[str, Any]] = field(default_factory=list)
127
  warnings: list[str] = field(default_factory=list)
 
122
  summary: dict[str, Any]
123
  latency: dict[str, Any]
124
  resource: dict[str, Any]
125
+ diagnostics: dict[str, Any] = field(default_factory=dict)
126
  requests: list[dict[str, Any]] = field(default_factory=list)
127
  timeline: list[dict[str, Any]] = field(default_factory=list)
128
  warnings: list[str] = field(default_factory=list)
py/inferscale/optimizer.py CHANGED
@@ -1,7 +1,7 @@
1
  from __future__ import annotations
2
 
3
  from copy import deepcopy
4
- from statistics import mean
5
 
6
  from .models import SimulationConfig
7
  from .simulator import run_simulation
@@ -14,24 +14,43 @@ def evaluate_rate(base: SimulationConfig, rate: float, repetitions: int = 3) ->
14
  cfg.request_rate_rps = rate
15
  cfg.seed = base.seed + rep * 101
16
  runs.append(run_simulation(cfg.to_dict()))
17
- attainment = mean(r["summary"]["slo_attainment"] for r in runs)
18
- goodput = mean(r["summary"]["goodput_rps"] for r in runs)
19
- p95_ttft = mean(r["latency"]["ttft_ms"]["p95"] for r in runs)
20
- p95_e2e = mean(r["latency"]["e2e_ms"]["p95"] for r in runs)
21
- unfinished = mean(r["summary"]["requests_unfinished"] for r in runs)
22
- passed = attainment >= base.slo_attainment_target and unfinished <= 0.0
 
 
 
 
 
23
  return {
24
  "rate_rps": rate,
25
  "passed": passed,
26
- "slo_attainment": attainment,
27
- "goodput_rps": goodput,
28
- "p95_ttft_ms": p95_ttft,
29
- "p95_e2e_ms": p95_e2e,
30
- "mean_unfinished": unfinished,
 
 
 
 
 
 
 
31
  }
32
 
33
 
34
- def capacity_search(config: dict, min_rate: float = 0.25, max_rate: float = 32.0, iterations: int = 8, repetitions: int = 2, headroom: float = 0.20) -> dict:
 
 
 
 
 
 
 
35
  base = SimulationConfig.from_dict(config)
36
  low = max(0.01, min_rate)
37
  high = max(low * 1.01, max_rate)
@@ -44,18 +63,21 @@ def capacity_search(config: dict, min_rate: float = 0.25, max_rate: float = 32.0
44
  "status": "no_feasible_rate",
45
  "capacity_rps": 0.0,
46
  "recommended_rps": 0.0,
 
 
47
  "trace": trace,
48
  }
49
 
50
  high_eval = evaluate_rate(base, high, repetitions)
51
  trace.append(high_eval)
52
  if high_eval["passed"]:
53
- capacity = high
54
  return {
55
  "status": "upper_bound_still_feasible",
56
- "capacity_rps": capacity,
57
- "recommended_rps": capacity * (1.0 - headroom),
58
- "trace": sorted(trace, key=lambda x: x["rate_rps"]),
 
 
59
  }
60
 
61
  best = low
@@ -74,7 +96,8 @@ def capacity_search(config: dict, min_rate: float = 0.25, max_rate: float = 32.0
74
  "capacity_rps": best,
75
  "recommended_rps": best * (1.0 - headroom),
76
  "headroom": headroom,
77
- "trace": sorted(trace, key=lambda x: x["rate_rps"]),
 
78
  }
79
 
80
 
@@ -92,15 +115,18 @@ def compare_schedulers(config: dict, schedulers: list[str] | None = None) -> dic
92
  cfg = deepcopy(base)
93
  cfg.scheduler = scheduler
94
  result = run_simulation(cfg.to_dict())
95
- rows.append({
96
- "scheduler": scheduler,
97
- "request_throughput_rps": result["summary"]["request_throughput_rps"],
98
- "goodput_rps": result["summary"]["goodput_rps"],
99
- "slo_attainment": result["summary"]["slo_attainment"],
100
- "p95_ttft_ms": result["latency"]["ttft_ms"]["p95"],
101
- "p95_e2e_ms": result["latency"]["e2e_ms"]["p95"],
102
- "peak_kv_utilization": result["resource"]["peak_kv_utilization"],
103
- "unfinished": result["summary"]["requests_unfinished"],
104
- })
105
- rows.sort(key=lambda r: (r["slo_attainment"], r["goodput_rps"]), reverse=True)
 
 
 
106
  return {"rows": rows}
 
1
  from __future__ import annotations
2
 
3
  from copy import deepcopy
4
+ from statistics import mean, pstdev
5
 
6
  from .models import SimulationConfig
7
  from .simulator import run_simulation
 
14
  cfg.request_rate_rps = rate
15
  cfg.seed = base.seed + rep * 101
16
  runs.append(run_simulation(cfg.to_dict()))
17
+
18
+ attainments = [run["summary"]["slo_attainment"] for run in runs]
19
+ goodputs = [run["summary"]["goodput_rps"] for run in runs]
20
+ ttfts = [run["latency"]["ttft_ms"]["p95"] for run in runs]
21
+ e2es = [run["latency"]["e2e_ms"]["p95"] for run in runs]
22
+ unfinished_values = [run["summary"]["requests_unfinished"] for run in runs]
23
+
24
+ passed = all(
25
+ attainment >= base.slo_attainment_target and unfinished == 0
26
+ for attainment, unfinished in zip(attainments, unfinished_values, strict=True)
27
+ )
28
  return {
29
  "rate_rps": rate,
30
  "passed": passed,
31
+ "slo_attainment": mean(attainments),
32
+ "slo_attainment_min": min(attainments),
33
+ "slo_attainment_max": max(attainments),
34
+ "slo_attainment_std": pstdev(attainments) if len(attainments) > 1 else 0.0,
35
+ "goodput_rps": mean(goodputs),
36
+ "goodput_std": pstdev(goodputs) if len(goodputs) > 1 else 0.0,
37
+ "p95_ttft_ms": mean(ttfts),
38
+ "p95_e2e_ms": mean(e2es),
39
+ "mean_unfinished": mean(unfinished_values),
40
+ "max_unfinished": max(unfinished_values),
41
+ "repetitions": repetitions,
42
+ "criterion": "all-repetitions-meet-target-and-drain",
43
  }
44
 
45
 
46
+ def capacity_search(
47
+ config: dict,
48
+ min_rate: float = 0.25,
49
+ max_rate: float = 32.0,
50
+ iterations: int = 8,
51
+ repetitions: int = 2,
52
+ headroom: float = 0.20,
53
+ ) -> dict:
54
  base = SimulationConfig.from_dict(config)
55
  low = max(0.01, min_rate)
56
  high = max(low * 1.01, max_rate)
 
63
  "status": "no_feasible_rate",
64
  "capacity_rps": 0.0,
65
  "recommended_rps": 0.0,
66
+ "headroom": headroom,
67
+ "criterion": "all-repetitions-meet-target-and-drain",
68
  "trace": trace,
69
  }
70
 
71
  high_eval = evaluate_rate(base, high, repetitions)
72
  trace.append(high_eval)
73
  if high_eval["passed"]:
 
74
  return {
75
  "status": "upper_bound_still_feasible",
76
+ "capacity_rps": high,
77
+ "recommended_rps": high * (1.0 - headroom),
78
+ "headroom": headroom,
79
+ "criterion": "all-repetitions-meet-target-and-drain",
80
+ "trace": sorted(trace, key=lambda item: item["rate_rps"]),
81
  }
82
 
83
  best = low
 
96
  "capacity_rps": best,
97
  "recommended_rps": best * (1.0 - headroom),
98
  "headroom": headroom,
99
+ "criterion": "all-repetitions-meet-target-and-drain",
100
+ "trace": sorted(trace, key=lambda item: item["rate_rps"]),
101
  }
102
 
103
 
 
115
  cfg = deepcopy(base)
116
  cfg.scheduler = scheduler
117
  result = run_simulation(cfg.to_dict())
118
+ rows.append(
119
+ {
120
+ "scheduler": scheduler,
121
+ "request_throughput_rps": result["summary"]["request_throughput_rps"],
122
+ "goodput_rps": result["summary"]["goodput_rps"],
123
+ "slo_attainment": result["summary"]["slo_attainment"],
124
+ "p95_ttft_ms": result["latency"]["ttft_ms"]["p95"],
125
+ "p95_e2e_ms": result["latency"]["e2e_ms"]["p95"],
126
+ "peak_kv_utilization": result["resource"]["peak_kv_utilization"],
127
+ "unfinished": result["summary"]["requests_unfinished"],
128
+ "bottleneck": result["diagnostics"]["label"],
129
+ }
130
+ )
131
+ rows.sort(key=lambda row: (row["slo_attainment"], row["goodput_rps"]), reverse=True)
132
  return {"rows": rows}
py/inferscale/simulator.py CHANGED
@@ -2,6 +2,7 @@ from __future__ import annotations
2
 
3
  from dataclasses import asdict
4
 
 
5
  from .kv_cache import KVCacheModel
6
  from .latency import AnalyticalLatencyModel
7
  from .metrics import summarize
@@ -296,9 +297,11 @@ class Simulator:
296
  "e2e_ms": (req.completion_time - req.arrival_time) * 1000.0 if req.completion_time is not None else None,
297
  })
298
 
 
 
299
  provenance = {
300
  "simulator": "InferScale-Sim",
301
- "version": "0.1.0",
302
  "latency_profile_type": "analytical-reference",
303
  "profile_warning": "Reference profiles are analytical proxies, not measured hardware benchmarks.",
304
  "model_profile_source": self.model.source,
@@ -311,6 +314,7 @@ class Simulator:
311
  summary=summary,
312
  latency=latency,
313
  resource=resource,
 
314
  requests=request_rows,
315
  timeline=[asdict(p) for p in self.timeline],
316
  warnings=self.warnings,
 
2
 
3
  from dataclasses import asdict
4
 
5
+ from .diagnostics import diagnose_run
6
  from .kv_cache import KVCacheModel
7
  from .latency import AnalyticalLatencyModel
8
  from .metrics import summarize
 
297
  "e2e_ms": (req.completion_time - req.arrival_time) * 1000.0 if req.completion_time is not None else None,
298
  })
299
 
300
+ diagnostics = diagnose_run(summary, latency, resource, self.cfg)
301
+
302
  provenance = {
303
  "simulator": "InferScale-Sim",
304
+ "version": "0.2.0",
305
  "latency_profile_type": "analytical-reference",
306
  "profile_warning": "Reference profiles are analytical proxies, not measured hardware benchmarks.",
307
  "model_profile_source": self.model.source,
 
314
  summary=summary,
315
  latency=latency,
316
  resource=resource,
317
+ diagnostics=diagnostics,
318
  requests=request_rows,
319
  timeline=[asdict(p) for p in self.timeline],
320
  warnings=self.warnings,
pyproject.toml CHANGED
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
 
5
  [project]
6
  name = "inferscale-sim"
7
- version = "0.1.0"
8
  description = "Interactive LLM serving simulator and SLO capacity planner"
9
  readme = "README.md"
10
  requires-python = ">=3.10"
 
4
 
5
  [project]
6
  name = "inferscale-sim"
7
+ version = "0.2.0"
8
  description = "Interactive LLM serving simulator and SLO capacity planner"
9
  readme = "README.md"
10
  requires-python = ">=3.10"
scripts/release_check.py CHANGED
@@ -9,7 +9,7 @@ README = (ROOT / "README.md").read_text()
9
  SRC = ROOT / "src"
10
  sys.path.insert(0, str(SRC))
11
 
12
- from inferscale import run_simulation # noqa: E402
13
 
14
  errors: list[str] = []
15
 
@@ -24,20 +24,29 @@ else:
24
 
25
  if "sdk: static" not in README:
26
  errors.append("README metadata must use sdk: static")
27
- if not (ROOT / "index.html").exists():
28
- errors.append("index.html missing")
29
- if not (ROOT / "worker.mjs").exists():
30
- errors.append("worker.mjs missing")
31
 
32
  src_files = sorted((ROOT / "src" / "inferscale").glob("*.py"))
33
  web_files = sorted((ROOT / "py" / "inferscale").glob("*.py"))
34
- if [p.name for p in src_files] != [p.name for p in web_files]:
35
  errors.append("browser Python mirror is stale; run python scripts/sync_web_python.py")
36
  else:
37
- for src, web in zip(src_files, web_files):
38
  if src.read_bytes() != web.read_bytes():
39
  errors.append(f"browser mirror differs for {src.name}; run sync_web_python.py")
40
 
 
 
 
 
 
 
 
 
 
 
 
41
  smoke_cfg = {
42
  "model": "Qwen2.5-3B",
43
  "accelerator": "L4",
@@ -53,6 +62,10 @@ try:
53
  errors.append("simulation smoke test completed zero requests")
54
  if smoke["provenance"]["latency_profile_type"] != "analytical-reference":
55
  errors.append("profile provenance guard is missing")
 
 
 
 
56
  except Exception as exc: # pragma: no cover - release diagnostic
57
  errors.append(f"simulation smoke test raised: {exc}")
58
 
@@ -63,7 +76,9 @@ if errors:
63
  raise SystemExit(1)
64
 
65
  print("InferScale release check: PASS")
 
66
  print(f"HF short_description: {len(short)}/60 characters")
67
  print(f"Python modules mirrored: {len(src_files)}")
68
  print(f"Smoke requests completed: {smoke['summary']['requests_completed']}")
69
  print(f"Profile provenance: {smoke['provenance']['latency_profile_type']}")
 
 
9
  SRC = ROOT / "src"
10
  sys.path.insert(0, str(SRC))
11
 
12
+ from inferscale import __version__, run_simulation # noqa: E402
13
 
14
  errors: list[str] = []
15
 
 
24
 
25
  if "sdk: static" not in README:
26
  errors.append("README metadata must use sdk: static")
27
+ if __version__ != "0.2.0":
28
+ errors.append(f"package version is {__version__}; expected 0.2.0")
 
 
29
 
30
  src_files = sorted((ROOT / "src" / "inferscale").glob("*.py"))
31
  web_files = sorted((ROOT / "py" / "inferscale").glob("*.py"))
32
+ if [path.name for path in src_files] != [path.name for path in web_files]:
33
  errors.append("browser Python mirror is stale; run python scripts/sync_web_python.py")
34
  else:
35
+ for src, web in zip(src_files, web_files, strict=True):
36
  if src.read_bytes() != web.read_bytes():
37
  errors.append(f"browser mirror differs for {src.name}; run sync_web_python.py")
38
 
39
+ worker_text = (ROOT / "worker.mjs").read_text()
40
+ for src in src_files:
41
+ if f'"{src.name}"' not in worker_text:
42
+ errors.append(f"worker module list is missing {src.name}")
43
+
44
+ for ui_file in (ROOT / "index.html", ROOT / "app.js"):
45
+ try:
46
+ ui_file.read_text().encode("ascii")
47
+ except UnicodeEncodeError:
48
+ errors.append(f"{ui_file.name} contains non-ASCII UI glyphs; use text labels for reliable rendering")
49
+
50
  smoke_cfg = {
51
  "model": "Qwen2.5-3B",
52
  "accelerator": "L4",
 
62
  errors.append("simulation smoke test completed zero requests")
63
  if smoke["provenance"]["latency_profile_type"] != "analytical-reference":
64
  errors.append("profile provenance guard is missing")
65
+ if smoke["diagnostics"].get("provenance") != "heuristic-simulator-diagnosis":
66
+ errors.append("v0.2 diagnosis provenance missing")
67
+ if "ttft_slo_attainment" not in smoke["summary"] or "e2e_slo_attainment" not in smoke["summary"]:
68
+ errors.append("component SLO metrics missing")
69
  except Exception as exc: # pragma: no cover - release diagnostic
70
  errors.append(f"simulation smoke test raised: {exc}")
71
 
 
76
  raise SystemExit(1)
77
 
78
  print("InferScale release check: PASS")
79
+ print(f"Version: {__version__}")
80
  print(f"HF short_description: {len(short)}/60 characters")
81
  print(f"Python modules mirrored: {len(src_files)}")
82
  print(f"Smoke requests completed: {smoke['summary']['requests_completed']}")
83
  print(f"Profile provenance: {smoke['provenance']['latency_profile_type']}")
84
+ print(f"Diagnosis: {smoke['diagnostics']['label']}")
src/inferscale/__init__.py CHANGED
@@ -12,4 +12,4 @@ __all__ = [
12
  "run_simulation",
13
  ]
14
 
15
- __version__ = "0.1.0"
 
12
  "run_simulation",
13
  ]
14
 
15
+ __version__ = "0.2.0"
src/inferscale/api.py CHANGED
@@ -7,7 +7,7 @@ from .simulator import SCHEDULERS, run_simulation
7
 
8
  def metadata() -> dict:
9
  return {
10
- "version": "0.1.0",
11
  "models": list(MODELS.keys()),
12
  "accelerators": list(ACCELERATORS.keys()),
13
  "schedulers": sorted(SCHEDULERS),
 
7
 
8
  def metadata() -> dict:
9
  return {
10
+ "version": "0.2.0",
11
  "models": list(MODELS.keys()),
12
  "accelerators": list(ACCELERATORS.keys()),
13
  "schedulers": sorted(SCHEDULERS),
src/inferscale/diagnostics.py ADDED
@@ -0,0 +1,81 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ from typing import Any
4
+
5
+ from .models import SimulationConfig
6
+
7
+
8
+ def diagnose_run(summary: dict[str, Any], latency: dict[str, Any], resource: dict[str, Any], cfg: SimulationConfig) -> dict[str, Any]:
9
+ """Explain the simulator's limiting regime from its own state."""
10
+ target = cfg.slo_attainment_target
11
+ attainment = float(summary.get("slo_attainment", 0.0))
12
+ ttft_attainment = float(summary.get("ttft_slo_attainment", 0.0))
13
+ e2e_attainment = float(summary.get("e2e_slo_attainment", 0.0))
14
+ unfinished = int(summary.get("requests_unfinished", 0))
15
+ busy = float(summary.get("busy_fraction", 0.0))
16
+ kv_util = float(resource.get("peak_kv_utilization", 0.0))
17
+ queue_p95 = float(latency.get("queue_ms", {}).get("p95", 0.0))
18
+
19
+ code = "mixed_pressure"
20
+ label = "Mixed SLO pressure"
21
+ explanation = "Several constraints are active; inspect queueing, KV pressure, and tail latency together."
22
+ recommendation = "Compare schedulers and run the capacity planner to isolate the limiting regime."
23
+
24
+ if unfinished > 0 and kv_util >= 0.85:
25
+ code, label = "kv_pressure", "KV memory pressure"
26
+ explanation = "Requests remain unfinished while the simulated KV cache approaches its configured capacity."
27
+ recommendation = "Reduce sequence pressure, lower offered load, use a smaller KV block, or choose a larger-memory profile."
28
+ elif unfinished > 0 and busy >= 0.92:
29
+ code, label = "over_capacity", "Offered load above capacity"
30
+ explanation = "The virtual device remains busy while the workload fails to drain."
31
+ recommendation = "Reduce offered load or use the capacity planner to find a sustainable operating point."
32
+ elif unfinished > 0:
33
+ code, label = "admission_stall", "Admission stall"
34
+ explanation = "Some requests cannot be admitted or completed under the current serving constraints."
35
+ recommendation = "Inspect batch-token and KV-cache limits, then compare a less restrictive scheduler configuration."
36
+ elif attainment >= target:
37
+ if busy >= 0.90:
38
+ code, label = "healthy_near_saturation", "Healthy, little headroom"
39
+ explanation = "The workload meets the configured SLO target, but simulated device busy time is already high."
40
+ recommendation = "Run the capacity planner before increasing traffic; the current point is close to saturation."
41
+ else:
42
+ code, label = "healthy", "Healthy"
43
+ explanation = "The workload drains and meets the configured SLO-attainment target with visible headroom."
44
+ recommendation = "Use the capacity planner to quantify how much additional offered load the profile can absorb."
45
+ elif kv_util >= 0.90:
46
+ code, label = "kv_pressure", "KV memory pressure"
47
+ explanation = "Peak simulated KV utilization is high enough to constrain admission and batching flexibility."
48
+ recommendation = "Reduce sequence pressure, use smaller KV blocks, or select a larger-memory accelerator profile."
49
+ elif ttft_attainment < target and queue_p95 >= max(25.0, cfg.slo_ttft_ms * 0.20):
50
+ code, label = "queue_prefill_pressure", "Queue / prefill pressure"
51
+ explanation = "TTFT misses coincide with substantial queueing before prefill begins."
52
+ recommendation = "Lower offered load, increase safe batching capacity, or try chunked prefill / SLO-aware scheduling."
53
+ elif ttft_attainment < target:
54
+ code, label = "ttft_pressure", "TTFT pressure"
55
+ explanation = "First-token latency is the dominant SLO failure even though queueing is not the only contributor."
56
+ recommendation = "Compare prefill-oriented schedulers and reduce prompt or offered-load pressure."
57
+ elif e2e_attainment < target:
58
+ code, label = "decode_pressure", "Decode / E2E pressure"
59
+ explanation = "Most first tokens arrive within target, but end-to-end latency still misses the configured SLO."
60
+ recommendation = "Reduce output-length pressure or offered load; larger decode batches may help when memory allows."
61
+ elif busy >= 0.95:
62
+ code, label = "compute_saturation", "Compute saturation"
63
+ explanation = "The simulated device is effectively saturated even though no single SLO component dominates."
64
+ recommendation = "Reduce load or switch to a higher-throughput accelerator profile."
65
+
66
+ return {
67
+ "code": code,
68
+ "label": label,
69
+ "explanation": explanation,
70
+ "recommendation": recommendation,
71
+ "evidence": {
72
+ "slo_attainment": attainment,
73
+ "ttft_slo_attainment": ttft_attainment,
74
+ "e2e_slo_attainment": e2e_attainment,
75
+ "busy_fraction": busy,
76
+ "peak_kv_utilization": kv_util,
77
+ "queue_p95_ms": queue_p95,
78
+ "unfinished_requests": unfinished,
79
+ },
80
+ "provenance": "heuristic-simulator-diagnosis",
81
+ }
src/inferscale/latency.py CHANGED
@@ -29,7 +29,7 @@ class AnalyticalLatencyModel:
29
  return self.model.params_b * self.weight_bytes_per_param
30
 
31
  def kv_bytes_per_token(self) -> float:
32
- # K + V, all layers, KV heads only. KV state is assumed fp16 in v0.1.
33
  return (
34
  2
35
  * self.model.layers
 
29
  return self.model.params_b * self.weight_bytes_per_param
30
 
31
  def kv_bytes_per_token(self) -> float:
32
+ # K + V, all layers, KV heads only. KV state is assumed fp16 in v0.2.
33
  return (
34
  2
35
  * self.model.layers
src/inferscale/metrics.py CHANGED
@@ -57,14 +57,23 @@ def summarize(completed: list[Request], cfg: SimulationConfig, makespan_s: float
57
  queue = [m.queue_ms for m in metrics]
58
  total_output = sum(r.output_tokens for r in completed)
59
  met = sum(m.met_all_slos for m in metrics)
 
 
 
 
 
60
  duration = max(makespan_s, 1e-9)
 
61
 
62
  summary = {
63
  "requests_completed": len(completed),
64
  "request_throughput_rps": len(completed) / duration,
65
  "output_throughput_tps": total_output / duration,
66
  "goodput_rps": met / duration,
67
- "slo_attainment": met / len(metrics) if metrics else 0.0,
 
 
 
68
  "simulated_makespan_s": makespan_s,
69
  "busy_fraction": min(1.0, busy_time_s / duration),
70
  "mean_prompt_tokens": mean([r.prompt_tokens for r in completed]) if completed else 0.0,
 
57
  queue = [m.queue_ms for m in metrics]
58
  total_output = sum(r.output_tokens for r in completed)
59
  met = sum(m.met_all_slos for m in metrics)
60
+ met_ttft = sum(m.met_ttft_slo for m in metrics)
61
+ met_e2e = sum(m.met_e2e_slo for m in metrics)
62
+ ttft_only_fail = sum((not m.met_ttft_slo) and m.met_e2e_slo for m in metrics)
63
+ e2e_only_fail = sum(m.met_ttft_slo and (not m.met_e2e_slo) for m in metrics)
64
+ both_fail = sum((not m.met_ttft_slo) and (not m.met_e2e_slo) for m in metrics)
65
  duration = max(makespan_s, 1e-9)
66
+ count = len(metrics)
67
 
68
  summary = {
69
  "requests_completed": len(completed),
70
  "request_throughput_rps": len(completed) / duration,
71
  "output_throughput_tps": total_output / duration,
72
  "goodput_rps": met / duration,
73
+ "slo_attainment": met / count if count else 0.0,
74
+ "ttft_slo_attainment": met_ttft / count if count else 0.0,
75
+ "e2e_slo_attainment": met_e2e / count if count else 0.0,
76
+ "slo_failure_breakdown": {"ttft_only": ttft_only_fail, "e2e_only": e2e_only_fail, "both": both_fail},
77
  "simulated_makespan_s": makespan_s,
78
  "busy_fraction": min(1.0, busy_time_s / duration),
79
  "mean_prompt_tokens": mean([r.prompt_tokens for r in completed]) if completed else 0.0,
src/inferscale/models.py CHANGED
@@ -122,6 +122,7 @@ class SimulationResult:
122
  summary: dict[str, Any]
123
  latency: dict[str, Any]
124
  resource: dict[str, Any]
 
125
  requests: list[dict[str, Any]] = field(default_factory=list)
126
  timeline: list[dict[str, Any]] = field(default_factory=list)
127
  warnings: list[str] = field(default_factory=list)
 
122
  summary: dict[str, Any]
123
  latency: dict[str, Any]
124
  resource: dict[str, Any]
125
+ diagnostics: dict[str, Any] = field(default_factory=dict)
126
  requests: list[dict[str, Any]] = field(default_factory=list)
127
  timeline: list[dict[str, Any]] = field(default_factory=list)
128
  warnings: list[str] = field(default_factory=list)
src/inferscale/optimizer.py CHANGED
@@ -1,7 +1,7 @@
1
  from __future__ import annotations
2
 
3
  from copy import deepcopy
4
- from statistics import mean
5
 
6
  from .models import SimulationConfig
7
  from .simulator import run_simulation
@@ -14,24 +14,43 @@ def evaluate_rate(base: SimulationConfig, rate: float, repetitions: int = 3) ->
14
  cfg.request_rate_rps = rate
15
  cfg.seed = base.seed + rep * 101
16
  runs.append(run_simulation(cfg.to_dict()))
17
- attainment = mean(r["summary"]["slo_attainment"] for r in runs)
18
- goodput = mean(r["summary"]["goodput_rps"] for r in runs)
19
- p95_ttft = mean(r["latency"]["ttft_ms"]["p95"] for r in runs)
20
- p95_e2e = mean(r["latency"]["e2e_ms"]["p95"] for r in runs)
21
- unfinished = mean(r["summary"]["requests_unfinished"] for r in runs)
22
- passed = attainment >= base.slo_attainment_target and unfinished <= 0.0
 
 
 
 
 
23
  return {
24
  "rate_rps": rate,
25
  "passed": passed,
26
- "slo_attainment": attainment,
27
- "goodput_rps": goodput,
28
- "p95_ttft_ms": p95_ttft,
29
- "p95_e2e_ms": p95_e2e,
30
- "mean_unfinished": unfinished,
 
 
 
 
 
 
 
31
  }
32
 
33
 
34
- def capacity_search(config: dict, min_rate: float = 0.25, max_rate: float = 32.0, iterations: int = 8, repetitions: int = 2, headroom: float = 0.20) -> dict:
 
 
 
 
 
 
 
35
  base = SimulationConfig.from_dict(config)
36
  low = max(0.01, min_rate)
37
  high = max(low * 1.01, max_rate)
@@ -44,18 +63,21 @@ def capacity_search(config: dict, min_rate: float = 0.25, max_rate: float = 32.0
44
  "status": "no_feasible_rate",
45
  "capacity_rps": 0.0,
46
  "recommended_rps": 0.0,
 
 
47
  "trace": trace,
48
  }
49
 
50
  high_eval = evaluate_rate(base, high, repetitions)
51
  trace.append(high_eval)
52
  if high_eval["passed"]:
53
- capacity = high
54
  return {
55
  "status": "upper_bound_still_feasible",
56
- "capacity_rps": capacity,
57
- "recommended_rps": capacity * (1.0 - headroom),
58
- "trace": sorted(trace, key=lambda x: x["rate_rps"]),
 
 
59
  }
60
 
61
  best = low
@@ -74,7 +96,8 @@ def capacity_search(config: dict, min_rate: float = 0.25, max_rate: float = 32.0
74
  "capacity_rps": best,
75
  "recommended_rps": best * (1.0 - headroom),
76
  "headroom": headroom,
77
- "trace": sorted(trace, key=lambda x: x["rate_rps"]),
 
78
  }
79
 
80
 
@@ -92,15 +115,18 @@ def compare_schedulers(config: dict, schedulers: list[str] | None = None) -> dic
92
  cfg = deepcopy(base)
93
  cfg.scheduler = scheduler
94
  result = run_simulation(cfg.to_dict())
95
- rows.append({
96
- "scheduler": scheduler,
97
- "request_throughput_rps": result["summary"]["request_throughput_rps"],
98
- "goodput_rps": result["summary"]["goodput_rps"],
99
- "slo_attainment": result["summary"]["slo_attainment"],
100
- "p95_ttft_ms": result["latency"]["ttft_ms"]["p95"],
101
- "p95_e2e_ms": result["latency"]["e2e_ms"]["p95"],
102
- "peak_kv_utilization": result["resource"]["peak_kv_utilization"],
103
- "unfinished": result["summary"]["requests_unfinished"],
104
- })
105
- rows.sort(key=lambda r: (r["slo_attainment"], r["goodput_rps"]), reverse=True)
 
 
 
106
  return {"rows": rows}
 
1
  from __future__ import annotations
2
 
3
  from copy import deepcopy
4
+ from statistics import mean, pstdev
5
 
6
  from .models import SimulationConfig
7
  from .simulator import run_simulation
 
14
  cfg.request_rate_rps = rate
15
  cfg.seed = base.seed + rep * 101
16
  runs.append(run_simulation(cfg.to_dict()))
17
+
18
+ attainments = [run["summary"]["slo_attainment"] for run in runs]
19
+ goodputs = [run["summary"]["goodput_rps"] for run in runs]
20
+ ttfts = [run["latency"]["ttft_ms"]["p95"] for run in runs]
21
+ e2es = [run["latency"]["e2e_ms"]["p95"] for run in runs]
22
+ unfinished_values = [run["summary"]["requests_unfinished"] for run in runs]
23
+
24
+ passed = all(
25
+ attainment >= base.slo_attainment_target and unfinished == 0
26
+ for attainment, unfinished in zip(attainments, unfinished_values, strict=True)
27
+ )
28
  return {
29
  "rate_rps": rate,
30
  "passed": passed,
31
+ "slo_attainment": mean(attainments),
32
+ "slo_attainment_min": min(attainments),
33
+ "slo_attainment_max": max(attainments),
34
+ "slo_attainment_std": pstdev(attainments) if len(attainments) > 1 else 0.0,
35
+ "goodput_rps": mean(goodputs),
36
+ "goodput_std": pstdev(goodputs) if len(goodputs) > 1 else 0.0,
37
+ "p95_ttft_ms": mean(ttfts),
38
+ "p95_e2e_ms": mean(e2es),
39
+ "mean_unfinished": mean(unfinished_values),
40
+ "max_unfinished": max(unfinished_values),
41
+ "repetitions": repetitions,
42
+ "criterion": "all-repetitions-meet-target-and-drain",
43
  }
44
 
45
 
46
+ def capacity_search(
47
+ config: dict,
48
+ min_rate: float = 0.25,
49
+ max_rate: float = 32.0,
50
+ iterations: int = 8,
51
+ repetitions: int = 2,
52
+ headroom: float = 0.20,
53
+ ) -> dict:
54
  base = SimulationConfig.from_dict(config)
55
  low = max(0.01, min_rate)
56
  high = max(low * 1.01, max_rate)
 
63
  "status": "no_feasible_rate",
64
  "capacity_rps": 0.0,
65
  "recommended_rps": 0.0,
66
+ "headroom": headroom,
67
+ "criterion": "all-repetitions-meet-target-and-drain",
68
  "trace": trace,
69
  }
70
 
71
  high_eval = evaluate_rate(base, high, repetitions)
72
  trace.append(high_eval)
73
  if high_eval["passed"]:
 
74
  return {
75
  "status": "upper_bound_still_feasible",
76
+ "capacity_rps": high,
77
+ "recommended_rps": high * (1.0 - headroom),
78
+ "headroom": headroom,
79
+ "criterion": "all-repetitions-meet-target-and-drain",
80
+ "trace": sorted(trace, key=lambda item: item["rate_rps"]),
81
  }
82
 
83
  best = low
 
96
  "capacity_rps": best,
97
  "recommended_rps": best * (1.0 - headroom),
98
  "headroom": headroom,
99
+ "criterion": "all-repetitions-meet-target-and-drain",
100
+ "trace": sorted(trace, key=lambda item: item["rate_rps"]),
101
  }
102
 
103
 
 
115
  cfg = deepcopy(base)
116
  cfg.scheduler = scheduler
117
  result = run_simulation(cfg.to_dict())
118
+ rows.append(
119
+ {
120
+ "scheduler": scheduler,
121
+ "request_throughput_rps": result["summary"]["request_throughput_rps"],
122
+ "goodput_rps": result["summary"]["goodput_rps"],
123
+ "slo_attainment": result["summary"]["slo_attainment"],
124
+ "p95_ttft_ms": result["latency"]["ttft_ms"]["p95"],
125
+ "p95_e2e_ms": result["latency"]["e2e_ms"]["p95"],
126
+ "peak_kv_utilization": result["resource"]["peak_kv_utilization"],
127
+ "unfinished": result["summary"]["requests_unfinished"],
128
+ "bottleneck": result["diagnostics"]["label"],
129
+ }
130
+ )
131
+ rows.sort(key=lambda row: (row["slo_attainment"], row["goodput_rps"]), reverse=True)
132
  return {"rows": rows}
src/inferscale/simulator.py CHANGED
@@ -2,6 +2,7 @@ from __future__ import annotations
2
 
3
  from dataclasses import asdict
4
 
 
5
  from .kv_cache import KVCacheModel
6
  from .latency import AnalyticalLatencyModel
7
  from .metrics import summarize
@@ -296,9 +297,11 @@ class Simulator:
296
  "e2e_ms": (req.completion_time - req.arrival_time) * 1000.0 if req.completion_time is not None else None,
297
  })
298
 
 
 
299
  provenance = {
300
  "simulator": "InferScale-Sim",
301
- "version": "0.1.0",
302
  "latency_profile_type": "analytical-reference",
303
  "profile_warning": "Reference profiles are analytical proxies, not measured hardware benchmarks.",
304
  "model_profile_source": self.model.source,
@@ -311,6 +314,7 @@ class Simulator:
311
  summary=summary,
312
  latency=latency,
313
  resource=resource,
 
314
  requests=request_rows,
315
  timeline=[asdict(p) for p in self.timeline],
316
  warnings=self.warnings,
 
2
 
3
  from dataclasses import asdict
4
 
5
+ from .diagnostics import diagnose_run
6
  from .kv_cache import KVCacheModel
7
  from .latency import AnalyticalLatencyModel
8
  from .metrics import summarize
 
297
  "e2e_ms": (req.completion_time - req.arrival_time) * 1000.0 if req.completion_time is not None else None,
298
  })
299
 
300
+ diagnostics = diagnose_run(summary, latency, resource, self.cfg)
301
+
302
  provenance = {
303
  "simulator": "InferScale-Sim",
304
+ "version": "0.2.0",
305
  "latency_profile_type": "analytical-reference",
306
  "profile_warning": "Reference profiles are analytical proxies, not measured hardware benchmarks.",
307
  "model_profile_source": self.model.source,
 
314
  summary=summary,
315
  latency=latency,
316
  resource=resource,
317
+ diagnostics=diagnostics,
318
  requests=request_rows,
319
  timeline=[asdict(p) for p in self.timeline],
320
  warnings=self.warnings,
styles.css CHANGED
@@ -1,79 +1,12 @@
1
- :root {
2
- --bg: #080b12;
3
- --panel: #10151f;
4
- --panel2: #151c28;
5
- --line: #242d3c;
6
- --text: #edf1f7;
7
- --muted: #95a0b3;
8
- --accent: #8b7cff;
9
- --accent2: #55c2ff;
10
- --good: #63d9a5;
11
- --warn: #f5c76e;
12
- --danger: #ff7b8a;
13
- --shadow: 0 22px 70px rgba(0,0,0,.28);
14
- }
15
- * { box-sizing: border-box; }
16
- html { background: var(--bg); color-scheme: dark; }
17
- body { margin: 0; font-family: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; color: var(--text); background: radial-gradient(circle at 70% -10%, rgba(82, 80, 210, .17), transparent 30rem), var(--bg); }
18
- button, input, select { font: inherit; }
19
- .topbar { height: 76px; padding: 0 34px; border-bottom: 1px solid rgba(255,255,255,.06); display:flex; align-items:center; justify-content:space-between; position:sticky; top:0; z-index:20; background:rgba(8,11,18,.88); backdrop-filter: blur(16px); }
20
- .brand-wrap { display:flex; align-items:center; gap:13px; }
21
- .logo { width:38px; height:38px; display:grid; place-items:center; border-radius:10px; background:linear-gradient(135deg,var(--accent),var(--accent2)); color:white; font-weight:800; letter-spacing:-.04em; }
22
- .brand { font-weight:750; letter-spacing:-.02em; }
23
- .subtitle { color:var(--muted); font-size:12px; margin-top:2px; }
24
- .runtime-pill { display:flex; gap:8px; align-items:center; padding:8px 12px; border:1px solid var(--line); border-radius:99px; color:var(--muted); font-size:12px; background:rgba(255,255,255,.025); }
25
- .dot { width:7px; height:7px; border-radius:50%; background:var(--warn); box-shadow:0 0 14px currentColor; }
26
- .runtime-pill.ready .dot { background:var(--good); }
27
- .runtime-pill.error .dot { background:var(--danger); }
28
- .shell { width:min(1480px, calc(100% - 44px)); margin:0 auto; padding:42px 0 60px; }
29
- .hero { display:grid; grid-template-columns:minmax(0,1.45fr) minmax(400px,.75fr); gap:44px; align-items:end; padding:26px 4px 34px; }
30
- .eyebrow,.section-kicker { font-size:11px; letter-spacing:.12em; text-transform:uppercase; color:#9f96ff; font-weight:750; }
31
- h1 { font-size:clamp(34px,4vw,60px); line-height:1.03; letter-spacing:-.045em; max-width:940px; margin:10px 0 18px; }
32
- .hero p { max-width:830px; color:var(--muted); line-height:1.65; font-size:16px; margin:0; }
33
- .hero-stat-grid { display:grid; grid-template-columns:1fr 1fr; border:1px solid var(--line); border-radius:16px; overflow:hidden; background:rgba(255,255,255,.025); }
34
- .hero-stat { padding:17px 18px; min-height:78px; border-right:1px solid var(--line); border-bottom:1px solid var(--line); }
35
- .hero-stat:nth-child(2n){border-right:0}.hero-stat:nth-child(n+3){border-bottom:0}
36
- .hero-stat span { display:block; color:var(--muted); font-size:11px; margin-bottom:8px; }
37
- .hero-stat strong { font-size:15px; }
38
- .notice { border:1px solid rgba(245,199,110,.27); background:rgba(245,199,110,.07); color:#d9cfb8; border-radius:12px; padding:13px 16px; font-size:13px; line-height:1.5; }
39
- .tabs { display:flex; gap:8px; padding:26px 0 16px; overflow:auto; }
40
- .tab { border:0; color:var(--muted); background:transparent; padding:10px 14px; cursor:pointer; border-radius:8px; font-weight:650; }
41
- .tab:hover { color:var(--text); background:rgba(255,255,255,.035); }
42
- .tab.active { color:var(--text); background:rgba(139,124,255,.12); }
43
- .tab-panel { display:none; }.tab-panel.active { display:block; }
44
- .workspace { display:grid; grid-template-columns:400px minmax(0,1fr); gap:16px; align-items:start; }
45
- .panel { background:linear-gradient(180deg,rgba(19,25,37,.96),rgba(14,19,29,.96)); border:1px solid var(--line); border-radius:16px; box-shadow:var(--shadow); }
46
- .controls-panel { padding:20px; position:sticky; top:94px; }
47
- .result-panel,.wide-panel { padding:22px; }
48
- .panel-title-row { display:flex; align-items:flex-start; justify-content:space-between; gap:14px; margin-bottom:18px; }
49
- .panel-title-row h2 { margin:0; font-size:18px; letter-spacing:-.02em; }
50
- .panel-title-row p { margin:6px 0 0; }
51
- .tag { display:inline-flex; align-items:center; border:1px solid var(--line); background:rgba(255,255,255,.03); color:var(--muted); border-radius:99px; padding:5px 8px; font-size:10px; white-space:nowrap; }
52
- .tag.good { color:var(--good); border-color:rgba(99,217,165,.3); background:rgba(99,217,165,.07); }.tag.bad { color:var(--danger); border-color:rgba(255,123,138,.28); }.tag.neutral{color:var(--muted)}
53
- label { display:block; color:#bdc5d2; font-size:11px; font-weight:650; position:relative; }
54
- select,input { width:100%; margin-top:7px; height:38px; border:1px solid var(--line); background:#0b1018; color:var(--text); border-radius:8px; padding:0 10px; outline:none; }
55
- select:focus,input:focus { border-color:#625bd0; box-shadow:0 0 0 3px rgba(98,91,208,.12); }
56
- .unit { position:absolute; right:9px; bottom:11px; color:#697488; font-size:9px; pointer-events:none; }
57
- .field-grid { display:grid; gap:11px; margin-bottom:11px; }.field-grid.two { grid-template-columns:1fr 1fr; }
58
- hr { border:0; border-top:1px solid var(--line); margin:18px 0; }
59
- .section-kicker { margin-bottom:11px; }
60
- button.primary,button.secondary { width:100%; border-radius:9px; height:42px; border:1px solid transparent; margin-top:12px; font-weight:750; cursor:pointer; transition:.16s ease; }
61
- button.primary { background:linear-gradient(135deg,#7568f5,#4775f2); color:white; box-shadow:0 10px 26px rgba(94,85,222,.20); }
62
- button.primary:hover:not(:disabled){transform:translateY(-1px);filter:brightness(1.06)}
63
- button.secondary { background:transparent; color:#c7cfdb; border-color:var(--line); }
64
- button:disabled { opacity:.45; cursor:not-allowed; }.compact { width:auto !important; min-width:180px; padding:0 18px; margin-top:0 !important; }
65
- .empty-state { min-height:420px; display:grid; place-content:center; text-align:center; color:var(--muted); padding:30px; }.empty-state.small { min-height:260px; }.empty-state h3{color:#cfd6e2;margin:10px 0 6px;font-size:16px}.empty-state p{max-width:480px;margin:0;line-height:1.55;font-size:13px}.empty-icon{width:44px;height:44px;border:1px solid var(--line);border-radius:12px;display:grid;place-items:center;margin:0 auto;font-size:20px;color:var(--accent2)}
66
- .hidden { display:none !important; }
67
- .metric-grid { display:grid; grid-template-columns:repeat(6,1fr); gap:10px; margin-bottom:16px; }.metric-grid.four{grid-template-columns:repeat(4,1fr)}
68
- .metric { min-height:88px; padding:14px; border:1px solid var(--line); border-radius:11px; background:#0c111a; }.metric span{display:block;color:var(--muted);font-size:10px;margin-bottom:10px}.metric strong{font-size:19px;letter-spacing:-.03em}.metric.emphasis{border-color:rgba(139,124,255,.38);background:rgba(139,124,255,.06)}
69
- .chart-grid { display:grid; grid-template-columns:1fr 1fr; gap:12px; }.chart-card { border:1px solid var(--line); border-radius:12px; background:#0c111a; padding:14px; min-height:310px; }.chart-card.full{margin-top:12px}.chart-title{color:#cfd6e2;font-size:11px;font-weight:700;margin-bottom:12px}
70
- canvas { max-height:280px; }
71
- .warnings { margin-top:12px; border:1px solid rgba(245,199,110,.24); padding:11px 13px; background:rgba(245,199,110,.06); border-radius:9px; color:#d8cdaF; font-size:12px; }
72
- .muted { color:var(--muted); font-size:13px; line-height:1.55; }
73
- .table-wrap { overflow:auto; margin-top:14px; border:1px solid var(--line); border-radius:10px; }
74
- table { width:100%; border-collapse:collapse; font-size:12px; min-width:760px; } th,td{padding:11px 12px;text-align:right;border-bottom:1px solid var(--line)}th:first-child,td:first-child{text-align:left}th{color:#7f8ba0;font-size:10px;text-transform:uppercase;letter-spacing:.06em;background:#0a0f17}td{color:#cbd3df}tbody tr:last-child td{border-bottom:0}.pass{color:var(--good)}.fail{color:var(--danger)}
75
- .planner-grid { grid-template-columns:370px minmax(0,1fr); }
76
- .method-grid { display:grid; grid-template-columns:1fr 1fr; gap:16px; }.prose{padding:26px}.prose h2{font-size:22px;margin:8px 0 12px}.prose p{color:var(--muted);line-height:1.7;font-size:14px}.prose code{color:#b7b0ff}.formula{font-family:ui-monospace,SFMono-Regular,Menlo,monospace;color:#b9c6dc;padding:12px;border:1px solid var(--line);border-radius:8px;background:#0b1018;font-size:12px}.wide-method{grid-column:1/-1}.paper-grid{display:grid;grid-template-columns:repeat(5,1fr);gap:10px;margin-top:20px}.paper-grid>div{padding:14px;border:1px solid var(--line);border-radius:10px;background:#0c111a}.paper-grid strong{display:block;font-size:12px;margin-bottom:8px}.paper-grid span{color:var(--muted);font-size:11px;line-height:1.55;display:block}
77
- footer { width:min(1480px,calc(100% - 44px)); margin:0 auto; padding:22px 0 36px; border-top:1px solid rgba(255,255,255,.055); display:flex; justify-content:space-between; gap:20px; color:#657085; font-size:11px; }
78
- @media (max-width:1100px){.hero{grid-template-columns:1fr}.workspace,.planner-grid{grid-template-columns:1fr}.controls-panel{position:static}.metric-grid{grid-template-columns:repeat(3,1fr)}.paper-grid{grid-template-columns:1fr 1fr}}
79
- @media (max-width:720px){.topbar{padding:0 18px}.subtitle{display:none}.runtime-pill{max-width:170px}.shell{width:min(100% - 24px,1480px);padding-top:20px}.hero{padding-top:8px;gap:20px}.hero-stat-grid{grid-template-columns:1fr 1fr}.field-grid.two,.chart-grid,.method-grid{grid-template-columns:1fr}.metric-grid,.metric-grid.four{grid-template-columns:1fr 1fr}.paper-grid{grid-template-columns:1fr}.panel-title-row{flex-direction:column}.compact{width:100%!important}.wide-method{grid-column:auto}footer{width:calc(100% - 24px);flex-direction:column}}
 
1
+ :root{--bg:#0b0e13;--surface:#11161e;--surface2:#0d1219;--line:#29313d;--line2:#202732;--text:#e8edf3;--muted:#929dab;--faint:#66717f;--accent:#79a7ff;--good:#69c99a;--warn:#dfb966;--danger:#e47e87}
2
+ *{box-sizing:border-box}html{background:var(--bg);color-scheme:dark}body{margin:0;font-family:ui-sans-serif,system-ui,-apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;color:var(--text);background:var(--bg);font-size:14px}button,input,select{font:inherit}
3
+ .topbar{height:68px;padding:0 28px;border-bottom:1px solid var(--line2);display:flex;align-items:center;justify-content:space-between;position:fixed;top:0;left:0;right:0;z-index:50;background:#0c1016;transition:transform .22s ease}body.runtime-ready .topbar{transform:translateY(calc(-100% + 6px))}body.runtime-ready .topbar:hover,body.runtime-ready .topbar:focus-within{transform:translateY(0)}.brand-wrap{display:flex;align-items:center;gap:11px}.logo{width:34px;height:34px;display:grid;place-items:center;border:1px solid #455164;border-radius:6px;color:#d9e2ee;font:700 12px/1 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace}.brand{font-weight:700;letter-spacing:-.015em}.subtitle{color:var(--muted);font-size:11px;margin-top:2px}.runtime-pill{display:flex;gap:8px;align-items:center;padding:7px 10px;border:1px solid var(--line);border-radius:6px;color:var(--muted);font:11px/1.2 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;background:var(--surface2)}.dot{width:7px;height:7px;border-radius:50%;background:var(--warn)}.runtime-pill.ready .dot{background:var(--good)}.runtime-pill.error .dot{background:var(--danger)}
4
+ .shell{width:min(1460px,calc(100% - 40px));margin:0 auto;padding:100px 0 56px;transition:padding-top .22s ease}body.runtime-ready .shell{padding-top:30px}.intro{display:grid;grid-template-columns:minmax(0,1.45fr) minmax(380px,.55fr);gap:48px;align-items:end;padding:20px 2px 26px}.eyebrow,.section-kicker{font:700 10px/1.2 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;letter-spacing:.08em;text-transform:uppercase;color:#8ea6c7}h1{font-size:clamp(32px,3.2vw,48px);line-height:1.06;letter-spacing:-.035em;max-width:900px;margin:10px 0 14px;font-weight:720}.intro p{max-width:840px;color:var(--muted);line-height:1.65;margin:0;font-size:15px}.project-facts{margin:0;border-top:1px solid var(--line);border-bottom:1px solid var(--line)}.project-facts div{display:grid;grid-template-columns:130px 1fr;gap:14px;padding:9px 0;border-bottom:1px solid var(--line2)}.project-facts div:last-child{border-bottom:0}.project-facts dt{color:var(--faint);font-size:11px}.project-facts dd{margin:0;color:#cbd4df;font-size:12px;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace}.reference-note{border-left:3px solid #8b7445;background:#12151a;color:#bcb5a5;padding:11px 13px;font-size:12px;line-height:1.5}.tabs{display:flex;gap:2px;padding:22px 0 12px;overflow:auto;border-bottom:1px solid var(--line2);margin-bottom:14px}.tab{border:0;border-bottom:2px solid transparent;color:var(--muted);background:transparent;padding:9px 12px;cursor:pointer;font-weight:650}.tab:hover{color:var(--text)}.tab.active{color:var(--text);border-bottom-color:var(--accent)}.tab-panel{display:none}.tab-panel.active{display:block}
5
+ .workspace{display:grid;grid-template-columns:390px minmax(0,1fr);gap:14px;align-items:start}.panel{background:var(--surface);border:1px solid var(--line);border-radius:8px}.controls-panel{padding:18px;position:sticky;top:18px}.result-panel,.wide-panel{padding:20px}.panel-title-row{display:flex;align-items:flex-start;justify-content:space-between;gap:14px;margin-bottom:16px}.panel-title-row h2{margin:0;font-size:17px;letter-spacing:-.02em}.panel-title-row p{margin:6px 0 0}.tag{display:inline-flex;align-items:center;border:1px solid var(--line);background:var(--surface2);color:var(--muted);border-radius:5px;padding:5px 7px;font-size:10px;white-space:nowrap}.tag.good{color:var(--good);border-color:#315e4a}.tag.bad{color:var(--danger);border-color:#643943}label{display:block;color:#b5bfcb;font-size:11px;font-weight:650;position:relative}select,input{width:100%;margin-top:7px;height:37px;border:1px solid var(--line);background:#0b1016;color:var(--text);border-radius:5px;padding:0 9px;outline:none}select:focus,input:focus{border-color:#5475a8;box-shadow:0 0 0 2px rgba(84,117,168,.17)}.unit{position:absolute;right:9px;bottom:11px;color:#697687;font-size:9px;pointer-events:none}.field-grid{display:grid;gap:10px;margin-bottom:10px}.field-grid.two{grid-template-columns:1fr 1fr}hr{border:0;border-top:1px solid var(--line2);margin:17px 0}.section-kicker{margin-bottom:10px}
6
+ button.primary,button.secondary{height:40px;border-radius:5px;font-weight:700;cursor:pointer;transition:.13s ease}button.primary{width:100%;border:1px solid #668fd5;background:#4f7fce;color:white;margin-top:11px}button.primary:hover:not(:disabled){background:#5b8bd8}button.secondary{border:1px solid var(--line);background:#0d1219;color:#c4ccd7}button.secondary:hover:not(:disabled),.mini-button:hover:not(:disabled),.chart-expand:hover{border-color:#536174;background:#151b24}button:disabled{opacity:.42;cursor:not-allowed}.compact{width:auto!important;min-width:178px;padding:0 17px;margin-top:0!important}.button-row{display:grid;grid-template-columns:1fr 1fr;gap:8px;margin-top:8px}
7
+ .empty-state{min-height:380px;display:grid;place-content:center;text-align:center;color:var(--muted);padding:30px}.empty-state.small{min-height:240px}.empty-state h3{color:#ccd4de;margin:0 0 6px;font-size:15px}.empty-state p{max-width:500px;margin:0;line-height:1.55;font-size:12px}.hidden{display:none!important}.metric-grid{display:grid;grid-template-columns:repeat(6,1fr);gap:8px;margin-bottom:12px}.metric-grid.four{grid-template-columns:repeat(4,1fr)}.metric{min-height:82px;padding:13px;border:1px solid var(--line2);background:var(--surface2)}.metric span{display:block;color:var(--muted);font-size:10px;margin-bottom:9px}.metric strong{font-size:18px;letter-spacing:-.025em}.metric.emphasis{border-color:#486da4;background:#101824}
8
+ .diagnostic-card{margin-bottom:12px;border:1px solid var(--line);background:#0e131a;padding:14px}.diagnostic-head{display:flex;align-items:baseline;justify-content:space-between;gap:16px}.diagnostic-head span{color:var(--muted);font-size:10px;text-transform:uppercase;letter-spacing:.06em}.diagnostic-head strong{font-size:14px}.diagnostic-card p{margin:8px 0 0;color:#b8c1cd;font-size:12px;line-height:1.55}.diagnostic-card .diagnostic-action{color:var(--muted)}.evidence-row{display:flex;flex-wrap:wrap;gap:6px;margin-top:10px}.evidence-row span{border:1px solid var(--line2);background:#0a0f15;padding:4px 6px;color:#8390a0;font:10px/1.2 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace}
9
+ .chart-grid{display:grid;grid-template-columns:1fr 1fr;gap:10px}.chart-card{border:1px solid var(--line);background:var(--surface2);min-height:300px;position:relative}.chart-card.full{margin-top:10px}.chart-head{height:42px;display:flex;align-items:center;justify-content:space-between;gap:12px;padding:0 12px;border-bottom:1px solid var(--line2)}.chart-title{color:#c7d0db;font-size:11px;font-weight:700}.chart-expand,.mini-button{border:1px solid var(--line);background:#0b1016;color:var(--muted);border-radius:4px;cursor:pointer;height:28px;padding:0 9px;font-size:10px}.chart-body{height:260px;padding:10px 12px 12px}.chart-body.large{height:340px}.chart-card.chart-expanded{position:fixed;inset:18px;z-index:100;background:#0b1016;border-color:#4c596b;box-shadow:0 0 0 9999px rgba(0,0,0,.72);min-height:0;margin:0}.chart-card.chart-expanded .chart-body{height:calc(100vh - 84px)}body.chart-open{overflow:hidden}canvas{width:100%!important;height:100%!important;max-height:none}.warnings{margin-top:10px;border-left:3px solid #7b6840;padding:9px 11px;background:#15140f;color:#c4bca9;font-size:11px;line-height:1.5}.muted{color:var(--muted);font-size:12px;line-height:1.55}
10
+ .table-toolbar{display:flex;align-items:center;justify-content:space-between;gap:12px;margin-top:14px;color:#aab4c1;font-size:11px}.table-toolbar>div{display:flex;gap:6px}.table-wrap{overflow:auto;margin-top:7px;border:1px solid var(--line)}table{width:100%;border-collapse:collapse;font-size:12px;min-width:900px}th,td{padding:10px 11px;text-align:right;border-bottom:1px solid var(--line2);white-space:nowrap}th:first-child,td:first-child{text-align:left}th{color:#7f8b9b;font-size:9px;text-transform:uppercase;letter-spacing:.06em;background:#0a0f15;position:sticky;top:0}td{color:#c7d0db}tbody tr:last-child td{border-bottom:0}tbody tr:hover td{background:#121821}.pass{color:var(--good)}.fail{color:var(--danger)}.best-label{display:inline-block;margin-left:7px;border:1px solid #4a6386;color:#9ab8df;padding:2px 5px;border-radius:3px;font-size:9px;text-transform:uppercase}.planner-grid{grid-template-columns:360px minmax(0,1fr)}.planner-note{color:var(--muted);border-left:2px solid #445a77;padding:7px 9px;margin-bottom:2px;font-size:11px;line-height:1.5}
11
+ .method-grid{display:grid;grid-template-columns:1fr 1fr;gap:12px}.prose{padding:22px}.prose h2{font-size:20px;margin:8px 0 11px}.prose p{color:var(--muted);line-height:1.7;font-size:13px}.prose code{color:#a8bfe4}.formula{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;color:#b7c3d2;padding:10px;border:1px solid var(--line);background:#0b1016;font-size:11px}.wide-method{grid-column:1/-1}.paper-grid{display:grid;grid-template-columns:repeat(5,1fr);gap:8px;margin-top:18px}.paper-grid>div{padding:12px;border:1px solid var(--line2);background:#0d1219}.paper-grid strong{display:block;font-size:11px;margin-bottom:7px}.paper-grid span{color:var(--muted);font-size:10px;line-height:1.55;display:block}footer{width:min(1460px,calc(100% - 40px));margin:0 auto;padding:20px 0 34px;border-top:1px solid var(--line2);display:flex;justify-content:space-between;gap:20px;color:var(--faint);font-size:10px}.toast{position:fixed;right:20px;bottom:20px;z-index:130;background:#17202b;border:1px solid #465467;color:#d5dce5;padding:9px 12px;border-radius:5px;font-size:11px;opacity:0;transform:translateY(8px);pointer-events:none;transition:.16s ease}.toast.show{opacity:1;transform:translateY(0)}
12
+ @media(max-width:1100px){.intro{grid-template-columns:1fr;gap:24px}.workspace,.planner-grid{grid-template-columns:1fr}.controls-panel{position:static}.metric-grid{grid-template-columns:repeat(3,1fr)}.paper-grid{grid-template-columns:1fr 1fr}}@media(max-width:720px){.topbar{padding:0 15px}.subtitle{display:none}.runtime-pill{max-width:170px}.shell{width:calc(100% - 20px);padding-top:88px}body.runtime-ready .shell{padding-top:20px}.intro{padding-top:10px}.project-facts div{grid-template-columns:100px 1fr}.field-grid.two,.chart-grid,.method-grid{grid-template-columns:1fr}.metric-grid,.metric-grid.four{grid-template-columns:1fr 1fr}.paper-grid{grid-template-columns:1fr}.panel-title-row{flex-direction:column}.arena-title-row .compact{width:100%!important}.wide-method{grid-column:auto}footer{width:calc(100% - 20px);flex-direction:column}.chart-card.chart-expanded{inset:6px}}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
tests/test_diagnostics.py ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from inferscale import run_simulation
2
+
3
+
4
+ def test_healthy_run_has_diagnosis():
5
+ result = run_simulation(
6
+ {
7
+ "duration_s": 6,
8
+ "request_rate_rps": 0.5,
9
+ "prompt_tokens_mean": 128,
10
+ "output_tokens_mean": 8,
11
+ "slo_ttft_ms": 2000,
12
+ "slo_e2e_ms": 20000,
13
+ "slo_attainment_target": 0.95,
14
+ }
15
+ )
16
+ assert result["diagnostics"]["code"] in {"healthy", "healthy_near_saturation"}
17
+ assert result["diagnostics"]["provenance"] == "heuristic-simulator-diagnosis"
18
+
19
+
20
+ def test_summary_exposes_component_slo_attainment():
21
+ result = run_simulation(
22
+ {
23
+ "duration_s": 6,
24
+ "request_rate_rps": 2,
25
+ "prompt_tokens_mean": 128,
26
+ "output_tokens_mean": 8,
27
+ }
28
+ )
29
+ summary = result["summary"]
30
+ assert 0 <= summary["ttft_slo_attainment"] <= 1
31
+ assert 0 <= summary["e2e_slo_attainment"] <= 1
32
+ assert set(summary["slo_failure_breakdown"]) == {"ttft_only", "e2e_only", "both"}
tests/test_optimizer.py CHANGED
@@ -1,4 +1,5 @@
1
- from inferscale.optimizer import capacity_search, compare_schedulers
 
2
 
3
 
4
  def test_capacity_search_returns_trace():
@@ -13,8 +14,16 @@ def test_capacity_search_returns_trace():
13
  out = capacity_search(cfg, min_rate=0.25, max_rate=3, iterations=3, repetitions=1)
14
  assert out["trace"]
15
  assert out["capacity_rps"] >= 0
 
16
 
17
 
18
- def test_scheduler_compare_has_rows():
 
 
 
 
 
 
19
  out = compare_schedulers({"duration_s": 4, "request_rate_rps": 1, "output_tokens_mean": 8})
20
  assert len(out["rows"]) == 5
 
 
1
+ from inferscale.models import SimulationConfig
2
+ from inferscale.optimizer import capacity_search, compare_schedulers, evaluate_rate
3
 
4
 
5
  def test_capacity_search_returns_trace():
 
14
  out = capacity_search(cfg, min_rate=0.25, max_rate=3, iterations=3, repetitions=1)
15
  assert out["trace"]
16
  assert out["capacity_rps"] >= 0
17
+ assert out["criterion"] == "all-repetitions-meet-target-and-drain"
18
 
19
 
20
+ def test_rate_evaluation_exposes_repetition_range():
21
+ out = evaluate_rate(SimulationConfig(duration_s=4, request_rate_rps=1), 1.0, repetitions=2)
22
+ assert out["slo_attainment_min"] <= out["slo_attainment"] <= out["slo_attainment_max"]
23
+ assert out["repetitions"] == 2
24
+
25
+
26
+ def test_scheduler_compare_has_rows_and_diagnostics():
27
  out = compare_schedulers({"duration_s": 4, "request_rate_rps": 1, "output_tokens_mean": 8})
28
  assert len(out["rows"]) == 5
29
+ assert all(row["bottleneck"] for row in out["rows"])
worker.mjs CHANGED
@@ -3,6 +3,7 @@ import { loadPyodide } from "https://cdn.jsdelivr.net/pyodide/v314.0.5/full/pyod
3
  const MODULES = [
4
  "__init__.py",
5
  "api.py",
 
6
  "kv_cache.py",
7
  "latency.py",
8
  "metrics.py",
 
3
  const MODULES = [
4
  "__init__.py",
5
  "api.py",
6
+ "diagnostics.py",
7
  "kv_cache.py",
8
  "latency.py",
9
  "metrics.py",