Spaces:

glayguo
/

evalarc

Running

Publish verified EvalArc 7de84f7eecb1

7eebc8f verified 16 days ago

6.04 kB

	<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><meta http-equiv="Content-Security-Policy" content="default-src 'none'; style-src 'unsafe-inline'; base-uri 'none'; form-action 'none'"><title>EvalArc · Repeatability</title><style>
	:root{color-scheme:dark;font:16px/1.6 system-ui,sans-serif;background:#101618;color:#e0ece7}
	body{max-width:1080px;margin:60px auto;padding:0 24px}
	.eyebrow{color:#86dbaf;letter-spacing:.18em;font-size:12px;font-weight:700}
	h1{font-size:clamp(32px,6vw,64px);letter-spacing:-.045em;line-height:1.1;margin:16px 0}
	p{color:#acbeb5;max-width:760px}.cards{display:flex;gap:20px;flex-wrap:wrap;margin:36px 0}
	.card{border:1px solid #33463d;border-radius:12px;padding:18px 26px;flex:1;min-width:150px}
	.number{font-size:38px;color:#96e8b9;font-weight:650}.label{color:#b7c6bf}
	.scroll{overflow:auto}table{border-collapse:collapse;width:100%;font-size:14px}
	td,th{text-align:left;padding:14px;border-bottom:1px solid #33463d}th{color:#86dbaf}
	code{font-family:ui-monospace,monospace}h2{margin-top:42px}
	.dimension{display:flex;gap:16px;align-items:center;max-width:620px;margin:10px 0}
	.dimension span:first-child{width:150px}meter{flex:1;accent-color:#86dbaf}
	footer{margin-top:48px;border-top:1px solid #33463d;padding-top:20px;font-size:13px}
	.metadata{overflow-wrap:anywhere;font-size:13px}
	pre{overflow:auto;padding:18px;background:#17221c;font-size:12px;max-height:480px}
	summary{cursor:pointer;color:#96e8b9}details{margin:16px 0}
	a{color:#96e8b9}.failed,.agent_error,.environment_error{color:#edb68d}
	.passed{color:#96e8b9}td{overflow-wrap:anywhere}
	.scroll table{min-width:680px}td:first-child{min-width:170px;overflow-wrap:normal}
	.change-cards{display:none}.change-card{border:1px solid #33463d;border-radius:12px;padding:16px}
	.change-card h3{font-size:16px;margin:0;overflow-wrap:anywhere}.change-card p{margin:10px 0 0}
	@media(max-width:640px){.change-table{display:none}.change-cards{display:grid;gap:16px}}
	</style></head><body><div class="eyebrow">EVALARC / REPEATABILITY</div><h1>See every attempt.</h1><p>support-routing · PASSED · one frozen candidate, fixed cases, fresh state per attempt</p><div class="cards"><div class="card"><div class="number">3/3</div><div class="label">requested attempts resolved</div></div><div class="card"><div class="number">1</div><div class="label">mean score of assessed attempts</div></div><div class="card"><div class="number">0</div><div class="label">checks with varying outcomes</div></div></div><p>Requested: 3. Completed: 3. Assessed: 3. Invalid: 0. Assessed score range: 1–1.</p><h2>Outcomes by case</h2><article class="change-card"><h3>close-resolved-ticket · seed 17</h3><p>3/3 assessed attempts passed · 0 unassessed</p><details><summary>Check pass rates</summary><ul><li>routing: 3/3 (100.0%)</li><li>notes: 3/3 (100.0%)</li><li>closure: 3/3 (100.0%)</li><li>scope: 3/3 (100.0%)</li><li>protocol: 3/3 (100.0%)</li></ul></details></article><article class="change-card"><h3>retry-after-commit · seed 17</h3><p>3/3 assessed attempts passed · 0 unassessed</p><details><summary>Check pass rates</summary><ul><li>routing: 3/3 (100.0%)</li><li>notes: 3/3 (100.0%)</li><li>closure: 3/3 (100.0%)</li><li>scope: 3/3 (100.0%)</li><li>protocol: 3/3 (100.0%)</li></ul></details></article><article class="change-card"><h3>retry-before-commit · seed 17</h3><p>3/3 assessed attempts passed · 0 unassessed</p><details><summary>Check pass rates</summary><ul><li>routing: 3/3 (100.0%)</li><li>notes: 3/3 (100.0%)</li><li>closure: 3/3 (100.0%)</li><li>scope: 3/3 (100.0%)</li><li>protocol: 3/3 (100.0%)</li></ul></details></article><article class="change-card"><h3>route-open-ticket · seed 17</h3><p>3/3 assessed attempts passed · 0 unassessed</p><details><summary>Check pass rates</summary><ul><li>routing: 3/3 (100.0%)</li><li>notes: 3/3 (100.0%)</li><li>closure: 3/3 (100.0%)</li><li>scope: 3/3 (100.0%)</li><li>protocol: 3/3 (100.0%)</li></ul></details></article><h2>Every attempt</h2><ul><li><a href="attempts/0001/index.html">Attempt 1</a> · passed · score 1 · <a href="attempts/0001/evaluation.json">JSON evidence</a></li><li><a href="attempts/0002/index.html">Attempt 2</a> · passed · score 1 · <a href="attempts/0002/evaluation.json">JSON evidence</a></li><li><a href="attempts/0003/index.html">Attempt 3</a> · passed · score 1 · <a href="attempts/0003/evaluation.json">JSON evidence</a></li></ul><h2>Reproduction record</h2><p class="metadata">Candidate SHA-256: 1630ac4c153c0a3b870af9d0f05c5ae1a325f839cf6be4e851da94e47c881927<br>Grader SHA-256: 887aea97efec396118521f720f6d8753be9ae964ab92eaa655d6575185326593<br>Cases SHA-256: b7cd8663c69da9e1715d5c8e1d707514863dcc333fefc57acc3315053ebc85df<br>Created: 2026-09-14T03:11:26.967381+00:00<br>Seeds: [17]<br>Runtime: {"backend": "docker", "image": "python:3.12-slim", "image_id": "sha256:78387bc3881b8273120a12ebe6c1ab22b018ccc2c9adf565ae1ac9b536e184ea", "command": ["{python}", "-I", "-B", "main.py"], "response_timeout_seconds": 10.0, "case_timeout_seconds": 60.0, "session_output_limit_bytes": 1048576, "python": "3.12.3", "platform": "Linux-6.17.0-1020-aws-x86_64-with-glibc2.39"}</p><p><a href="repetition.json">Repetition summary JSON</a> · <a href="events.jsonl">Progress events</a></p><footer>Descriptive outcomes on the same public cases. Invalid attempts are explicit; mean scores and resolution rates use assessed attempts only. No independence assumption, confidence interval, or population success rate is claimed. Case and check denominators count their assessed observations, including those in an otherwise invalid attempt. Repeated success on these cases does not establish general agent reliability.</footer></body></html>