`;
}
function recordingPilotProgress(){
const p=data.recording_pilot;if(!p)return '';
return `Codex + RoboBridge · GPT-6 Astra / high · GPU recording pilot
${number(p.completed)} / ${number(p.planned)} independent attempts published: LIBERO, RoboTwin, and two RoboDojo recording conditions. This pilot does not cover the full benchmark.
LIBERO and RoboTwin use independent 720-high spectator cameras. RoboDojo compares native camera recording with online HD recording. Both keep native agent camera resolution, but the RTX control test found that extra HD cameras change subsequent native image pixels. The HD control is shown separately.
Videos play at 4× simulation speed. Session galleries contain images actually submitted to the agent. One model episode per condition cannot establish a success-rate effect.
${c.quota_waiting?'Waiting for model-service quota recovery · ':''}${number(c.published_episodes)} episode records published · Results and protocol ↗
`;
}
function methodology() {
return `Reading the Hard benchmarkProtocol, metrics & provenance+
Baseline and four standalone conditions
All 50 Hard tasks are evaluated for the VLAct OFT 50k baseline and for progVLA-yz/progVLA-jy in both modes: 250 episodes. Every condition receives the same native unseen instruction and frozen scene seed for each task. Pending episodes are excluded from success-rate denominators.
VLAct OFT baseline
The released VLAct Qwen3-VL-OFT 50k checkpoint receives the three images and the native instruction, with the same cameras, 32-action chunks and native step limits: 19/50 (38%). An earlier native-instruction run of the same checkpoint (16/50, 32%) used per-run expert checks and its own seeds on 4 tasks; it and the 22% RLE-instruction result remain in Previous attempts.
Mode 1 · images and instruction
Pose encoders and pose modulation are disabled. Trained action projections and decoders are retained. This is an inference-time conditioning ablation, not a separately trained vision-only model. Progress heads are disabled in both modes.
Mode 2 · reference pose
Current pose is the learner's measured world TCP. Target pose comes from the unchanged successful same-scene native expert reference. progVLA-yz uses the next subgoal; progVLA-jy samples its training window [phase+64, phase+96). This condition uses privileged reference information.
Native execution
Three RGB views, released image buckets, model seed 42, full 32-action chunks and native TOPP execution are fixed. Native success and task step limits decide outcomes. Valid failures are retained. The phase tracker changes neither actions nor termination.
Agent evaluation context
Native agent conditions retain their own seeds, admission and execution protocols. They do not isolate the effect of standalone pose conditioning. Removed RLE display rows remain in the archived evidence.
`;
}
const expandedTaskVariants = new Set();
const visibleTaskVariants = () => data.variants.filter(v => expandedTaskVariants.has(v.id));
function taskColumnPicker() {
return `
Choose methods to compare
All method columns are collapsed. Select one or more to compare.
${data.variants.map(v=>``).join('')}
`;
}
function taskColumnHeaders() {
return '
Task
' + visibleTaskVariants().map(v=>`
${esc(methodLabel(v))}
${esc(v.model)} · ${esc(v.effort)}Native result · Estimated USD
`).join('');
}
function syncTaskColumns() {
if(current.type==='runs')return;
const variants=visibleTaskVariants(), table=$('#task-comparison');
table.querySelector('thead tr').innerHTML=taskColumnHeaders();
table.classList.toggle('has-methods',variants.length>0);
table.style.setProperty('--method-count',variants.length);
$$('[data-task-column].task-column-toggle').forEach(button=>{
const expanded=expandedTaskVariants.has(button.dataset.taskColumn);
button.setAttribute('aria-expanded',expanded);
button.querySelector('.column-toggle-icon').textContent=expanded?'−':'+';
});
$('#task-column-status').textContent=variants.length?`${variants.length} of ${data.variants.length} method columns expanded. Select another method to compare.`:'All method columns are collapsed. Select one or more to compare.';
$('[data-task-columns="none"]').disabled=variants.length===0;
$('[data-task-columns="all"]').disabled=variants.length===data.variants.length;
}
function taskBrowser(variant=null) {
return `
`;
$('#task-count').textContent=`${tasks.length} of ${data.tasks.length} tasks`;
$('#filter-context').textContent=variant?variantName(variant):'Outcome filter & usage/cost sort: '+variantName(filters.variant);
}
function researchRerunNote() {
const r=data.research_update;if(!r)return '';
return ``;
}
function overview() {
current={type:'overview'};
$('#main').innerHTML=`
✦ RLE-Bench · Embodied agent evaluation
RoboTwin 2.0. Native instruction.
demo_randomized · one episode per selected task. Final 50k checkpoints · baseline and four conditions · 250 completed episodes · ${data.standalone_native_campaign?.status==='scheduler_unavailable'?'scheduler unavailable':data.standalone_native_campaign?.status==='waiting_for_gpu'?'queued, waiting for GPU capacity':data.standalone_native_campaign?.status==='loading_model'?'loading models':data.standalone_native_campaign?.status==='running'?'running':data.standalone_native_campaign?.status==='paused'?'paused':'complete'}. VLAct OFT 50k baseline: 19/50 (38%).
${data.families.length}environment
${data.tasks.length}distinct tasks
${data.variants.length}agent–model variants
${data.benchmark_complete?'Evaluation complete':'Evaluation in progress'} · selected seeds
`;
}
// Align words and punctuation while preserving the exact original whitespace.
function instructionDiff(original, revised) {
const split = text => text.match(/\s+|[\p{L}\p{N}_]+|[^\s]/gu) || [];
const a=split(original), b=split(revised);
const lcs=Array.from({length:a.length+1},()=>new Uint16Array(b.length+1));
for(let i=a.length-1;i>=0;i--)for(let j=b.length-1;j>=0;j--)
lcs[i][j]=a[i]===b[j]?lcs[i+1][j+1]+1:Math.max(lcs[i+1][j],lcs[i][j+1]);
const before=[],after=[];
const append=(list,text,changed)=>{const last=list.at(-1);if(last&&last.changed===changed)last.text+=text;else list.push({text,changed});};
let i=0,j=0;
while(i=lcs[i+1][j]))append(after,b[j++],true);
else append(before,a[i++],true);
}
const html=(parts,kind)=>parts.map(p=>p.changed&&p.text.trim()?`<${kind==='removed'?'del':'mark'} class="instruction-${kind}">${esc(p.text)}${kind==='removed'?'del':'mark'}>`:esc(p.text)).join('');
return {original:html(before,'removed'),revised:html(after,'added')};
}
function instructionPair(task, ep) {
const candidate = ep ? measuredInstructions.episodes[ep.id] : null;
const record = candidate && candidate.initial_goal === ep.instruction && candidate.task_key === task.key ? candidate : null;
const recordedNative = record?.observed_native || (ep ? ep.native_instruction : task.native_instruction);
const measured = record?.text || (ep ? (ep.instruction || (ep.instruction_policy === 'original_native' ? recordedNative : null)) : null);
const reference = catalogInstructionAudit.tasks.find(item => item.published_task_key === task.key && item.id === task.native_id);
const nativeReference = !recordedNative && reference?.native ? reference : null;
const native = recordedNative || nativeReference?.native;
return {record, native, measured, nativeReference};
}
function instructionPanel(task,ep) {
const {record,native,measured:recorded,nativeReference}=instructionPair(task,ep);
const measured=ep?recorded:(task.catalog_instruction_by_variant?.[current.variant]||task.catalog_instruction);
const initial=record&&record.initial_goal!==measured?record.initial_goal:null;
const changed=Boolean(native&&measured&&native!==measured);
const diff=changed?instructionDiff(native,measured):null;
const choices=[['measured',ep?'Measured':'Planned',measured],['native','Original native',native],['current','Current catalog',null]].filter(([key,,text])=>text&&(key!=='current'||(ep&&text!==measured&&text!==initial)));
const currentChanged=native&&task.catalog_instruction&&native!==task.catalog_instruction;
const description=!ep?'The current catalog instruction for the planned evaluation.':record?.kind==='scene_bound_goal'?"The measured goal with this episode's objects and targets filled in from the agent's session.":record?.kind==='native_received'?'The native instruction received by the agent during this episode.':'The task instruction supplied for this episode.';
return `
Task instruction
${description}
${choices.map(([k,label],i)=>``).join('')}
${nativeReference?`
Native comparison uses the pinned upstream full_description reference${ep?'; native text was not recorded in this episode':''}. ${external(nativeReference.source_url,'Native source')}
`:''}${choices.map(([k,,text],i)=>{
const comparison=k==='current'?(currentChanged?instructionDiff(native,text):null):diff;
const html=comparison?(k==='native'?comparison.original:comparison.revised):esc(text);
const note=comparison?(k==='native'?'Struck-through text was removed or replaced in the '+(ep?'measured':'planned')+' instruction.':'Highlighted text was added or replaced relative to the original native instruction.'):(native?'No changes from the original native instruction.':'Original native text is unavailable for comparison.');
return `
${comparison?``:''}${note}${k==='current'?' This catalog text is not the measured episode instruction.':''}
${html}
`;
}).join('')}${record&&initial?`
${record.kind==='scene_bound_goal'?'Object names are resolved for display; the measured completion requirements are preserved.':'The initial input contained a generic placeholder; this native instruction was received in the recorded session.'} Original instruction record ↗ · Session evidence ↗
Original harness verdict: invalid. Recording manifest: incomplete. This attempt is excluded from formal success metrics; its usage is retained separately.
${seedLimit?.exhausted?'No native-valid seed at or below '+number(seedLimit.max_seed):initializationBlocked?'Native seed 0 could not initialize':'No recorded episode yet'}
${seedLimit?.exhausted?'Native checks exhausted seeds 1–'+number(seedLimit.max_seed)+'. No Codex model attempt was launched. The original Kinex result is retained; this unavailable Codex task is excluded from success rates.':initializationBlocked?'No model was called. The authored seed could not initialize; a seed-selection decision is pending. This case is excluded from native success rates.':'This task is included in the planned catalog. Results and usage will appear after evaluation.'}
${m.playback_kind==='official_action_index'?'Head + left wrist + right wrist · 4× official recording, not simulation time. Wrist views update only at inference boundaries. Native 320 × 240 views are shown side by side; action-limit recordings omit the post-final-action frame.':ep.recording_mode==='native'?'Native camera recording · cached agent-resolution frames · '+(m.speed||1)+'× simulation speed.':ep.provider==='robobridge'&&m.playback_kind==='live_spectator'?'Independent spectator cameras · 720 px high per view · overview + wrist views · '+(m.speed||1)+'× simulation speed. Agent images retain native resolution and orientation.':m.playback_kind==='native_observation_snapshots'?'Native observation snapshots · 10 FPS playback; this is not simulation time.':`${m.source_fps?m.source_fps+' FPS source capture · ':''}${m.speed||1}× simulation speed.`} Session timestamps use wall time and are not synchronized to video frames.
No standalone tool or memo files were saved in this episode. The original session and workspace evidence remain available.
'}
${evidenceCard('Execution evidence','Original verdict, protocol and runtime identity.',[['verdict','Native verdict'],['protocol','Protocol'],['provenance','Source identity']],ep)}${evidenceCard('Session & interactions','Original tool calls, observations and edited workspace.',[['native_session','Native session'],['trace','Trace'],['workspace_changes','Workspace changes']],ep)}
`;
}
async function fetchText(url) {
if(fileCache.has(url))return fileCache.get(url);
const r=await fetch(url);if(!r.ok)throw Error('HTTP '+r.status);
if(Number(r.headers.get('content-length'))>3e6)throw Error('File exceeds the 3 MB preview limit; use Open file');
const text=await r.text();if(text.length>3e6)throw Error('File exceeds the preview limit; use Open file');
fileCache.set(url,text);if(fileCache.size>20)fileCache.delete(fileCache.keys().next().value);return text;
}
async function loadResource(index,token) {
const r=current.ep?.resources?.[index];if(!r)return;
current.resource=index;$('#resource-name').textContent=r.name;$('#resource-open').href=r.file;$('#resource-kind').textContent=r.kind||'Final workspace snapshot';$('#resource-preview').textContent='Loading file…';
$$('[data-resource]').forEach(b=>{b.classList.toggle('active',+b.dataset.resource===index);b.setAttribute('aria-pressed',+b.dataset.resource===index);});
try{const text=await fetchText(r.file);if(token===epoch&¤t.resource===index){$('#resource-preview').textContent=text;current.resourceText=text;}}
catch(e){if(token===epoch&¤t.resource===index){$('#resource-preview').textContent='Preview unavailable: '+e.message;current.resourceText=null;}}
}
const evidenceGroups=[
['Native evaluation','Owner verdicts, measured task and control protocol.',[['verdict','Native verdict'],['timing','Episode timing'],['protocol','Measured protocol'],['native_goal','Task instructions'],['owner_journal','Owner action journal']]],
['Session & workspace','The original agent trajectory and final files.',[['native_session','Native session'],['transcript','Readable transcript'],['trace','Agent trace'],['workspace','Workspace archive'],['workspace_changes','Workspace manifest']]],
['Recording & review','Published media and checks against the recorded evidence.',[['native_video','Original head recording'],['video','Video'],['poster','Poster'],['terminal_frame','Terminal frame'],['recording_manifest','Recording manifest'],['media_check','Media validation'],['analysis','Episode analysis']]],
['Usage & provenance','Provider counters and the actual code used by the episode.',[['usage','Token usage'],['provider_usage','Provider usage ledger'],['provenance','Source identities'],['tool_errors','Tool errors']]]
];
function evidenceCard(title,description,items,ep) {
return `
${esc(record.contract.image_preprocessing||"Model images follow the recorded checkpoint preprocessing.")} Additional information reaches the model only when present in the request and rendered language below.
${jsonBlock('Complete model-service request',c.request)}${c.model_input_text!==undefined?jsonBlock('Language passed to predict_action · recorded',c.model_input_text,true):''}${c.backbone_messages?jsonBlock('Backbone message · reconstructed from executed code',c.backbone_messages):''}
Model-service output
${response?.waypoints?'Untimed joint waypoints consumed by the recorded caller. Kinex or agent-authored code determines their physical timing.':'The recorded action response consumed by the caller.'} Joint values are absolute radians; grippers use −1 open, +1 closed.
${jsonBlock('Recorded model output · full precision',c.service_response)}${c.execution?jsonBlock('Actual execution · target count and world time',{executor:c.planner,...c.execution},true):''}${jsonBlock('Internal diagnostic · denormalized training-order actions',c.internal_diagnostics.denormalized_actions)}${caller?`
Kinex tool call & result
This inference is joined to the recorded tool call and result below. Agent-authored code may consume the model output directly; the language agent receives the displayed tool response.
${jsonBlock('Agent tool call',caller.tool_call,true)}${jsonBlock('Tool result received by the language agent',{...caller.tool_result,parts:textParts(caller.tool_result.parts)},true)}
${c.executed_tool_source?jsonBlock('Agent-authored code at this call',c.executed_tool_source):''}`:'
An exact caller-to-inference join is unavailable. Inspect the recorded Kinex session and authored code.
'}Recording coverage & model contract
See the per-run contract below for recorded and reconstructed fields. Original requests and rendered language are retained in the current rich-input run; older runs have different recording coverage.
${jsonBlock('Complete recorded StarVLA response · full precision',c.raw_response)}${jsonBlock('Reordered native actions · full precision',c.service_response.actions)}
Standalone execution; no language-agent session or tool call. Token and API costs cover the language agent only; local GPU cost is not reported.
${jsonBlock('Recording coverage & model contract',r.contract)}
${esc(c.call_id)}${c.response.inference_s.toFixed(3)} s · ${c.actions.length} × ${cols.length} actions · progress off
Native task instruction
${esc(c.request.instruction)}
${c.instruction_matches_task?'Exactly matches the original native instruction.':'The recorded model request differs from the original native instruction; this outcome is retained.'}
${images}
Measured current and agent-chosen target poses
world:base · metres · [x, y, z, qx, qy, qz, qw]
Arm
Current TCP pose
Target TCP pose
${poses}
${jsonBlock('Complete recorded model request',c.request)}
Model output
Absolute joint radians; gripper opening: 0 closed, 1 open. The device returns joint trajectories and four gripper ticks at 40 ms each. The Kinex session records their execution.