# m1_mix — Evaluation Summary

A single Mem-0 execution-module checkpoint (`checkpoint/m1_mix_final_step50000.pt`),
trained jointly on all five RMBench **M1** tasks (`m1_mix` dataset), evaluated on each
task in turn. Same checkpoint and same `m1_mix` normalization stats are used for every
task — only `--task_name` and the per-task `--global_task` instruction change.

## Results (task_config = `demo_clean`, instruction_type = `unseen`, 100 episodes/task)

| Task                | Success Rate | Reward | Eval timestamp        |
|---------------------|:------------:|:------:|-----------------------|
| put_back_block      | **1.00**     | 1.00   | 2026-06-22 20:00:54   |
| rearrange_blocks    | **0.86**     | 0.86   | 2026-06-22 09:35:33   |
| swap_blocks         | **0.81**     | 0.81   | 2026-06-22 09:34:12   |
| swap_T              | **0.13**     | 0.13   | 2026-06-22 20:01:49   |
| observe_and_pickup  | **0.03**     | 0.00   | 2026-06-23 05:03:56   |
| **Average**         | **0.566**    | —      |                       |

Block-manipulation tasks (put_back / rearrange / swap_blocks) are strong. The two weak
tasks are **swap_T** (fine T-block pose alignment — both position and orientation) and
**observe_and_pickup** (cross-view target identification after occlusion, then pickup).

## Provenance — identical across all five evaluations

- **Checkpoint:** `checkpoints/m1_mix/final_step50000.pt` (global_step 50000)
- **Normalization stats:** `policy/Mem-0/assets/m1_mix/norm_stats.json` (min-max → [-1, 1])
- **Action horizon:** 30
- **Entry point:** `script/eval_policy.py --config policy/Mem-0/deploy_policy.yml --overrides ...`
- **vLLM / planner:** not used (M1 tasks set a single global instruction directly)

The exact `--global_task` string for each task is in `../task_instructions.json` and is
reproduced verbatim (including the original `traies` spelling in `swap_blocks`).

## Per-task artifacts

Each `eval_results/<task>/` folder contains:

- `_result.txt` — final success rate and reward
- `eval_log.txt` — per-episode `episode_id, seed, result=Success/Fail` (seeds start at 100000)
- `episode<N>.mp4` — head-camera rollout video for all 100 evaluated episodes

> Note: the evaluation loop counts an episode only after it passes the simulator's
> expert-feasibility check, so the 100 episodes correspond to seeds drawn from 100000
> upward until 100 feasible inits are collected.