README.md

---
license: mit
tags:
- robotics
- libero
- oat
- dense-visual-memory
- imitation-learning
---

# OAT Dense LIBERO-10 — Checkpoint Epoch 950

Hugging Face model repository for a **dense cross-attention OAT policy** trained on
**LIBERO-10 (N500)**. Snapshot at **epoch 950** during run `oat_dense_with_uid_long_0530_220204`.

This checkpoint is the best point on the training ladder after resume from ep-0700
(in-loop peak **52.67%**). **Phase B confirm: 50.60% ± 0.76%** on LIBERO-10.

## Quick download

```bash
huggingface-cli download hackhackhack66666/oat-dense-blt-950 ep-0950.ckpt \
  --local-dir ./checkpoints
```

## Files

| File | Description |
|------|-------------|
| `ep-0950.ckpt` | PyTorch workspace checkpoint (~729 MB) |
| `training_logs.jsonl` | Full training JSONL (train/val curves) |
| `training_metrics_dashboard.png` | Training loss / val / reconst MSE dashboard |
| `training_snapshot_ep0950.json` | Offline metrics at epoch 950 |
| `sim_eval/` | Phase A screen eval (30 ep/task) |
| `sim_eval_phase_b/` | Phase B confirm eval (50 ep/task, 3 seeds) |
| `experiment_log_dense_visual_memory.md` | Experiment journal |

## Metrics @ epoch 950

| Train loss | Val loss | Reconst MSE | Phase A SR | Phase B SR (confirm) |
|-----------:|---------:|------------:|-----------:|---------------------:|
| 2.0182993412017822 | 5.783568859100342 | 0.07388626039028168 | **52.7%** | **50.60%** ± 0.76% |

In-loop eval at ep 950 (same run, 30 ep/task): **52.67%** — noisier than Phase B; use Phase B as the calibrated reference.

## Visualizations

### Training (offline)

![Training metrics dashboard](training_metrics_dashboard.png)

### Sim eval — Phase A screen (30 ep/task)

**Mean success rate: 52.7%** — 30 episodes/task, 300 total rollouts, seed 1000 (single experiment).

![Phase A sim eval dashboard](sim_eval/sim_eval_dashboard.png)

Details: [`sim_eval/eval_summary.md`](sim_eval/eval_summary.md) · [`sim_eval/eval_log.json`](sim_eval/eval_log.json)


| Task | SR |
|------|-----|
| `KITCHEN_SCENE6_put_the_yellow_and_white_mug_in_the_microwave…` | 86.7% |
| `KITCHEN_SCENE4_put_the_black_bowl_in_the_bottom_drawer_of_th…` | 86.7% |
| `KITCHEN_SCENE3_turn_on_the_stove_and_put_the_moka_pot_on_it` | 83.3% |
| `LIVING_ROOM_SCENE2_put_both_the_cream_cheese_box_and_the_but…` | 73.3% |
| `STUDY_SCENE1_pick_up_the_book_and_place_it_in_the_back_compa…` | 66.7% |
| `LIVING_ROOM_SCENE5_put_the_white_mug_on_the_left_plate_and_p…` | 53.3% |
| `KITCHEN_SCENE8_put_both_moka_pots_on_the_stove` | 53.3% |
| `LIVING_ROOM_SCENE6_put_the_white_mug_on_the_plate_and_put_th…` | 20.0% |
| `LIVING_ROOM_SCENE2_put_both_the_alphabet_soup_and_the_tomato…` | 3.3% |
| `LIVING_ROOM_SCENE1_put_both_the_alphabet_soup_and_the_cream_…` | 0.0% |


### Sim eval — Phase B confirm (50 ep/task, 3 seeds)

**Mean success rate: 50.60% ± 0.76%** — official-style protocol for comparison with OAT paper (~56.3%).

Seeds 1000 / 1500 / 2000 (`seed_stride=500`), 500 rollouts total. Per-seed mean SR: **52.0%** · **49.4%** · **50.4%**.

Compared to ep-0700 Phase B confirm (**47.60% ± 1.75%**): **+3.0 pp** on the calibrated protocol.

![Phase B confirm dashboard](sim_eval_phase_b/phase_b_confirm_pt50_ep-0950_dashboard.png)

Details: [`sim_eval_phase_b/eval_summary.md`](sim_eval_phase_b/eval_summary.md) · [`sim_eval_phase_b/eval_log.json`](sim_eval_phase_b/eval_log.json)


| Task | SR |
|------|-----|
| `KITCHEN_SCENE4_put_the_black_bowl_in_the_bottom_drawer_of_th…` | 94.7% |
| `KITCHEN_SCENE3_turn_on_the_stove_and_put_the_moka_pot_on_it` | 88.7% |
| `KITCHEN_SCENE6_put_the_yellow_and_white_mug_in_the_microwave…` | 74.0% |
| `STUDY_SCENE1_pick_up_the_book_and_place_it_in_the_back_compa…` | 66.0% |
| `LIVING_ROOM_SCENE2_put_both_the_cream_cheese_box_and_the_but…` | 56.7% |
| `LIVING_ROOM_SCENE5_put_the_white_mug_on_the_left_plate_and_p…` | 48.0% |
| `KITCHEN_SCENE8_put_both_moka_pots_on_the_stove` | 43.3% |
| `LIVING_ROOM_SCENE6_put_the_white_mug_on_the_plate_and_put_th…` | 34.0% |
| `LIVING_ROOM_SCENE2_put_both_the_alphabet_soup_and_the_tomato…` | 0.7% |
| `LIVING_ROOM_SCENE1_put_both_the_alphabet_soup_and_the_cream_…` | 0.0% |

## Ladder context

| Epoch | Phase A SR | Phase B confirm | HF repo |
|------:|-----------:|----------------:|---------|
| 300 | 39.7% | — | [OAT-BLT-LIBERO-300](https://huggingface.co/hackhackhack66666/OAT-BLT-LIBERO-300) |
| 500 | 38.0% | — | [OAT-BLT-LIBERO-500](https://huggingface.co/hackhackhack66666/OAT-BLT-LIBERO-500) |
| 700 | 51.7% | 47.60% ± 1.75% | [OAT-BLT-Libero-700](https://huggingface.co/hackhackhack66666/OAT-BLT-Libero-700) |
| **950** | **52.7%** | **50.60% ± 0.76%** | **this repo** |

Paper **OAT8** on LIBERO-10: **~56.3%** mean success rate (external reference).

## Model configuration

- **Policy:** OAT with `use_dense_visual_memory=true` (spatial visual tokens + cross-attn)
- **State memory:** enabled (`use_state_memory_tokens=true`)
- **Task UID:** enabled in state tokens
- **Dataset:** [`libero10_N500.zarr`](https://huggingface.co/datasets/chaoqi-liu/libero10_N500.zarr)
- **Embed dim:** 256
- **Tokenizer:** `tokenizer_ep-0950_mse-0.002.ckpt` (Mirageinv/oat)

## Citation

If you use this checkpoint, please cite **OAT: Ordered Action Tokenization** and specify
epoch **950** of the dense LIBERO-10 ladder.

Source: [GadzhiAskhabaliev/OAT-BLT-Dense](https://github.com/GadzhiAskhabaliev/OAT-BLT-Dense) (`BLT-OAT-dense` branch).