Update README.md
Browse files
README.md
CHANGED
|
@@ -57,8 +57,8 @@ Sigma can be seen as **π0.5 + telepathic head + LoRA adapters**:
|
|
| 57 |
- **Language–semantic stream**
|
| 58 |
- take text tokens, vision tokens, and state tokens into a shared MLLM backbone;
|
| 59 |
- derive:
|
| 60 |
-
- a **semantic memory**
|
| 61 |
-
- an **intent vector**
|
| 62 |
- pooled **semantic factors** aligned with the text embedding space.
|
| 63 |
|
| 64 |
- **Action stream (three branches)**
|
|
@@ -112,7 +112,7 @@ python train_sigma_telepathy_vla_lora.py \
|
|
| 112 |
Key aspects:
|
| 113 |
|
| 114 |
- freeze backbone weights from `lerobot/pi05_base`;
|
| 115 |
-
- attach **LoRA** on key projections (
|
| 116 |
- jointly optimize:
|
| 117 |
- **three control losses**:
|
| 118 |
- `L_act_vec` for per-step action vectors,
|
|
@@ -156,7 +156,7 @@ A lightweight adapter (`sigma_adapter.py`) controls how much the telepathy resid
|
|
| 156 |
- baseline π0.5 actions (`base_action_vector`, …),
|
| 157 |
- Sigma residuals,
|
| 158 |
- telepathy diagnostics (norms, cosine alignments),
|
| 159 |
-
- computes a **risk-aware scaling factor** in
|
| 160 |
- blends:
|
| 161 |
|
| 162 |
```python
|
|
|
|
| 57 |
- **Language–semantic stream**
|
| 58 |
- take text tokens, vision tokens, and state tokens into a shared MLLM backbone;
|
| 59 |
- derive:
|
| 60 |
+
- a **semantic memory** m_t that accumulates cross-time information,
|
| 61 |
+
- an **intent vector** z_intent,
|
| 62 |
- pooled **semantic factors** aligned with the text embedding space.
|
| 63 |
|
| 64 |
- **Action stream (three branches)**
|
|
|
|
| 112 |
Key aspects:
|
| 113 |
|
| 114 |
- freeze backbone weights from `lerobot/pi05_base`;
|
| 115 |
+
- attach **LoRA** on key projections (q, k, v, o) and the telepathy heads;
|
| 116 |
- jointly optimize:
|
| 117 |
- **three control losses**:
|
| 118 |
- `L_act_vec` for per-step action vectors,
|
|
|
|
| 156 |
- baseline π0.5 actions (`base_action_vector`, …),
|
| 157 |
- Sigma residuals,
|
| 158 |
- telepathy diagnostics (norms, cosine alignments),
|
| 159 |
+
- computes a **risk-aware scaling factor** in min_scale, max_scale,
|
| 160 |
- blends:
|
| 161 |
|
| 162 |
```python
|