Update to post-retrain hp-consistent 5-condition checkpoints; minimal README
Browse files- README.md +64 -6
- foveated/ckpt.36.pth → blind/ckpt.15.pth +2 -2
- coarse/ckpt.10.pth +2 -2
- coarse/ckpt.20.pth +2 -2
- coarse/ckpt.30.pth +2 -2
- coarse/ckpt.40.pth +2 -2
- coarse/ckpt.49.pth +2 -2
- foveated/ckpt.10.pth +2 -2
- foveated/ckpt.20.pth +2 -2
- foveated/ckpt.30.pth +2 -2
- foveated/ckpt.40.pth +3 -0
- foveated/ckpt.49.pth +3 -0
- foveated_logpolar/ckpt.10.pth +3 -0
- foveated_logpolar/ckpt.20.pth +3 -0
- foveated_logpolar/ckpt.30.pth +3 -0
- foveated_logpolar/ckpt.40.pth +3 -0
- foveated_logpolar/ckpt.49.pth +3 -0
- uniform/ckpt.10.pth +2 -2
- uniform/ckpt.20.pth +2 -2
- uniform/ckpt.30.pth +2 -2
- uniform/ckpt.40.pth +2 -2
- uniform/ckpt.49.pth +2 -2
README.md
CHANGED
|
@@ -1,11 +1,69 @@
|
|
| 1 |
-
-
|
| 2 |
-
license: cc-by-4.0
|
| 3 |
-
---
|
| 4 |
|
| 5 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
|
| 7 |
```python
|
| 8 |
import torch
|
| 9 |
-
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
```
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Spatial-memory checkpoints (5 visual conditions)
|
|
|
|
|
|
|
| 2 |
|
| 3 |
+
Frozen post-training DD-PPO PointNav agents on Habitat for five visual sensor
|
| 4 |
+
conditions on a shared ResNet-18 + 3-layer LSTM (512-d) backbone. Hidden
|
| 5 |
+
state `h_2` (top LSTM layer) is the canonical 512-d cognitive-map readout.
|
| 6 |
+
|
| 7 |
+
| folder | encoder | final ckpt |
|
| 8 |
+
| ------------------- | ------------------------------------------------------ | ---------- |
|
| 9 |
+
| `blind/` | no visual encoder | `ckpt.34.pth` |
|
| 10 |
+
| `coarse/` | 48 x 48 RGB, 1 x 1 encoder feature map | `ckpt.49.pth` |
|
| 11 |
+
| `foveated/` | 256 x 256 RGB, eccentricity Gaussian blur, 4 x 4 map | `ckpt.49.pth` |
|
| 12 |
+
| `foveated_logpolar/`| 64 x 64 log-polar resampled, ~2 x 2 map | `ckpt.49.pth` |
|
| 13 |
+
| `uniform/` | 256 x 256 RGB, no blur, 4 x 4 map | `ckpt.49.pth` |
|
| 14 |
+
|
| 15 |
+
Intermediate ckpts (training-trajectory snapshots) are also included:
|
| 16 |
+
sighted at `ckpt.{10,20,30,40,49}.pth`; blind at
|
| 17 |
+
`ckpt.{5,10,15,20,25,30,34}.pth`.
|
| 18 |
+
|
| 19 |
+
## Load a checkpoint
|
| 20 |
|
| 21 |
```python
|
| 22 |
import torch
|
| 23 |
+
from huggingface_hub import hf_hub_download
|
| 24 |
+
|
| 25 |
+
cond = "foveated" # or: blind | coarse | uniform | foveated_logpolar
|
| 26 |
+
ckpt_path = hf_hub_download(
|
| 27 |
+
repo_id="alunxu/spatial-memory-checkpoints",
|
| 28 |
+
filename=f"{cond}/ckpt.49.pth",
|
| 29 |
+
)
|
| 30 |
+
|
| 31 |
+
ckpt = torch.load(ckpt_path, map_location="cpu", weights_only=False)
|
| 32 |
+
state_dict = ckpt["state_dict"] # actor-critic policy weights
|
| 33 |
+
config = ckpt["config"] # habitat-baselines OmegaConf
|
| 34 |
+
```
|
| 35 |
+
|
| 36 |
+
Each `.pth` is a habitat-baselines checkpoint with keys `state_dict`,
|
| 37 |
+
`config`, and `extra_state` (training step counters).
|
| 38 |
+
|
| 39 |
+
## Rebuild the policy and read `h_2`
|
| 40 |
+
|
| 41 |
+
```python
|
| 42 |
+
from habitat_baselines.common.baseline_registry import baseline_registry
|
| 43 |
+
from habitat_baselines.utils.common import get_action_space_info
|
| 44 |
+
from habitat_baselines.config.default import get_config
|
| 45 |
+
from habitat.config.default_structured_configs import HabitatConfigPlugin
|
| 46 |
+
|
| 47 |
+
# 1. Build the same env the policy was trained on (for obs/action spaces).
|
| 48 |
+
env_config = config.habitat # already inside ckpt
|
| 49 |
+
# ... construct the eval env from env_config (see code repo) ...
|
| 50 |
+
|
| 51 |
+
# 2. Instantiate the policy class registered for this config and load weights.
|
| 52 |
+
policy_cls = baseline_registry.get_policy(
|
| 53 |
+
config.habitat_baselines.rl.policy.name)
|
| 54 |
+
policy = policy_cls.from_config(
|
| 55 |
+
config=config,
|
| 56 |
+
observation_space=env.observation_space,
|
| 57 |
+
action_space=env.action_space,
|
| 58 |
+
)
|
| 59 |
+
policy.load_state_dict(state_dict)
|
| 60 |
+
policy.eval()
|
| 61 |
+
|
| 62 |
+
# 3. Run a rollout. The recurrent hidden state has shape
|
| 63 |
+
# (num_envs, num_layers=3, hidden=512). h_2 is the top layer:
|
| 64 |
+
# h_2 = recurrent_hidden_states[:, 2, :]
|
| 65 |
+
# Pass `recurrent_hidden_states` back into `policy.act(...)` each step.
|
| 66 |
```
|
| 67 |
+
|
| 68 |
+
Code, configs, and the deterministic-rollout probing pipeline that produced
|
| 69 |
+
this release: <https://github.com/alunxu/foveated-cog-map>.
|
foveated/ckpt.36.pth → blind/ckpt.15.pth
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:753d2653cedc480dda74f71f505844426f1225e58a8aa8c45c6e36c01e54a2f1
|
| 3 |
+
size 22413164
|
coarse/ckpt.10.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6b7a809a51fe23fb7508cd1fbf3734b45f1954cd08345750922cfe777bf70a0c
|
| 3 |
+
size 60926368
|
coarse/ckpt.20.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f7b9ede10b865d63a06aec24af384a5b06f8589350e49cae05087d9a90c7eddb
|
| 3 |
+
size 60926368
|
coarse/ckpt.30.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:df0ae6d2b0f16368b30212740dbac9c5086759126e1e1d90e8e1ec7da5d1877e
|
| 3 |
+
size 60926368
|
coarse/ckpt.40.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:28a2adf4c982a7ec732d4602552210ceaa224b41819c678ac91f4aa320302827
|
| 3 |
+
size 60926368
|
coarse/ckpt.49.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:99d459395cf3415690f9f1eba76f6cc77d08f49e679f1f05aab41ef270e52af4
|
| 3 |
+
size 60926368
|
foveated/ckpt.10.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:59adbc90d5fe7a72bb6a7847418afd50373e8148f9f296c1896df4a44d2790c4
|
| 3 |
+
size 43349076
|
foveated/ckpt.20.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7d555d5dbc4d0b85375a4e34ae2e50f7ef9da23d7d74be4647923ad2afe056cf
|
| 3 |
+
size 43349076
|
foveated/ckpt.30.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:38b45c99f6a2b418fa7cde92199d7e38784f76bc0f8b369943f3844ae86396f1
|
| 3 |
+
size 43349076
|
foveated/ckpt.40.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8482714097ae138634b66f9750d9eb243328bf0b9ffec4694ef2577cbf722c07
|
| 3 |
+
size 43349076
|
foveated/ckpt.49.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3487e0f463b1b2e27f08a1a2c2f7c70b4bfa7e37faacf8c7ac7348144ccd055c
|
| 3 |
+
size 43349076
|
foveated_logpolar/ckpt.10.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:074709eaeafeafbdb71fa95cbc3de910151d0007490f7594a02bb794153326b3
|
| 3 |
+
size 40103904
|
foveated_logpolar/ckpt.20.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1fc92f69d1cc0d2aea7d918b89524b35cd374b63e8b29f4265e6847f9a6a97f9
|
| 3 |
+
size 40103904
|
foveated_logpolar/ckpt.30.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5862e039c3c2f8dfef6431c5a95f71be6ef0eb24ca7b74bbfb63a657511be9d1
|
| 3 |
+
size 40103904
|
foveated_logpolar/ckpt.40.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:46a04a58966699af80bc92472822744654f835db8af9f802c7b34c3d2347ab7a
|
| 3 |
+
size 40103904
|
foveated_logpolar/ckpt.49.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dfed11bf6a2483e5ecdb8ec41a6ac47068175291c12e117712eaa3ce0c9fbe38
|
| 3 |
+
size 40103904
|
uniform/ckpt.10.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e1d078c09f677c1cb5ff7a32807a51796262ba154729dfa85df676c662efda29
|
| 3 |
+
size 43216288
|
uniform/ckpt.20.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:685d555f3b2fb3404b01d9f981c1c173a49b090df0c2124633764658a68cbb33
|
| 3 |
+
size 43216288
|
uniform/ckpt.30.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b8e02ecd308ec308e79579aa84b4b65076f3c1731869368fa6fd1fa168f0f571
|
| 3 |
+
size 43216288
|
uniform/ckpt.40.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f9264bdfa824134de30c9a0cdd85e421991f8927b4b1655685ed051474b8d31b
|
| 3 |
+
size 43216288
|
uniform/ckpt.49.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ef4408b5ccbacbce1b0b8f1852de978988c88c71c079ea294858630e9558b1e1
|
| 3 |
+
size 43216288
|