| # IMUDINO |
|
|
| Official PyTorch release of the IMUDINO IMU encoder. The checkpoint `imudino.pt` |
| contains only backbone weights. DINO/iBOT heads, the training mask token, and |
| reconstruction code have been removed. |
|
|
| ## Download the checkpoint |
|
|
| The weights are hosted on Hugging Face: [Octopus1/IMUDINO](https://huggingface.co/Octopus1/IMUDINO). |
|
|
| Download `imudino.pt` into this repository's root directory with either: |
|
|
| ```bash |
| # Option 1: huggingface-cli (pip install "huggingface_hub[cli]") |
| huggingface-cli download Octopus1/IMUDINO imudino.pt --local-dir . |
| |
| # Option 2: plain download |
| wget https://huggingface.co/Octopus1/IMUDINO/resolve/main/imudino.pt |
| ``` |
|
|
| Then follow the usage instructions below; the example scripts expect |
| `imudino.pt` next to them by default. |
|
|
| The backbone uses 16-sample patches, 12 transformer layers, 24 attention heads, |
| 768-dimensional features, and 4 register tokens. It embeds all six sensor channels |
| jointly and uses standard GELU MLPs. Frequency-aware positional encoding always |
| uses the supplied sampling rates and a fixed 100 Hz reference. The factory retains |
| its original name, `imuformer_huge_patch16`, for compatibility. |
| The implementation lives in `imudino.py`; constructing `IMUFormer()` directly |
| uses the same settings as the factory. |
|
|
| Install PyTorch and timm in a Python 3.10+ environment: |
|
|
| ```bash |
| pip install -r requirements.txt |
| python example_forward.py |
| ``` |
|
|
| Use `--device cuda` for a CUDA device or `--checkpoint /path/to/imudino.pt` for a |
| different checkpoint location. The example loads weights strictly, switches to |
| evaluation mode, and runs inference on a synthetic batch with padding. Tested |
| with Python 3.12, PyTorch 2.15.0, and timm 1.0.30. |
|
|
| To use the model in your own code: |
|
|
| ```python |
| import torch |
| from model_factory import imuformer_huge_patch16 |
| |
| model = imuformer_huge_patch16() |
| model.load_state_dict( |
| torch.load("imudino.pt", map_location="cpu", weights_only=True), strict=True |
| ) |
| model.eval() |
| |
| with torch.inference_mode(): |
| features = model({ |
| "imu_data": torch.randn(2, 1024, 6), |
| "freqs": torch.tensor([100.0, 200.0]), |
| "masks": None, |
| }) |
| ``` |
|
|
| Input `imu_data` has shape `[B, L, 6]`, with accelerometer XYZ followed by gyroscope |
| XYZ. `L` must be a positive multiple of 16. `freqs` contains one positive sampling |
| frequency in Hz per sequence. Use floating-point inputs on the same device as the |
| model. The example generates synthetic data; it does not perform sensor |
| calibration or normalization. |
|
|
| The optional `masks` tensor has shape `[B, L]`: `True`/`1` means valid and |
| `False`/`0` means padding. A patch is valid only when all 16 samples are valid. |
| Omit `masks` or pass `None` for unpadded input. Padded patches remain in the output; |
| exclude them when pooling or using patch features downstream. |
|
|
| The output contains only the final transformer layer's tokens after layer normalization: |
|
|
| | Key | Shape | Description | |
| | --- | --- | --- | |
| | `cls_token` | `[B, 768]` | Sequence representation | |
| | `reg_tokens` | `[B, 4, 768]` | Register representations | |
| | `patch_tokens` | `[B, L / 16, 768]` | Temporal patch representations | |
|
|
| For the example batch, these shapes are `(2, 768)`, `(2, 4, 768)`, and `(2, 64, 768)`. |
|
|
