| --- |
| license: gpl-3.0 |
| library_name: pytorch |
| tags: |
| - imu |
| - time-series |
| - self-supervised-learning |
| - dino |
| - feature-extraction |
| - encoder |
| --- |
| |
| # IMUDINO |
|
|
| [](https://github.com/Cambrian-yzt/IMUDINO-Release) |
| [](http://arxiv.org/abs/2611.07437) |
|
|
| Official PyTorch release of the IMUDINO IMU encoder. This repository hosts the |
| checkpoint `imudino.pt`; see the [GitHub repository](https://github.com/Cambrian-yzt/IMUDINO-Release) |
| for code, and the [paper](http://arxiv.org/abs/2611.07437) for details. |
|
|
| ## Architecture |
|
|
| - 16-sample patches, 12 transformer layers, 24 attention heads, 768-dimensional features |
| - 4 register tokens |
| - All six sensor channels embedded jointly, standard GELU MLPs |
| - Frequency-aware positional encoding using the supplied sampling rates and a fixed 100 Hz reference |
| - The factory retains its original name, `imuformer_huge_patch16`, for compatibility |
|
|
| ## Usage |
|
|
| Install PyTorch and timm in a Python 3.10+ environment: |
|
|
| ```bash |
| pip install torch timm==1.0.30 |
| ``` |
|
|
| Clone the code from GitHub and download `imudino.pt` into it: |
|
|
| ```bash |
| huggingface-cli download Octopus1/IMUDINO imudino.pt --local-dir . |
| ``` |
|
|
| Then load the checkpoint: |
|
|
| ```python |
| import torch |
| from model_factory import imuformer_huge_patch16 |
| |
| model = imuformer_huge_patch16() |
| model.load_state_dict( |
| torch.load("imudino.pt", map_location="cpu", weights_only=True), strict=True |
| ) |
| model.eval() |
| |
| with torch.inference_mode(): |
| features = model({ |
| "imu_data": torch.randn(2, 1024, 6), |
| "freqs": torch.tensor([100.0, 200.0]), |
| "masks": None, |
| }) |
| ``` |
|
|
| A runnable end-to-end example (`example_forward.py`) is available in the GitHub |
| repository. |
|
|
| ## Input / Output |
|
|
| Input `imu_data` has shape `[B, L, 6]`, with accelerometer XYZ followed by |
| gyroscope XYZ. `L` must be a positive multiple of 16. `freqs` contains one |
| positive sampling frequency in Hz per sequence. Use floating-point inputs on the |
| same device as the model. |
|
|
| The optional `masks` tensor has shape `[B, L]`: `True`/`1` means valid and |
| `False`/`0` means padding. A patch is valid only when all 16 samples are valid. |
| Omit `masks` or pass `None` for unpadded input. Padded patches remain in the |
| output; exclude them when pooling or using patch features downstream. |
|
|
| The output contains only the final transformer layer's tokens after layer |
| normalization: |
|
|
| | Key | Shape | Description | |
| | --- | --- | --- | |
| | `cls_token` | `[B, 768]` | Sequence representation | |
| | `reg_tokens` | `[B, 4, 768]` | Register representations | |
| | `patch_tokens` | `[B, L / 16, 768]` | Temporal patch representations | |
|
|
| For the example batch, these shapes are `(2, 768)`, `(2, 4, 768)`, and `(2, 64, 768)`. |
|
|
| ## License |
|
|
| This model is released under the [GPL-3.0 license](https://www.gnu.org/licenses/gpl-3.0.en.html). |
|
|
