File size: 2,915 Bytes
7e96221 d351722 ecbc1e0 d351722 7e96221 ecbc1e0 7e96221 ecbc1e0 7e96221 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 | ---
license: gpl-3.0
library_name: pytorch
tags:
- imu
- time-series
- self-supervised-learning
- dino
- feature-extraction
- encoder
---
# IMUDINO
[](https://github.com/Cambrian-yzt/IMUDINO-Release)
[](http://arxiv.org/abs/2611.07437)
Official PyTorch release of the IMUDINO IMU encoder. This repository hosts the
checkpoint `imudino.pt`; see the [GitHub repository](https://github.com/Cambrian-yzt/IMUDINO-Release)
for code, and the [paper](http://arxiv.org/abs/2611.07437) for details.
## Architecture
- 16-sample patches, 12 transformer layers, 24 attention heads, 768-dimensional features
- 4 register tokens
- All six sensor channels embedded jointly, standard GELU MLPs
- Frequency-aware positional encoding using the supplied sampling rates and a fixed 100 Hz reference
- The factory retains its original name, `imuformer_huge_patch16`, for compatibility
## Usage
Install PyTorch and timm in a Python 3.10+ environment:
```bash
pip install torch timm==1.0.30
```
Clone the code from GitHub and download `imudino.pt` into it:
```bash
huggingface-cli download Octopus1/IMUDINO imudino.pt --local-dir .
```
Then load the checkpoint:
```python
import torch
from model_factory import imuformer_huge_patch16
model = imuformer_huge_patch16()
model.load_state_dict(
torch.load("imudino.pt", map_location="cpu", weights_only=True), strict=True
)
model.eval()
with torch.inference_mode():
features = model({
"imu_data": torch.randn(2, 1024, 6),
"freqs": torch.tensor([100.0, 200.0]),
"masks": None,
})
```
A runnable end-to-end example (`example_forward.py`) is available in the GitHub
repository.
## Input / Output
Input `imu_data` has shape `[B, L, 6]`, with accelerometer XYZ followed by
gyroscope XYZ. `L` must be a positive multiple of 16. `freqs` contains one
positive sampling frequency in Hz per sequence. Use floating-point inputs on the
same device as the model.
The optional `masks` tensor has shape `[B, L]`: `True`/`1` means valid and
`False`/`0` means padding. A patch is valid only when all 16 samples are valid.
Omit `masks` or pass `None` for unpadded input. Padded patches remain in the
output; exclude them when pooling or using patch features downstream.
The output contains only the final transformer layer's tokens after layer
normalization:
| Key | Shape | Description |
| --- | --- | --- |
| `cls_token` | `[B, 768]` | Sequence representation |
| `reg_tokens` | `[B, 4, 768]` | Register representations |
| `patch_tokens` | `[B, L / 16, 768]` | Temporal patch representations |
For the example batch, these shapes are `(2, 768)`, `(2, 4, 768)`, and `(2, 64, 768)`.
## License
This model is released under the [GPL-3.0 license](https://www.gnu.org/licenses/gpl-3.0.en.html).
|