---
license: gpl-3.0
library_name: pytorch
tags:
- imu
- time-series
- self-supervised-learning
- dino
- feature-extraction
- encoder
---

# IMUDINO

[![GitHub](https://img.shields.io/badge/GitHub-IMUDINO--Release-181717?logo=github&logoColor=white)](https://github.com/Cambrian-yzt/IMUDINO-Release)
[![arXiv](https://img.shields.io/badge/arXiv-2611.07437-b31b1b)](http://arxiv.org/abs/2611.07437)

Official PyTorch release of the IMUDINO IMU encoder. This repository hosts the
checkpoint `imudino.pt`; see the [GitHub repository](https://github.com/Cambrian-yzt/IMUDINO-Release)
for code, and the [paper](http://arxiv.org/abs/2611.07437) for details.

## Architecture

- 16-sample patches, 12 transformer layers, 24 attention heads, 768-dimensional features
- 4 register tokens
- All six sensor channels embedded jointly, standard GELU MLPs
- Frequency-aware positional encoding using the supplied sampling rates and a fixed 100 Hz reference
- The factory retains its original name, `imuformer_huge_patch16`, for compatibility

## Usage

Install PyTorch and timm in a Python 3.10+ environment:

```bash
pip install torch timm==1.0.30
```

Clone the code from GitHub and download `imudino.pt` into it:

```bash
huggingface-cli download Octopus1/IMUDINO imudino.pt --local-dir .
```

Then load the checkpoint:

```python
import torch
from model_factory import imuformer_huge_patch16

model = imuformer_huge_patch16()
model.load_state_dict(
    torch.load("imudino.pt", map_location="cpu", weights_only=True), strict=True
)
model.eval()

with torch.inference_mode():
    features = model({
        "imu_data": torch.randn(2, 1024, 6),
        "freqs": torch.tensor([100.0, 200.0]),
        "masks": None,
    })
```

A runnable end-to-end example (`example_forward.py`) is available in the GitHub
repository.

## Input / Output

Input `imu_data` has shape `[B, L, 6]`, with accelerometer XYZ followed by
gyroscope XYZ. `L` must be a positive multiple of 16. `freqs` contains one
positive sampling frequency in Hz per sequence. Use floating-point inputs on the
same device as the model.

The optional `masks` tensor has shape `[B, L]`: `True`/`1` means valid and
`False`/`0` means padding. A patch is valid only when all 16 samples are valid.
Omit `masks` or pass `None` for unpadded input. Padded patches remain in the
output; exclude them when pooling or using patch features downstream.

The output contains only the final transformer layer's tokens after layer
normalization:

| Key | Shape | Description |
| --- | --- | --- |
| `cls_token` | `[B, 768]` | Sequence representation |
| `reg_tokens` | `[B, 4, 768]` | Register representations |
| `patch_tokens` | `[B, L / 16, 768]` | Temporal patch representations |

For the example batch, these shapes are `(2, 768)`, `(2, 4, 768)`, and `(2, 64, 768)`.

## License

This model is released under the [GPL-3.0 license](https://www.gnu.org/licenses/gpl-3.0.en.html).