README.git.md

IMUDINO

Official PyTorch release of the IMUDINO IMU encoder. The checkpoint imudino.pt contains only backbone weights. DINO/iBOT heads, the training mask token, and reconstruction code have been removed.

Download the checkpoint

The weights are hosted on Hugging Face: Octopus1/IMUDINO.

Download imudino.pt into this repository's root directory with either:

# Option 1: huggingface-cli (pip install "huggingface_hub[cli]")
huggingface-cli download Octopus1/IMUDINO imudino.pt --local-dir .

# Option 2: plain download
wget https://huggingface.co/Octopus1/IMUDINO/resolve/main/imudino.pt

Then follow the usage instructions below; the example scripts expect imudino.pt next to them by default.

The backbone uses 16-sample patches, 12 transformer layers, 24 attention heads, 768-dimensional features, and 4 register tokens. It embeds all six sensor channels jointly and uses standard GELU MLPs. Frequency-aware positional encoding always uses the supplied sampling rates and a fixed 100 Hz reference. The factory retains its original name, imuformer_huge_patch16, for compatibility. The implementation lives in imudino.py; constructing IMUFormer() directly uses the same settings as the factory.

Install PyTorch and timm in a Python 3.10+ environment:

pip install -r requirements.txt
python example_forward.py

Use --device cuda for a CUDA device or --checkpoint /path/to/imudino.pt for a different checkpoint location. The example loads weights strictly, switches to evaluation mode, and runs inference on a synthetic batch with padding. Tested with Python 3.12, PyTorch 2.15.0, and timm 1.0.30.

To use the model in your own code:

import torch
from model_factory import imuformer_huge_patch16

model = imuformer_huge_patch16()
model.load_state_dict(
    torch.load("imudino.pt", map_location="cpu", weights_only=True), strict=True
)
model.eval()

with torch.inference_mode():
    features = model({
        "imu_data": torch.randn(2, 1024, 6),
        "freqs": torch.tensor([100.0, 200.0]),
        "masks": None,
    })

Input imu_data has shape [B, L, 6], with accelerometer XYZ followed by gyroscope XYZ. L must be a positive multiple of 16. freqs contains one positive sampling frequency in Hz per sequence. Use floating-point inputs on the same device as the model. The example generates synthetic data; it does not perform sensor calibration or normalization.

The optional masks tensor has shape [B, L]: True/1 means valid and False/0 means padding. A patch is valid only when all 16 samples are valid. Omit masks or pass None for unpadded input. Padded patches remain in the output; exclude them when pooling or using patch features downstream.

The output contains only the final transformer layer's tokens after layer normalization:

Key	Shape	Description
cls_token	[B, 768]	Sequence representation
reg_tokens	[B, 4, 768]	Register representations
patch_tokens	[B, L / 16, 768]	Temporal patch representations

For the example batch, these shapes are (2, 768), (2, 4, 768), and (2, 64, 768).