IMUDINO / README.md

Add GitHub and paper badges to model card

d351722 verified 1 day ago

2.92 kB

license: gpl-3.0
library_name: pytorch
tags:
  - imu
  - time-series
  - self-supervised-learning
  - dino
  - feature-extraction
  - encoder

IMUDINO

Official PyTorch release of the IMUDINO IMU encoder. This repository hosts the checkpoint imudino.pt; see the GitHub repository for code, and the paper for details.

Architecture

16-sample patches, 12 transformer layers, 24 attention heads, 768-dimensional features
4 register tokens
All six sensor channels embedded jointly, standard GELU MLPs
Frequency-aware positional encoding using the supplied sampling rates and a fixed 100 Hz reference
The factory retains its original name, imuformer_huge_patch16, for compatibility

Usage

Install PyTorch and timm in a Python 3.10+ environment:

pip install torch timm==1.0.30

Clone the code from GitHub and download imudino.pt into it:

huggingface-cli download Octopus1/IMUDINO imudino.pt --local-dir .

Then load the checkpoint:

import torch
from model_factory import imuformer_huge_patch16

model = imuformer_huge_patch16()
model.load_state_dict(
    torch.load("imudino.pt", map_location="cpu", weights_only=True), strict=True
)
model.eval()

with torch.inference_mode():
    features = model({
        "imu_data": torch.randn(2, 1024, 6),
        "freqs": torch.tensor([100.0, 200.0]),
        "masks": None,
    })

A runnable end-to-end example (example_forward.py) is available in the GitHub repository.

Input / Output

Input imu_data has shape [B, L, 6], with accelerometer XYZ followed by gyroscope XYZ. L must be a positive multiple of 16. freqs contains one positive sampling frequency in Hz per sequence. Use floating-point inputs on the same device as the model.

The optional masks tensor has shape [B, L]: True/1 means valid and False/0 means padding. A patch is valid only when all 16 samples are valid. Omit masks or pass None for unpadded input. Padded patches remain in the output; exclude them when pooling or using patch features downstream.

The output contains only the final transformer layer's tokens after layer normalization:

Key	Shape	Description
`cls_token`	`[B, 768]`	Sequence representation
`reg_tokens`	`[B, 4, 768]`	Register representations
`patch_tokens`	`[B, L / 16, 768]`	Temporal patch representations

For the example batch, these shapes are (2, 768), (2, 4, 768), and (2, 64, 768).

License

This model is released under the GPL-3.0 license.