SAM-MM-HAR

SAM-MM-HAR is a lightweight multimodal Human Activity Recognition model built by AMEFORGE Lab (Amega Mike) on a proprietary sparse Transformer architecture. It classifies 40 daily activities from privacy-preserving non-RGB sensors: Depth, Skeleton, IMU, mmWave Radar, IR and Thermal.

Developed for the CUHK-X Multimodal Human Activity Challenge (co-located with UbiComp 2026).

Key specs

Property	Value
Architecture	Sparse Transformer (proprietary — AMEFORGE)
Parameters	{n_params:,} (~{n_params/1e6:.1f}M)
Size on disk	{size:.1f} MB
Classes	40 daily activities
Modalities	Depth · Skeleton · IMU · mmWave · IR · Thermal
Val accuracy	{val_acc:.1f}% (cross-subject)
Edge ready	✅ CPU inference < 100 MB

Modalities

The model handles missing modalities gracefully — any subset works at inference.

Modality	Encoder type
Depth	Patch Conv2D + sparse attention
IR / Thermal	Patch Conv2D + sparse attention
Skeleton	Joint linear + sparse attention
IMU (6-axis)	Conv1D temporal
mmWave Radar	Patch Conv2D + sparse attention

A MotionCore temporal world-model (GRU over per-frame embeddings) models human movement dynamics across frames — the key advantage over standard frame-by-frame classifiers.

Classes (40)

Wash_face · Brush_teeth · Wash_hands · Comb_hair · Put/Take_off_glasses · Put/Take_off_clothes · Put/Take_off_shoes · Drink_water · Eat · Read_book · Write · Use_phone · Use_laptop · Sit_down · Stand_up · Lie_down · Get_up · Walk · Run · Jump · Clap · Wave · Point · Throw · Kick · Pick_up · Put_down · Open/Close_door · Turn_on/off_light · Sweep_floor · Vacuum · Fall_down · Check_time · Take_body_temperature

Inference

import torch
from huggingface_hub import hf_hub_download

ckpt = hf_hub_download("AMFORGE/sam-mm-har", "best.pt")
# Load with inference.py from the repo
# python inference.py --checkpoint best.pt --clip /path/to/clip_folder

Citation

If you use SAM-MM-HAR, please cite:

@misc{{sam_mm_har,
  title  = {{SAM-MM-HAR: Multimodal Human Activity Recognition
             on Privacy-Preserving Sensors}},
  author = {{AM},
  year   = {{2026}},
  note   = {{AMEFORGE Lab. Built on a proprietary sparse Transformer architecture.}},
}}

---

Architecture internals are proprietary and not disclosed. © AMEFORGE Lab 2026 """