English
tinymyo
emg
bio-signals
foundation-model
MatteoFasulo commited on
Commit
a24bc5a
·
unverified ·
1 Parent(s): d9b6f8e

refactor: remove unused safetensors files and add new configurations for EMG models

Browse files
DB5/DB5_finetune_5sec.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3f155360b53600d266b12f5aff473094971a551c3f1e828e88dde83a0cd076a
3
+ size 14927820
DB5/config.json ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_target_": "models.TinyMyo.TinyMyo",
3
+ "img_size": 1000,
4
+ "patch_size": 20,
5
+ "in_chans": 16,
6
+ "embed_dim": 192,
7
+ "n_layer": 8,
8
+ "n_head": 3,
9
+ "mlp_ratio": 4,
10
+ "qkv_bias": true,
11
+ "attn_drop": 0.1,
12
+ "proj_drop": 0.1,
13
+ "drop_path": 0.1,
14
+ "num_classes": 53,
15
+ "task": "classification"
16
+ }
DB8/DB8_finetune_500ms.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4ee6743c584e3e1d0791be95ab9beb5b3937d20484551c22fc249b51f7dcc4b
3
+ size 17432748
DB8/config.json ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_target_": "models.TinyMyo.TinyMyo",
3
+ "img_size": 1000,
4
+ "patch_size": 20,
5
+ "in_chans": 16,
6
+ "embed_dim": 192,
7
+ "n_layer": 8,
8
+ "n_head": 3,
9
+ "mlp_ratio": 4,
10
+ "qkv_bias": true,
11
+ "attn_drop": 0.1,
12
+ "proj_drop": 0.1,
13
+ "drop_path": 0.1,
14
+ "num_classes": 5,
15
+ "task": "regression"
16
+ }
EPN612/EPN_finetune_5sec.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:07593aa6579a99e3c9035db75fe697f7ce133bc9c3aa1ba64935a646c7b6a349
3
+ size 14313224
EPN612/config.json ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_target_": "models.TinyMyo.TinyMyo",
3
+ "img_size": 1000,
4
+ "patch_size": 20,
5
+ "in_chans": 8,
6
+ "embed_dim": 192,
7
+ "n_layer": 8,
8
+ "n_head": 3,
9
+ "mlp_ratio": 4,
10
+ "qkv_bias": true,
11
+ "attn_drop": 0.1,
12
+ "proj_drop": 0.1,
13
+ "drop_path": 0.1,
14
+ "num_classes": 6,
15
+ "task": "classification"
16
+ }
README.md CHANGED
@@ -1,170 +1,62 @@
1
- ---
2
- license: cc-by-nd-4.0
3
- language:
4
- - en
5
- model-index:
6
- - name: TinyMyo
7
- results:
8
- - task:
9
- type: gesture-classification
10
- dataset:
11
- type: ninapro_db5
12
- name: Ninapro DB5
13
- metrics:
14
- - name: acc@1
15
- type: acc@1
16
- value: 0.8941
17
- verified: false
18
- - name: f1
19
- type: f1
20
- value: 0.7797
21
- verified: false
22
- - task:
23
- type: gesture-classification
24
- dataset:
25
- type: epn612
26
- name: EPN-612
27
- metrics:
28
- - name: acc@1
29
- type: acc@1
30
- value: 0.9674
31
- verified: false
32
- - name: f1
33
- type: f1
34
- value: 0.9674
35
- verified: false
36
- - task:
37
- type: gesture-classification
38
- dataset:
39
- type: uci_emg
40
- name: UCI-EMG
41
- metrics:
42
- - name: acc@1
43
- type: acc@1
44
- value: 0.9756
45
- verified: false
46
- - name: f1
47
- type: f1
48
- value: 0.9755
49
- verified: false
50
- - task:
51
- type: gesture-classification
52
- dataset:
53
- type: gni_meta_rl
54
- name: Generic Neuromotor Interface (Discrete Gesture)
55
- metrics:
56
- - name: CLER
57
- type: classification-error-rate
58
- value: 0.153
59
- verified: false
60
- - task:
61
- type: kinematic-regression
62
- dataset:
63
- type: ninapro_db8
64
- name: Ninapro DB8
65
- metrics:
66
- - name: MAE
67
- type: mean-absolute-error
68
- value: 8.77
69
- verified: false
70
- - name: RMSE
71
- type: root-mean-square-error
72
- value: 13.35
73
- verified: false
74
- - name: R2
75
- type: r2
76
- value: 0.62
77
- verified: false
78
- - task:
79
- type: speech-synthesis
80
- dataset:
81
- type: gaddy_silent_speech
82
- name: Gaddy Silent Speech (MFCC to Audio)
83
- metrics:
84
- - name: WER
85
- type: word-error-rate
86
- value: 0.3354
87
- verified: false
88
- - task:
89
- type: speech-recognition
90
- dataset:
91
- type: gaddy_silent_speech
92
- name: Gaddy Silent Speech (EMG to Text)
93
- metrics:
94
- - name: WER
95
- type: word-error-rate
96
- value: 0.3395
97
- verified: false
98
- tags:
99
- - emg
100
- - bio-signals
101
- - foundation-model
102
- ---
103
 
104
  <div align="center">
105
  <img src="https://raw.githubusercontent.com/MatteoFasulo/BioFoundation/refs/heads/TinyMyo/docs/model/logo/TinyMyo_logo.png" alt="TinyMyo Logo" width="400" />
106
- <h1>TinyMyo: a Tiny Foundation Model for Flexible EMG Signal Processing at the Edge</h1>
107
  </div>
 
108
  <p align="center">
109
  <a href="https://github.com/pulp-bio/BioFoundation"><img src ="https://img.shields.io/github/stars/pulp-bio/BioFoundation?color=ccf" alt="Github"></a>
110
  <a href="https://creativecommons.org/licenses/by-nd/4.0/"><img src="https://img.shields.io/badge/License-CC_BY--ND_4.0-lightgrey.svg" alt="License"></a>
111
  <a href="https://arxiv.org/abs/2512.15729"><img src="https://img.shields.io/badge/arXiv-2512.15729-b31b1b.svg" alt="Paper"></a>
112
  </p>
113
 
114
- **TinyMyo** is a **3.6M-parameter** Transformer foundation model for surface EMG (sEMG), optimized for ultra-low-power edge deployment (GAP9 MCU). It demonstrates state-of-the-art performance across gesture classification, kinematic regression, and speech synthesis.
115
-
116
- ---
117
 
118
- ## 🚀 Quick Start
 
 
 
 
119
 
120
- TinyMyo is built as a specialized model within the [BioFoundation](https://github.com/pulp-bio/BioFoundation) framework.
 
 
 
 
121
 
122
- ### 1. Requirements
123
- - **Preprocessing:** Dependencies for data scripts are in `scripts/requirements.txt`.
124
- - **BioFoundation:** Full framework requirements for training/inference are in the [GitHub repository](https://github.com/pulp-bio/BioFoundation/blob/main/requirements.txt).
125
 
126
- ### 2. Preprocessing
127
- Process raw datasets into HDF5 format:
128
- ```bash
129
- python scripts/db5.py --data_dir $DATA_PATH/raw/ --save_dir $DATA_PATH/h5/ --seq_len 200 --stride 50
130
- ```
131
- *See [scripts/README.md](scripts/README.md) for all dataset commands.*
132
-
133
- ### 3. Fine-tuning
134
- ```bash
135
- python run_train.py +experiment=TinyMyo_finetune pretrained_safetensors_path=/path/to/base.safetensors
136
- ```
137
-
138
- ---
139
-
140
- ## 🧠 Architecture & Pretraining
141
- - **Core:** 8-layer Transformer encoder (192-dim embeddings, 3 heads).
142
- - **Tokenization:** Channel-independent patching (20 samples/patch) with RoPE.
143
- - **Data:** Pretrained on >480 GB of EMG (NinaPro DB6/7, EMG2Pose).
144
- - **Specs:** 3.6M parameters, 4.0 GFLOPs.
145
 
146
- ## 🎯 Benchmarks
 
 
 
 
147
 
148
- | Task | Dataset | Metric | TinyMyo |
149
- | :--- | :--- | :--- | :--- |
150
- | **Gesture** | NinaPro DB5 | Accuracy | **89.41%** |
151
- | **Gesture** | EPN-612 | Accuracy | **96.74%** |
152
- | **Gesture** | UCI EMG | Accuracy | **97.56%** |
153
- | **Regression**| NinaPro DB8 | MAE | **8.77°** |
154
- | **Speech** | Gaddy (Speech Synthesis) | WER | **33.54%** |
155
- | **Speech** | Gaddy (Speech Recognition) | WER | **33.95%** |
156
 
157
- ---
 
158
 
159
- ## ⚡ Edge Performance (GAP9 MCU)
160
- - **Inference:** 0.785 s
161
- - **Energy:** 44.91 mJ
162
- - **Power:** 57.18 mW
163
 
164
- ---
 
 
165
 
166
  ## 📜 License & Citation
167
- Weights are licensed under **CC BY-ND 4.0**. See [LICENSE](LICENSE) for details.
168
 
169
  ```bibtex
170
  @misc{fasulo2026tinymyotinyfoundationmodel,
@@ -173,7 +65,6 @@ Weights are licensed under **CC BY-ND 4.0**. See [LICENSE](LICENSE) for details.
173
  year={2026},
174
  eprint={2512.15729},
175
  archivePrefix={arXiv},
176
- primaryClass={eess.SP},
177
- url={https://arxiv.org/abs/2512.15729},
178
  }
179
  ```
 
1
+ # TinyMyo: Tiny Foundation Model for EMG Signal Processing
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
 
3
  <div align="center">
4
  <img src="https://raw.githubusercontent.com/MatteoFasulo/BioFoundation/refs/heads/TinyMyo/docs/model/logo/TinyMyo_logo.png" alt="TinyMyo Logo" width="400" />
 
5
  </div>
6
+
7
  <p align="center">
8
  <a href="https://github.com/pulp-bio/BioFoundation"><img src ="https://img.shields.io/github/stars/pulp-bio/BioFoundation?color=ccf" alt="Github"></a>
9
  <a href="https://creativecommons.org/licenses/by-nd/4.0/"><img src="https://img.shields.io/badge/License-CC_BY--ND_4.0-lightgrey.svg" alt="License"></a>
10
  <a href="https://arxiv.org/abs/2512.15729"><img src="https://img.shields.io/badge/arXiv-2512.15729-b31b1b.svg" alt="Paper"></a>
11
  </p>
12
 
13
+ ## 📖 Overview
14
+ **TinyMyo** is a lightweight (3.6M parameters), Transformer-based foundation model designed specifically for surface electromyography (sEMG) signal processing. Unlike large-scale models, TinyMyo is purpose-built for **ultra-low-power edge deployment**, enabling real-time motor intent decoding, neuromuscular assessment, and human-machine interaction directly on microcontrollers like the GAP9.
 
15
 
16
+ ## 🚀 Key Highlights
17
+ * **Generalist Foundation:** Pre-trained on a massive, heterogeneous corpus of >480 GB of EMG data (NinaPro DB6/7, EMG2Pose) using self-supervised masked reconstruction.
18
+ * **Edge-Ready:** The first EMG foundation model demonstrated on an ultra-low-power MCU (GAP9).
19
+ * **Highly Efficient:** Just 3.6M parameters, ensuring low latency and high energy efficiency (44.91 mJ per inference).
20
+ * **Versatile:** Achieves state-of-the-art (SoA) performance across hand gesture classification, kinematic regression, and speech processing.
21
 
22
+ ## 🧠 Model Architecture
23
+ * **Core:** 8-layer bidirectional Transformer encoder.
24
+ * **Embeddings:** 192-dimensional latent space with 3 attention heads.
25
+ * **Tokenization:** Channel-independent patching (20 samples per patch) utilizing Rotary Position Embeddings (RoPE) to preserve temporal alignment across channels.
26
+ * **Deployment:** Optimized via multi-level tiling and INT8 quantization for execution on resource-constrained hardware.
27
 
28
+ ## 📊 Performance Benchmarks
 
 
29
 
30
+ | Task | Dataset | Metric | TinyMyo Result |
31
+ | :--- | :--- | :--- | :--- |
32
+ | **Gesture Classification** | NinaPro DB5 | Accuracy | **89.41%** |
33
+ | **Gesture Classification** | EPN-612 | Accuracy | **96.74%** |
34
+ | **Gesture Classification** | UCI EMG | Accuracy | **97.56%** |
35
+ | **Kinematic Regression** | NinaPro DB8 | MAE | **8.77°** |
36
+ | **Speech Synthesis** | Gaddy | WER | **33.54%** |
37
+ | **Speech Recognition** | Gaddy | WER | **33.95%** |
 
 
 
 
 
 
 
 
 
 
 
38
 
39
+ ## Deployment (GAP9 MCU)
40
+ TinyMyo bridges the gap between high-performance deep learning and wearable constraints:
41
+ * **Inference Time:** 0.785 s
42
+ * **Energy Consumption:** 44.91 mJ
43
+ * **Power Envelope:** 57.18 mW
44
 
45
+ ## 🛠️ Getting Started
46
+ TinyMyo is part of the [BioFoundation](https://github.com/pulp-bio/BioFoundation) ecosystem.
 
 
 
 
 
 
47
 
48
+ ### Prerequisites
49
+ Install the required dependencies from the [BioFoundation repository](https://github.com/pulp-bio/BioFoundation).
50
 
51
+ ### Fine-tuning
52
+ You can easily fine-tune the pre-trained weights for your specific task:
 
 
53
 
54
+ ```bash
55
+ python run_train.py +experiment=TinyMyo_finetune pretrained_safetensors_path={*.safetensors}
56
+ ```
57
 
58
  ## 📜 License & Citation
59
+ This model is licensed under **CC BY-ND 4.0**. If you find TinyMyo useful in your research, please cite our paper:
60
 
61
  ```bibtex
62
  @misc{fasulo2026tinymyotinyfoundationmodel,
 
65
  year={2026},
66
  eprint={2512.15729},
67
  archivePrefix={arXiv},
68
+ primaryClass={eess.SP}
 
69
  }
70
  ```
UCI_EMG/UCI_finetune_5sec.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f4349d063bc4d8a04bcdb70b919c6dcf2b6c809075d9ca1d849adce6feb4ebe
3
+ size 14313224
UCI_EMG/config.json ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_target_": "models.TinyMyo.TinyMyo",
3
+ "img_size": 1000,
4
+ "patch_size": 20,
5
+ "in_chans": 8,
6
+ "embed_dim": 192,
7
+ "n_layer": 8,
8
+ "n_head": 3,
9
+ "mlp_ratio": 4,
10
+ "qkv_bias": true,
11
+ "attn_drop": 0.1,
12
+ "proj_drop": 0.1,
13
+ "drop_path": 0.1,
14
+ "num_classes": 6,
15
+ "task": "classification"
16
+ }
ckpt_to_safetensor.py ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import argparse
2
+ import json
3
+ import os
4
+
5
+ import torch
6
+ from omegaconf import OmegaConf
7
+ from safetensors.torch import load_file, save_file
8
+
9
+ if __name__ == "__main__":
10
+ parser = argparse.ArgumentParser(
11
+ description="Convert a PyTorch Lightning checkpoint to a safetensors file."
12
+ )
13
+ parser.add_argument("ckpt_path", type=str, help="Path to .ckpt file.")
14
+ parser.add_argument(
15
+ "--exclude_keys", type=str, nargs="*", default=[], help="Keys to exclude."
16
+ )
17
+ parser.add_argument(
18
+ "--strip_prefix",
19
+ type=str,
20
+ default=None,
21
+ help="Prefix to remove from keys (e.g., 'model.').",
22
+ )
23
+ parser.add_argument(
24
+ "--verbose", action="store_true", help="Print keys being saved."
25
+ )
26
+
27
+ args = parser.parse_args()
28
+
29
+ # Load checkpoint
30
+ ckpt = torch.load(args.ckpt_path, map_location="cpu", weights_only=False)
31
+ state_dict = ckpt["state_dict"]
32
+ hparams = ckpt["hyper_parameters"]
33
+
34
+ # Process: Exclude keys and strip prefixes
35
+ parameters = {}
36
+ for k, v in state_dict.items():
37
+ if any(k.startswith(excl) for excl in args.exclude_keys):
38
+ continue
39
+
40
+ new_key = k
41
+ if args.strip_prefix and k.startswith(f"{args.strip_prefix}."):
42
+ new_key = k.replace(f"{args.strip_prefix}.", "", 1)
43
+
44
+ parameters[new_key] = v
45
+
46
+ if args.verbose:
47
+ print("The following keys will be saved:")
48
+ for key in parameters.keys():
49
+ print(f" - {key}")
50
+
51
+ # Save safetensors
52
+ output_path = args.ckpt_path.replace(".ckpt", ".safetensors")
53
+ save_file(parameters, output_path)
54
+ print(f"Safetensors file saved to {output_path}")
55
+
56
+ # Export config.json
57
+ hparams_dict = OmegaConf.to_container(hparams, resolve=False)
58
+
59
+ # We only save the 'model' key to keep the config clean
60
+ config_data = hparams_dict.get("model", hparams_dict)
61
+
62
+ config_path = os.path.join(os.path.dirname(output_path), "config.json")
63
+ with open(config_path, "w", encoding="utf-8") as f:
64
+ json.dump(config_data, f, indent=2)
65
+ print(f"Configuration saved to {config_path}")
66
+
67
+ # Verification
68
+ try:
69
+ loaded_params = load_file(output_path)
70
+ assert len(parameters) == len(loaded_params), "Mismatch in parameter count!"
71
+ for k in parameters:
72
+ # We verify the tensor shape and dtype, as torch.equal can be slow/strict
73
+ assert parameters[k].shape == loaded_params[k].shape, f"Shape mismatch: {k}"
74
+ assert parameters[k].dtype == loaded_params[k].dtype, f"Dtype mismatch: {k}"
75
+ print("Verification successful: File is valid.")
76
+ except Exception as e:
77
+ print(f"Verification failed: {e}")
pretraining/TinyMyo.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:33c44fb4db05b9673227400b1fcf89e4dee03e0bfaf1573772fc760a6c2287f6
3
+ size 14291784
pretraining/config.json ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_target_": "models.TinyMyo.TinyMyo",
3
+ "img_size": 1000,
4
+ "patch_size": 20,
5
+ "in_chans": 16,
6
+ "embed_dim": 192,
7
+ "n_layer": 8,
8
+ "n_head": 3,
9
+ "mlp_ratio": 4,
10
+ "qkv_bias": true,
11
+ "attn_drop": 0.1,
12
+ "proj_drop": 0.1,
13
+ "drop_path": 0.0,
14
+ "num_classes": 0,
15
+ "task": "pretraining"
16
+ }
scripts/avespeech.py ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import argparse
2
+ import os
3
+ from huggingface_hub import snapshot_download
4
+
5
+
6
+ def download_emg_only(save_dir: str):
7
+ """
8
+ Download only EMG folders from the AVE-Speech dataset.
9
+ Requires HF authentication if the dataset is gated.
10
+ """
11
+
12
+ repo_id = "MML-Group/AVE-Speech"
13
+
14
+ # Patterns to include only EMG folders across splits
15
+ allow_patterns = [
16
+ "Train/EMG/**",
17
+ "Dev/EMG/**",
18
+ "Test/EMG/**",
19
+ ]
20
+
21
+ snapshot_download(
22
+ repo_id=repo_id,
23
+ repo_type="dataset",
24
+ local_dir=save_dir,
25
+ allow_patterns=allow_patterns,
26
+ resume_download=True,
27
+ )
28
+
29
+
30
+ if __name__ == "__main__":
31
+ parser = argparse.ArgumentParser(description="Download AVE-Speech EMG data only")
32
+ parser.add_argument("--save_dir", type=str, required=True)
33
+ args = parser.parse_args()
34
+
35
+ os.makedirs(args.save_dir, exist_ok=True)
36
+
37
+ download_emg_only(args.save_dir)