---
license: cc-by-nc-4.0
language:
- en
pipeline_tag: feature-extraction
tags:
- embeddings
- tactile
- pressure
- force
- fsr
- taxel
- sensor
- robotics
- qwen3-vl
base_model: EximiusLabs/fusion-embedding-2-2b-preview
---
# fusion-embedding-2-tactus
[](https://github.com/Eximius-Labs/fusion-embedding)
[](https://github.com/Eximius-Labs/fusion-embedding)
[](#license)
[](#)
[](https://github.com/Eximius-Labs/fusion-embedding)
**Tactus** is the tactile sensor pack for Eximius Labs' fusion-embedding stack. It maps a
short window of pressure-array frames (a 32x32 taxel grid, the signal class produced by
resistive/FSR gloves, e-skins, and instrumented robot hands) into the
[Qwen3-VL-Embedding-2B](https://huggingface.co/Qwen/Qwen3-VL-Embedding-2B) text embedding
space, so touch becomes searchable in plain language: recognition is a text query, not a
trained classifier head.
Tactus reads **low-dimensional pressure arrays**, not optical tactile images. Optical
sensors (GelSight, DIGIT) already have strong models (TVL, UniTouch, Sparsh); the cheap,
widely-shipped resistive arrays did not. To our knowledge Tactus is the first open model
to put this sensor class in a text-aligned, cross-modal embedding space.
Tactus is part of the **fusion-embedding family** alongside
[Tactus Mat](https://huggingface.co/EximiusLabs/fusion-embedding-2-tactus-mat)
(the same pack trained for a 64x32 body pressure mat),
[Tremor](https://huggingface.co/EximiusLabs/fusion-embedding-2-tremor) (motion) and
[Ember](https://huggingface.co/EximiusLabs/fusion-embedding-2-ember) (thermal). Its
embeddings target the canonical readout of
[fusion-embedding-2](https://huggingface.co/EximiusLabs/fusion-embedding-2-2b-preview), so
tactile windows are directly comparable to that model's text, image, video, and audio in
one 2048-d space, and drop into the [Engram](https://github.com/Eximius-Labs/engram)
memory layer (`pip install engram-robomem`) as a first-class sense.
[GitHub](https://github.com/Eximius-Labs/fusion-embedding) | [fusion-embedding-2](https://huggingface.co/EximiusLabs/fusion-embedding-2-2b-preview) | [Live playground](https://www.eximiuslabs.com/playground) | [Family report (arXiv:2607.18666)](https://arxiv.org/abs/2607.18666) | Tactus report: arXiv link lands with this week's submission
## Model Overview
Tactus is a **trained CNN trunk plus projector** over pressure windows. Each 32x32 frame
passes through a ResNet-18-width trunk (3x3 stem, four stages; 32x32 -> 4x4 spatial map);
the K frames of a grasp window are fused by a learned 1x1 convolution over their
concatenated feature maps, pooled, and projected into the frozen base's 2048-d text space.
The trunk is initialized by masked-autoencoder pretraining (mask 0.6, per-patch normalized
targets) on 144k unlabeled STAG-family pressure frames, then fine-tuned contrastively
against canonical text embeddings of natural grasp phrases.
The design choice that matters is the data path: pressure is normalized with the sensor's
own calibration affine (`clip((raw - 500) / 150, 0, 1)`, the STAG reference preprocessing),
and pretraining stays same-sensor. In our ablations, correct normalization and same-sensor
MAE were worth more than every architecture change combined, while cross-sensor pretraining
pooling gave nothing, consistent with published findings (HTT, TacVerse).
| Feature | Value |
| --- | --- |
| Parameters | ~2.06B frozen Qwen base (text side); **16.2M trained** (13.5M trunk + 2.6M projector) |
| Modality | tactile pressure (32x32 taxel array; 548 active sensors in the training glove) |
| Supported tasks | `zero-shot object recognition from touch`, `text -> touch retrieval` |
| Input | one grasp window `[F, 32, 32]` (F frames, any F; trained at K=8) or a single frame |
| Input scale | STAG calibration affine `clip((raw - 500) / 150, 0, 1)`; uint8 0-255 maps accepted |
| Embedding dimension | 2048 (canonical whitened readout; directly comparable across modalities) |
| Pooling strategy | last-token pooling (text side) |
| Base model | Qwen/Qwen3-VL-Embedding-2B via fusion-embedding-2-2b-preview (frozen) |
| Pretraining | same-sensor MAE, 144k frames incl. unlabeled; supervised test frames excluded |
| Trained components | trunk + conv frame-fusion + projector, 16.2M; shipped as `model.safetensors` |
| Distribution | ~65 MB trained head; the frozen base downloads from its own repository |
## See it in action
**Real held-out grasps, recognized from pressure alone.** Each panel is a genuine STAG test
frame (the most active frame of that class in the held-out split, by total pressure) with
the text query the model matches it against: no camera, no trained classifier head. Across
the full test split the model averages 0.77 top-1 and 0.94 top-3 over 27 such queries.
## Training and Evaluation
Tactus trains in two stages on the [STAG](https://stag.csail.mit.edu/) datasets
([Sundaram et al., Nature 2019](https://www.nature.com/articles/s41586-019-1234-z)): a
masked-autoencoder pretrain over every STAG-family pressure frame (classification +
blindfolded + weights + handposes, 144k frames including unlabeled ones, supervised test
frames excluded), then contrastive fine-tuning of the whole head against canonical text
embeddings of grasp phrases, with STAG-style cluster sampling (each training window draws
diverse frames from across a recording rather than consecutive near-duplicates).
Evaluation is 27-way object recognition on **fully held-out test recordings**, scored as
cosine ranking against text queries (open-vocabulary; the model never trains a classifier
head).
| | top-1 (27-way) | top-3 | recording-level top-1 |
| --- | ---: | ---: | ---: |
| **This checkpoint** | **0.817** | **0.951** | 0.741 |
| Recipe mean (4 independent runs) | 0.771 +/- 0.062 | 0.935 | 0.722 |
| Training from scratch (no MAE), mean of 3 | 0.705 | 0.905 | 0.691 |
| STAG 2019 supervised closed-set CNN | 0.76 | - | - |
| chance | 0.037 | 0.111 | 0.037 |
Interpreting these numbers: the recipe's mean exceeds the original paper's supervised CNN
while performing a harder task (open-vocabulary text queries against a frozen language
space, versus a 27-way trained classifier), though by less than one standard error; we
describe the result as **matching to exceeding the original baseline, with best runs at
0.83**, rather than claiming a definitive margin. Top-3 accuracy is stable across every
run. Our evaluation mirrors STAG's cluster-sampling test protocol but is not their
byte-identical harness. Same-sensor MAE pretraining accounts for about +7 points over
training from scratch. Full recipe, ablations, and negative results: `results.json` and
the GitHub repository.
## Usage
Requirements
- `torch` (CUDA recommended), `numpy`, `safetensors`
- `pip install fusion-embedding[hf]` for the text side (the canonical whitened readout
Tactus was trained against; embedding text any other way will misrank)
- The frozen base downloads from `EximiusLabs/fusion-embedding-2-2b-preview`.
via inference.py (this repository)
```python
import numpy as np
from inference import TactusEmbedder
ta = TactusEmbedder.from_pretrained("EximiusLabs/fusion-embedding-2-tactus",
revision="v0.1-preview")
# a grasp window: [F, 32, 32] pressure frames (uint8 0-255 or floats in [0, 1]);
# for raw sensor counts pass raw="stag" to apply the calibration affine
window = np.load("grasp.npy")
for text, score in ta.rank(window, ["a mug", "scissors", "a full soda can", "an empty hand"]):
print(f"{score:+.3f} {text}")
# or embed both sides into the shared space directly
p = ta.embed_pressure(window) # 2048-d, L2-normalized
t = ta.embed_text(["holding a mug"]) # canonical text embedding, same space
```
Pressure embeddings land in the same space as fusion-embedding-2's text, image, video, and
audio, and as Tremor's motion, so cross-modal search over a robot session works out of the
box through [Engram](https://github.com/Eximius-Labs/engram). Match text against pressure
through this API rather than embedding text with the raw base model; Tactus was trained
against the canonical whitened readout, and other text paths will misrank.
## Related models
Tactus joins the fusion-embedding sense packs, all built on
[fusion-embedding-2](https://huggingface.co/EximiusLabs/fusion-embedding-2-2b-preview):
| Model | Sense | Signal |
| --- | --- | --- |
| **This model** | touch | 32x32 pressure/taxel arrays |
| [fusion-embedding-2-tremor](https://huggingface.co/EximiusLabs/fusion-embedding-2-tremor) | motion | 3-axis accelerometer windows |
| [fusion-embedding-2-tremor-g1](https://huggingface.co/EximiusLabs/fusion-embedding-2-tremor-g1) | motion (Unitree G1 head) | robot IMU |
| [fusion-embedding-2-ember](https://huggingface.co/EximiusLabs/fusion-embedding-2-ember) | heat | thermal infrared images |
All packs embed into one 2048-d space, so a query can match across senses. The
[Engram](https://github.com/Eximius-Labs/engram) memory layer (`pip install engram-robomem`)
wires them into a searchable robot session memory with temporal reasoning.
## License
The trained weights in this repository are released under
**[CC-BY-NC-4.0](https://creativecommons.org/licenses/by-nc/4.0/)** (non-commercial).
This reflects the training data's lineage: Tactus is trained on the
[STAG](https://stag.csail.mit.edu/) datasets, which are released for non-commercial
research use. A commercially-clean retrain (on permissively licensed pressure corpora) is
future work; a commercial license may follow.
## Limitations
- **Run-to-run variance.** The training recipe's top-1 varies +/-0.06 across seeds
(0.70-0.83 over four runs). The released checkpoint is a strong draw, and the mean is
reported alongside it. Seed stabilization is active work.
- **One sensor family.** Trained on one glove (STAG's 32x32 grid, 548 taxels). Our
cross-sensor experiments show transfer to other taxel geometries needs fine-tuning, not
zero-shot use; the input path accepts any [F,32,32] window, and other resolutions must
be resampled.
- **27-object vocabulary at eval.** Open-vocabulary means text queries, not tested
open-set generalization to arbitrary unseen object categories; treat novel-category
recognition as unvalidated.
- **Research preview.** Not a production classifier. The intended use is
language-addressable touch inside a multimodal memory, not high-stakes recognition.
- **English text only**, through the canonical readout (`fusion-embedding` package); do
not embed text with the raw base model.
## Citation
If you use Tactus, please cite this repository and the dataset it builds on:
```bibtex
@misc{tactus2026,
title = {Tactus: a tactile pressure sensor pack for the fusion-embedding space},
author = {Tonmoy, Abdul Basit},
year = {2026},
note = {Eximius Labs. Model weights CC-BY-NC-4.0.},
url = {https://huggingface.co/EximiusLabs/fusion-embedding-2-tactus}
}
```
Tactus trains on **STAG** (Sundaram et al., *Learning the signatures of the human grasp
using a scalable tactile glove*, Nature 2019); please cite that work when using the
benchmark numbers. The text space is **Qwen3-VL-Embedding-2B**.