README.md

---
license: odc-by
library_name: fusion-embedding
pipeline_tag: feature-extraction
tags:
  - tactile
  - pressure-sensing
  - open-vocabulary
  - healthcare
  - fusion-embedding
base_model: EximiusLabs/fusion-embedding-2-2b-preview
---

# Tactus Mat: open-vocabulary posture from a body pressure mat

A body pressure mat streams a 64x32 grid of numbers. Tactus Mat turns that grid into a
point in a text embedding space, so you recognize what is happening on the mat by
**ranking sentences** rather than by running a classifier trained on a fixed label set.

![posture recognition from a pressure mat](assets/bedmat_hero.gif)

**Try it on real held-out data in a notebook** (no dataset download, no hardware):
[demo.ipynb](https://huggingface.co/EximiusLabs/fusion-embedding-2-tactus-mat/blob/main/demo.ipynb)
· [open in Colab](https://colab.research.google.com/#fileId=https%3A//huggingface.co/EximiusLabs/fusion-embedding-2-tactus-mat/blob/main/demo.ipynb). Type your own sentence and
it ranks real mat windows against it.

This is the mat sensor profile of the [Tactus](https://huggingface.co/EximiusLabs/fusion-embedding-2-tactus)
tactile pack. Same architecture, same recipe, different sensor: Tactus reads a 32x32
pressure glove and names grasped objects, Tactus Mat reads a 64x32 body mat and names
postures. Both project into the frozen text space of
[fusion-embedding-2](https://huggingface.co/EximiusLabs/fusion-embedding-2-2b-preview),
which is never modified.

## What it does

```python
from inference import TactusMatEmbedder

tm = TactusMatEmbedder.from_pretrained("EximiusLabs/fusion-embedding-2-tactus-mat",
                                       revision="v0.1-preview")
window = mat_frames                       # [F,64,32] or one [64,32] frame
for text, score in tm.rank(window, ["a person lying flat on their back",
                                    "a person lying on their left side",
                                    "a person in the fetal position",
                                    "an empty bed"]):
    print(f"{score:+.3f}  {text}")
```

Raw sensor counts go in with `raw="fsa"` (the training normalization is
`clip(raw/1000, 0, 1)` for the Vista Medical FSA range).

## Results

Held-out subjects (11, 12, 13 of PhysioNet PmatData), never seen in training.
Top-1 carries one sample standard deviation across independent runs.

| Metric | Result | Chance |
|---|---:|---:|
| 17-position top-1, open-vocabulary, mean of 5 runs | **0.957 ± 0.016** | 0.059 |
| 17-position top-3, mean of 5 runs | 0.993 ± 0.010 | 0.176 |
| 17-position top-1, single frame (one run) | 0.947 | 0.059 |
| 3-class (supine / right / left) via the 17-way ranking (one run) | 0.987 | 0.333 |
| 3-class scored against **unseen coarse phrases**, all 7 runs | 0.882 | 0.333 |

The last row is the open-vocabulary property in one number: the model was trained against
17 fine-grained posture phrases, then scored against three coarse phrases it never saw
during training, with no retraining and no classifier.

That score is identical in every run, MAE-initialized and from scratch alike, because its
errors are systematic rather than noise. 98.7% of them are the two positions where a
subject lies on their back with one knee raised, which the model ranks as lying on that
side. Excluding those two positions, coarse accuracy is 0.987. Whether a supine subject
with the right knee up should answer to "a person lying on their right side" is a question
about the dataset's label taxonomy, not about the model.

Published supervised baselines on this dataset report 99.6% on the 3-class task (ResNet-18,
leave-one-subject-out) and 82.7% in the dataset paper. Those are closed-set classifiers:
they answer three fixed labels and cannot answer a new phrase.

<details>
<summary>Real held-out frames with their ranked queries</summary>

![gallery](assets/bedmat_gallery.png)

Selection policy: for each posture shown, the held-out frame with the largest margin
between the correct query and its nearest competitor.
</details>

<details>
<summary>Temporal queries over a session</summary>

![timeline](assets/bedmat_timeline.png)

Because the mat streams at 1 Hz, posture becomes a time series and questions like "when did
they last move?" are answerable directly. The night shown is a disclosed composition: real
recordings from one held-out subject, sessions concatenated in a fixed order; per-frame
agreement with the recorded posture is 99%.
</details>

## Model

| Field | Value |
|---|---|
| Input | `[F,64,32]` pressure frames scaled to [0,1] (8 frames at training time) |
| Output | 2048-d L2-normalized vector in the fusion-embedding canonical text space |
| Trained parameters | 16.2M (13.5M ResNet-18-width trunk, 2.6M projector) |
| Frozen | the entire 2B language side; existing embeddings are unchanged |
| Training | 4000 steps, AdamW, cosine schedule, contrastive against 17 posture phrases |
| Cost | about half an hour on one A10G |

Masked-autoencoder initialization shows **no clear gain on this sensor**: 0.957 ± 0.016
over five MAE-initialized runs against 0.945 ± 0.007 over three from-scratch runs
(+1.2 points, Welch t = 1.4, p = 0.22, under five test windows). The +6.6-point same-sensor
pretraining gain reported for the glove profile does not reproduce here, most likely
because this task saturates near 0.96 on 20k frames from a single mat while the glove task
sat far from its ceiling.

The released checkpoint is one MAE-initialized run scoring 0.961, near the five-run mean;
the artifacts on this page come from it.

## Adding your own sensor

This model exists to show that the expensive part is already done. A new pressure sensor
does not need a new foundation model, it needs a small head trained against the same frozen
text space: hours of compute, not a research program. If you build or sell pressure arrays
(bed, seat, wheelchair, insole, industrial mat) and want your sensor to answer language
queries, the recipe and code are open, and we are glad to help.

## Data, license, and limits

Trained on [PhysioNet PmatData](https://physionet.org/content/pmd/1.0.0/) (pmd 1.0.0)
experiment I: 13 subjects, 17 in-bed positions, ~20k frames at 1 Hz on a Vista Medical FSA
SoftFlex 2048 (32x64 sensels), ODC-By v1.0. Please cite Pouyan et al., "A pressure map
dataset for posture and subject analytics," IEEE BHI 2017, along with PhysioNet.

Weights are released under ODC-By 1.0, matching the training data.

Limits worth knowing before you rely on this:

- Thirteen subjects, one mat, one lab. Transfer to a different mat geometry needs a short
  fine-tune; our own cross-sensor ablation on the glove profile predicts that pooling
  across sensor families does not work.
- The test split is 408 windows, so one window is 0.25 accuracy points. Small differences
  on this dataset are noise.
- **Not a medical device.** This is a research demonstration, not a diagnostic or
  monitoring product, and it must not be used to make clinical decisions.

## Paper

The Tactus recipe both profiles build on is described in [arXiv:2608.04043](https://arxiv.org/abs/2608.04043).

## Family

| Model | Sense |
|---|---|
| [fusion-embedding-2](https://huggingface.co/EximiusLabs/fusion-embedding-2-2b-preview) | text, image, video, audio |
| [Tactus](https://huggingface.co/EximiusLabs/fusion-embedding-2-tactus) | touch, 32x32 pressure glove |
| **Tactus Mat** (this model) | touch, 64x32 body pressure mat |
| [Tremor](https://huggingface.co/EximiusLabs/fusion-embedding-2-tremor) | inertial motion |
| [Ember](https://huggingface.co/EximiusLabs/fusion-embedding-2-ember) | thermal |

The memory layer these plug into is [Engram](https://github.com/Eximius-Labs/engram)
(`pip install engram-robomem`), which does language and temporal search over recorded
sensor sessions.