README.md

---
license: mit
tags:
  - robotics
  - tactile
  - visuo-tactile
  - representation-learning
library_name: pytorch
---

# NeoForce: A Unified Force Tactile Representation Model

[Technical Report](https://research.neoteai.com/assets/n0-foundation-paper.pdf) · [Project Website](http://research.neoteai.com/n0-foundation) · [Code](https://github.com/SparkleXFantasy/N0-Foundation-Preview) · [OpenNeoData](https://huggingface.co/datasets/NeoteAIEmbodied/OpenNeoData)

Tactile hardware is fragmented, so a model built on one signal format is bound to the device that produced it. NeoForce instead describes every tactile observation as a **dense three-axis force field** over the sensing surface, capturing shear and pressure in a form that is physically grounded and shared across sensors, and learns a temporally structured representation on top of it.

The model is a joint visual-tactile ViT: RGB frames and dual-sensor force fields are patched onto one shared token grid and run through a single trunk, so the modalities attend to each other directly instead of being fused after the fact. A temporal transformer over per-frame CLS pairs supplies context, and a progressive decoder reconstructs the force field and contact mask at the full input resolution, supervised at every scale it passes through.

This repository holds the weights. The code, together with the data preparation, training and evaluation guides, lives in the [code repository](https://github.com/SparkleXFantasy/N0-Foundation-Preview).

## Files

| File | Size | What it is |
|---|---|---|
| `neoforce/neoforce.pt` | 1.25 GB | the NeoForce model |
| `visuo_tactile_conversion/visuo_tactile_conversion.pt` | 49 MB | the tactile conversion model |

### `neoforce/neoforce.pt`

The trained NeoForce model at step 100,000, kept as a full training state rather than inference weights alone:

| Key | Contents |
|---|---|
| `model` | the student: the visual-tactile encoder plus the JEPA predictor, 156.14 M parameters |
| `teacher` | the EMA copy the JEPA and DINO targets are drawn from |
| `model_cfg` | the model configuration, so nothing has to be restated to rebuild it |
| `data_cfg`, `force_norm` | the per-channel scale the force targets were divided by |
| `step` | the training step the weights come from |

### `visuo_tactile_conversion/visuo_tactile_conversion.pt`

The frozen tactile conversion model, which turns raw tactile camera frames into the force field NeoForce consumes:

```
gel image -> bird-view correction -> optical flow based on the reference frame
          -> conversion network -> masked, scaled force field
```

## Setup

Each file has a fixed path the code looks in:

```bash
git clone https://github.com/SparkleXFantasy/N0-Foundation-Preview
cd N0-Foundation-Preview/neoforce

mkdir -p weights neoforce/visuo_tactile_conversion/model
curl -L -o weights/neoforce.pt \
  https://huggingface.co/NeoteAI/NeoForce/resolve/main/neoforce/neoforce.pt
curl -L -o neoforce/visuo_tactile_conversion/model/visuo_tactile_conversion.pt \
  https://huggingface.co/NeoteAI/NeoForce/resolve/main/visuo_tactile_conversion/visuo_tactile_conversion.pt
```

The [Installation Guide](https://github.com/SparkleXFantasy/N0-Foundation-Preview/blob/main/neoforce/docs/Installation.md) covers the rest of the setup.

## Model

| | |
|---|---|
| Backbone | ViT-B, 12 layers, 768-dim, patch 20 |
| Input | 4 frames @ 30 fps, RGB 360×640 + tactile representation `(6, 360, 640)` |
| Output | tactile representation `(6, 360, 640)`, contact mask `(6, 360, 640)`, global force `(6,)` |

The six force channels are `[left fx, fy, fz, right fx, fy, fz]`, one triplet per tactile sensor.

Trained on [OpenNeoData](https://huggingface.co/datasets/NeoteAIEmbodied/OpenNeoData), the 5,000-hour open-source subset of NeoData.

## Citation

```bibtex
@misc{n0foundation,
      title={N0-Foundation: Towards the Age of Tactile Intelligence},
      author={NeoteAI Team and TEAI Team},
      year={2026},
      url={https://research.neoteai.com/assets/n0-foundation-paper.pdf},
      note={Technical Report}
}
```

## License

[MIT](https://opensource.org/licenses/MIT)