Instructions to use prosoro/fingernet with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use prosoro/fingernet with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("prosoro/fingernet", dtype="auto") - Notebooks
- Google Colab
- Kaggle
han-xudong commited on
Commit ·
e326c19
1
Parent(s): 8ba6f46
modified: README.md
Browse filesnew file: __init__.py
modified: config.json
deleted: config_surf.json
deleted: fingernet_surf.onnx
renamed: fingernet.onnx -> model.onnx
new file: model.safetensors
new file: modeling.py
- README.md +47 -25
- __init__.py +1 -0
- config.json +12 -14
- config_surf.json +0 -17
- fingernet.onnx → model.onnx +0 -0
- fingernet_surf.onnx → model.safetensors +2 -2
- modeling.py +52 -0
README.md
CHANGED
|
@@ -1,6 +1,15 @@
|
|
| 1 |
---
|
| 2 |
license: bsd-3-clause
|
| 3 |
pipeline_tag: robotics
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
---
|
| 5 |
|
| 6 |
# Model Card for FingerNet
|
|
@@ -16,10 +25,12 @@ pipeline_tag: robotics
|
|
| 16 |
|
| 17 |
## Model Description
|
| 18 |
|
| 19 |
-
FingerNet is
|
| 20 |
|
| 21 |
-
|
| 22 |
-
|
|
|
|
|
|
|
| 23 |
- License: BSD-3-Clause
|
| 24 |
- Resources for more information:
|
| 25 |
- [Project page](https://doc.ancoraspring.com/asfinger)
|
|
@@ -31,27 +42,39 @@ This model is intended for researchers and developers working in robotics and ta
|
|
| 31 |
|
| 32 |
See the [project page](https://doc.ancoraspring.com/asfinger) for more details.
|
| 33 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
## Training Data
|
| 35 |
|
| 36 |
-
The model was trained on the
|
| 37 |
|
| 38 |
## Citation
|
| 39 |
|
| 40 |
If you use this model in your research, please cite the following papers:
|
| 41 |
|
| 42 |
-
```bibtex
|
| 43 |
-
@article{guo2024proprioceptive,
|
| 44 |
-
title={Proprioceptive State Estimation for Amphibious Tactile Sensing},
|
| 45 |
-
author={Guo, Ning and Han, Xudong and Zhong, Shuqiao and Zhou, Zhiyuan and Lin, Jian and Dai, Jian S and Wan, Fang and Song, Chaoyang},
|
| 46 |
-
journal={IEEE Transactions on Robotics},
|
| 47 |
-
volume={40},
|
| 48 |
-
pages={4684-4698},
|
| 49 |
-
year={2024},
|
| 50 |
-
publisher={IEEE},
|
| 51 |
-
doi={10.1109/TRO.2024.3463509}
|
| 52 |
-
}
|
| 53 |
-
```
|
| 54 |
-
|
| 55 |
```bibtex
|
| 56 |
@article{liu2024proprioceptive,
|
| 57 |
title={Proprioceptive learning with soft polyhedral networks},
|
|
@@ -66,14 +89,13 @@ If you use this model in your research, please cite the following papers:
|
|
| 66 |
}
|
| 67 |
```
|
| 68 |
|
|
|
|
|
|
|
| 69 |
```bibtex
|
| 70 |
-
@article{
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
year={2025},
|
| 76 |
-
publisher={Wiley Online Library},
|
| 77 |
-
doi={10.1002/aisy.202500444}
|
| 78 |
}
|
| 79 |
```
|
|
|
|
| 1 |
---
|
| 2 |
license: bsd-3-clause
|
| 3 |
pipeline_tag: robotics
|
| 4 |
+
tags:
|
| 5 |
+
- fingernet
|
| 6 |
+
- asfinger
|
| 7 |
+
- multimodal
|
| 8 |
+
- onnx
|
| 9 |
+
- pytorch
|
| 10 |
+
library_name: transformers
|
| 11 |
+
datasets:
|
| 12 |
+
- asRobotics/fingernet-100k
|
| 13 |
---
|
| 14 |
|
| 15 |
# Model Card for FingerNet
|
|
|
|
| 25 |
|
| 26 |
## Model Description
|
| 27 |
|
| 28 |
+
FingerNet is an MLP model designed for the asFinger. It can predict both 6D force and 3D shape (mesh nodes) from the 6D motion of the asFinger.
|
| 29 |
|
| 30 |
+
Try it out on the [Spaces Demo](https://huggingface.co/spaces/asRobotics/fingernet-demo)!
|
| 31 |
+
|
| 32 |
+
- Developer: Xudong Han, Ning Guo, Xiaobo Liu, Tianyu Wu, Fang Wan, and Chaoyang Song.
|
| 33 |
+
- Model type: MLP
|
| 34 |
- License: BSD-3-Clause
|
| 35 |
- Resources for more information:
|
| 36 |
- [Project page](https://doc.ancoraspring.com/asfinger)
|
|
|
|
| 42 |
|
| 43 |
See the [project page](https://doc.ancoraspring.com/asfinger) for more details.
|
| 44 |
|
| 45 |
+
To load the model:
|
| 46 |
+
|
| 47 |
+
```python
|
| 48 |
+
# Example code to load safetensors
|
| 49 |
+
from transformers import AutoModel
|
| 50 |
+
|
| 51 |
+
model = AutoModel.from_pretrained("asRobotics/fingernet", trust_remote_code=True)
|
| 52 |
+
x = torch.zeros((1, 6)) # Example input: batch size of 1, 6D motion
|
| 53 |
+
output = model(x)
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
Or to load the ONNX version:
|
| 57 |
+
|
| 58 |
+
```python
|
| 59 |
+
# Example code to load onnx
|
| 60 |
+
import onnxruntime as ort
|
| 61 |
+
import numpy as np
|
| 62 |
+
from huggingface_hub import hf_hub_download
|
| 63 |
+
|
| 64 |
+
onnx_model_path = hf_hub_download(repo_id="asRobotics/fingernet", filename="model.onnx")
|
| 65 |
+
ort_session = ort.InferenceSession(onnx_model_path)
|
| 66 |
+
x = np.zeros((1, 6)).astype(np.float32) # Example input: batch size of 1, 6D motion
|
| 67 |
+
outputs = ort_session.run(None, {"motion": x})
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
## Training Data
|
| 71 |
|
| 72 |
+
The model was trained on the [FingerNet-100K](https://huggingface.co/datasets/asRobotics/fingernet-100k), which includes a variety of motion, force, and shape data collected by finite element simulations.
|
| 73 |
|
| 74 |
## Citation
|
| 75 |
|
| 76 |
If you use this model in your research, please cite the following papers:
|
| 77 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 78 |
```bibtex
|
| 79 |
@article{liu2024proprioceptive,
|
| 80 |
title={Proprioceptive learning with soft polyhedral networks},
|
|
|
|
| 89 |
}
|
| 90 |
```
|
| 91 |
|
| 92 |
+
[](https://arxiv.org/abs/2308.08538)
|
| 93 |
+
|
| 94 |
```bibtex
|
| 95 |
+
@article{wu2025magiclaw,
|
| 96 |
+
title={MagiClaw: A Dual-Use, Vision-Based Soft Gripper for Bridging the Human Demonstration to Robotic Deployment Gap},
|
| 97 |
+
author={Wu, Tianyu and Han, Xudong and Sun, Haoran and Zhang, Zishang and Huang, Bangchao and Song, Chaoyang and Wan, Fang},
|
| 98 |
+
journal={arXiv preprint arXiv:2509.19169},
|
| 99 |
+
year={2025}
|
|
|
|
|
|
|
|
|
|
| 100 |
}
|
| 101 |
```
|
__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
from .modeling import FingerNet, FingerNetConfig
|
config.json
CHANGED
|
@@ -1,17 +1,15 @@
|
|
| 1 |
{
|
| 2 |
-
"
|
| 3 |
-
"
|
| 4 |
-
"
|
| 5 |
-
"
|
| 6 |
-
"
|
| 7 |
-
"
|
| 8 |
-
"
|
| 9 |
-
"
|
| 10 |
-
"
|
| 11 |
-
"
|
| 12 |
-
"
|
| 13 |
-
|
| 14 |
-
"dependencies": {
|
| 15 |
-
"onnxruntime": ">=1.16.0"
|
| 16 |
}
|
| 17 |
}
|
|
|
|
| 1 |
{
|
| 2 |
+
"_name_or_path": "asRobotics/fingernet",
|
| 3 |
+
"architectures": ["FingerNet"],
|
| 4 |
+
"model_type": "fingernet",
|
| 5 |
+
"x_dim": [6],
|
| 6 |
+
"y_dim": [6, 1800],
|
| 7 |
+
"h1_dim": [100, 1000],
|
| 8 |
+
"h2_dim": [100, 1000],
|
| 9 |
+
"torch_dtype": "float32",
|
| 10 |
+
"layer_norm": false,
|
| 11 |
+
"use_activation": "relu",
|
| 12 |
+
"auto_map": {
|
| 13 |
+
"AutoModel": "modeling.FingerNet"
|
|
|
|
|
|
|
| 14 |
}
|
| 15 |
}
|
config_surf.json
DELETED
|
@@ -1,17 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"model_name": "FingerNet(Surface)",
|
| 3 |
-
"architecture": "dual-branch-mlp",
|
| 4 |
-
"framework": "onnx",
|
| 5 |
-
"input_motion": 6,
|
| 6 |
-
"output_force_dim": 6,
|
| 7 |
-
"output_shape_dim": 1803,
|
| 8 |
-
"quantized": false,
|
| 9 |
-
"version": "v1.0",
|
| 10 |
-
"description": "Dual-branch MLP model for force (6D) and shape (1803D) prediction from motion of the asFinger with surface.",
|
| 11 |
-
"author": "AncoraSpring",
|
| 12 |
-
"license": "BSD-3-Clause",
|
| 13 |
-
"training_dataset": "asFinger Dataset",
|
| 14 |
-
"dependencies": {
|
| 15 |
-
"onnxruntime": ">=1.16.0"
|
| 16 |
-
}
|
| 17 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
fingernet.onnx → model.onnx
RENAMED
|
File without changes
|
fingernet_surf.onnx → model.safetensors
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3209d7f93ef4059f4cf9108139e7a4f26cce35ff8b0cdc654f7a2abea7617a53
|
| 3 |
+
size 11285832
|
modeling.py
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import torch
|
| 2 |
+
import torch.nn as nn
|
| 3 |
+
from transformers import PreTrainedModel, PretrainedConfig
|
| 4 |
+
|
| 5 |
+
class FingerNetConfig(PretrainedConfig):
|
| 6 |
+
model_type = "fingernet"
|
| 7 |
+
|
| 8 |
+
def __init__(
|
| 9 |
+
self,
|
| 10 |
+
x_dim=[6],
|
| 11 |
+
y_dim=[6, 1800],
|
| 12 |
+
h1_dim=[100, 1000],
|
| 13 |
+
h2_dim=[100, 1000],
|
| 14 |
+
**kwargs,
|
| 15 |
+
):
|
| 16 |
+
super().__init__(**kwargs)
|
| 17 |
+
self.x_dim = x_dim
|
| 18 |
+
self.y_dim = y_dim
|
| 19 |
+
self.h1_dim = h1_dim
|
| 20 |
+
self.h2_dim = h2_dim
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
class FingerNet(PreTrainedModel):
|
| 24 |
+
config_class = FingerNetConfig
|
| 25 |
+
|
| 26 |
+
def __init__(self, config):
|
| 27 |
+
super().__init__(config)
|
| 28 |
+
self.x_dim = config.x_dim
|
| 29 |
+
self.y_dim = config.y_dim
|
| 30 |
+
self.h1_dim = config.h1_dim
|
| 31 |
+
self.h2_dim = config.h2_dim
|
| 32 |
+
|
| 33 |
+
# build sub-networks
|
| 34 |
+
self.branches = nn.ModuleList()
|
| 35 |
+
for i in range(len(self.y_dim)):
|
| 36 |
+
net = nn.Sequential(
|
| 37 |
+
nn.Linear(self.x_dim[0], self.h1_dim[i]),
|
| 38 |
+
nn.ReLU(),
|
| 39 |
+
nn.Linear(self.h1_dim[i], self.h2_dim[i]),
|
| 40 |
+
nn.ReLU(),
|
| 41 |
+
nn.Linear(self.h2_dim[i], self.y_dim[i]),
|
| 42 |
+
)
|
| 43 |
+
self.branches.append(net)
|
| 44 |
+
|
| 45 |
+
# initialize weights
|
| 46 |
+
self.post_init()
|
| 47 |
+
|
| 48 |
+
def forward(self, x):
|
| 49 |
+
if isinstance(x, (list, tuple)):
|
| 50 |
+
x = torch.tensor(x, dtype=torch.float32)
|
| 51 |
+
outputs = [branch(x) for branch in self.branches]
|
| 52 |
+
return tuple(outputs)
|