Upload folder using huggingface_hub
Browse files- .gitattributes +5 -0
- README.md +70 -25
- checkpoints/best_cls.pth +3 -0
- checkpoints/best_det.pth +3 -0
- configs/paper_cls.toml +23 -0
- configs/paper_det.toml +23 -0
- logs/metrics_cls.json +11 -0
- logs/metrics_det.json +11 -0
- onnx/rfpar_cls_v1.onnx +3 -0
- onnx/rfpar_det_v1.onnx +3 -0
- paper.pdf +3 -0
- pytorch/rfpar_cls_v1.pth +3 -0
- pytorch/rfpar_cls_v1.safetensors +3 -0
- pytorch/rfpar_det_v1.pth +3 -0
- pytorch/rfpar_det_v1.safetensors +3 -0
- tensorrt/rfpar_cls_v1_fp16.trt +3 -0
- tensorrt/rfpar_cls_v1_fp32.trt +3 -0
- tensorrt/rfpar_det_v1_fp16.trt +3 -0
- tensorrt/rfpar_det_v1_fp32.trt +3 -0
.gitattributes
CHANGED
|
@@ -35,3 +35,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
agent_fp16.trt filter=lfs diff=lfs merge=lfs -text
|
| 37 |
agent_fp32.trt filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
agent_fp16.trt filter=lfs diff=lfs merge=lfs -text
|
| 37 |
agent_fp32.trt filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
paper.pdf filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
tensorrt/rfpar_cls_v1_fp16.trt filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
tensorrt/rfpar_cls_v1_fp32.trt filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
tensorrt/rfpar_det_v1_fp16.trt filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
tensorrt/rfpar_det_v1_fp32.trt filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,62 +1,107 @@
|
|
| 1 |
---
|
| 2 |
-
library_name: pytorch
|
| 3 |
tags:
|
|
|
|
|
|
|
| 4 |
- adversarial-attacks
|
| 5 |
- reinforcement-learning
|
| 6 |
- pixel-attack
|
| 7 |
- image-classification
|
| 8 |
- object-detection
|
| 9 |
-
-
|
| 10 |
-
|
|
|
|
|
|
|
| 11 |
datasets:
|
| 12 |
- imagenet-1k
|
| 13 |
- coco
|
| 14 |
---
|
| 15 |
|
| 16 |
-
# DEF-rfpar
|
|
|
|
|
|
|
| 17 |
|
| 18 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
|
| 23 |
-
- **
|
| 24 |
-
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
## Results
|
| 27 |
|
| 28 |
### Classification (ImageNet-1K, ResNeXt50)
|
|
|
|
| 29 |
| Metric | Value |
|
| 30 |
|--------|-------|
|
| 31 |
-
| Attack Success Rate | 93.0% |
|
| 32 |
-
| Images Deceived | 186/200 |
|
| 33 |
| Mean L0 | 143.9 |
|
| 34 |
| Mean L2 | 6.26 |
|
| 35 |
-
|
|
| 36 |
| Forget Iterations | 100 |
|
| 37 |
-
| Time | 419s (NVIDIA L4) |
|
| 38 |
|
| 39 |
-
##
|
| 40 |
|
| 41 |
-
|
| 42 |
-
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
## Usage
|
| 48 |
|
| 49 |
```python
|
| 50 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
|
| 52 |
-
|
| 53 |
-
|
|
|
|
|
|
|
| 54 |
```
|
| 55 |
|
| 56 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
|
| 58 |
-
|
|
|
|
|
|
|
| 59 |
|
| 60 |
## License
|
| 61 |
|
| 62 |
-
|
|
|
|
|
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
tags:
|
| 3 |
+
- robotics
|
| 4 |
+
- anima
|
| 5 |
- adversarial-attacks
|
| 6 |
- reinforcement-learning
|
| 7 |
- pixel-attack
|
| 8 |
- image-classification
|
| 9 |
- object-detection
|
| 10 |
+
- robot-flow-labs
|
| 11 |
+
library_name: pytorch
|
| 12 |
+
pipeline_tag: robotics
|
| 13 |
+
license: apache-2.0
|
| 14 |
datasets:
|
| 15 |
- imagenet-1k
|
| 16 |
- coco
|
| 17 |
---
|
| 18 |
|
| 19 |
+
# DEF-rfpar — RFPAR: Remember and Forget Pixel Attack using RL
|
| 20 |
+
|
| 21 |
+
Part of the [ANIMA Perception Suite](https://robotflowlabs.com) by Robot Flow Labs.
|
| 22 |
|
| 23 |
+
## Paper
|
| 24 |
+
|
| 25 |
+
**RFPAR: Remember and Forget Pixel Attack using Reinforcement Learning** ([ArXiv 2502.07821](https://arxiv.org/abs/2502.07821))
|
| 26 |
+
Dongsu Song, Daehwa Ko, Jay Hoon Jung — Korea Aerospace University
|
| 27 |
+
|
| 28 |
+
## Architecture
|
| 29 |
|
| 30 |
+
RFPAR uses a REINFORCE policy network (Conv2d + FC) to select optimal pixel perturbations for black-box adversarial attacks. The key innovation is the **Remember and Forget** process:
|
| 31 |
|
| 32 |
+
- **Remember**: Train RL agent to find effective pixel perturbations via one-step policy gradient
|
| 33 |
+
- **Forget**: Reset agent parameters, keep successful perturbations in memory, re-optimize from new starting point
|
| 34 |
+
|
| 35 |
+
Two variants are provided:
|
| 36 |
+
- **Classification** (`rfpar_cls_v1`): Attacks image classifiers (ResNeXt50 on ImageNet)
|
| 37 |
+
- **Detection** (`rfpar_det_v1`): Attacks object detectors (YOLO on COCO)
|
| 38 |
|
| 39 |
## Results
|
| 40 |
|
| 41 |
### Classification (ImageNet-1K, ResNeXt50)
|
| 42 |
+
|
| 43 |
| Metric | Value |
|
| 44 |
|--------|-------|
|
| 45 |
+
| Attack Success Rate | **93.0%** |
|
| 46 |
+
| Images Deceived | 186 / 200 |
|
| 47 |
| Mean L0 | 143.9 |
|
| 48 |
| Mean L2 | 6.26 |
|
| 49 |
+
| Average Queries | 463 |
|
| 50 |
| Forget Iterations | 100 |
|
|
|
|
| 51 |
|
| 52 |
+
### Detection (COCO, YOLO11n)
|
| 53 |
|
| 54 |
+
| Metric | Value |
|
| 55 |
+
|--------|-------|
|
| 56 |
+
| Boxes Eliminated | 169 total |
|
| 57 |
+
| Box Elimination Rate | **112%** |
|
| 58 |
+
| Mean L0 | 234.8 |
|
| 59 |
+
| Mean L2 | 1885.2 |
|
| 60 |
+
| Average Queries | 494 |
|
| 61 |
+
| Forget Iterations | 5 |
|
| 62 |
+
|
| 63 |
+
## Exported Formats
|
| 64 |
+
|
| 65 |
+
| Format | Classification | Detection | Use Case |
|
| 66 |
+
|--------|---------------|-----------|----------|
|
| 67 |
+
| PyTorch (.pth) | `pytorch/rfpar_cls_v1.pth` | `pytorch/rfpar_det_v1.pth` | Training, fine-tuning |
|
| 68 |
+
| SafeTensors | `pytorch/rfpar_cls_v1.safetensors` | `pytorch/rfpar_det_v1.safetensors` | Fast loading, safe |
|
| 69 |
+
| ONNX | `onnx/rfpar_cls_v1.onnx` | `onnx/rfpar_det_v1.onnx` | Cross-platform inference |
|
| 70 |
+
| TensorRT FP16 | `tensorrt/rfpar_cls_v1_fp16.trt` | `tensorrt/rfpar_det_v1_fp16.trt` | Edge deployment (Jetson/L4) |
|
| 71 |
+
| TensorRT FP32 | `tensorrt/rfpar_cls_v1_fp32.trt` | `tensorrt/rfpar_det_v1_fp32.trt` | Full precision inference |
|
| 72 |
|
| 73 |
## Usage
|
| 74 |
|
| 75 |
```python
|
| 76 |
+
import torch
|
| 77 |
+
from anima_rfpar.agent import REINFORCEAgent
|
| 78 |
+
|
| 79 |
+
# Load classification agent
|
| 80 |
+
agent = REINFORCEAgent(224, 224, 3, detector_mode=False)
|
| 81 |
+
ckpt = torch.load("pytorch/rfpar_cls_v1.pth", weights_only=False)
|
| 82 |
+
agent.load_state_dict(ckpt["agent_state_dict"])
|
| 83 |
+
agent.eval()
|
| 84 |
|
| 85 |
+
# Get pixel perturbation action for an image
|
| 86 |
+
image = torch.randn(1, 3, 224, 224) # [0, 1] normalized
|
| 87 |
+
action_mean, action_std = agent(image)
|
| 88 |
+
# action_mean: (1, 5) -> sigmoid -> (x, y, r, g, b)
|
| 89 |
```
|
| 90 |
|
| 91 |
+
## Training
|
| 92 |
+
|
| 93 |
+
- **Hardware**: NVIDIA L4 (23GB VRAM)
|
| 94 |
+
- **CUDA**: 12.0 / PyTorch cu128
|
| 95 |
+
- **Framework**: PyTorch 2.11
|
| 96 |
+
- **Config**: See `configs/` directory
|
| 97 |
+
- **Time**: 419s (classification), 1581s (detection)
|
| 98 |
|
| 99 |
+
## Product Stack
|
| 100 |
+
|
| 101 |
+
ATLAS / ORACLE — Defense Marketplace
|
| 102 |
|
| 103 |
## License
|
| 104 |
|
| 105 |
+
Apache 2.0 — Robot Flow Labs / AIFLOW LABS LIMITED
|
| 106 |
+
|
| 107 |
+
Built with ANIMA by Robot Flow Labs
|
checkpoints/best_cls.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2b99506f7ce6615a97d11865a8a2c79681ab50899fd70bdfc2784b724f938b0c
|
| 3 |
+
size 411145941
|
checkpoints/best_det.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f8cd89dd3eeb73931cd47f6fc2ff7235556da97f32ed8e7d3281b42c409e63d5
|
| 3 |
+
size 411145941
|
configs/paper_cls.toml
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[module]
|
| 2 |
+
name = "DEF-rfpar"
|
| 3 |
+
seed = 2
|
| 4 |
+
|
| 5 |
+
[execution]
|
| 6 |
+
backend = "cuda"
|
| 7 |
+
mode = "classification"
|
| 8 |
+
dry_run = false
|
| 9 |
+
output_dir = "/mnt/artifacts-datai/checkpoints/DEF-rfpar"
|
| 10 |
+
|
| 11 |
+
[paths]
|
| 12 |
+
reference_repo = "repositories/RFPAR"
|
| 13 |
+
imagenet_root = "repositories/RFPAR/ImageNet"
|
| 14 |
+
coco_root = "repositories/RFPAR/COCO/images/val"
|
| 15 |
+
argoverse_root = "/mnt/forge-data/datasets/argoverse"
|
| 16 |
+
|
| 17 |
+
[attack]
|
| 18 |
+
max_iterations = 100
|
| 19 |
+
bound_threshold_eta = 0.05
|
| 20 |
+
alpha = 0.01
|
| 21 |
+
convergence_duration_t = 3
|
| 22 |
+
query_budget = 1700
|
| 23 |
+
yolo_conf_threshold = 0.5
|
configs/paper_det.toml
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[module]
|
| 2 |
+
name = "DEF-rfpar"
|
| 3 |
+
seed = 2
|
| 4 |
+
|
| 5 |
+
[execution]
|
| 6 |
+
backend = "cuda"
|
| 7 |
+
mode = "detection"
|
| 8 |
+
dry_run = false
|
| 9 |
+
output_dir = "/mnt/artifacts-datai/checkpoints/DEF-rfpar"
|
| 10 |
+
|
| 11 |
+
[paths]
|
| 12 |
+
reference_repo = "repositories/RFPAR"
|
| 13 |
+
imagenet_root = "repositories/RFPAR/ImageNet"
|
| 14 |
+
coco_root = "repositories/RFPAR/COCO/images/val"
|
| 15 |
+
argoverse_root = "/mnt/forge-data/datasets/argoverse"
|
| 16 |
+
|
| 17 |
+
[attack]
|
| 18 |
+
max_iterations = 100
|
| 19 |
+
bound_threshold_eta = 0.05
|
| 20 |
+
alpha = 0.05
|
| 21 |
+
convergence_duration_t = 20
|
| 22 |
+
query_budget = 1700
|
| 23 |
+
yolo_conf_threshold = 0.5
|
logs/metrics_cls.json
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"mode": "classification",
|
| 3 |
+
"total_images": 200,
|
| 4 |
+
"total_deceived": 186,
|
| 5 |
+
"success_rate": 0.93,
|
| 6 |
+
"mean_l0": 143.86559139784947,
|
| 7 |
+
"mean_l2": 6.261905570824941,
|
| 8 |
+
"mean_queries": 462.595,
|
| 9 |
+
"forget_iterations": 100,
|
| 10 |
+
"elapsed_sec": 419.3310854434967
|
| 11 |
+
}
|
logs/metrics_det.json
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"mode": "detection",
|
| 3 |
+
"total_images": 50,
|
| 4 |
+
"total_deceived": 169,
|
| 5 |
+
"success_rate": 1.1192053359075886,
|
| 6 |
+
"mean_l0": 234.76,
|
| 7 |
+
"mean_l2": 1885.1656958007814,
|
| 8 |
+
"mean_queries": 494.0,
|
| 9 |
+
"forget_iterations": 5,
|
| 10 |
+
"elapsed_sec": 1581.1591651439667
|
| 11 |
+
}
|
onnx/rfpar_cls_v1.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ea339a66a2b496131a8a1b5bd904465b59b6b7409b99781fc4e146b34591f692
|
| 3 |
+
size 411156009
|
onnx/rfpar_det_v1.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5e4b18d7d9ee62dd56f7da8674bcf86ce0803a693370a239c9c8f35a934a2aee
|
| 3 |
+
size 411156777
|
paper.pdf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e3f93a17aaecb78a63380596000d2224e2b7be8e2d8c03977343c62bac12da4f
|
| 3 |
+
size 24130597
|
pytorch/rfpar_cls_v1.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2b99506f7ce6615a97d11865a8a2c79681ab50899fd70bdfc2784b724f938b0c
|
| 3 |
+
size 411145941
|
pytorch/rfpar_cls_v1.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9056062898ff6d33bc2ca3f6fbba796a1c1daee7b7cbcbef16c361cf4ba89af4
|
| 3 |
+
size 411142704
|
pytorch/rfpar_det_v1.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f8cd89dd3eeb73931cd47f6fc2ff7235556da97f32ed8e7d3281b42c409e63d5
|
| 3 |
+
size 411145941
|
pytorch/rfpar_det_v1.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:336d9f18315ae072954a8a7f586beda1b792c0190130b35b71bb3c91fa8825ea
|
| 3 |
+
size 411142704
|
tensorrt/rfpar_cls_v1_fp16.trt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8909c4f418659c3ebdc195ec249e6b090bb72995356ffaaad7d5ffdb246aec2d
|
| 3 |
+
size 205651772
|
tensorrt/rfpar_cls_v1_fp32.trt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:42f7805abcc8a92147307dbba5a1ab10c39f0471e515779f2e7ac7e285fea410
|
| 3 |
+
size 411201724
|
tensorrt/rfpar_det_v1_fp16.trt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:50fe82bb26f7c11687f44f7d4c5fbeea807477831eca70efa4a5edae5ae88b34
|
| 3 |
+
size 205660620
|
tensorrt/rfpar_det_v1_fp32.trt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:95bc1dfc008eba84133fa06ced2ba352d2dee1a981bb2661299cc19a51388c84
|
| 3 |
+
size 411268436
|