ilessio-aiflowlab commited on
Commit
6702f87
·
verified ·
1 Parent(s): 5d4e5e8

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -35,3 +35,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  agent_fp16.trt filter=lfs diff=lfs merge=lfs -text
37
  agent_fp32.trt filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  agent_fp16.trt filter=lfs diff=lfs merge=lfs -text
37
  agent_fp32.trt filter=lfs diff=lfs merge=lfs -text
38
+ paper.pdf filter=lfs diff=lfs merge=lfs -text
39
+ tensorrt/rfpar_cls_v1_fp16.trt filter=lfs diff=lfs merge=lfs -text
40
+ tensorrt/rfpar_cls_v1_fp32.trt filter=lfs diff=lfs merge=lfs -text
41
+ tensorrt/rfpar_det_v1_fp16.trt filter=lfs diff=lfs merge=lfs -text
42
+ tensorrt/rfpar_det_v1_fp32.trt filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -1,62 +1,107 @@
1
  ---
2
- library_name: pytorch
3
  tags:
 
 
4
  - adversarial-attacks
5
  - reinforcement-learning
6
  - pixel-attack
7
  - image-classification
8
  - object-detection
9
- - anima
10
- license: mit
 
 
11
  datasets:
12
  - imagenet-1k
13
  - coco
14
  ---
15
 
16
- # DEF-rfpar: RFPAR — Remember and Forget Pixel Attack using RL
 
 
17
 
18
- ANIMA defense module implementing the RFPAR adversarial attack algorithm from [ArXiv 2502.07821](https://arxiv.org/abs/2502.07821).
 
 
 
 
 
19
 
20
- ## Model Description
21
 
22
- RFPAR uses a REINFORCE policy network to select optimal pixel perturbations for black-box adversarial attacks. The "Remember and Forget" process:
23
- - **Remember**: Train RL agent to find effective pixel perturbations
24
- - **Forget**: Reset agent, keep successful perturbations in memory
 
 
 
25
 
26
  ## Results
27
 
28
  ### Classification (ImageNet-1K, ResNeXt50)
 
29
  | Metric | Value |
30
  |--------|-------|
31
- | Attack Success Rate | 93.0% |
32
- | Images Deceived | 186/200 |
33
  | Mean L0 | 143.9 |
34
  | Mean L2 | 6.26 |
35
- | Avg Queries | 463 |
36
  | Forget Iterations | 100 |
37
- | Time | 419s (NVIDIA L4) |
38
 
39
- ## Available Formats
40
 
41
- - `agent.pth` PyTorch checkpoint
42
- - `agent.safetensors` — Safetensors format
43
- - `agent.onnx` ONNX format (opset 18)
44
- - `agent_fp16.trt` TensorRT FP16
45
- - `agent_fp32.trt` TensorRT FP32
 
 
 
 
 
 
 
 
 
 
 
 
 
46
 
47
  ## Usage
48
 
49
  ```python
50
- from anima_rfpar.export import load_agent_from_checkpoint
 
 
 
 
 
 
 
51
 
52
- agent = load_agent_from_checkpoint("best.pth")
53
- mean, std = agent(image_tensor) # Returns (x, y, r, g, b) action distribution
 
 
54
  ```
55
 
56
- ## Paper
 
 
 
 
 
 
57
 
58
- Song, D., Ko, D., & Jung, J. H. (2025). RFPAR: Remember and Forget Pixel Attack using Reinforcement Learning. *arXiv:2502.07821*.
 
 
59
 
60
  ## License
61
 
62
- MITBuilt with ANIMA by Robot Flow Labs
 
 
 
1
  ---
 
2
  tags:
3
+ - robotics
4
+ - anima
5
  - adversarial-attacks
6
  - reinforcement-learning
7
  - pixel-attack
8
  - image-classification
9
  - object-detection
10
+ - robot-flow-labs
11
+ library_name: pytorch
12
+ pipeline_tag: robotics
13
+ license: apache-2.0
14
  datasets:
15
  - imagenet-1k
16
  - coco
17
  ---
18
 
19
+ # DEF-rfpar — RFPAR: Remember and Forget Pixel Attack using RL
20
+
21
+ Part of the [ANIMA Perception Suite](https://robotflowlabs.com) by Robot Flow Labs.
22
 
23
+ ## Paper
24
+
25
+ **RFPAR: Remember and Forget Pixel Attack using Reinforcement Learning** ([ArXiv 2502.07821](https://arxiv.org/abs/2502.07821))
26
+ Dongsu Song, Daehwa Ko, Jay Hoon Jung — Korea Aerospace University
27
+
28
+ ## Architecture
29
 
30
+ RFPAR uses a REINFORCE policy network (Conv2d + FC) to select optimal pixel perturbations for black-box adversarial attacks. The key innovation is the **Remember and Forget** process:
31
 
32
+ - **Remember**: Train RL agent to find effective pixel perturbations via one-step policy gradient
33
+ - **Forget**: Reset agent parameters, keep successful perturbations in memory, re-optimize from new starting point
34
+
35
+ Two variants are provided:
36
+ - **Classification** (`rfpar_cls_v1`): Attacks image classifiers (ResNeXt50 on ImageNet)
37
+ - **Detection** (`rfpar_det_v1`): Attacks object detectors (YOLO on COCO)
38
 
39
  ## Results
40
 
41
  ### Classification (ImageNet-1K, ResNeXt50)
42
+
43
  | Metric | Value |
44
  |--------|-------|
45
+ | Attack Success Rate | **93.0%** |
46
+ | Images Deceived | 186 / 200 |
47
  | Mean L0 | 143.9 |
48
  | Mean L2 | 6.26 |
49
+ | Average Queries | 463 |
50
  | Forget Iterations | 100 |
 
51
 
52
+ ### Detection (COCO, YOLO11n)
53
 
54
+ | Metric | Value |
55
+ |--------|-------|
56
+ | Boxes Eliminated | 169 total |
57
+ | Box Elimination Rate | **112%** |
58
+ | Mean L0 | 234.8 |
59
+ | Mean L2 | 1885.2 |
60
+ | Average Queries | 494 |
61
+ | Forget Iterations | 5 |
62
+
63
+ ## Exported Formats
64
+
65
+ | Format | Classification | Detection | Use Case |
66
+ |--------|---------------|-----------|----------|
67
+ | PyTorch (.pth) | `pytorch/rfpar_cls_v1.pth` | `pytorch/rfpar_det_v1.pth` | Training, fine-tuning |
68
+ | SafeTensors | `pytorch/rfpar_cls_v1.safetensors` | `pytorch/rfpar_det_v1.safetensors` | Fast loading, safe |
69
+ | ONNX | `onnx/rfpar_cls_v1.onnx` | `onnx/rfpar_det_v1.onnx` | Cross-platform inference |
70
+ | TensorRT FP16 | `tensorrt/rfpar_cls_v1_fp16.trt` | `tensorrt/rfpar_det_v1_fp16.trt` | Edge deployment (Jetson/L4) |
71
+ | TensorRT FP32 | `tensorrt/rfpar_cls_v1_fp32.trt` | `tensorrt/rfpar_det_v1_fp32.trt` | Full precision inference |
72
 
73
  ## Usage
74
 
75
  ```python
76
+ import torch
77
+ from anima_rfpar.agent import REINFORCEAgent
78
+
79
+ # Load classification agent
80
+ agent = REINFORCEAgent(224, 224, 3, detector_mode=False)
81
+ ckpt = torch.load("pytorch/rfpar_cls_v1.pth", weights_only=False)
82
+ agent.load_state_dict(ckpt["agent_state_dict"])
83
+ agent.eval()
84
 
85
+ # Get pixel perturbation action for an image
86
+ image = torch.randn(1, 3, 224, 224) # [0, 1] normalized
87
+ action_mean, action_std = agent(image)
88
+ # action_mean: (1, 5) -> sigmoid -> (x, y, r, g, b)
89
  ```
90
 
91
+ ## Training
92
+
93
+ - **Hardware**: NVIDIA L4 (23GB VRAM)
94
+ - **CUDA**: 12.0 / PyTorch cu128
95
+ - **Framework**: PyTorch 2.11
96
+ - **Config**: See `configs/` directory
97
+ - **Time**: 419s (classification), 1581s (detection)
98
 
99
+ ## Product Stack
100
+
101
+ ATLAS / ORACLE — Defense Marketplace
102
 
103
  ## License
104
 
105
+ Apache 2.0 Robot Flow Labs / AIFLOW LABS LIMITED
106
+
107
+ Built with ANIMA by Robot Flow Labs
checkpoints/best_cls.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b99506f7ce6615a97d11865a8a2c79681ab50899fd70bdfc2784b724f938b0c
3
+ size 411145941
checkpoints/best_det.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8cd89dd3eeb73931cd47f6fc2ff7235556da97f32ed8e7d3281b42c409e63d5
3
+ size 411145941
configs/paper_cls.toml ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [module]
2
+ name = "DEF-rfpar"
3
+ seed = 2
4
+
5
+ [execution]
6
+ backend = "cuda"
7
+ mode = "classification"
8
+ dry_run = false
9
+ output_dir = "/mnt/artifacts-datai/checkpoints/DEF-rfpar"
10
+
11
+ [paths]
12
+ reference_repo = "repositories/RFPAR"
13
+ imagenet_root = "repositories/RFPAR/ImageNet"
14
+ coco_root = "repositories/RFPAR/COCO/images/val"
15
+ argoverse_root = "/mnt/forge-data/datasets/argoverse"
16
+
17
+ [attack]
18
+ max_iterations = 100
19
+ bound_threshold_eta = 0.05
20
+ alpha = 0.01
21
+ convergence_duration_t = 3
22
+ query_budget = 1700
23
+ yolo_conf_threshold = 0.5
configs/paper_det.toml ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [module]
2
+ name = "DEF-rfpar"
3
+ seed = 2
4
+
5
+ [execution]
6
+ backend = "cuda"
7
+ mode = "detection"
8
+ dry_run = false
9
+ output_dir = "/mnt/artifacts-datai/checkpoints/DEF-rfpar"
10
+
11
+ [paths]
12
+ reference_repo = "repositories/RFPAR"
13
+ imagenet_root = "repositories/RFPAR/ImageNet"
14
+ coco_root = "repositories/RFPAR/COCO/images/val"
15
+ argoverse_root = "/mnt/forge-data/datasets/argoverse"
16
+
17
+ [attack]
18
+ max_iterations = 100
19
+ bound_threshold_eta = 0.05
20
+ alpha = 0.05
21
+ convergence_duration_t = 20
22
+ query_budget = 1700
23
+ yolo_conf_threshold = 0.5
logs/metrics_cls.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "mode": "classification",
3
+ "total_images": 200,
4
+ "total_deceived": 186,
5
+ "success_rate": 0.93,
6
+ "mean_l0": 143.86559139784947,
7
+ "mean_l2": 6.261905570824941,
8
+ "mean_queries": 462.595,
9
+ "forget_iterations": 100,
10
+ "elapsed_sec": 419.3310854434967
11
+ }
logs/metrics_det.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "mode": "detection",
3
+ "total_images": 50,
4
+ "total_deceived": 169,
5
+ "success_rate": 1.1192053359075886,
6
+ "mean_l0": 234.76,
7
+ "mean_l2": 1885.1656958007814,
8
+ "mean_queries": 494.0,
9
+ "forget_iterations": 5,
10
+ "elapsed_sec": 1581.1591651439667
11
+ }
onnx/rfpar_cls_v1.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ea339a66a2b496131a8a1b5bd904465b59b6b7409b99781fc4e146b34591f692
3
+ size 411156009
onnx/rfpar_det_v1.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5e4b18d7d9ee62dd56f7da8674bcf86ce0803a693370a239c9c8f35a934a2aee
3
+ size 411156777
paper.pdf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e3f93a17aaecb78a63380596000d2224e2b7be8e2d8c03977343c62bac12da4f
3
+ size 24130597
pytorch/rfpar_cls_v1.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b99506f7ce6615a97d11865a8a2c79681ab50899fd70bdfc2784b724f938b0c
3
+ size 411145941
pytorch/rfpar_cls_v1.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9056062898ff6d33bc2ca3f6fbba796a1c1daee7b7cbcbef16c361cf4ba89af4
3
+ size 411142704
pytorch/rfpar_det_v1.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8cd89dd3eeb73931cd47f6fc2ff7235556da97f32ed8e7d3281b42c409e63d5
3
+ size 411145941
pytorch/rfpar_det_v1.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:336d9f18315ae072954a8a7f586beda1b792c0190130b35b71bb3c91fa8825ea
3
+ size 411142704
tensorrt/rfpar_cls_v1_fp16.trt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8909c4f418659c3ebdc195ec249e6b090bb72995356ffaaad7d5ffdb246aec2d
3
+ size 205651772
tensorrt/rfpar_cls_v1_fp32.trt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:42f7805abcc8a92147307dbba5a1ab10c39f0471e515779f2e7ac7e285fea410
3
+ size 411201724
tensorrt/rfpar_det_v1_fp16.trt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:50fe82bb26f7c11687f44f7d4c5fbeea807477831eca70efa4a5edae5ae88b34
3
+ size 205660620
tensorrt/rfpar_det_v1_fp32.trt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95bc1dfc008eba84133fa06ced2ba352d2dee1a981bb2661299cc19a51388c84
3
+ size 411268436