File size: 1,887 Bytes
ff12d64 a4775fa 7d1360c 3cc974e 7d1360c 3cc974e 7d1360c a4775fa |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
---
license: mit
---
**Spiking Neural Network as Adaptive Event Stream Slicer (NeurIPS'24)**
*Abstract:*
Event-based cameras are attracting significant interest as they provide rich edge information, high dynamic range, and high temporal resolution. Many state-of-the-art event-based algorithms rely on splitting the events into fixed groups, resulting in the omission of crucial temporal information, particularly when dealing with diverse motion scenarios (e.g., high/low speed). In this work, we propose SpikeSlicer, a
novel-designed plug-and-play event processing method capable of splitting events
stream adaptively. SpikeSlicer utilizes a low-energy spiking neural network (SNN)
to trigger event slicing. To guide the SNN to fire spikes at optimal time steps, we
propose the Spiking Position-aware Loss (SPA-Loss) to modulate the neuron’s state.
Additionally, we develop a Feedback-Update training strategy that refines the slicing decisions using feedback from the downstream artificial neural network (ANN).
Extensive experiments demonstrate that our method yields significant performance
improvements in event-based object tracking and recognition. Notably, SpikeSlicer
provides a brand-new SNN-ANN cooperation paradigm, where the SNN acts as an
efficient, low-energy data processor to assist the ANN in improving downstream
performance, injecting new perspectives and potential avenues of exploration.
<p align="left">
<img src="spikeslicer.gif" alt="Logo" width="80%">
</p>
More details can be found at [paper](https://arxiv.org/pdf/2410.02249) and [code](https://github.com/AndyCao1125/SpikeSlicer).
We provide example Test Logs and Weights in this HF hub:
* Logs: TransT_baseline_test_log.zip; TransT_SpikeSlicer_Small_test_log.zip; TransT_SpikeSlicer_Base_test_log.zip
* Weights: TransT_baseline_pth; TransT_SpikeSlicer_Small_pth; TransT_SpikeSlicer_Base_pth |