File size: 1,668 Bytes
07b1bb7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | # AnalysisObjectTransformer Model
This repository contains the implementation of the AnalysisObjectTransformer model, a deep learning architecture designed for event classification with reconstructed-object inputs. MultiHeadAttention is used to extract the correlation between jets (hadrons) in the final state. Achieves state-of-the-art performance on final states which can be summarized as jets accompanied by missing transverse energy.
## Model Overview
The AnalysisObjectTransformer model is structured to process jet-level features (energy, mass, area, btag score) in any order (permutation invariance) and event-level features (angle analysis of missing energy and leading jets) to classify signal from background processes to enhance the sensitivity to rare BSM signatures.
### Components
- **Embedding Layers**: Transform input data into a higher-dimensional space for subsequent processing.
- **Attention Blocks (AttBlock)**: Utilize multi-head attention to capture dependencies between different elements of the input data.
- **Class Blocks (ClassBlock)**: Extend attention mechanisms to incorporate class tokens, enabling the model to focus on class-relevant features. Implementation based on "Going deeper with transformers": https://arxiv.org/abs/2103.17239
- **MLP Head**: A sequence of fully connected layers that maps the output of the transformer blocks to the final prediction targets.
## Usage
```python
from particle_transformer import AnalysisObjectTransformer
model = AnalysisObjectTransformer(input_dim_obj=..., input_dim_event=..., embed_dims=..., linear_dims1=..., linear_dims2=..., mlp_hidden_1=..., mlp_hidden_2=..., num_heads=...)
|