Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,141 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# VAAS: Vision-Attention Anomaly Scoring
|
| 2 |
+
|
| 3 |
+
## Model Summary
|
| 4 |
+
|
| 5 |
+
VAAS (Vision-Attention Anomaly Scoring) is a dual-module vision framework for image anomaly detection and localisation.
|
| 6 |
+
It combines global attention-based reasoning with patch-level self-consistency analysis to produce a continuous and interpretable anomaly score alongside spatial anomaly maps.
|
| 7 |
+
|
| 8 |
+
The model is designed to indicate **where anomalies occur** and **how strongly they deviate from expected visual consistency**, supporting explainable image analysis and integrity assessment.
|
| 9 |
+
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
## Architecture Overview
|
| 13 |
+
|
| 14 |
+
VAAS consists of two complementary components:
|
| 15 |
+
|
| 16 |
+
- **Global Attention Module (Fx)**
|
| 17 |
+
A Vision Transformer backbone that captures global semantic and structural irregularities using attention distributions.
|
| 18 |
+
|
| 19 |
+
- **Patch-Level Module (Px)**
|
| 20 |
+
A SegFormer-based segmentation model that identifies local inconsistencies in texture, boundaries, and regions.
|
| 21 |
+
|
| 22 |
+
These components are combined via a hybrid scoring mechanism:
|
| 23 |
+
|
| 24 |
+
- `S_F`: Global attention fidelity score
|
| 25 |
+
- `S_P`: Patch-level plausibility score
|
| 26 |
+
- `S_H`: Final hybrid anomaly score
|
| 27 |
+
|
| 28 |
+
`S_H` provides a continuous measure of anomaly intensity rather than a binary decision.
|
| 29 |
+
|
| 30 |
+
---
|
| 31 |
+
|
| 32 |
+
## Model Variant
|
| 33 |
+
|
| 34 |
+
This release corresponds to:
|
| 35 |
+
|
| 36 |
+
- **VAAS v1**
|
| 37 |
+
- Trained on **10% of the DF2023 dataset**
|
| 38 |
+
- Input resolution: `224 × 224`
|
| 39 |
+
- Outputs:
|
| 40 |
+
- Global anomaly score (`S_H`)
|
| 41 |
+
- Component scores (`S_F`, `S_P`)
|
| 42 |
+
- Dense anomaly map (`224 × 224`)
|
| 43 |
+
|
| 44 |
+
Future releases will scale training data size, include cross-dataset evaluation, and explore model compression.
|
| 45 |
+
|
| 46 |
+
---
|
| 47 |
+
|
| 48 |
+
## Intended Use
|
| 49 |
+
|
| 50 |
+
This model can be used for:
|
| 51 |
+
|
| 52 |
+
- Image anomaly detection
|
| 53 |
+
- Visual integrity assessment
|
| 54 |
+
- Explainable inspection of irregular regions
|
| 55 |
+
- Research on attention-based anomaly scoring
|
| 56 |
+
- Prototyping anomaly-aware vision systems
|
| 57 |
+
|
| 58 |
+
It supports both **CPU-only inference** & **GPU-only inference** , though GPU is recommended for faster processing.
|
| 59 |
+
|
| 60 |
+
---
|
| 61 |
+
|
| 62 |
+
## Usage
|
| 63 |
+
|
| 64 |
+
### Load the pipeline
|
| 65 |
+
|
| 66 |
+
```python
|
| 67 |
+
from vaas.inference.pipeline import VAASPipeline
|
| 68 |
+
from PIL import Image
|
| 69 |
+
|
| 70 |
+
pipeline = VAASPipeline.from_pretrained(
|
| 71 |
+
"OBA-Research/vaas-v1-df2023",
|
| 72 |
+
device="cpu",
|
| 73 |
+
alpha=0.5
|
| 74 |
+
)
|
| 75 |
+
|
| 76 |
+
image = Image.open("example.jpg").convert("RGB")
|
| 77 |
+
result = pipeline(image)
|
| 78 |
+
|
| 79 |
+
print(result["S_H"])
|
| 80 |
+
anomaly_map = result["anomaly_map"]
|
| 81 |
+
```
|
| 82 |
+
|
| 83 |
+
### Output Format
|
| 84 |
+
|
| 85 |
+
```python
|
| 86 |
+
{
|
| 87 |
+
"S_F": float,
|
| 88 |
+
"S_P": float,
|
| 89 |
+
"S_H": float,
|
| 90 |
+
"anomaly_map": numpy.ndarray # shape (224, 224)
|
| 91 |
+
}
|
| 92 |
+
```
|
| 93 |
+
|
| 94 |
+
---
|
| 95 |
+
|
| 96 |
+
## Training Data
|
| 97 |
+
|
| 98 |
+
The model was trained on a reproducible 10% subset of DF2023.
|
| 99 |
+
The exact filenames used for training are released to support experiment reproducibility.
|
| 100 |
+
|
| 101 |
+
---
|
| 102 |
+
|
| 103 |
+
## Limitations
|
| 104 |
+
|
| 105 |
+
- Trained on a subset of a single dataset
|
| 106 |
+
- Does not classify anomaly types
|
| 107 |
+
- Performance may degrade on out-of-distribution imagery
|
| 108 |
+
|
| 109 |
+
Users are encouraged to fine-tune or retrain for domain-specific applications.
|
| 110 |
+
|
| 111 |
+
---
|
| 112 |
+
|
| 113 |
+
## Ethical Considerations
|
| 114 |
+
|
| 115 |
+
VAAS is intended for research and inspection purposes.
|
| 116 |
+
It should not be used as a standalone decision-making system in high-stakes settings.
|
| 117 |
+
|
| 118 |
+
---
|
| 119 |
+
|
| 120 |
+
## Citation
|
| 121 |
+
|
| 122 |
+
If you use this model, please cite:
|
| 123 |
+
|
| 124 |
+
```
|
| 125 |
+
Bamigbade, O., Scanlon, M., Sheppard, J.
|
| 126 |
+
Vision-Attention Anomaly Scoring (VAAS).
|
| 127 |
+
Forensic Science International: Digital Investigation, 2026.
|
| 128 |
+
```
|
| 129 |
+
|
| 130 |
+
---
|
| 131 |
+
|
| 132 |
+
## License
|
| 133 |
+
|
| 134 |
+
MIT License
|
| 135 |
+
|
| 136 |
+
---
|
| 137 |
+
|
| 138 |
+
## Maintainers
|
| 139 |
+
|
| 140 |
+
OBA-Research
|
| 141 |
+
https://huggingface.co/OBA-Research
|