oliverlevn commited on
Commit
3af30c8
Β·
verified Β·
1 Parent(s): bfe55df

update readme file

Browse files
Files changed (1) hide show
  1. README.md +73 -162
README.md CHANGED
@@ -1,199 +1,110 @@
1
- ---
2
- license: apache-2.0
3
- tags:
4
- - time-series
5
- - anomaly-detection
6
- - zero-shot
7
- - pytorch
8
- - transformers
9
- library_name: transformers
10
- pipeline_tag: time-series-classification
11
- ---
12
-
13
- # Time-RCD: Zero-Shot Time Series Anomaly Detection
14
-
15
- Time-RCD is a transformer-based model for zero-shot anomaly detection in time series data.
16
-
17
- ## ⚠️ IMPORTANT: Custom Model Loading
18
-
19
- **This model uses a custom architecture not built into transformers.**
20
- You **MUST** include `trust_remote_code=True` when loading:
21
-
22
- ```python
23
- # βœ… CORRECT - Will work
24
- from transformers import AutoModel
25
- model = AutoModel.from_pretrained("your-repo/Time-RCD", trust_remote_code=True)
26
-
27
- # ❌ WRONG - Will throw KeyError: 'time_rcd'
28
- model = AutoModel.from_pretrained("your-repo/Time-RCD") # Missing trust_remote_code=True
29
- ```
30
 
31
- ## Quick Start
32
-
33
- ```python
34
- from transformers import AutoModel, AutoConfig
35
- import numpy as np
36
-
37
- # Load model (trust_remote_code=True is REQUIRED!)
38
- model = AutoModel.from_pretrained(
39
- "thu-sail-lab/Time_RCD",
40
- trust_remote_code=True
41
- )
42
-
43
- # Load processor
44
- from transformers import AutoProcessor
45
- processor = AutoProcessor.from_pretrained(
46
- "thu-sail-lab/Time_RCD",
47
- trust_remote_code=True
48
- )
49
-
50
- # Prepare data
51
- data = np.random.randn(10000, 1) # [n_samples, n_features]
52
-
53
- # Process data
54
- processed = processor(
55
- data,
56
- return_tensors="pt"
57
- )
58
-
59
- # Get anomaly scores
60
- outputs = model(**processed)
61
- anomaly_scores = outputs.anomaly_scores.numpy()
62
- ```
63
 
64
- ## Model Details
65
 
66
- - **Architecture:** Transformer encoder with patch embedding
67
- - **Parameters:** ~5M parameters
68
- - **Patch Size:** 4
69
- - **Hidden Dimension:** 512
70
- - **Projection Dimension:** 256
71
- - **Layers:** 8 transformer layers
72
- - **Attention Heads:** 8 heads
 
 
 
 
 
73
 
74
- ## Features
75
 
76
- βœ… **Zero-shot detection** - No training required
77
- βœ… **Multi-variate support** - Handle multiple features
78
- βœ… **Flexible windows** - Configurable window sizes
79
- βœ… **Robust normalization** - Built-in preprocessing
80
 
81
- ## Usage Examples
82
 
83
- ### Basic Anomaly Detection
84
 
85
- ```python
86
- from transformers import AutoModel
87
- import numpy as np
 
88
 
89
- # IMPORTANT: trust_remote_code=True is required!
90
- model = AutoModel.from_pretrained("thu-sail-lab/Time_RCD", trust_remote_code=True)
91
 
92
- # Your time series (n_samples, n_features)
93
- data = np.random.randn(10000, 1)
 
 
 
94
 
95
- # Get anomaly scores using the zero_shot method
96
- scores, logits = model.zero_shot(data)
97
 
98
- # Flatten scores from list of batches
99
- import numpy as np
100
- all_scores = np.concatenate(scores, axis=0).flatten()
101
 
102
- # Detect anomalies (e.g., top 5%)
103
- threshold = np.percentile(all_scores, 95)
104
- anomalies = all_scores > threshold
 
 
 
 
 
105
  ```
106
 
107
- ### With Custom Processing
108
-
109
- ```python
110
- from transformers import AutoModel
111
- from processing_time_rcd import TimeRCDProcessor
112
- import numpy as np
113
 
114
- # IMPORTANT: trust_remote_code=True is required!
115
- model = AutoModel.from_pretrained("thu-sail-lab/Time_RCD", trust_remote_code=True)
116
 
117
- # Create and configure processor
118
- processor = TimeRCDProcessor(win_size=5000, normalize=True)
 
 
119
 
120
- # Process data
121
- data = np.random.randn(10000, 1)
122
- processed = processor(data, return_tensors="pt")
123
 
124
- # Get predictions
125
- outputs = model(**processed)
126
- anomaly_scores = outputs.anomaly_scores
127
  ```
128
 
129
- ## Configuration
130
 
131
- | Parameter | Default | Description |
132
- |-----------|---------|-------------|
133
- | patch_size | 4 | Patch size for embedding |
134
- | d_model | 512 | Model dimension |
135
- | d_proj | 256 | Projection dimension |
136
- | num_layers | 8 | Transformer layers |
137
- | num_heads | 8 | Attention heads |
138
- | use_rope | True | Rotary position embeddings |
139
 
140
- ## Performance
 
 
 
141
 
142
- Evaluated on various time series anomaly detection benchmarks.
143
 
144
- ## Limitations
145
 
146
- - Requires sufficient data (> window size)
147
- - Performance varies by domain
148
- - High-dimensional data may need preprocessing
149
 
150
- ## Troubleshooting
151
 
152
- ### KeyError: 'time_rcd'
153
 
154
- If you see this error:
155
- ```
156
- KeyError: 'time_rcd'
157
- The checkpoint you are trying to load has model type `time_rcd` but Transformers does not recognize this architecture.
158
  ```
159
 
160
- **Solution:** Add `trust_remote_code=True` to your loading code:
 
161
 
162
- ```python
163
- # This will fix the error
164
- model = AutoModel.from_pretrained("your-repo/Time-RCD", trust_remote_code=True)
 
 
165
  ```
166
 
167
- This is required because Time-RCD is a custom architecture not built into the transformers library. The `trust_remote_code=True` flag tells transformers to load and execute the custom model code from the repository.
168
-
169
- ### Other Common Issues
170
-
171
- **Issue:** `ModuleNotFoundError: No module named 'einops'`
172
- **Solution:** Install einops: `pip install einops`
173
 
174
- **Issue:** Model runs slowly on CPU
175
- **Solution:** Move model to GPU: `model = model.to('cuda')`
176
-
177
- **Issue:** Out of memory errors
178
- **Solution:** Reduce window size or batch size in the processor:
179
- ```python
180
- processor = TimeRCDProcessor(win_size=2000, batch_size=32)
181
- ```
182
-
183
- ## Citation
184
-
185
- ```bibtex
186
- @misc{lan2025foundationmodelszeroshottime,
187
- title={Towards Foundation Models for Zero-Shot Time Series Anomaly Detection: Leveraging Synthetic Data and Relative Context Discrepancy},
188
- author={Tian Lan and Hao Duong Le and Jinbo Li and Wenjun He and Meng Wang and Chenghao Liu and Chen Zhang},
189
- year={2025},
190
- eprint={2509.21190},
191
- archivePrefix={arXiv},
192
- primaryClass={cs.LG},
193
- url={https://arxiv.org/abs/2509.21190},
194
- }
195
- ```
196
 
197
- ## License
198
 
199
- Apache 2.0
 
1
+ # Towards Foundation Models for Zero-Shot Time Series Anomaly Detection: Leveraging Synthetic Data and Relative Context Discrepancy
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
 
3
+ This repository contains the implementation of Time-RCD for time series anomaly detection, integrated with the TSB-AD (Time Series Benchmark for Anomaly Detection) datasets.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
 
5
+ ## Project Structure
6
 
7
+ ```
8
+ .
9
+ β”œβ”€β”€ checkpoints/ # Pre-trained model checkpoints
10
+ β”œβ”€β”€ datasets/ # TSB-AD datasets (univariate and multivariate)
11
+ β”œβ”€β”€ evaluation/ # Evaluation metrics and visualization tools
12
+ β”œβ”€β”€ models/ # Model implementations
13
+ β”‚ └── time_rcd/ # Time-RCD model components
14
+ β”œβ”€β”€ utils/ # Utility functions
15
+ β”œβ”€β”€ testing.py # Main entry point
16
+ β”œβ”€β”€ model_wrapper.py # Model wrapper for different algorithms
17
+ └── README.md # This file
18
+ ```
19
 
20
+ ## Prerequisites
21
 
22
+ - Python 3.10
23
+ - conda (recommended for environment management)
24
+ - Git
 
25
 
26
+ ## Installation
27
 
28
+ ### 1. Create and Activate Conda Environment
29
 
30
+ ```bash
31
+ conda create -n Time-RCD python=3.10
32
+ conda activate Time-RCD
33
+ ```
34
 
35
+ ### 2. Download the Repository
 
36
 
37
+ ```bash
38
+ wget https://anonymous.4open.science/api/repo/TimeRCD-5BE1/zip -O Time-RCD.zip
39
+ unzip Time-RCD.zip -d Time-RCD
40
+ ```
41
+ or dowload from the link: https://anonymous.4open.science/r/TimeRCD-5BE1 and unzip
42
 
43
+ ### 3. Download TSB-AD Datasets
 
44
 
45
+ Create the datasets directory and download the TSB-AD-U (univariate) and TSB-AD-M (multivariate) datasets:
 
 
46
 
47
+ ```bash
48
+ mkdir -p "datasets" \
49
+ && wget -O "datasets/TSB-AD-U.zip" "https://www.thedatum.org/datasets/TSB-AD-U.zip" \
50
+ && wget -O "datasets/TSB-AD-M.zip" "https://www.thedatum.org/datasets/TSB-AD-M.zip" \
51
+ && cd datasets \
52
+ && unzip TSB-AD-U.zip && rm TSB-AD-U.zip \
53
+ && unzip TSB-AD-M.zip && rm TSB-AD-M.zip \
54
+ && cd ..
55
  ```
56
 
57
+ ### 4. Install Python Dependencies
 
 
 
 
 
58
 
59
+ #### Option A: Fast Install (using uv)
 
60
 
61
+ ```bash
62
+ pip install uv
63
+ uv pip install jaxtyping einops pandas numpy scikit-learn transformers torch torchvision statsmodels matplotlib seaborn -U "huggingface_hub[cli]"
64
+ ```
65
 
66
+ #### Option B: Normal Install
 
 
67
 
68
+ ```bash
69
+ pip install jaxtyping einops pandas numpy scikit-learn transformers torch torchvision statsmodels matplotlib seaborn -U "huggingface_hub[cli]"
 
70
  ```
71
 
72
+ ### 5. Download Pre-trained Checkpoints
73
 
74
+ Download the pre-trained model checkpoints from Hugging Face:
 
 
 
 
 
 
 
75
 
76
+ ```bash
77
+ huggingface-cli download thu-sail-lab/Time-RCD checkpoints.zip --local-dir ./
78
+ unzip checkpoints.zip
79
+ ```
80
 
81
+ ### Single Variable Time Series
82
 
83
+ To run anomaly detection on univariate time series:
84
 
85
+ ```bash
86
+ python testing.py
87
+ ```
88
 
89
+ ### Multi-Variable Time Series
90
 
91
+ To run anomaly detection on multivariate time series:
92
 
93
+ ```bash
94
+ python testing.py --mode multi
 
 
95
  ```
96
 
97
+ <!--
98
+ ### 6. Download Training Datasets
99
 
100
+ Download Training Datasets (Optional -for retraining models)
101
+ ```bash
102
+ mkdir training_data
103
+ huggingface-cli download thu-sail-lab/Time-RCD training_data.zip --local-dir ./
104
+ unzip training_data.zip
105
  ```
106
 
107
+ -->
 
 
 
 
 
108
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
109
 
 
110