Upload 17 files

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+figures/main_result.png filter=lfs diff=lfs merge=lfs -text
+figures/main_result2.png filter=lfs diff=lfs merge=lfs -text
+figures/overview.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,93 @@
+# Wave-Mask/Mix: Exploring Wavelet-Based Augmentations for Time Series Forecasting <a href="https://www.arxiv.org/abs/2408.10951" title="Read the paper on arXiv"><img src="https://img.shields.io/badge/arXiv-2408.10951-b31b1b.svg" margin-left="5px" height="20" align="center"></a>
+
+<a href="https://www.arxiv.org/abs/2408.10951" style="color: #4285F4; font-size: 24px; font-weight: bold; text-decoration: none;">Paper(arXiv)</a>
+
+The figure depicts a framework of the training stages incorporating wavelet augmentations, which involve the concatenation of
+the look-back window and the forecasting horizon prior to transformation and augmentation. Batch sampling of the generated synthetic data is conducted according to a predefined hyperparameter called the sampling rate. These batches are subsequently used to split the data into the look-back window and target horizon, after which they are concatenated with the original data. Wavelet augmentations are Wavelet Masking (WaveMask) and Wavelet Mixing (WaveMix). These techniques utilize the discrete wavelet transform (DWT) to obtain wavelet coefficients (both approximation and detail coefficients) by breaking down the signal and adjusting these coefficients, in line with modifying frequency components across different time scales. 
+
+WaveMask selectively eliminates specific wavelet coefficients at each decomposition level, thereby introducing variability in the augmented data. Conversely, WaveMix exchanges wavelet coefficients from two distinct instances of the dataset, thereby enhancing the diversity of the augmented data. 
+
+To the best of our knowledge, this is the first study to conduct extensive experiments on multivariate time series using Discrete Wavelet Transform as an augmentation technique.
+
+
+
+<div align=center>
+<img src="./figures/overview.png" alt="Overview" width="700" style="margin-bottom: 40px; margin-top: 40px;"/>
+</div>
+
+
+
+Tables present comparisons between our methods and baselines in terms of the metrics Mean Squared Error (MSE) and Mean Absolute Error (MAE). The best result is indicated in bold, while the second most favorable outcome is underlined.
+
+<div align=center>
+<img src="./figures/main_result.png" alt="Main Results" width="1000" height="500" style="margin-bottom: 40px; margin-top: 40px;"/>
+</div>
+
+
+
+<div align=center>
+<img src="./figures/main_result2.png" alt="Main Results" width="1000" height="500" style="margin-bottom: 40px; margin-top: 40px;"/>
+</div>
+
+
+## Dataset
+
+You can obtain all datasets under https://drive.google.com/drive/folders/1ZOYpTUa82_jCcxIdTmyr0LXQfvaM9vIy. All of them are ready for training. 
+
+```
+  mkdir dataset
+```
+Please place all of them within the ```./dataset ``` directory.
+
+## Quick Start
+
+Clone the project
+
+```bash
+  git clone https://github.com/jafarbakhshaliyev/Wave-Augs.git
+```
+
+Go to the project directory
+
+```bash
+  cd Wave-Augs
+```
+
+Install dependencies
+
+```bash
+  pip install -r requirements.txt
+```
+
+Train: 
+
+```bash
+  sh scripts/etth1.sh
+  sh scripts/etth2.sh
+  sh scripts/weather.sh
+  sh scripts/ili.sh
+```
+
+You can change ```percentage``` to down-sample training dataset for ablation study.
+
+## Citation
+
+If you find the code useful, please cite our paper:
+
+```
+@misc{waveaug2024,
+      title={Wave-Mask/Mix: Exploring Wavelet-Based Augmentations for Time Series Forecasting}, 
+      author={Dona Arabi and Jafar Bakhshaliyev and Ayse Coskuner and Kiran Madhusudhanan and Kami Serdar Uckardes},
+      year={2024},
+      eprint={2408.10951},
+      archivePrefix={arXiv},
+      primaryClass={cs.LG},
+      url={https://arxiv.org/abs/2408.10951}, 
+}
+```
+
+Please remember to cite all the datasets and compared methods if you use them in your experiments.
+
+## Acknowledgements
+
+We would like to express our gratitude to [Zheng et al.](https://arxiv.org/abs/2205.13504) for providing datasets used in this project. Additionally, we also acknowledge [Chen et al.](https://arxiv.org/abs/2302.09292) and [Zhang et al.](https://arxiv.org/abs/2303.14254)  for their code frameworks, which served as the foundation for our codebase.

augmentation/aug.py

@@ -0,0 +1,248 @@
+# =============================================================================
+# The freq_mask, freq_mix methods are adapted from the following sources:
+#  Chen, M., Xu, Z., Zeng, A., & Xu, Q. (2023). "FrAug: Frequency Domain Augmentation for Time Series Forecasting".
+#  arXiv preprint arXiv:2302.09292.
+#
+# The emd_aug and mix_aug for STAug method are adapted from the following source:
+#  https://github.com/xiyuanzh/STAug/tree/main
+# =============================================================================
+
+import numpy as np
+import torch
+import pywt
+from pytorch_wavelets import DWT1DForward, DWT1DInverse
+from typing import List, Tuple
+
+class augmentation():
+
+  """
+    A class for data augmentation techniques used for Time Series Forecasting.
+
+    Attributes:
+    None
+
+    Methods:
+    - freq_mask: Apply frequency masking to input data.
+    - freq_mix: Mix two input signals in the frequency domain.
+    - wave_mask: Apply wavelet-based masking to input data.
+    - wave_mix: Mix two input signals using wavelet transformation.
+    - emd_aug: Apply empirical mode decomposition (EMD) based augmentation.
+    - mix_aug: Mix two batches of data with a random interpolation factor.
+    """
+
+  def __init__(self):
+        pass
+
+  @staticmethod
+  def freq_mask(x: torch.Tensor, y: torch.Tensor, rate: float = 0.5, dim: int = 1) -> torch.Tensor:
+    """
+        Apply frequency masking to input data.
+
+        Args:
+        - x (torch.Tensor): Look-back window.
+        - y (torch.Tensor): Target horizon.
+        - rate (float): Mask rate.
+        - dim (int): Dimension along to concatenate and apply Fourier Transform.
+
+        Returns:
+        - torch.Tensor: Masked synthetic data tensor.
+    """
+
+    xy = torch.cat([x,y],dim=1)
+    xy_f = torch.fft.rfft(xy,dim=dim)
+    m = torch.FloatTensor(xy_f.shape).uniform_() < rate
+    freal = xy_f.real.masked_fill(m,0)
+    fimag = xy_f.imag.masked_fill(m,0)
+    xy_f = torch.complex(freal,fimag)
+    xy = torch.fft.irfft(xy_f,dim=dim)
+    return xy
+  
+  @staticmethod
+  def freq_mix(x: torch.Tensor, y: torch.Tensor, rate: float = 0.5, dim: int = 1) -> torch.Tensor:
+    """
+        Mix two input signals in the frequency domain.
+
+        Args:
+        - x (torch.Tensor): Look-back window.
+        - y (torch.Tensor): Target horizon.
+        - rate (float): Mix rate.
+        - dim (int): Dimension along to concatenate and apply Fourier Transform.
+
+        Returns:
+        - torch.Tensor: Mixed synthetic data tensor.
+    """
+
+    xy = torch.cat([x,y],dim=dim)
+    xy_f = torch.fft.rfft(xy,dim=dim)
+
+    m = torch.FloatTensor(xy_f.shape).uniform_() < rate
+    amp = abs(xy_f)
+    _,index = amp.sort(dim=dim, descending=True)
+    dominant_mask = index > 2
+    m = torch.bitwise_and(m, dominant_mask)
+    freal = xy_f.real.masked_fill(m,0)
+    fimag = xy_f.imag.masked_fill(m,0)
+
+    b_idx = np.arange(x.shape[0])
+    np.random.shuffle(b_idx)
+    x2, y2 = x[b_idx], y[b_idx]
+    xy2 = torch.cat([x2,y2],dim=dim)
+    xy2_f = torch.fft.rfft(xy2,dim=dim)
+
+    m = torch.bitwise_not(m)
+    freal2 = xy2_f.real.masked_fill(m,0)
+    fimag2 = xy2_f.imag.masked_fill(m,0)
+
+    freal += freal2
+    fimag += fimag2
+
+    xy_f = torch.complex(freal,fimag)
+
+    xy = torch.fft.irfft(xy_f,dim=dim)
+
+    return xy
+  
+  
+  @staticmethod
+  def wave_mask(x: torch.Tensor, y: torch.Tensor, rates: List[float], wavelet: str = 'db1', level: int = 2, dim: int= 1) -> torch.Tensor:
+      """
+          Apply wavelet-based masking to input data.
+
+          Args:
+          - x (torch.Tensor): Look-back window.
+          - y (torch.Tensor): Target horizon.
+          - rates (list of floats): List of mask rates for each wavelet level.
+          - wavelet (str): Type of wavelet to use.
+          - level (int): Number of decomposition levels.
+          - dim (int): Dimension along which to concatenate and apply DWT.
+
+          Returns:
+          - torch.Tensor: Masked synthetic data tensor.
+      """
+
+      xy = torch.cat([x, y], dim=1)  # Concatenate along the time dimension
+      rate_tensor = torch.tensor(rates, device=x.device) # Convert rates to tensor
+
+      # Permute dimensions to match expected input: (batch_size, num_features, seq_len)
+      xy = xy.permute(0, 2, 1).to(x.device)
+
+      # Initialize & perform the DWT
+      dwt = DWT1DForward(J=level, wave=wavelet, mode='symmetric').to(x.device)
+      cA, cDs = dwt(xy)
+
+      mask = torch.rand_like(cA).to(cA.device) < rate_tensor[0]
+      cA = cA.masked_fill(mask, 0)
+
+      # Apply masking to detail coefficients
+      masked_cDs = []
+      for i, cD in enumerate(cDs, 1):
+          mask_cD = torch.rand_like(cD).to(cD.device) < rate_tensor[i]  # Create mask
+          cD = cD.masked_fill(mask_cD, 0)
+          masked_cDs.append(cD)
+
+      # Initialize the inverse DWT & reconstruct the signal
+      idwt = DWT1DInverse(wave=wavelet, mode='symmetric').to(x.device)
+      reconstructed = idwt((cA, masked_cDs))
+
+      # Permute back to original shape: (batch_size, seq_len, num_features)
+      reconstructed = reconstructed.permute(0, 2, 1)
+
+      return reconstructed
+  
+  @staticmethod
+  def wave_mix(x: torch.Tensor, y: torch.Tensor, rates: List[float], wavelet: str = 'db1', level: int = 2, dim: int = 1) -> torch.Tensor:
+      """
+          Mix two input signals using wavelet transformation.
+
+          Args:
+          - x (torch.Tensor): Look-back window.
+          - y (torch.Tensor): Target horizon.
+          - rates (list of floats): List of mix rates for each wavelet level.
+          - wavelet (str): Type of wavelet to use.
+          - level (int): Number of decomposition levels.
+          - dim (int): Dimension along which to concatenate and apply DWT.
+
+          Returns:
+          - torch.Tensor: Mixed synthetic data tensor.
+      """
+
+      xy = torch.cat([x, y], dim=1)  # Concatenate along the time dimension
+      batch_size, _, _ = xy.shape
+      rate_tensor = torch.tensor(rates, device=x.device) # Convert rates to tensor
+
+      # Permute dimensions to match expected input: (batch_size, num_features, seq_len)
+      xy = xy.permute(0, 2, 1).to(x.device)
+
+      # Shuffle the batch for mixing
+      b_idx = torch.randperm(batch_size)
+      xy2 = xy[b_idx]
+
+      # Initialize & perform the DWT on the both signals
+      dwt = DWT1DForward(J=level, wave=wavelet, mode='symmetric').to(x.device)
+      cA1, cDs1 = dwt(xy)
+      cA2, cDs2 = dwt(xy2)
+
+      # Mix the approximation coefficients
+      mask = torch.rand_like(cA1).to(cA1.device) < rate_tensor[0] # Create mask
+      cA_mixed = cA1.masked_fill(mask, 0) + cA2.masked_fill(~mask, 0)
+
+      # Mix the coefficients
+      mixed_cDs = []
+      for i, (cD1, cD2) in enumerate(zip(cDs1, cDs2), 1):
+          mask = torch.rand_like(cD1).to(cD1.device) < rate_tensor[i] # Create mask
+          cD_mixed = cD1.masked_fill(mask, 0) + cD2.masked_fill(~mask, 0)
+          mixed_cDs.append(cD_mixed)
+
+      # Initialize the inverse DWT & reconstruct the signal
+      idwt = DWT1DInverse(wave=wavelet, mode='symmetric').to(x.device)
+      reconstructed = idwt((cA_mixed, mixed_cDs))
+
+      # Permute back to original shape: (batch_size, seq_len, num_features)
+      reconstructed = reconstructed.permute(0, 2, 1)
+
+      return reconstructed
+
+  # StAug: frequency-domain augmentation 
+  def emd_aug(self, x: torch.Tensor) -> torch.Tensor:
+      """
+        Apply augmentation on empirical mode decomposition (EMD).
+
+        Args:
+        - x (torch.Tensor): Input tensor.
+
+        Returns:
+        - torch.Tensor: Augmented tensor.
+      """
+
+      b,n_imf,t,c = x.size()
+      inp = x.permute(0,2,1,3).reshape(b,t,n_imf*c) #b,t,n_imf,c -> b,t,n_imf*c
+      if(torch.rand(1) >= 0.5):
+          w = 2 * torch.rand((b,1,n_imf*c)).cuda()
+      else:
+          w = torch.ones((b,1,n_imf*c)).cuda()
+      w_exp = w.expand(-1,t,-1) #b,t,n_imf*c
+      out = w_exp * inp
+      out = out.reshape(b,t,n_imf,c).sum(dim=2) #b,t,c
+      
+      return out
+
+# StAug: time-domain augmentation
+  def mix_aug(self, batch_x: np.ndarray, batch_y: np.ndarray, lambd: float = 0.5) -> Tuple[np.ndarray, np.ndarray]:
+      """
+        Mix two batches of data with a random interpolation factor.
+
+        Args:
+        - batch_x (numpy.ndarray): Input batch 1.
+        - batch_y (numpy.ndarray): Input batch 2.
+        - lambd (float): Beta distribution parameter for interpolation.
+
+        Returns:
+        - numpy.ndarray: Mixed augmented batches.
+      """
+
+      inds2 = np.random.permutation(len(batch_x))
+      lam = np.random.beta(lambd, lambd)
+      batch_x = lam * batch_x[inds2] + (1-lam) * batch_x
+      batch_y = lam * batch_y[inds2] + (1-lam) * batch_y
+
+      return batch_x, batch_y

dataset_loader/datasetloader.py

@@ -0,0 +1,431 @@
+# =============================================================================
+# The code is originated from
+#  Chen, M., Xu, Z., Zeng, A., & Xu, Q. (2023). "FrAug: Frequency Domain Augmentation for Time Series Forecasting".
+#  arXiv preprint arXiv:2302.09292.
+# =============================================================================
+
+from torch.utils.data import Dataset, DataLoader
+import pandas as pd
+import numpy as np
+import os
+from sklearn.preprocessing import StandardScaler
+from decompositions.decomposition import emd_augment
+
+class Dataset_ETT_hour(Dataset):
+    def __init__(self, root_path, flag='train', size=None,
+                 features='S', data_path='ETTh1.csv',
+                 target='OT', scale=True, freq='h', n_imf = 500, percentage = 100, params = None):
+        # size [seq_len, label_len, pred_len]
+        # info
+        if size == None:
+            self.seq_len = 24 * 4 * 4
+            self.label_len = 24 * 4
+            self.pred_len = 24 * 4
+        else:
+            self.seq_len = size[0]
+            self.label_len = size[1]
+            self.pred_len = size[2]
+        # init
+        assert flag in ['train', 'test', 'val']
+        type_map = {'train': 0, 'val': 1, 'test': 2}
+        self.set_type = type_map[flag]
+
+        self.features = features
+        self.target = target
+        self.scale = scale
+        self.freq = freq
+        self.n_imf = n_imf
+        self.percentage = percentage
+
+        self.root_path = root_path
+        self.data_path = data_path
+        self.params = params
+        self.__read_data__()
+
+    def __read_data__(self):
+        self.scaler = StandardScaler()
+        df_raw = pd.read_csv(os.path.join(self.root_path,
+                                          self.data_path))
+
+
+        border1s = [0, 12 * 30 * 24 - self.seq_len, 12 * 30 * 24 + 4 * 30 * 24 - self.seq_len]
+        border2s = [12 * 30 * 24, 12 * 30 * 24 + 4 * 30 * 24, 12 * 30 * 24 + 8 * 30 * 24]
+        border1 = border1s[self.set_type]
+        border2 = border2s[self.set_type]
+
+        if self.features == 'M' or self.features == 'MS':
+            cols_data = df_raw.columns[1:]
+            df_data = df_raw[cols_data]
+        elif self.features == 'S':
+            df_data = df_raw[[self.target]]
+
+        if self.scale:
+            train_data = df_data[border1s[0]:border2s[0]]
+            train_length = int((self.percentage / 100) * len(train_data))
+            train_data = train_data[-train_length:]
+            self.scaler.fit(train_data.values)
+            data = self.scaler.transform(df_data.values)
+        else:
+            data = df_data.values
+
+        if self.set_type == 0 and self.params.aug_type == 5:
+            self.aug_data = emd_augment(data[border1:border2][-len(train_data):], self.seq_len+self.pred_len, n_IMF = self.n_imf)
+        else: 
+            self.aug_data = np.zeros_like(data[border1:border2])    
+    
+        if self.set_type == 0:
+            self.data_x = data[border1:border2][-len(train_data):]
+            self.data_y = data[border1:border2][-len(train_data):]
+        else: 
+            self.data_x = data[border1:border2]
+            self.data_y = data[border1:border2]
+
+
+    def __getitem__(self, index):
+        s_begin = index
+        s_end = s_begin + self.seq_len
+        r_begin = s_end - self.label_len
+        r_end = r_begin + self.label_len + self.pred_len
+
+        seq_x = self.data_x[s_begin:s_end]
+        seq_y = self.data_y[r_begin:r_end]
+
+        if self.params.aug_type == 5:
+            aug_data = self.aug_data[s_begin]
+        else:
+            aug_data = np.array([])
+            
+        return seq_x, seq_y, aug_data
+
+    def __len__(self):
+        return len(self.data_x) - self.seq_len - self.pred_len + 1
+
+    def inverse_transform(self, data):
+        return self.scaler.inverse_transform(data)
+    
+class Dataset_ETT_minute(Dataset):
+    def __init__(self, root_path, flag='train', size=None,
+                 features='S', data_path='ETTm1.csv',
+                 target='OT', scale=True, freq='t', n_imf = 500, percentage = 100, params = None):
+        # size [seq_len, label_len, pred_len]
+        # info
+        if size == None:
+            self.seq_len = 24 * 4 * 4
+            self.label_len = 24 * 4
+            self.pred_len = 24 * 4
+        else:
+            self.seq_len = size[0]
+            self.label_len = size[1]
+            self.pred_len = size[2]
+        # init
+        assert flag in ['train', 'test', 'val']
+        type_map = {'train': 0, 'val': 1, 'test': 2}
+        self.set_type = type_map[flag]
+
+        self.features = features
+        self.target = target
+        self.scale = scale
+        self.freq = freq
+        self.n_imf = n_imf
+
+        self.root_path = root_path
+        self.data_path = data_path
+        self.percentage = percentage
+        self.params = params
+        self.__read_data__()
+
+    def __read_data__(self):
+        self.scaler = StandardScaler()
+        df_raw = pd.read_csv(os.path.join(self.root_path,
+                                          self.data_path))
+
+        border1s = [0, 12 * 30 * 24 * 4 - self.seq_len, 12 * 30 * 24 * 4 + 4 * 30 * 24 * 4 - self.seq_len]
+        border2s = [12 * 30 * 24 * 4, 12 * 30 * 24 * 4 + 4 * 30 * 24 * 4, 12 * 30 * 24 * 4 + 8 * 30 * 24 * 4]
+        border1 = border1s[self.set_type]
+        border2 = border2s[self.set_type]
+
+        if self.features == 'M' or self.features == 'MS':
+            cols_data = df_raw.columns[1:]
+            df_data = df_raw[cols_data]
+        elif self.features == 'S':
+            df_data = df_raw[[self.target]]
+
+        if self.scale:
+            train_data = df_data[border1s[0]:border2s[0]]
+            train_length = int((self.percentage / 100) * len(train_data))
+            train_data = train_data[-train_length:]
+            self.scaler.fit(train_data.values)
+            data = self.scaler.transform(df_data.values)
+        else:
+            data = df_data.values
+
+        if self.set_type == 0 and self.params.aug_type == 5:
+            self.aug_data = emd_augment(data[border1:border2][-len(train_data):], self.seq_len+self.pred_len, n_IMF = self.n_imf)
+        else: 
+            self.aug_data = np.zeros_like(data[border1:border2])
+
+        if self.set_type == 0:
+            self.data_x = data[border1:border2][-len(train_data):]
+            self.data_y = data[border1:border2][-len(train_data):]
+        else: 
+            self.data_x = data[border1:border2]
+            self.data_y = data[border1:border2]
+
+    def __getitem__(self, index):
+        s_begin = index
+        s_end = s_begin + self.seq_len
+        r_begin = s_end - self.label_len
+        r_end = r_begin + self.label_len + self.pred_len
+
+        seq_x = self.data_x[s_begin:s_end]
+        seq_y = self.data_y[r_begin:r_end]
+
+        if self.params.aug_type == 5:
+            aug_data = self.aug_data[s_begin]
+        else:
+            aug_data = np.array([])
+
+        return seq_x, seq_y, aug_data
+        
+    def __len__(self):
+        return len(self.data_x) - self.seq_len - self.pred_len + 1
+
+    def inverse_transform(self, data):
+        return self.scaler.inverse_transform(data)
+
+
+class Dataset_Custom(Dataset):
+    def __init__(self, root_path, flag='train', size=None,
+                 features='S', data_path='ETTh1.csv', scale = True,
+                 target='OT', freq='h', n_imf = 500, percentage = 100, params=None):
+        # size [seq_len, label_len, pred_len]
+        # info
+
+        self.seq_len = size[0]
+        self.label_len = size[1]
+        self.pred_len = size[2]
+        # init
+        assert flag in ['train', 'test', 'val']
+        type_map = {'train': 0, 'val': 1, 'test': 2}
+        self.set_type = type_map[flag]
+
+        self.features = features
+        self.target = target
+        self.freq = freq
+        self.scale = scale
+        self.n_imf = n_imf
+        self.percentage = percentage
+        self.root_path = root_path
+        self.data_path = data_path
+        self.params = params
+
+        self.__read_data__()
+
+    def __read_data__(self):
+
+        self.scaler = StandardScaler()
+        df_raw = pd.read_csv(os.path.join(self.root_path,
+                                          self.data_path))
+        '''
+        df_raw.columns: ['date', ...(other features), target feature]
+        '''
+        cols = list(df_raw.columns)
+        cols.remove(self.target)
+        cols.remove('date')
+        df_raw = df_raw[['date'] + cols + [self.target]]
+
+        num_train = int(len(df_raw) * 0.7)
+        num_test = int(len(df_raw) * 0.2)
+        num_vali = len(df_raw) - num_train - num_test
+        border1s = [0, num_train - self.seq_len, len(df_raw) - num_test - self.seq_len]
+        border2s = [num_train, num_train + num_vali, len(df_raw)]
+        border1 = border1s[self.set_type]
+        border2 = border2s[self.set_type]
+
+        if self.features == 'M' or self.features == 'MS':
+            cols_data = df_raw.columns[1:]
+            df_data = df_raw[cols_data]
+        elif self.features == 'S':
+            df_data = df_raw[[self.target]]
+
+        if self.scale:
+            train_data = df_data[border1s[0]:border2s[0]]
+            train_length = int((self.percentage / 100) * len(train_data))
+            train_data = train_data[-train_length:]
+            self.scaler.fit(train_data.values)
+            data = self.scaler.transform(df_data.values)
+        else:
+            data = df_data.values
+
+        if self.set_type == 0 and self.params.aug_type == 5:
+            self.aug_data = emd_augment(data[border1:border2][-len(train_data):], self.seq_len+self.pred_len, n_IMF = self.n_imf)
+        
+        else: 
+            self.aug_data = np.zeros_like(data[border1:border2])      
+
+        if self.set_type == 0:
+            self.data_x = data[border1:border2][-len(train_data):]
+            self.data_y = data[border1:border2][-len(train_data):]
+        else: 
+            self.data_x = data[border1:border2]
+            self.data_y = data[border1:border2]
+
+    def __getitem__(self, index):
+        s_begin = index # 0
+        s_end = s_begin + self.seq_len
+        r_begin = s_end - self.label_len 
+        r_end = r_begin + self.label_len + self.pred_len
+
+        seq_x = self.data_x[s_begin:s_end] 
+        seq_y = self.data_y[r_begin:r_end] 
+
+        if self.params.aug_type == 5:
+            aug_data = self.aug_data[s_begin]
+        else:
+            aug_data = np.array([])
+        
+        return seq_x, seq_y, aug_data
+
+    def __len__(self):
+        return len(self.data_x) - self.seq_len - self.pred_len + 1
+
+    def inverse_transform(self, data):
+        return self.scaler.inverse_transform(data)
+
+
+class Dataset_Pred(Dataset):
+
+    def __init__(self, root_path, flag='pred', size=None,
+                 features='S', data_path='ETTh1.csv',
+                 target='OT', scale=True, inverse=False, freq='15min', cols=None):
+        # size [seq_len, label_len, pred_len]
+        # info
+
+        self.seq_len = size[0]
+        self.label_len = size[1]
+        self.pred_len = size[2]
+        # init
+        assert flag in ['pred']
+
+        self.features = features
+        self.target = target
+        self.scale = scale
+        self.inverse = inverse
+        self.freq = freq
+        self.cols = cols
+        self.root_path = root_path
+        self.data_path = data_path
+        self.__read_data__()
+
+    def __read_data__(self):
+        self.scaler = StandardScaler()
+        df_raw = pd.read_csv(os.path.join(self.root_path,
+                                          self.data_path))
+        '''
+        df_raw.columns: ['date', ...(other features), target feature]
+        '''
+        if self.cols:
+            cols = self.cols.copy()
+            cols.remove(self.target)
+        else:
+            cols = list(df_raw.columns)
+            cols.remove(self.target)
+            cols.remove('date')
+        df_raw = df_raw[['date'] + cols + [self.target]]
+        border1 = len(df_raw) - self.seq_len
+        border2 = len(df_raw)
+
+        if self.features == 'M' or self.features == 'MS':
+            cols_data = df_raw.columns[1:]
+            df_data = df_raw[cols_data]
+        elif self.features == 'S':
+            df_data = df_raw[[self.target]]
+
+        if self.scale:
+            self.scaler.fit(df_data.values)
+            data = self.scaler.transform(df_data.values)
+        else:
+            data = df_data.values
+
+
+        self.data_x = data[border1:border2]
+        if self.inverse:
+            self.data_y = df_data.values[border1:border2]
+        else:
+            self.data_y = data[border1:border2]
+
+    def __getitem__(self, index):
+        s_begin = index
+        s_end = s_begin + self.seq_len
+        r_begin = s_end - self.label_len
+        r_end = r_begin + self.label_len + self.pred_len
+
+        seq_x = self.data_x[s_begin:s_end]
+        if self.inverse:
+            seq_y = self.data_x[r_begin:r_begin + self.label_len]
+        else:
+            seq_y = self.data_y[r_begin:r_begin + self.label_len]
+
+        aug_data = np.array([])
+
+        return seq_x, seq_y, aug_data
+
+
+    def __len__(self):
+        return len(self.data_x) - self.seq_len + 1
+
+    def inverse_transform(self, data):
+        return self.scaler.inverse_transform(data)
+
+data_dict = {
+    'ETTh1': Dataset_ETT_hour,
+    'ETTh2': Dataset_ETT_hour,
+    'ETTm1': Dataset_ETT_minute,
+    'ETTm2': Dataset_ETT_minute,
+    'custom': Dataset_Custom,
+}
+
+
+def data_provider(args, flag):
+    Data = data_dict[args.data]
+
+    if flag == 'test':
+        shuffle_flag = False
+        drop_last = True
+        batch_size = args.batch_size
+        freq = args.freq
+        nIMF = args.nIMF
+
+    elif flag == 'pred':
+        shuffle_flag = False
+        drop_last = False
+        batch_size = 1
+        freq = args.freq
+        Data = Dataset_Pred
+    else:
+        shuffle_flag = True
+        drop_last = True
+        batch_size = args.batch_size
+        freq = args.freq
+        nIMF = args.nIMF
+
+
+    data_set = Data(
+        root_path=args.root_path,
+        data_path=args.data_path,
+        flag=flag,
+        size=[args.seq_len, args.label_len, args.pred_len],
+        features=args.features,
+        target=args.target,
+        freq=freq,
+        n_imf = nIMF,
+        percentage = args.percentage,
+        params = args
+    )
+    data_loader = DataLoader(
+        data_set,
+        batch_size=batch_size,
+        shuffle=shuffle_flag,
+        num_workers=args.num_workers,
+        drop_last=drop_last)
+    return data_set, data_loader

decompositions/decomposition.py

@@ -0,0 +1,28 @@
+# =============================================================================
+# The code is originated from
+# https://github.com/xiyuanzh/STAug/tree/main
+# =============================================================================
+
+import numpy as np
+from PyEMD import EMD
+
+def emd_augment(data, sequence_length, n_IMF = 500):
+    n_imf, channel_num = n_IMF, data.shape[1]
+    emd_data = np.zeros((n_imf,data.shape[0],channel_num))
+    max_imf = 0
+    for ci in range(channel_num):
+        s = data[:, ci]
+        IMF = EMD().emd(s)
+        r_s = np.zeros((n_imf, data.shape[0]))
+        if len(IMF) > max_imf:
+            max_imf = len(IMF)
+        for i in range(len(IMF)):
+            r_s[i] = IMF[len(IMF)-1-i] 
+        if(len(IMF)==0): r_s[0] = s
+        emd_data[:,:,ci] = r_s
+    if max_imf < n_imf:
+        emd_data = emd_data[:max_imf,:,:]
+    train_data_new = np.zeros((len(data)-sequence_length+1,max_imf,sequence_length,channel_num))
+    for i in range(len(data)-sequence_length+1):
+        train_data_new[i] = emd_data[:,i:i+sequence_length,:]
+    return train_data_new

main_run/train.py

@@ -0,0 +1,290 @@
+# =============================================================================
+# The code is originated from
+# Chen, M., Xu, Z., Zeng, A., & Xu, Q. (2023). "FrAug: Frequency Domain Augmentation for Time Series Forecasting".
+# arXiv preprint arXiv:2302.09292.
+# =============================================================================
+
+import os
+import torch
+import torch.nn.functional as F
+import numpy as np
+
+from dataset_loader.datasetloader import data_provider
+from models import DLinear
+from utils.tools import EarlyStopping, adjust_learning_rate, visual, test_params_flop
+from utils.metrics import metric
+
+import torch
+import torch.nn as nn
+from torch import optim
+import time
+import warnings
+import matplotlib.pyplot as plt
+from augmentation.aug import augmentation
+
+
+class Exp_Basic(object):
+    def __init__(self, args):
+        self.args = args
+        self.device = self._acquire_device()
+        self.model = self._build_model().to(self.device)
+
+    def _build_model(self):
+        raise NotImplementedError
+        return None
+
+    def _acquire_device(self):
+        if self.args.use_gpu:
+            os.environ["CUDA_VISIBLE_DEVICES"] = str(
+                self.args.gpu) if not self.args.use_multi_gpu else self.args.devices
+            device = torch.device('cuda:{}'.format(self.args.gpu))
+            print('Use GPU: cuda:{}'.format(self.args.gpu))
+        else:
+            device = torch.device('cpu')
+            print('Use CPU')
+        return device
+
+    def _get_data(self):
+        pass
+
+    def vali(self):
+        pass
+
+    def train(self):
+        pass
+
+    def test(self):
+        pass
+
+warnings.filterwarnings('ignore')
+
+TYPES = {0: 'None',
+         1: 'Freq-Mask',
+         2: 'Freq-Mix',
+         3: 'Wave-Mask',
+         4: 'Wave-Mix',
+         5: 'StAug'}
+
+class Exp_Main(Exp_Basic):
+
+    def __init__(self, args):
+        super(Exp_Main, self).__init__(args)
+
+    def _build_model(self):
+         
+        model_dict = {
+            'DLinear': DLinear
+         }
+        model = model_dict[self.args.model].Model(self.args).float()
+        if self.args.use_multi_gpu and self.args.use_gpu:
+            model = nn.DataParallel(model, device_ids=self.args.device_ids)
+        return model
+
+    def _get_data(self, flag):
+        data_set, data_loader = data_provider(self.args, flag)
+        return data_set, data_loader
+
+    def _select_optimizer(self):
+        model_optim = optim.Adam(self.model.parameters(), lr=self.args.learning_rate)
+        return model_optim
+
+    def _select_criterion(self):
+        criterion = nn.MSELoss()
+        return criterion
+
+    def vali(self, vali_data, vali_loader, criterion):
+
+        total_loss = []
+        self.model.eval()
+        with torch.no_grad():
+            for i, (batch_x, batch_y, _) in enumerate(vali_loader):
+                batch_x = batch_x.float().to(self.device)
+                batch_y = batch_y.float()
+
+                outputs = self.model(batch_x)
+
+                f_dim = -1 if self.args.features == 'MS' else 0
+                outputs = outputs[:, -self.args.pred_len:, f_dim:]
+                batch_y = batch_y[:, -self.args.pred_len:, f_dim:].to(self.device)
+
+                pred = outputs.detach().cpu()
+                true = batch_y.detach().cpu()
+
+                loss = criterion(pred, true)
+
+                total_loss.append(loss)
+        total_loss = np.average(total_loss)
+        self.model.train()
+        return total_loss
+
+    def train(self, setting):
+        train_data, train_loader = self._get_data(flag='train')
+        vali_data, vali_loader = self._get_data(flag='val')
+        test_data, test_loader = self._get_data(flag='test')
+
+        path = os.path.join(self.args.checkpoints, setting)
+        if not os.path.exists(path):
+            os.makedirs(path)
+
+        train_steps = len(train_loader)
+        early_stopping = EarlyStopping(patience=self.args.patience, verbose=True)
+        time_now = time.time()
+
+        model_optim = self._select_optimizer()
+        criterion = self._select_criterion()
+        time_now = time.time()
+
+        for epoch in range(self.args.train_epochs):
+            iter_count = 0
+            train_loss = []
+
+            self.model.train()
+            epoch_time = time.time()
+
+    
+            for i, (batch_x, batch_y, aug_data) in enumerate(train_loader):
+
+
+                iter_count += 1
+                model_optim.zero_grad()
+                if self.args.aug_type == 5:
+                    aug_data = aug_data.float().to(self.device)
+                else:
+                    aug_data = None
+                
+                if self.args.aug_type:
+                  aug = augmentation()
+                  if self.args.aug_type == 1:
+                    xy = aug.freq_mask(batch_x, batch_y[:, -self.args.pred_len:, :], rate=self.args.aug_rate, dim=1)
+                    batch_x2, batch_y2 = xy[:, :self.args.seq_len, :], xy[:, -self.args.label_len-self.args.pred_len:, :]
+                    batch_x = torch.cat([batch_x,batch_x2],dim=0)
+                    batch_y = torch.cat([batch_y,batch_y2],dim=0)
+                  elif self.args.aug_type  == 2:
+                    xy = aug.freq_mix(batch_x, batch_y[:, -self.args.pred_len:, :], rate=self.args.aug_rate, dim=1)
+                    batch_x2, batch_y2 = xy[:, :self.args.seq_len, :], xy[:, -self.args.label_len-self.args.pred_len:, :]
+                    batch_x = torch.cat([batch_x,batch_x2],dim=0)
+                    batch_y = torch.cat([batch_y,batch_y2],dim=0)
+                  elif self.args.aug_type == 3:
+                    xy = aug.wave_mask(batch_x, batch_y[:, -self.args.pred_len:, :] ,rates = self.args.rates, wavelet =self.args.wavelet, level = self.args.level, dim = 1)
+                    batch_x2, batch_y2 = xy[:, :self.args.seq_len, :], xy[:, -self.args.label_len-self.args.pred_len:, :]
+                    sampling_steps = int(batch_x2.shape[0] * self.args.sampling_rate)
+                    indices = torch.randperm(batch_x2.shape[0])[:sampling_steps]
+                    batch_x2 = batch_x2[indices,:,:]
+                    batch_y2 = batch_y2[indices,:,:]
+                    batch_x = torch.cat([batch_x,batch_x2],dim=0)
+                    batch_y = torch.cat([batch_y,batch_y2],dim=0)
+                  elif self.args.aug_type == 4:
+                    batch_x = batch_x.float().to(self.device)
+                    batch_y = batch_y.float().to(self.device)
+                    xy = aug.wave_mix(batch_x, batch_y[:, -self.args.pred_len:, :] ,rates = self.args.rates, wavelet = self.args.wavelet, level = self.args.level, dim = 1)
+                    batch_x2, batch_y2 = xy[:, :self.args.seq_len, :], xy[:, -self.args.label_len-self.args.pred_len:, :]
+                    sampling_steps = int(batch_x2.shape[0] * self.args.sampling_rate)
+                    indices = torch.randperm(batch_x2.shape[0])[:sampling_steps]
+                    batch_x2 = batch_x2[indices,:,:]
+                    batch_y2 = batch_y2[indices,:,:]
+                    batch_x = torch.cat([batch_x,batch_x2],dim=0)
+                    batch_y = torch.cat([batch_y,batch_y2],dim=0)
+                  elif self.args.aug_type == 5:
+                    batch_x = batch_x.float().to(self.device)
+                    batch_y = batch_y.float().to(self.device)
+                    weighted_xy = aug.emd_aug(aug_data)
+                    weighted_x, weighted_y = weighted_xy[:,:self.args.seq_len,:], weighted_xy[:,-self.args.label_len-self.args.pred_len:,:]
+                    batch_x, batch_y = aug.mix_aug(weighted_x, weighted_y, lambd = self.args.aug_rate)
+                     
+
+                batch_x = batch_x.float().to(self.device)
+                batch_y = batch_y.float().to(self.device)
+
+                outputs = self.model(batch_x)
+
+                f_dim = -1 if self.args.features == 'MS' else 0
+                outputs = outputs[:, -self.args.pred_len:, f_dim:] 
+                batch_y = batch_y[:, -self.args.pred_len:, f_dim:].to(self.device) 
+                loss = criterion(outputs, batch_y)
+                train_loss.append(loss.item())
+
+                if (i + 1) % 100 == 0:
+                    print("\titers: {0}, epoch: {1} | loss: {2:.7f}".format(i + 1, epoch + 1, loss.item()))
+                    speed = (time.time() - time_now) / iter_count
+                    left_time = speed * ((self.args.train_epochs - epoch) * train_steps - i)
+                    print('\tspeed: {:.4f}s/iter; left time: {:.4f}s'.format(speed, left_time))
+                    iter_count = 0
+                    time_now = time.time()
+
+                loss.backward()
+                model_optim.step()
+
+            print("Epoch: {} cost time: {}".format(epoch + 1, time.time() - epoch_time))
+            train_loss = np.average(train_loss)
+            vali_loss = self.vali(vali_data, vali_loader, criterion)
+            test_loss = self.vali(test_data, test_loader, criterion)
+
+            print("Epoch: {0}, Steps: {1} | Train Loss: {2:.7f} Vali Loss: {3:.7f} Test Loss: {4:.7f}".format(
+                epoch + 1, train_steps, train_loss, vali_loss, test_loss))
+            
+            early_stopping(vali_loss, self.model, path)
+            if early_stopping.early_stop:
+                print("Early stopping")
+                break
+            
+            adjust_learning_rate(model_optim, epoch + 1, self.args)
+
+        best_model_path = path + '/' + 'checkpoint.pth'
+        self.model.load_state_dict(torch.load(best_model_path))
+        min_val_loss = early_stopping.get_val_loss_min()
+        
+        return self.model, min_val_loss
+
+    def test(self, setting,  test=1):
+        test_data, test_loader = self._get_data(flag='test')
+
+        if test:
+            print('loading model')
+            self.model.load_state_dict(torch.load(os.path.join('./checkpoints/' + setting, 'checkpoint.pth')))
+
+        preds = []
+        trues = []
+        inputx = []
+        
+        self.model.eval()
+        with torch.no_grad():
+            for i, (batch_x, batch_y, _) in enumerate(test_loader):
+                batch_x = batch_x.float().to(self.device)
+                batch_y = batch_y.float().to(self.device)
+
+                outputs = self.model(batch_x)
+
+                f_dim = -1 if self.args.features == 'MS' else 0
+                outputs = outputs[:, -self.args.pred_len:, f_dim:]
+                batch_y = batch_y[:, -self.args.pred_len:, f_dim:].to(self.device)
+                outputs = outputs.detach().cpu().numpy()
+                batch_y = batch_y.detach().cpu().numpy()
+
+                pred = outputs  #
+                true = batch_y  
+
+                preds.append(pred)
+                trues.append(true)
+                inputx.append(batch_x.detach().cpu().numpy())
+                    
+        if self.args.test_flop:
+            test_params_flop((batch_x.shape[1],batch_x.shape[2]))
+            exit()
+
+        preds = np.array(preds)
+        trues = np.array(trues)
+        inputx = np.array(inputx)
+
+        preds = preds.reshape(-1, preds.shape[-2], preds.shape[-1])
+        trues = trues.reshape(-1, trues.shape[-2], trues.shape[-1])
+        inputx = inputx.reshape(-1, inputx.shape[-2], inputx.shape[-1])
+
+        mae, mse, rmse, mape, mspe, rse, corr = metric(preds, trues)
+        print('mse:{}, mae:{}, rse:{}, corr:{}'.format(mse, mae, rse, corr))
+        f = open(self.args.des + self.args.data + ".txt", 'a')
+        f.write(" \n")
+        f.write('{} --- Pred {} -> mse:{}, mae:{}, rse:{}'.format(TYPES[self.args.aug_type], self.args.pred_len, mse, mae, rse))
+        f.write('\n')
+        f.close()
+
+        return mse, mae, rse    

models/DLinear.py

@@ -0,0 +1,91 @@
+# =============================================================================
+# The code is originated from
+#  Chen, M., Xu, Z., Zeng, A., & Xu, Q. (2023). "FrAug: Frequency Domain Augmentation for Time Series Forecasting".
+#  arXiv preprint arXiv:2302.09292.
+# =============================================================================
+
+import torch
+import torch.nn as nn
+
+class moving_avg(nn.Module):
+    """
+    Moving average block to highlight the trend of time series
+    """
+    def __init__(self, kernel_size, stride):
+        super(moving_avg, self).__init__()
+        self.kernel_size = kernel_size
+        self.avg = nn.AvgPool1d(kernel_size=kernel_size, stride=stride, padding=0)
+
+    def forward(self, x):
+        # padding on the both ends of time series
+        front = x[:, 0:1, :].repeat(1, (self.kernel_size - 1) // 2, 1)
+        end = x[:, -1:, :].repeat(1, (self.kernel_size - 1) // 2, 1)
+        x = torch.cat([front, x, end], dim=1)
+        x = self.avg(x.permute(0, 2, 1))
+        x = x.permute(0, 2, 1)
+        return x
+
+
+class series_decomp(nn.Module):
+    """
+    Series decomposition block
+    """
+    def __init__(self, kernel_size):
+        super(series_decomp, self).__init__()
+        self.moving_avg = moving_avg(kernel_size, stride=1)
+
+    def forward(self, x):
+        moving_mean = self.moving_avg(x)
+        res = x - moving_mean
+        return res, moving_mean
+
+class Model(nn.Module):
+    """
+    Decomposition-Linear
+    """
+    def __init__(self, args):
+        super(Model, self).__init__()
+        self.seq_len = args.seq_len
+        self.pred_len = args.pred_len
+
+        # Decompsition Kernel Size
+        kernel_size = 25
+        self.decompsition = series_decomp(kernel_size)
+        self.individual = args.individual
+        self.channels = args.enc_in
+
+        if self.individual:
+            self.Linear_Seasonal = nn.ModuleList()
+            self.Linear_Trend = nn.ModuleList()
+
+            for i in range(self.channels):
+                self.Linear_Seasonal.append(nn.Linear(self.seq_len,self.pred_len))
+                self.Linear_Trend.append(nn.Linear(self.seq_len,self.pred_len))
+
+                # Use this two lines if you want to visualize the weights
+                # self.Linear_Seasonal[i].weight = nn.Parameter((1/self.seq_len)*torch.ones([self.pred_len,self.seq_len]))
+                # self.Linear_Trend[i].weight = nn.Parameter((1/self.seq_len)*torch.ones([self.pred_len,self.seq_len]))
+        else:
+            self.Linear_Seasonal = nn.Linear(self.seq_len,self.pred_len)
+            self.Linear_Trend = nn.Linear(self.seq_len,self.pred_len)
+
+            # Use this two lines if you want to visualize weights
+            # self.Linear_Seasonal.weight = nn.Parameter((1/self.seq_len)*torch.ones([self.pred_len,self.seq_len]))
+            # self.Linear_Trend.weight = nn.Parameter((1/self.seq_len)*torch.ones([self.pred_len,self.seq_len]))
+
+    def forward(self, x):
+        # x: [Batch, Input length, Channel]
+        seasonal_init, trend_init = self.decompsition(x)
+        seasonal_init, trend_init = seasonal_init.permute(0,2,1), trend_init.permute(0,2,1)
+        if self.individual:
+            seasonal_output = torch.zeros([seasonal_init.size(0),seasonal_init.size(1),self.pred_len],dtype=seasonal_init.dtype).to(seasonal_init.device)
+            trend_output = torch.zeros([trend_init.size(0),trend_init.size(1),self.pred_len],dtype=trend_init.dtype).to(trend_init.device)
+            for i in range(self.channels):
+                seasonal_output[:,i,:] = self.Linear_Seasonal[i](seasonal_init[:,i,:])
+                trend_output[:,i,:] = self.Linear_Trend[i](trend_init[:,i,:])
+        else:
+            seasonal_output = self.Linear_Seasonal(seasonal_init)
+            trend_output = self.Linear_Trend(trend_init)
+
+        x = seasonal_output + trend_output
+        return x.permute(0,2,1) # to [Batch, Output length, Channel]

requirements.txt

@@ -0,0 +1,10 @@
+numpy
+matplotlib
+pandas
+scikit-learn
+torch==1.9.0
+einops
+pywavelets
+EMD-signal
+tensorflow==2.11.0
+pytorch_wavelets

run_main.py

@@ -0,0 +1,150 @@
+# =============================================================================
+# The code is originated from
+# Chen, M., Xu, Z., Zeng, A., & Xu, Q. (2023). "FrAug: Frequency Domain Augmentation for Time Series Forecasting".
+# arXiv preprint arXiv:2302.09292.
+# =============================================================================
+
+import argparse
+import os
+import ast
+import torch
+from main_run.train import Exp_Main
+import random
+import numpy as np
+
+fix_seed = 2024
+random.seed(fix_seed)
+torch.manual_seed(fix_seed)
+np.random.seed(fix_seed)
+
+TYPES = {0: 'None',
+         1: 'Freq-Mask',
+         2: 'Freq-Mix',
+         3: 'Wave-Mask',
+         4: 'Wave-Mix',
+         5: 'StAug'}
+
+
+parser = argparse.ArgumentParser(description='Augmentations for Time Series Forecasting')
+
+# basic config 
+parser.add_argument('--model', type=str, required=True, default='DLinear',
+                    help='model name, options: [DLinear]')
+
+# data loader 
+parser.add_argument('--is_training', type=int, required=True, default=1, help='status')
+parser.add_argument('--data', type=str, required=True, default='ETTh1', help='dataset type')
+parser.add_argument('--root_path', type=str, default='./dataset/', help='root path of the data file')
+parser.add_argument('--data_path', type=str, default='ETTh1.csv', help='data file')
+parser.add_argument('--features', type=str, default='M',
+                    help='forecasting task, options:[M, S, MS]; M:multivariate predict multivariate, S:univariate predict univariate, MS:multivariate predict univariate')
+parser.add_argument('--target', type=str, default='OT', help='target feature in S or MS task')
+parser.add_argument('--freq', type=str, default='h',
+                    help='freq for time features encoding, options:[s:secondly, t:minutely, h:hourly, d:daily, b:business days, w:weekly, m:monthly], you can also use more detailed freq like 15min or 3h')
+parser.add_argument('--checkpoints', type=str, default='./checkpoints/', help='location of model checkpoints')
+parser.add_argument('--percentage', type=int, default=100, help='percentage of train data as a downsampling ratio')
+parser.add_argument('--patience', type=int, default=12, help='early stopping patience')
+
+# forecasting task 
+parser.add_argument('--seq_len', type=int, default=336, help='input sequence length')
+parser.add_argument('--label_len', type=int, default=0, help='start token length')
+parser.add_argument('--pred_len', type=int, default=96, help='prediction sequence length')
+
+
+# DLinear 
+parser.add_argument('--individual', action='store_true', default=False, help='DLinear: a linear layer for each variate(channel) individually')
+parser.add_argument('--enc_in', type=int, default=7, help='encoder input size') # DLinear with --individual, use this hyperparameter as the number of channels
+
+# optimization
+parser.add_argument('--num_workers', type=int, default=10, help='data loader num workers')
+parser.add_argument('--itr', type=int, default=2, help='experiments times')
+parser.add_argument('--train_epochs', type=int, default=30, help='train epochs')
+parser.add_argument('--batch_size', type=int, default=64, help='batch size of train input data')
+parser.add_argument('--learning_rate', type=float, default=0.0001, help='optimizer learning rate')
+parser.add_argument('--des', type=str, default='test', help='exp description')
+parser.add_argument('--lradj', type=str, default='type1', help='adjust learning rate')
+parser.add_argument('--use_amp', action='store_true', help='use automatic mixed precision training', default=False)
+
+# GPU
+parser.add_argument('--use_gpu', type=bool, default=True, help='use gpu')
+parser.add_argument('--gpu', type=int, default=0, help='gpu')
+parser.add_argument('--use_multi_gpu', action='store_true', help='use multiple gpus', default=False)
+parser.add_argument('--devices', type=str, default='0,1,2,3', help='device ids of multile gpus')
+parser.add_argument('--test_flop', action='store_true', default=False, help='See utils/tools for usage')
+
+# Augmentation
+parser.add_argument('--aug_type', type=int, default=0, help='0: No augmentation, 1: Frequency Masking 2: Frequency Mixing  3: Wave Masking  4: Wave Mixing 5: StAug ')
+parser.add_argument('--aug_rate', type=float, default=0.5, help='rate for FreqMask, FreqMix, and STAug')
+parser.add_argument('--wavelet', type=str, default='db2', help='wavelet form for DWT')
+parser.add_argument('--level', type=int, default=2, help='level for DWT')
+parser.add_argument('--rates', type=str, default="[0.2, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]",
+                        help='List of float rates as a string, e.g., "[0.2, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]"')
+parser.add_argument('--nIMF', type=int, default=500, help='number of IMFs for EMD (STAug)')
+parser.add_argument('--sampling_rate', type=float, default=0.5, help='sampling rate for WaveMask and WaveMix')
+
+
+
+
+args = parser.parse_args()
+args.rates = ast.literal_eval(args.rates)
+
+args.use_gpu = True if torch.cuda.is_available() and args.use_gpu else False
+
+if args.use_gpu and args.use_multi_gpu:
+    args.dvices = args.devices.replace(' ', '')
+    device_ids = args.devices.split(',')
+    args.device_ids = [int(id_) for id_ in device_ids]
+    args.gpu = args.device_ids[0]
+
+print('Args in experiment:')
+print(args)
+
+Exp = Exp_Main
+
+if args.is_training:
+    mse_avg, mae_avg, rse_avg = np.zeros(args.itr), np.zeros(args.itr), np.zeros(args.itr)
+    for ii in range(args.itr):
+        # setting record of experiments
+        setting = '{}_{}_ft{}_sl{}_ll{}_pl{}_{}_{}_{}_{}'.format(
+            args.model,
+            args.data,
+            args.features,
+            args.seq_len,
+            args.label_len,
+            args.pred_len,
+            args.des, args.aug_type, args.percentage, ii)
+
+        exp = Exp(args)  # set experiments
+        print('>>>>>>>start training : {}>>>>>>>>>>>>>>>>>>>>>>>>>>'.format(setting))
+        exp.train(setting)
+
+        print('>>>>>>>testing : {}<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<'.format(setting))
+        mse, mae, rse = exp.test(setting)
+        mse_avg[ii] = mse
+        mae_avg[ii] = mae
+        rse_avg[ii] = rse
+
+    f = open("result-" + args.des + args.data + ".txt", 'a')
+    f.write('\n')
+    f.write('\n')
+    f.write("-------START FROM HERE-----")
+    f.write(TYPES[args.aug_type] + "  \n")
+    f.write('avg mse:{}, avg mae:{} avg rse:{}  std mse:{}, std mae:{} std rse:{}'.format(mse_avg.mean(), mae_avg.mean(), rse_avg.mean(), mse_avg.std(), mae_avg.std(), rse_avg.std()))
+    f.write('\n')
+    f.write('\n')
+    f.close()
+    torch.cuda.empty_cache()
+else:
+    ii = 0
+    setting = '{}_{}_ft{}_sl{}_ll{}_pl{}_{}_{}_{}'.format(args.model,
+                                                       args.data,
+                                                       args.features,
+                                                    args.seq_len,
+                                                    args.label_len,
+                                                    args.pred_len,
+                                                    args.des, args.aug_type, ii)
+
+    exp = Exp(args)  # set experiments
+    print('>>>>>>>testing : {}<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<'.format(setting))
+    exp.test(setting, test=1)
+    torch.cuda.empty_cache()

scripts/etth1.sh

@@ -0,0 +1,335 @@
+
+if [ ! -d "./logs" ]; then
+    mkdir ./logs
+fi
+
+if [ ! -d "./logs/onerun" ]; then
+    mkdir ./logs/onerun
+fi
+
+seq_len=336
+percentage=100
+model_name=DLinear
+
+# aug 0: None
+# aug 1: Frequency Masking 
+# aug 2: Frequency Mixing
+# aug 3: Wave Masking
+# aug 4: Wave Mixing
+# aug 5: STAug
+
+pred_lens=(96 192 336 720)
+
+
+# For Aug 0: None
+
+for pred_len in "${pred_lens[@]}"; do
+    python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len $pred_len \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 0 --aug_rate 0.0 >logs/onerun/$model_name'_'Etth1_$seq_len'_'$pred_len'_'0.0'_'$percentage'_'None.log
+done
+
+
+# For Aug 1: Freq-Masking
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data ETTh1 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  96 \
+    --enc_in 7 \
+    --des '100p-h1-'  \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 1 --aug_rate 0.1 >logs/onerun/$model_name'_'Etth1_$seq_len'_'96'_'0.4'_'$percentage'_'FreqMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data ETTh1 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  192 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 1 --aug_rate 0.5 >logs/onerun/$model_name'_'Etth1_$seq_len'_'192'_'0.4'_'$percentage'_'FreqMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data ETTh1 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  336 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 1 --aug_rate 0.5 >logs/onerun/$model_name'_'Etth1_$seq_len'_'336'_'0.4'_'$percentage'_'FreqMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data ETTh1 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  720 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 1 --aug_rate 0.4 >logs/onerun/$model_name'_'Etth1_$seq_len'_'720'_'0.4'_'$percentage'_'FreqMask.log
+
+
+
+# For Aug 2: Freq-Mixing
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 96 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 2 --aug_rate 0.2 >logs/onerun/$model_name'_'Etth1_$seq_len'_'96'_'0.4'_'$percentage'_'FreqMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 192 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 2 --aug_rate 0.1 >logs/onerun/$model_name'_'Etth1_$seq_len'_'192'_'0.4'_'$percentage'_'FreqMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 336 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 2 --aug_rate 0.1 >logs/onerun/$model_name'_'Etth1_$seq_len'_'336'_'0.4'_'$percentage'_'FreqMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 720 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 2 --aug_rate 0.6 >logs/onerun/$model_name'_'Etth1_$seq_len'_'720'_'0.4'_'$percentage'_'FreqMix.log
+
+# For Aug 3: Wave Masking
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 96 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 3 --rates "[0.5, 0.3, 0.9, 0.9, 0.0, 0.5, 0.3]" --wavelet 'db2' --level 3 --sampling_rate 0.2 >logs/onerun/$model_name'_'Etth1_$seq_len'_'96'_'0.0'_'$percentage'_'WaveMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 192 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 3 --rates "[0.0, 1.0, 0.2, 1.0, 0.4, 0.9, 0.3]" --wavelet 'db3' --level 1 --sampling_rate 0.2 >logs/onerun/$model_name'_'Etth1_$seq_len'_'192'_'0.0'_'$percentage'_'WaveMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 336 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 3 --rates "[0.1, 0.9, 0.4, 0.8, 0.4, 0.9, 0.3]" --wavelet 'db25' --level 1 --sampling_rate 0.8 >logs/onerun/$model_name'_'Etth1_$seq_len'_'336'_'0.0'_'$percentage'_'WaveMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 720 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 3 --rates "[0.4, 0.9, 0.5, 1.0, 0.3, 0.9, 0.3]" --wavelet 'db1' --level 1 --sampling_rate 0.2 >logs/onerun/$model_name'_'Etth1_$seq_len'_'720'_'0.0'_'$percentage'_'WaveMask.log
+
+# For Aug 4: Wave Mixing
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 96 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 4 --rates "[0.0, 0.9, 0.7, 0.7, 0.5, 0.7, 0.6]" --wavelet 'db3' --level 1 --sampling_rate 0.2 >logs/onerun/$model_name'_'Etth1_$seq_len'_'96'_'0.0'_'$percentage'_'WaveMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 192 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 4 --rates "[1.0, 0.4, 0.6, 0.6, 0.3, 0.8, 0.1]" --wavelet 'db3' --level 1 --sampling_rate 0.8 >logs/onerun/$model_name'_'Etth1_$seq_len'_'192'_'0.0'_'$percentage'_'WaveMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 336 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 4 --rates "[0.0, 0.9, 0.2, 0.2, 0.2, 0.1, 0.7]" --wavelet 'db3' --level 1 --sampling_rate 0.8 >logs/onerun/$model_name'_'Etth1_$seq_len'_'336'_'0.0'_'$percentage'_'WaveMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 720 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 4 --rates "[0.1, 0.9, 0.5, 0.7, 0.1, 0.2, 0.8]" --wavelet 'db5' --level 1 --sampling_rate 0.8 >logs/onerun/$model_name'_'Etth1_$seq_len'_'720'_'0.0'_'$percentage'_'WaveMix.log
+
+
+# For Aug 5: STAug
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 96 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 5 --aug_rate 0.9 --nIMF 100 >logs/onerun/$model_name'_'Etth1_$seq_len'_'96'_'0.9'_'$percentage'_'StAug.log
+
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 192 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 5 --aug_rate 0.9 --nIMF 900 >logs/onerun/$model_name'_'Etth1_$seq_len'_'192'_'0.9'_'$percentage'_'StAug.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 336 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 5 --aug_rate 0.8 --nIMF 200 >logs/onerun/$model_name'_'Etth1_$seq_len'_'336'_'0.9'_'$percentage'_'StAug.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh1.csv \
+    --model $model_name \
+    --data  ETTh1 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 720 \
+    --enc_in 7 \
+    --des '100p-h1-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 5 --aug_rate 0.7 --nIMF 1000 >logs/onerun/$model_name'_'Etth1_$seq_len'_'720'_'0.9'_'$percentage'_'StAug.log

scripts/etth2.sh

@@ -0,0 +1,341 @@
+
+if [ ! -d "./logs" ]; then
+    mkdir ./logs
+fi
+
+if [ ! -d "./logs/onerun" ]; then
+    mkdir ./logs/onerun
+fi
+
+seq_len=336
+percentage=100
+model_name=DLinear
+
+# aug 0: None
+# aug 1: Frequency Masking 
+# aug 2: Frequency Mixing
+# aug 3: Wave Masking
+# aug 4: Wave Mixing
+# aug 5: STAug
+
+pred_lens=(96 192 336 720)
+
+
+# For Aug 0: None
+
+for pred_len in "${pred_lens[@]}"; do
+    python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len $pred_len \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 0 --aug_rate 0.0 >logs/onerun/$model_name'_'Etth2_$seq_len'_'$pred_len'_'0.0'_'$percentage'_'None.log
+done
+
+
+# For Aug 1: Freq-Masking
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data ETTh2 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  96 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 1 --aug_rate 0.6 >logs/onerun/$model_name'_'Etth2_$seq_len'_'96'_'0.4'_'$percentage'_'FreqMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data ETTh2 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  192 \
+    --enc_in 7 \
+    --des '100p-h2-'  \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 1 --aug_rate 0.6 >logs/onerun/$model_name'_'Etth2_$seq_len'_'192'_'0.4'_'$percentage'_'FreqMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data ETTh2 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  336 \
+    --enc_in 7 \
+    --des '100p-h2-'  \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 1 --aug_rate 0.1 >logs/onerun/$model_name'_'Etth2_$seq_len'_'336'_'0.4'_'$percentage'_'FreqMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data ETTh2 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  720 \
+    --enc_in 7 \
+    --des '100p-h2-'  \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 1 --aug_rate 0.1 >logs/onerun/$model_name'_'Etth2_$seq_len'_'720'_'0.4'_'$percentage'_'FreqMask.log
+
+
+
+# For Aug 2: Freq-Mixing
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 96 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 2 --aug_rate 0.9 >logs/onerun/$model_name'_'Etth2_$seq_len'_'96'_'0.4'_'$percentage'_'FreqMix.log
+
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 192 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 2 --aug_rate 0.8 >logs/onerun/$model_name'_'Etth2_$seq_len'_'192'_'0.4'_'$percentage'_'FreqMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 336 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 2 --aug_rate 0.8 >logs/onerun/$model_name'_'Etth2_$seq_len'_'336'_'0.4'_'$percentage'_'FreqMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 720 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 2 --aug_rate 0.1 >logs/onerun/$model_name'_'Etth2_$seq_len'_'720'_'0.4'_'$percentage'_'FreqMix.log
+
+
+
+
+# For Aug 3: Wave Masking
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 96 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 3 --rates "[0.4, 0.9, 0.0, 0.8, 0.3, 0.7, 0.2]" --wavelet 'db26' --level 2 --sampling_rate 0.5 >logs/onerun/$model_name'_'Etth2_$seq_len'_'96'_'0.0'_'$percentage'_'WaveMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 192 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 3 --rates "[0.6, 0.7, 0.5, 0.7, 0.8, 0.3, 0.5]" --wavelet 'db26' --level 2 --sampling_rate 0.8 >logs/onerun/$model_name'_'Etth2_$seq_len'_'192'_'0.0'_'$percentage'_'WaveMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 336 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 3 --rates "[0.2, 0.7, 0.9, 0.4, 0.4, 0.1, 1.0]" --wavelet 'db1' --level 3 --sampling_rate 0.8 >logs/onerun/$model_name'_'Etth2_$seq_len'_'336'_'0.0'_'$percentage'_'WaveMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 720 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 3 --rates "[0.8, 0.9, 0.4, 0.9, 0.4, 0.4, 0.5]" --wavelet 'db5' --level 4 --sampling_rate 0.2 >logs/onerun/$model_name'_'Etth2_$seq_len'_'720'_'0.0'_'$percentage'_'WaveMask.log
+
+# For Aug 4: Wave Mixing
+
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 96 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 4 --rates "[0.9, 0.9, 0.1, 0.5, 0.9, 0.6, 0.2]" --wavelet 'db25' --level 2 --sampling_rate 0.2 >logs/onerun/$model_name'_'Etth2_$seq_len'_'96'_'0.0'_'$percentage'_'WaveMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 192 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 4 --rates "[0.9, 0.4, 0.1, 0.8, 0.9, 0.5, 0.4]" --wavelet 'db1' --level 3 --sampling_rate 0.5 >logs/onerun/$model_name'_'Etth2_$seq_len'_'192'_'0.0'_'$percentage'_'WaveMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 336 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 4 --rates "[0.9, 0.1, 0.2, 0.5, 1.0, 0.5, 0.5]" --wavelet 'db25' --level 3 --sampling_rate 0.8 >logs/onerun/$model_name'_'Etth2_$seq_len'_'336'_'0.0'_'$percentage'_'WaveMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 720 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 4 --rates "[0.5, 0.1, 0.2, 0.7, 0.4, 0.9, 0.5]" --wavelet 'db5' --level 1 --sampling_rate 1.0 >logs/onerun/$model_name'_'Etth2_$seq_len'_'720'_'0.0'_'$percentage'_'WaveMix.log
+
+
+
+# For Aug 5: STAug
+    
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 96 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 5 --aug_rate 0.4 --nIMF 2000 >logs/onerun/$model_name'_'Etth2_$seq_len'_'96'_'0.9'_'$percentage'_'StAug.log
+
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 192 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 5 --aug_rate 0.9 --nIMF 200 >logs/onerun/$model_name'_'Etth2_$seq_len'_'192'_'0.9'_'$percentage'_'StAug.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 336 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 5 --aug_rate 0.6 --nIMF 100 >logs/onerun/$model_name'_'Etth2_$seq_len'_'336'_'0.9'_'$percentage'_'StAug.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path ETTh2.csv \
+    --model $model_name \
+    --data  ETTh2 \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 720 \
+    --enc_in 7 \
+    --des '100p-h2-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 5 --aug_rate 0.4 --nIMF 700 >logs/onerun/$model_name'_'Etth2_$seq_len'_'720'_'0.9'_'$percentage'_'StAug.log

scripts/ili.sh

@@ -0,0 +1,342 @@
+
+if [ ! -d "./logs" ]; then
+    mkdir ./logs
+fi
+
+if [ ! -d "./logs/onerun" ]; then
+    mkdir ./logs/onerun
+fi
+
+seq_len=36
+percentage=100
+model_name=DLinear
+
+# aug 0: None
+# aug 1: Frequency Masking 
+# aug 2: Frequency Mixing
+# aug 3: Wave Masking
+# aug 4: Wave Mixing
+# aug 5: STAug
+
+pred_lens=(24 36 48 60)
+
+
+
+# For Aug 0: None
+
+for pred_len in "${pred_lens[@]}"; do
+    python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv \
+    --model $model_name \
+    --data  custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len $pred_len \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 0 --aug_rate 0.0 >logs/onerun/$model_name'_'ill_$seq_len'_'$pred_len'_'0.0'_'$percentage'_'None.log
+done
+
+
+# For Aug 1: Freq-Masking
+
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  24 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 1 --aug_rate 0.2 >logs/onerun/$model_name'_'ill_$seq_len'_'96'_'0.4'_'$percentage'_'FreqMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  36 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 1 --aug_rate 0.1 >logs/onerun/$model_name'_'ill_$seq_len'_'36'_'0.4'_'$percentage'_'FreqMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  48 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 1 --aug_rate 0.1 >logs/onerun/$model_name'_'ill_$seq_len'_'48'_'0.4'_'$percentage'_'FreqMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  60 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 1 --aug_rate 0.1 >logs/onerun/$model_name'_'ill_$seq_len'_'60'_'0.4'_'$percentage'_'FreqMask.log
+
+
+
+# For Aug 2: Freq-Mixing
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 24 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 2 --aug_rate 0.1 >logs/onerun/$model_name'_'ill_$seq_len'_'24'_'0.4'_'$percentage'_'FreqMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 36 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 2 --aug_rate 0.1 >logs/onerun/$model_name'_'ill_$seq_len'_'36'_'0.4'_'$percentage'_'FreqMix.log
+
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 48 \
+    --enc_in 7 \
+    --des '100p-ili-'  \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 2 --aug_rate 0.1 >logs/onerun/$model_name'_'ill_$seq_len'_'48'_'0.4'_'$percentage'_'FreqMix.log
+
+
+python3 -u ./run_main2.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 60 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 2 --aug_rate 0.1 >logs/onerun/$model_name'_'ill_$seq_len'_'60'_'0.4'_'$percentage'_'FreqMix.log
+
+
+
+# For Aug 3: Wave Masking
+
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 24 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 3 --rates "[0.4, 0.8, 0.9, 0.7, 0.9, 0.0, 0.5]" --wavelet 'db25' --level 1 --sampling_rate 0.2 >logs/onerun/$model_name'_'ill_$seq_len'_'24'_'0.0'_'$percentage'_'WaveMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 36 \
+    --enc_in 7 \
+    --des '100p-ili-'  \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 3 --rates "[0.6, 0.8, 0.3, 0.1, 0.9, 0.0, 0.5]" --wavelet 'db25' --level 1 --sampling_rate 0.2 >logs/onerun/$model_name'_'ill_$seq_len'_'36'_'0.0'_'$percentage'_'WaveMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 48 \
+    --enc_in 7 \
+    --des '100p-ili-'  \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 3 --rates "[0.2, 0.7, 1.0, 0.4, 0.4, 0.0, 0.5]" --wavelet 'db2' --level 1 --sampling_rate 0.2 >logs/onerun/$model_name'_'ill_$seq_len'_'48'_'0.0'_'$percentage'_'WaveMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 60 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 3 --rates "[0.2, 0.8, 0.5, 0.1, 0.9, 0.0, 0.5]" --wavelet 'db25' --level 1 --sampling_rate 0.2 >logs/onerun/$model_name'_'ill_$seq_len'_'60'_'0.0'_'$percentage'_'WaveMask.log
+
+# For Aug 4: Wave Mixing
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 24 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 4 --rates "[0.1, 0.8, 1.0, 0.0, 0.5, 0.7, 0.1]" --wavelet 'db1' --level 1 --sampling_rate 0.2 >logs/onerun/$model_name'_'ill_$seq_len'_'24'_'0.0'_'$percentage'_'WaveMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 36 \
+    --enc_in 7 \
+    --des '100p-ili-'  \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 4 --rates "[0.1, 1.0, 0.9, 0.2, 0.1, 0.7, 0.1]" --wavelet 'db25' --level 1 --sampling_rate 0.8 >logs/onerun/$model_name'_'ill_$seq_len'_'36'_'0.0'_'$percentage'_'WaveMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 48 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 4 --rates "[0.1, 1.0, 0.4, 0.5, 0.1, 0.6, 0.1]" --wavelet 'db3' --level 1 --sampling_rate 1.0 >logs/onerun/$model_name'_'ill_$seq_len'_'48'_'0.0'_'$percentage'_'WaveMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv  \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 60 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 4 --rates "[0.1, 0.9, 0.3, 0.9, 0.5, 0.7, 0.1]" --wavelet 'db1' --level 1 --sampling_rate 0.5 >logs/onerun/$model_name'_'ill_$seq_len'_'60'_'0.0'_'$percentage'_'WaveMix.log
+
+
+# For Aug 5: STAug
+    
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv \
+    --model $model_name \
+    --data  custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 24 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 5 --aug_rate 0.7 --nIMF 200 >logs/onerun/$model_name'_'ill_$seq_len'_'24'_'0.9'_'$percentage'_'StAug.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv \
+    --model $model_name \
+    --data  custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 36 \
+    --enc_in 7 \
+    --des '100p-ili-'  \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 5 --aug_rate 0.3 --nIMF 300 >logs/onerun/$model_name'_'ill_$seq_len'_'36'_'0.9'_'$percentage'_'StAug.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv \
+    --model $model_name \
+    --data  custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 48 \
+    --enc_in 7 \
+    --des '100p-ili-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 5 --aug_rate 0.9 --nIMF 300 >logs/onerun/$model_name'_'ill_$seq_len'_'48'_'0.9'_'$percentage'_'StAug.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path national_illness.csv \
+    --model $model_name \
+    --data  custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 60 \
+    --enc_in 7 \
+    --des '100p-ili-'  \
+    --percentage $percentage \
+    --itr 10 --batch_size 32 --learning_rate 0.01 --aug_type 5 --aug_rate 0.7 --nIMF 1000 >logs/onerun/$model_name'_'ill_$seq_len'_'60'_'0.9'_'$percentage'_'StAug.log
+

scripts/weather.sh

@@ -0,0 +1,278 @@
+
+if [ ! -d "./logs" ]; then
+    mkdir ./logs
+fi
+
+if [ ! -d "./logs/onerun" ]; then
+    mkdir ./logs/onerun
+fi
+
+seq_len=336
+percentage=100
+model_name=DLinear
+
+# aug 0: None
+# aug 1: Frequency Masking 
+# aug 2: Frequency Mixing
+# aug 3: Wave Masking
+# aug 4: Wave Mixing
+# aug 5: STAug
+
+pred_lens=(96 192 336 720)
+
+# For Aug 0: None
+
+for pred_len in "${pred_lens[@]}"; do
+    python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv \
+    --model $model_name \
+    --data  custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len $pred_len \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 0 --aug_rate 0.0 >logs/onerun/$model_name'_'weather_$seq_len'_'$pred_len'_'0.0'_'$percentage'_'None.log
+done
+
+
+# For Aug 1: Freq-Masking
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv  \
+    --model $model_name \
+    --data custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  96 \
+    --enc_in 21 \
+    --des '100p-whr-'  \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 1 --aug_rate 0.1 >logs/onerun/$model_name'_'weather_$seq_len'_'96'_'0.4'_'$percentage'_'FreqMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv  \
+    --model $model_name \
+    --data custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  192 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 1 --aug_rate 0.1 >logs/onerun/$model_name'_'weather_$seq_len'_'192'_'0.4'_'$percentage'_'FreqMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv  \
+    --model $model_name \
+    --data custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  336 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 1 --aug_rate 0.1 >logs/onerun/$model_name'_'weather_$seq_len'_'336'_'0.4'_'$percentage'_'FreqMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv  \
+    --model $model_name \
+    --data custom \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len  720 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 1 --aug_rate 0.9 >logs/onerun/$model_name'_'weather_$seq_len'_'720'_'0.4'_'$percentage'_'FreqMask.log
+
+
+
+# For Aug 2: Freq-Mixing
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 96 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 2 --aug_rate 0.9 >logs/onerun/$model_name'_'weather_$seq_len'_'96'_'0.4'_'$percentage'_'FreqMix.log
+
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 192 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 2 --aug_rate 0.1 >logs/onerun/$model_name'_'weather_$seq_len'_'192'_'0.4'_'$percentage'_'FreqMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 336 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 2 --aug_rate 0.1 >logs/onerun/$model_name'_'weather_$seq_len'_'336'_'0.4'_'$percentage'_'FreqMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 720 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 2 --aug_rate 0.9 >logs/onerun/$model_name'_'weather_$seq_len'_'720'_'0.4'_'$percentage'_'FreqMix.log
+
+# For Aug 3: Wave Masking
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 96 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 3 --rates "[0.2, 1.0, 0.4, 0.4, 0.9, 0.0, 0.5]" --wavelet 'db2' --level 2 --sampling_rate 0.5 >logs/onerun/$model_name'_'weather_$seq_len'_'96'_'0.0'_'$percentage'_'WaveMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 192 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 3 --rates "[0.1, 0.7, 0.1, 0.4, 0.5, 0.1, 0.7]" --wavelet 'db2' --level 1 --sampling_rate 0.5 >logs/onerun/$model_name'_'weather_$seq_len'_'192'_'0.0'_'$percentage'_'WaveMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 336 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 3 --rates "[1.0, 1.0, 0.0, 0.0, 0.3, 0.4, 0.2]" --wavelet 'db1' --level 1 --sampling_rate 1.0 >logs/onerun/$model_name'_'weather_$seq_len'_'336'_'0.0'_'$percentage'_'WaveMask.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 720 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 3 --rates "[1.0, 0.8, 0.6, 0.0, 0.2, 0.6, 0.5]" --wavelet 'db2' --level 1 --sampling_rate 0.5 >logs/onerun/$model_name'_'weather_$seq_len'_'720'_'0.0'_'$percentage'_'WaveMask.log
+
+# For Aug 4: Wave Mixing
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 96 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 4 --rates "[0.1, 0.5, 0.1, 0.2, 0.4, 0.7, 0.1]" --wavelet 'db3' --level 1 --sampling_rate 1.0 >logs/onerun/$model_name'_'weather_$seq_len'_'96'_'0.0'_'$percentage'_'WaveMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 192 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 4 --rates "[0.2, 0.7, 1.0, 0.3, 0.2, 0.3, 0.1]" --wavelet 'db3' --level 1 --sampling_rate 1.0 >logs/onerun/$model_name'_'weather_$seq_len'_'192'_'0.0'_'$percentage'_'WaveMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 336 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 4 --rates "[0.8, 0.6, 0.8, 0.6, 0.1, 1.0, 0.3]" --wavelet 'db2' --level 1 --sampling_rate 1.0 >logs/onerun/$model_name'_'weather_$seq_len'_'336'_'0.0'_'$percentage'_'WaveMix.log
+
+python3 -u ./run_main.py \
+    --is_training 1 \
+    --root_path ./dataset/ \
+    --data_path weather.csv \
+    --model $model_name \
+    --data  custom  \
+    --features M \
+    --seq_len $seq_len \
+    --pred_len 720 \
+    --enc_in 21 \
+    --des '100p-whr-' \
+    --percentage $percentage \
+    --itr 10 --batch_size 64 --learning_rate 0.01 --aug_type 4 --rates "[0.1, 0.1, 0.7, 0.5, 0.6, 0.5, 0.1]" --wavelet 'db1' --level 1 --sampling_rate 1.0 >logs/onerun/$model_name'_'weather_$seq_len'_'720'_'0.0'_'$percentage'_'WaveMix.log
+
+
+

utils/metrics.py

@@ -0,0 +1,49 @@
+# =============================================================================
+# The code is originated from
+#  Chen, M., Xu, Z., Zeng, A., & Xu, Q. (2023). "FrAug: Frequency Domain Augmentation for Time Series Forecasting".
+#  arXiv preprint arXiv:2302.09292.
+# =============================================================================
+
+import numpy as np
+
+def RSE(pred, true):
+    return np.sqrt(np.sum((true - pred) ** 2)) / np.sqrt(np.sum((true - true.mean()) ** 2))
+
+
+def CORR(pred, true):
+    u = ((true - true.mean(0)) * (pred - pred.mean(0))).sum(0)
+    d = np.sqrt(((true - true.mean(0)) ** 2 * (pred - pred.mean(0)) ** 2).sum(0))
+    d += 1e-12
+    return 0.01*(u / d).mean(-1)
+
+
+def MAE(pred, true):
+    return np.mean(np.abs(pred - true))
+
+
+def MSE(pred, true):
+    return np.mean((pred - true) ** 2)
+
+
+def RMSE(pred, true):
+    return np.sqrt(MSE(pred, true))
+
+
+def MAPE(pred, true):
+    return np.mean(np.abs((pred - true) / true))
+
+
+def MSPE(pred, true):
+    return np.mean(np.square((pred - true) / true))
+
+
+def metric(pred, true):
+    mae = MAE(pred, true)
+    mse = MSE(pred, true)
+    rmse = RMSE(pred, true)
+    mape = MAPE(pred, true)
+    mspe = MSPE(pred, true)
+    rse = RSE(pred, true)
+    corr = CORR(pred, true)
+
+    return mae, mse, rmse, mape, mspe, rse, corr

utils/tools.py

@@ -0,0 +1,102 @@
+# =============================================================================
+# The code is originated from
+#  Chen, M., Xu, Z., Zeng, A., & Xu, Q. (2023). "FrAug: Frequency Domain Augmentation for Time Series Forecasting".
+#  arXiv preprint arXiv:2302.09292.
+# =============================================================================
+
+import torch
+import numpy as np
+import matplotlib.pyplot as plt
+
+plt.switch_backend('agg')
+
+def adjust_learning_rate(optimizer, epoch, args):
+    # lr = args.learning_rate * (0.2 ** (epoch // 2))
+    if args.lradj == 'type1':
+        lr_adjust = {epoch: args.learning_rate * (0.5 ** ((epoch - 1) // 1))}
+    elif args.lradj == 'type2':
+        lr_adjust = {
+            2: 5e-5, 4: 1e-5, 6: 5e-6, 8: 1e-6,
+            10: 5e-7, 15: 1e-7, 20: 5e-8
+        }
+    elif args.lradj == '3':
+        lr_adjust = {epoch: args.learning_rate  if epoch < 10 else args.learning_rate*0.1}
+    elif args.lradj  == '4':
+        lr_adjust = {epoch: args.learning_rate  if epoch < 15 else args.learning_rate*0.1}
+    elif args.lradj == '5':
+        lr_adjust = {epoch: args.learning_rate  if epoch < 25 else args.learning_rate*0.1}
+    elif args.lradj == '6':
+        lr_adjust = {epoch: args.learning_rate  if epoch < 5 else args.learning_rate*0.1}
+    if epoch in lr_adjust.keys():
+        lr = lr_adjust[epoch]
+        for param_group in optimizer.param_groups:
+            param_group['lr'] = lr
+        print('Updating learning rate to {}'.format(lr))
+
+
+class EarlyStopping:
+    def __init__(self, patience=7, verbose=False, delta=0):
+        self.patience = patience
+        self.verbose = verbose
+        self.counter = 0
+        self.best_score = None
+        self.early_stop = False
+        self.val_loss_min = np.Inf
+        self.delta = delta
+
+    def __call__(self, val_loss, model, path):
+        score = -val_loss
+        if self.best_score is None:
+            self.best_score = score
+            self.save_checkpoint(val_loss, model, path)
+        elif score < self.best_score + self.delta:
+            self.counter += 1
+            print(f'EarlyStopping counter: {self.counter} out of {self.patience}')
+            if self.counter >= self.patience:
+                self.early_stop = True
+        else:
+            self.best_score = score
+            self.save_checkpoint(val_loss, model, path)
+            self.counter = 0
+
+    def save_checkpoint(self, val_loss, model, path):
+        if self.verbose:
+            print(f'Validation loss decreased ({self.val_loss_min:.6f} --> {val_loss:.6f}).  Saving model ...')
+        torch.save(model.state_dict(), path + '/' + 'checkpoint.pth')
+        self.val_loss_min = val_loss
+
+    def get_val_loss_min(self):
+        return self.val_loss_min    
+
+def visual(true, preds=None, name='./pic/test.pdf'):
+    """
+    Results visualization
+    """
+    plt.figure()
+    plt.plot(true, label='GroundTruth', linewidth=2)
+    if preds is not None:
+        plt.plot(preds, label='Prediction', linewidth=2)
+    plt.legend()
+    plt.savefig(name, bbox_inches='tight')
+
+def test_params_flop(model,x_shape):
+    """
+    If you want to thest former's flop, you need to give default value to inputs in model.forward(), the following code can only pass one argument to forward()
+    """
+    model_params = 0
+    for parameter in model.parameters():
+        model_params += parameter.numel()
+        print('INFO: Trainable parameter count: {:.2f}M'.format(model_params / 1000000.0))
+    from ptflops import get_model_complexity_info
+    with torch.cuda.device(0):
+        macs, params = get_model_complexity_info(model.cuda(), x_shape, as_strings=True, print_per_layer_stat=True)
+        # print('Flops:' + flops)
+        # print('Params:' + params)
+        print('{:<30}  {:<8}'.format('Computational complexity: ', macs))
+        print('{:<30}  {:<8}'.format('Number of parameters: ', params))
+
+
+
+
+
+