| --- |
| license: cc-by-nc-4.0 |
| --- |
| |
| # SpecEncoder for Galaxy spectra🌕 |
| GitHub: [GalaxySVLM](https://github.com/Marker-Inc-Korea/GalaxySVLM/tree/main/SpecEncoder) |
| Weight file (recommended): [specformer-epoch=440-val.ckpt](https://huggingface.co/Markr-AI/SpecEncoder/blob/main/specformer-epoch%3D440-val.ckpt) |
| Weight file (last): [last.ckpt](https://huggingface.co/Markr-AI/SpecEncoder/blob/main/last.ckpt) |
|
|
| > [!NOTE] |
| > After finishing the project, we will share all the code on GitHub. |
|
|
| ## Hyperparameters |
| | Parameters | Value | |
| | --- | --- | |
| | Epoch | 500 | |
| | Batch | 512 | |
| | LR | 1e-4 | |
| | Scheduler | CosineAnnealingWithWarmupLR | |
| | Warmup_step | 1000 | |
| | Optimizer | AdamW | |
| > 2 NVIDIA RTX PRO 6000 |
| |
| ## *Data Preprocessing |
| 1. Flux (spectrum) Overlapping |
| ```python |
| def preprocess_flux(self, b_wave, r_wave, z_wave, b_flux, r_flux, z_flux): |
| ''' |
| simple overlapping (kyuiin method.) |
| ''' |
| |
| # indices to keep |
| r_start = np.searchsorted(r_wave, b_wave[-1], side="right") |
| z_start = np.searchsorted(z_wave, r_wave[-1], side="right") |
| |
| #print("Remove from R:", r_start) |
| #print("Remove from Z:", z_start) |
| |
| wave = np.concatenate([ |
| b_wave, |
| r_wave[r_start:], |
| z_wave[z_start:] |
| ]) |
| |
| flux = np.concatenate([ |
| b_flux, |
| r_flux[r_start:], |
| z_flux[z_start:] |
| ]) |
| |
| #print("Final length:", len(wave)) |
| assert len(wave)==7781, 'manually spectrum length 7781' |
| |
| return wave, flux |
| ``` |
| |
| 2. Each Standardization |
| ```python |
| std, mean = x.std(1, keepdim=True).clip_(0.2), x.mean(1, keepdim=True) |
| x = (x - mean) / std |
| ``` |
|
|
| 3. slicing |
| - Overlap length = 10 |
| - window = 20 |
|
|
| 4. Add standardization information (mean/std) |
| ```python |
| x = F.pad(x, pad=(2, 0, 1, 0), mode="constant", value=0) |
| x[:, 0, 0] = (mean.squeeze() - 2) / 2 |
| x[:, 0, 1] = (std.squeeze() - 2) / 8 |
| #print(x.shape) # (B, 778, 22) |
| ``` |
|
|
| 5. masking (for self-supervised learning) |
| - mask_chunk_width: 50 |
| - mask_num_chunks: 6 |
|
|
|
|
| # Reference |
| - [AstroCLIP](https://github.com/PolymathicAI/AstroCLIP/tree/main) |
