Hugging Face's logo

Markr-AI
/
SpecEncoder

Model card Files
xet
 Community
SpecEncoder
File size: 2,048 Bytes
756bd12
 
 
df41f4e
 
 
4055958
8cbcf10
df41f4e
55ea957
 
 
df41f4e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
---
license: cc-by-nc-4.0
---

# SpecEncoder for Galaxy spectra🌕
GitHub: [GalaxySVLM](https://github.com/Marker-Inc-Korea/GalaxySVLM/tree/main/SpecEncoder)  
Weight file (recommended): [specformer-epoch=440-val.ckpt](https://huggingface.co/Markr-AI/SpecEncoder/blob/main/specformer-epoch%3D440-val.ckpt)  
Weight file (last): [last.ckpt](https://huggingface.co/Markr-AI/SpecEncoder/blob/main/last.ckpt)

> [!NOTE]
> After finishing the project, we will share all the code on GitHub.

## Hyperparameters
| Parameters | Value |
| --- | --- |
| Epoch | 500 |
| Batch | 512 |
| LR | 1e-4 |
| Scheduler | CosineAnnealingWithWarmupLR |
| Warmup_step | 1000 |
| Optimizer | AdamW |
> 2 NVIDIA RTX PRO 6000
   
## *Data Preprocessing
1. Flux (spectrum) Overlapping
```python
def preprocess_flux(self, b_wave, r_wave, z_wave, b_flux, r_flux, z_flux):
    '''
    simple overlapping (kyuiin method.)
    '''

    # indices to keep
    r_start = np.searchsorted(r_wave, b_wave[-1], side="right")
    z_start = np.searchsorted(z_wave, r_wave[-1], side="right")

    #print("Remove from R:", r_start)
    #print("Remove from Z:", z_start)

    wave = np.concatenate([
        b_wave,
        r_wave[r_start:],
        z_wave[z_start:]
    ])

    flux = np.concatenate([
        b_flux,
        r_flux[r_start:],
        z_flux[z_start:]
    ])

    #print("Final length:", len(wave))
    assert len(wave)==7781, 'manually spectrum length 7781'

    return wave, flux
```

2. Each Standardization
```python
std, mean = x.std(1, keepdim=True).clip_(0.2), x.mean(1, keepdim=True)
x = (x - mean) / std
```

3. slicing
    - Overlap length = 10
    - window = 20

4. Add standardization information (mean/std)
```python
x = F.pad(x, pad=(2, 0, 1, 0), mode="constant", value=0)
x[:, 0, 0] = (mean.squeeze() - 2) / 2
x[:, 0, 1] = (std.squeeze() - 2) / 8
#print(x.shape) # (B, 778, 22)
```

5. masking (for self-supervised learning)
    - mask_chunk_width: 50
    - mask_num_chunks: 6


# Reference
- [AstroCLIP](https://github.com/PolymathicAI/AstroCLIP/tree/main)