Spektron

Foundation model for vibrational spectroscopy using D-LinOSS (Damped Linear Oscillatory State Space) models.

Model Description

Spektron is a state space model designed specifically for vibrational spectroscopy (IR, Raman). It combines:

D-LinOSS Architecture: Second-order oscillatory SSM with damped sinusoid basis, naturally suited for vibrational spectra
Masked Pretraining: Self-supervised learning on 130K quantum chemistry spectra (QM9S dataset)
Variational Information Bottleneck (VIB): Disentangles chemical information from instrument factors
Cross-Spectral Prediction: Novel capability to predict Raman from IR spectra and vice versa

Key Features

🔬 Spectroscopy-Native: D-LinOSS oscillatory dynamics match the physics of molecular vibrations
📊 Data-Efficient: Achieves strong performance with <100 labeled samples via masked pretraining
🔄 Bidirectional Scanning: Forward + backward D-LinOSS layers for full spectral context
⚡ Efficient: 6.6M parameters, O(n) complexity (vs O(n²) for Transformers)
🎯 Interpretable: Transfer function H(z) analysis reveals learned spectral filters

Model Architecture

Spectrum (B, 2048)
  → RawSpectralEmbedding    # Conv1d + wavenumber PE + [CLS] + [DOMAIN]
  → D-LinOSS Backbone       # 4 bidirectional layers, 256 hidden dim
  → MixtureOfExperts        # 4 experts, top-2 gating
  → TransformerEncoder      # 2 blocks
  → VIBHead                 # z_chem (128d) + z_inst (64d)
  → Task Heads              # Reconstruction | CrossSpectral | Regression

Parameters: 6.6M total (4.7M backbone + 1.9M heads)

Training Data

Dataset: QM9S - 130K organic molecules with DFT-computed IR/Raman spectra
Spectral Range: 500-4000 cm⁻¹, 2048 points
Method: B3LYP/def2-TZVP (Gaussian 16)
Molecules: Non-centrosymmetric (99.93%), C/H/N/O/F composition

Performance

Task	Dataset	Metric	Score
Spectral Reconstruction	QM9S	MSRP	0.0741
Masked Prediction	QM9S	OT Loss	0.0152
VIB Disentanglement	QM9S	Domain Acc	52.1% (≈chance)
Chemistry Preservation	QM9S	Chem Acc	94.7%

Intended Use

Primary Applications:

Spectral preprocessing and denoising
Cross-spectral prediction (IR ↔ Raman)
Calibration transfer (instrument adaptation)
Few-shot learning for new spectroscopic tasks
Feature extraction for downstream chemometrics

Research Areas:

Vibrational spectroscopy
Chemical analysis
Materials characterization
Quality control
Pharmaceutical analysis

Limitations

Trained only on gas-phase DFT spectra (single molecule, no intermolecular effects)
May not generalize to condensed-phase spectra (liquids, solids) without fine-tuning
Limited to organic molecules from QM9 distribution (C/H/N/O/F, ≤9 heavy atoms)
Cross-spectral prediction works best for non-centrosymmetric molecules

How to Use

import torch
from spektron import Spektron, SpektronConfig

# Load pretrained model
config = SpektronConfig.from_pretrained("ktubhyam/Spektron")
model = Spektron(config)
model.load_state_dict(torch.load("checkpoint.pt"))
model.eval()

# Predict Raman from IR spectrum
with torch.no_grad():
    ir_spectrum = torch.randn(1, 2048)  # Your IR spectrum
    raman_pred = model.predict_cross_spectral(
        ir_spectrum,
        source_domain="ir",
        target_domain="raman"
    )

Training Details

Pretraining

Objective: Masked spectral reconstruction + optimal transport loss
Mask Ratio: Progressive 15% → 40% over first 40% of training
Batch Size: 64 (effective, with gradient accumulation)
Optimizer: AdamW (lr=3e-4, weight decay=0.05 on weight matrices only)
Schedule: Cosine annealing with 5% warmup, min LR = 0.01× max
Epochs: 50K steps (~23 GPU-hours on 2× RTX 5060 Ti)
Precision: BF16 AMP (D-LinOSS forced to FP32 for stability)

Augmentation

Gaussian noise (σ=0.02, p=0.5)
Baseline drift (polynomial order 3, p=0.5)
Intensity scaling (0.8-1.2×, p=0.5)
Wavelength shift (±5 cm⁻¹, p=0.5)

VIB Annealing

β: 0.1 → 1e-3 (cosine decay over first 60% of training)
Promotes diverse representations early, tight bottleneck late

Citation

If you use Spektron in your research, please cite:

@software{karthikeyan2026spektron,
  author = {Karthikeyan, Tubhyam},
  title = {Spektron: Foundation Model for Vibrational Spectroscopy},
  year = {2026},
  publisher = {Zenodo},
  version = {v0.1.0},
  doi = {10.5281/zenodo.18802440},
  url = {https://github.com/ktubhyam/Spektron}
}

License

MIT License - see LICENSE

Author

Tubhyam Karthikeyan

Institute of Chemical Technology, Mumbai
ORCID: 0009-0001-4424-7297
Website: https://tubhyam.dev
Email: takarthikeyan25@gmail.com

Downloads last month: -; Downloads are not tracked for this model. How to track

ktubhyam
/

Spektron