File size: 1,457 Bytes
89d3393 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 | ---
tags:
- pytorch
- safetensors
license: mit
---
# dm_qwen4b_noise_emulator
Laplacian kernel regression noise model for `std_math` prediction in data mixture optimization.
Predicts per-configuration std_math (across seeds) given data mixture proportions,
used as a heteroscedastic noise model in Bayesian optimization.
## Architecture
- Kernel: Laplacian `K(x, x') = exp(-γ · ||x - x'||₁)`
- Support points: 50 training configs
- Input features (3): `[if_prop1, math_prop1, math_prop2]` (values in [0, 1])
- Output: predicted std_math (scalar)
## Usage
```python
import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
class KernelRegressionModel(torch.nn.Module):
def __init__(self, dual_coef, X_fit, gamma=0.1):
super().__init__()
self.gamma = gamma
self.register_buffer("dual_coef", dual_coef)
self.register_buffer("X_fit", X_fit)
def forward(self, x):
dist = torch.cdist(x, self.X_fit, p=1)
K = torch.exp(-self.gamma * dist)
return K @ self.dual_coef
path = hf_hub_download("chewwt/dm_qwen4b_noise_emulator", "noise_model.safetensors")
tensors = load_file(path)
model = KernelRegressionModel(tensors["dual_coef"], tensors["X_fit"])
model.eval()
# x: (batch, 3) float64 tensor, features in [0, 1]
x = torch.tensor([[0.3, 0.4, 0.2]], dtype=torch.float64)
with torch.no_grad():
sigma = model(x) # predicted std_math
```
|