cascade custom_miner generator v1
Browse files- README.md +74 -0
- config.json +18 -0
- generator.py +377 -0
- requirements.txt +8 -0
README.md
ADDED
|
@@ -0,0 +1,74 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# custom_miner β the generator you submit
|
| 2 |
+
|
| 3 |
+
A cascade `DataGenerator` (`generator.Generator`) that turns one integer `seed`
|
| 4 |
+
into a corpus of univariate float series. The subnet holds the model, seeds, and
|
| 5 |
+
compute identical between king and challenger, so **the only thing that moves
|
| 6 |
+
your score is the distribution this generator emits.**
|
| 7 |
+
|
| 8 |
+
## The idea: compete on prior *diversity + realism*
|
| 9 |
+
|
| 10 |
+
The reference/genesis generators win by covering the shapes a forecaster must
|
| 11 |
+
handle. This one mixes **10 process families**, each fully seed-deterministic and
|
| 12 |
+
vectorised:
|
| 13 |
+
|
| 14 |
+
| family | what it contributes |
|
| 15 |
+
|--------|--------------------|
|
| 16 |
+
| `trend_seasonal_ar` | level + slope + multi-seasonal sinusoids + AR(1) noise |
|
| 17 |
+
| `regime_shift` | piecewise level & variance regimes (structural breaks) |
|
| 18 |
+
| `multiplicative` | positive level Γ seasonal factor Γ multiplicative noise |
|
| 19 |
+
| `ar2` | AR(2), stationarity-guaranteed (Levinson-Durbin), incl. near-unit-root |
|
| 20 |
+
| `integrated` | I(1)/I(2) random walks with drift |
|
| 21 |
+
| `threshold_ar` | SETAR β regime-switching nonlinear recurrence |
|
| 22 |
+
| `chaotic` | bounded chaotic maps (logistic / sine) |
|
| 23 |
+
| `rff_gp` | smooth GP-like samples via random Fourier features |
|
| 24 |
+
| `intermittent` | zero-inflated / intermittent demand |
|
| 25 |
+
| `pulse_outlier` | smooth base + sparse pulses/outliers + flat gaps |
|
| 26 |
+
|
| 27 |
+
The mixture weights (`family_weights` in [`config.json`](config.json)) were tuned
|
| 28 |
+
with `local_validator` against a broad multi-domain eval: strong seasonal
|
| 29 |
+
coverage matters (most real series are seasonal) while every family keeps
|
| 30 |
+
meaningful mass so the prior generalises across non-seasonal domains too.
|
| 31 |
+
|
| 32 |
+
## Contract compliance (what `cascade verify` checks)
|
| 33 |
+
|
| 34 |
+
* **Determinism** β every value comes from one `np.random.default_rng(seed)` in a
|
| 35 |
+
fixed draw order β byte-identical corpus at a fixed seed (the property the
|
| 36 |
+
trainer audits by building twice).
|
| 37 |
+
* **Code-only** β no shipped weights, no network, no clock; imports are numpy +
|
| 38 |
+
`cascade.interface` only (on the allowlist, clear of the static-guard blocklist).
|
| 39 |
+
* **Bounded + finite** β 1-D float64 series, length in `[min_length, max_length]`,
|
| 40 |
+
finite; `_sanitize` is the hard backstop.
|
| 41 |
+
* **Fast** β vectorised per family (a batched time-axis recurrence, never a
|
| 42 |
+
per-series Python loop), so draining the full `corpus_n_series` (16384) stays
|
| 43 |
+
well under `max_generate_seconds`.
|
| 44 |
+
|
| 45 |
+
Verify it yourself:
|
| 46 |
+
|
| 47 |
+
```bash
|
| 48 |
+
python -m cascade.miner.cli verify custom/custom_miner
|
| 49 |
+
# OK: generator would be accepted by the trainer.
|
| 50 |
+
# corpus_digest (seed=0): 8ecf44e7ebbb601f⦠[deterministic]
|
| 51 |
+
```
|
| 52 |
+
|
| 53 |
+
## How to make it yours
|
| 54 |
+
|
| 55 |
+
1. **Tune the mixture** β edit `family_weights` in `config.json` (no code change),
|
| 56 |
+
then re-run `python -m custom.local_validator` and watch the LCB / per-domain
|
| 57 |
+
win-rate move. This is the cheapest, safest lever.
|
| 58 |
+
2. **Add/replace a family** β add a `_yourfamily(rng, n, L) -> (n, L)` builder,
|
| 59 |
+
register it in `_FAMILIES` / `_DEFAULT_WEIGHTS` / the `builders` tuple. Keep it
|
| 60 |
+
vectorised and seed-deterministic; `_sanitize` guarantees finiteness.
|
| 61 |
+
3. **Re-verify + re-A/B** every change: `verify` must stay green and you want the
|
| 62 |
+
local KOTH verdict trending up before you deploy.
|
| 63 |
+
|
| 64 |
+
## Files
|
| 65 |
+
|
| 66 |
+
```
|
| 67 |
+
custom_miner/
|
| 68 |
+
generator.py class Generator(DataGenerator) β the mixture-of-priors
|
| 69 |
+
config.json name/description, length band, family_weights
|
| 70 |
+
requirements.txt hash-locked deps (numpy only). The trainer only FORMAT-checks
|
| 71 |
+
this file (it does not reinstall β the sandbox ships the
|
| 72 |
+
allowlisted stack), so the placeholder zero-hash is accepted,
|
| 73 |
+
as in the shipped reference generators. Real hashes optional.
|
| 74 |
+
```
|
config.json
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"name": "custom-mixture-of-priors-v1",
|
| 3 |
+
"description": "Diverse deterministic prior: trend/seasonal/AR, regime shifts, multiplicative seasonality, AR(2), integrated walks, threshold-AR, bounded chaos, RFF-GP, intermittent demand, and pulse/outlier structure. Fully seed-deterministic.",
|
| 4 |
+
"min_length": 64,
|
| 5 |
+
"max_length": 2048,
|
| 6 |
+
"family_weights": {
|
| 7 |
+
"trend_seasonal_ar": 0.26,
|
| 8 |
+
"regime_shift": 0.10,
|
| 9 |
+
"multiplicative": 0.16,
|
| 10 |
+
"ar2": 0.10,
|
| 11 |
+
"integrated": 0.08,
|
| 12 |
+
"threshold_ar": 0.06,
|
| 13 |
+
"chaotic": 0.05,
|
| 14 |
+
"rff_gp": 0.07,
|
| 15 |
+
"intermittent": 0.03,
|
| 16 |
+
"pulse_outlier": 0.03
|
| 17 |
+
}
|
| 18 |
+
}
|
generator.py
ADDED
|
@@ -0,0 +1,377 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""custom_miner β a diverse, deterministic synthetic time-series generator.
|
| 2 |
+
|
| 3 |
+
This is the artifact a cascade miner actually competes with: a subclass of
|
| 4 |
+
``cascade.interface.DataGenerator`` that turns a single integer ``seed`` into a
|
| 5 |
+
corpus of univariate float series. The subnet holds the model, seeds, and
|
| 6 |
+
compute budget byte-identical between the king and every challenger, so the
|
| 7 |
+
*only* thing that moves the forecast score is the distribution this file emits.
|
| 8 |
+
The competitive lever is therefore **prior diversity + realism**: a corpus that
|
| 9 |
+
covers more of the shapes a real forecaster must handle (trend, multi-seasonal,
|
| 10 |
+
regime shifts, integrated/near-unit-root dynamics, smooth GP-like curves,
|
| 11 |
+
nonlinear/chaotic recurrences, intermittent demand, outliers) trains a stronger
|
| 12 |
+
zero-shot model than the reference generator's trend+seasonal+AR(1) mix.
|
| 13 |
+
|
| 14 |
+
Design constraints this file respects (all from the contract in
|
| 15 |
+
``cascade.interface``):
|
| 16 |
+
|
| 17 |
+
* **Determinism is load-bearing.** Every value is drawn from one
|
| 18 |
+
``np.random.default_rng(seed)`` in a fixed draw order, so two runs at the same
|
| 19 |
+
seed produce byte-identical corpora β the property ``cascade verify`` audits
|
| 20 |
+
by building the corpus twice and comparing digests.
|
| 21 |
+
* **Code-only.** No shipped weights, no network, no clock, no un-seeded RNG.
|
| 22 |
+
Imports stay on the dependency allowlist (numpy only here) and clear of the
|
| 23 |
+
static-guard blocklist.
|
| 24 |
+
* **Bounded + finite.** Each series is 1-D ``(L,)`` float64, length in
|
| 25 |
+
``[min_length, max_length]``, finite (no NaN/inf). ``_sanitize`` is the last
|
| 26 |
+
gate so a numerically unlucky draw can never poison a training run.
|
| 27 |
+
|
| 28 |
+
Everything is **vectorised per family** (a batched time-axis recurrence, never a
|
| 29 |
+
per-series Python loop over time), so draining the full ``corpus_n_series``
|
| 30 |
+
(16384 on mainnet) is fast enough to stay well under ``max_generate_seconds``.
|
| 31 |
+
"""
|
| 32 |
+
|
| 33 |
+
from __future__ import annotations
|
| 34 |
+
|
| 35 |
+
import json
|
| 36 |
+
from collections.abc import Iterator
|
| 37 |
+
from pathlib import Path
|
| 38 |
+
|
| 39 |
+
import numpy as np
|
| 40 |
+
|
| 41 |
+
from cascade.interface import DataGenerator
|
| 42 |
+
|
| 43 |
+
# Series generated per vectorised batch. Bounds peak memory to O(_CHUNK Β· max_len)
|
| 44 |
+
# so streaming feed modes (which request millions of series and stop early) never
|
| 45 |
+
# materialise more than one chunk. ~256 keeps the vectorisation win without a big
|
| 46 |
+
# working set: 256 Γ 2048 Γ 8 B β 4 MB per family buffer.
|
| 47 |
+
_CHUNK = 256
|
| 48 |
+
|
| 49 |
+
# ββ family mixture ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 50 |
+
# Names are the process families the corpus mixes over; the default weights are
|
| 51 |
+
# a deliberate spread (no single family dominates). Override with
|
| 52 |
+
# ``"family_weights": {"chaotic": 0.2, ...}`` in config.json to tune the prior
|
| 53 |
+
# without touching code β unspecified families keep their default weight.
|
| 54 |
+
_FAMILIES: tuple[str, ...] = (
|
| 55 |
+
"trend_seasonal_ar", # level + slope + multi-seasonal + AR(1) noise (rich reference)
|
| 56 |
+
"regime_shift", # piecewise level/variance regimes with structural breaks
|
| 57 |
+
"multiplicative", # positive level Γ seasonal factor Γ multiplicative noise
|
| 58 |
+
"ar2", # AR(2), stationarity-guaranteed, incl. near-unit-root
|
| 59 |
+
"integrated", # I(1)/I(2) random walks with drift
|
| 60 |
+
"threshold_ar", # SETAR β regime-switching nonlinear recurrence
|
| 61 |
+
"chaotic", # bounded chaotic maps (logistic / sine)
|
| 62 |
+
"rff_gp", # smooth GP-like sample via random Fourier features
|
| 63 |
+
"intermittent", # zero-inflated / intermittent demand
|
| 64 |
+
"pulse_outlier", # smooth base + sparse pulses/outliers + flat gaps
|
| 65 |
+
)
|
| 66 |
+
# Tuned with local_validator against a broad multi-domain eval: strong seasonal
|
| 67 |
+
# coverage (trend_seasonal_ar + multiplicative) matters because most real series
|
| 68 |
+
# are seasonal, but every family keeps meaningful mass so the prior stays diverse
|
| 69 |
+
# (the diversity is what generalises across non-seasonal domains). Override per
|
| 70 |
+
# submission via ``"family_weights"`` in config.json.
|
| 71 |
+
_DEFAULT_WEIGHTS: dict[str, float] = {
|
| 72 |
+
"trend_seasonal_ar": 0.26,
|
| 73 |
+
"regime_shift": 0.10,
|
| 74 |
+
"multiplicative": 0.16,
|
| 75 |
+
"ar2": 0.10,
|
| 76 |
+
"integrated": 0.08,
|
| 77 |
+
"threshold_ar": 0.06,
|
| 78 |
+
"chaotic": 0.05,
|
| 79 |
+
"rff_gp": 0.07,
|
| 80 |
+
"intermittent": 0.03,
|
| 81 |
+
"pulse_outlier": 0.03,
|
| 82 |
+
}
|
| 83 |
+
|
| 84 |
+
|
| 85 |
+
class Generator(DataGenerator):
|
| 86 |
+
"""A mixture-of-priors generator. Submit as ``generator.Generator``."""
|
| 87 |
+
|
| 88 |
+
def __init__(self, config_dir: str, *, seed: int) -> None:
|
| 89 |
+
cfg_path = Path(config_dir) / "config.json"
|
| 90 |
+
cfg = json.loads(cfg_path.read_text(encoding="utf-8")) if cfg_path.is_file() else {}
|
| 91 |
+
self._cfg = cfg
|
| 92 |
+
self._seed = int(seed)
|
| 93 |
+
self._min_len = int(cfg.get("min_length", 64))
|
| 94 |
+
self._max_len = int(cfg.get("max_length", 2048))
|
| 95 |
+
if self._min_len < 1 or self._max_len < self._min_len:
|
| 96 |
+
raise ValueError(f"invalid length band [{self._min_len}, {self._max_len}]")
|
| 97 |
+
weights = dict(_DEFAULT_WEIGHTS)
|
| 98 |
+
for k, v in dict(cfg.get("family_weights", {})).items():
|
| 99 |
+
if k in weights:
|
| 100 |
+
weights[k] = float(v)
|
| 101 |
+
w = np.asarray([weights[f] for f in _FAMILIES], dtype=np.float64)
|
| 102 |
+
if not np.all(np.isfinite(w)) or w.min() < 0 or w.sum() <= 0:
|
| 103 |
+
raise ValueError("family_weights must be finite, non-negative, and not all zero")
|
| 104 |
+
self._weights = w / w.sum()
|
| 105 |
+
|
| 106 |
+
@property
|
| 107 |
+
def name(self) -> str:
|
| 108 |
+
return str(self._cfg.get("name", "custom-mixture-of-priors-v1"))
|
| 109 |
+
|
| 110 |
+
def generate(self, n_series: int) -> Iterator[np.ndarray]:
|
| 111 |
+
# Lazy, chunked generation. This is REQUIRED for the streaming feed
|
| 112 |
+
# modes (chain.toml ``corpus_mode = "stream_cpu"``): the trainer calls
|
| 113 |
+
# ``generate(n_upper)`` with ``n_upper = token_budget // min_length + 2``
|
| 114 |
+
# β often millions β and stops pulling once the token budget is hit
|
| 115 |
+
# (see cascade/trainer/stream.py). Materialising all ``n_series`` up
|
| 116 |
+
# front would OOM before the first yield. Generating one CHUNK at a time
|
| 117 |
+
# keeps memory at O(CHUNK) and stops early when the consumer stops,
|
| 118 |
+
# while a fixed draw order keeps the whole sequence seed-deterministic.
|
| 119 |
+
if n_series <= 0:
|
| 120 |
+
return
|
| 121 |
+
rng = np.random.default_rng(self._seed)
|
| 122 |
+
max_len = self._max_len
|
| 123 |
+
builders = (
|
| 124 |
+
_trend_seasonal_ar, _regime_shift, _multiplicative, _ar2,
|
| 125 |
+
_integrated, _threshold_ar, _chaotic, _rff_gp,
|
| 126 |
+
_intermittent, _pulse_outlier,
|
| 127 |
+
)
|
| 128 |
+
produced = 0
|
| 129 |
+
while produced < n_series:
|
| 130 |
+
# Always draw a FULL _CHUNK (yielding only what's still needed), so
|
| 131 |
+
# chunk boundaries fall at fixed multiples of _CHUNK regardless of
|
| 132 |
+
# the total requested. Then series i is a pure function of (seed, i):
|
| 133 |
+
# a run at any n >= i produces the identical series i. That makes a
|
| 134 |
+
# smaller local build a true prefix of the mainnet corpus, and keeps
|
| 135 |
+
# cross-mode/cross-party runs reproducible.
|
| 136 |
+
lengths = rng.integers(self._min_len, max_len + 1, size=_CHUNK)
|
| 137 |
+
fam_ids = rng.choice(len(_FAMILIES), size=_CHUNK, p=self._weights)
|
| 138 |
+
chunk: list[np.ndarray | None] = [None] * _CHUNK
|
| 139 |
+
for fam in range(len(_FAMILIES)):
|
| 140 |
+
idx = np.nonzero(fam_ids == fam)[0]
|
| 141 |
+
if idx.size == 0:
|
| 142 |
+
continue
|
| 143 |
+
block = _sanitize(builders[fam](rng, int(idx.size), max_len))
|
| 144 |
+
for row, series_i in enumerate(idx):
|
| 145 |
+
L = int(lengths[series_i])
|
| 146 |
+
chunk[series_i] = np.ascontiguousarray(block[row, :L], dtype=np.float64)
|
| 147 |
+
take = min(_CHUNK, n_series - produced)
|
| 148 |
+
for arr in chunk[:take]:
|
| 149 |
+
# Every slot is filled: fam_ids partitions [0, _CHUNK). Guard
|
| 150 |
+
# anyway (survives python -O) so a logic slip fails loud.
|
| 151 |
+
if arr is None: # pragma: no cover - defensive
|
| 152 |
+
raise RuntimeError("internal: unfilled series slot")
|
| 153 |
+
yield arr
|
| 154 |
+
produced += take
|
| 155 |
+
|
| 156 |
+
|
| 157 |
+
# ββ shared vectorised primitives ββββββββββββββββββββββββββββββββββββββββββββ
|
| 158 |
+
|
| 159 |
+
|
| 160 |
+
def _ar1_batch(innov: np.ndarray, phi: np.ndarray) -> np.ndarray:
|
| 161 |
+
"""AR(1) filter applied along the time axis of a (n, L) innovation block.
|
| 162 |
+
|
| 163 |
+
``x[:, t] = phi * x[:, t-1] + innov[:, t]``. The loop is over time (L
|
| 164 |
+
iterations, vectorised across the batch), never over the n series.
|
| 165 |
+
"""
|
| 166 |
+
n, L = innov.shape
|
| 167 |
+
x = np.empty((n, L), dtype=np.float64)
|
| 168 |
+
x[:, 0] = innov[:, 0]
|
| 169 |
+
p = phi.reshape(n)
|
| 170 |
+
for t in range(1, L):
|
| 171 |
+
x[:, t] = p * x[:, t - 1] + innov[:, t]
|
| 172 |
+
return x
|
| 173 |
+
|
| 174 |
+
|
| 175 |
+
def _ar2_batch(innov: np.ndarray, a1: np.ndarray, a2: np.ndarray) -> np.ndarray:
|
| 176 |
+
"""AR(2) filter: ``x_t = a1 x_{t-1} + a2 x_{t-2} + e_t`` (batched over n)."""
|
| 177 |
+
n, L = innov.shape
|
| 178 |
+
x = np.empty((n, L), dtype=np.float64)
|
| 179 |
+
x[:, 0] = innov[:, 0]
|
| 180 |
+
if L > 1:
|
| 181 |
+
x[:, 1] = a1 * x[:, 0] + innov[:, 1]
|
| 182 |
+
for t in range(2, L):
|
| 183 |
+
x[:, t] = a1 * x[:, t - 1] + a2 * x[:, t - 2] + innov[:, t]
|
| 184 |
+
return x
|
| 185 |
+
|
| 186 |
+
|
| 187 |
+
def _seasonal(rng: np.random.Generator, n: int, L: int, k_max: int = 3) -> np.ndarray:
|
| 188 |
+
"""Sum of 1..k_max sinusoids with per-series random period/amp/phase."""
|
| 189 |
+
t = np.arange(L, dtype=np.float64)[None, :]
|
| 190 |
+
periods = np.array([4, 7, 12, 24, 30, 52, 96, 144, 168, 336], dtype=np.float64)
|
| 191 |
+
k = rng.integers(1, k_max + 1, size=n)
|
| 192 |
+
out = np.zeros((n, L), dtype=np.float64)
|
| 193 |
+
for j in range(k_max):
|
| 194 |
+
active = (k > j).astype(np.float64)[:, None]
|
| 195 |
+
per = rng.choice(periods, size=n)[:, None]
|
| 196 |
+
amp = rng.uniform(0.2, 2.0, size=n)[:, None]
|
| 197 |
+
phase = rng.uniform(0.0, 2.0 * np.pi, size=n)[:, None]
|
| 198 |
+
out += active * amp * np.sin(2.0 * np.pi * t / per + phase)
|
| 199 |
+
return out
|
| 200 |
+
|
| 201 |
+
|
| 202 |
+
def _sparse_jumps(rng: np.random.Generator, n: int, L: int, rate: float, scale) -> np.ndarray:
|
| 203 |
+
"""A (n, L) block of mostly-zero values with occasional N(0, scale) jumps.
|
| 204 |
+
|
| 205 |
+
``cumsum`` over this yields a piecewise-constant level; ``exp(cumsum)`` of a
|
| 206 |
+
scaled version yields a piecewise-constant positive multiplier.
|
| 207 |
+
"""
|
| 208 |
+
mask = rng.random((n, L)) < rate
|
| 209 |
+
mag = rng.normal(0.0, 1.0, size=(n, L))
|
| 210 |
+
s = np.asarray(scale, dtype=np.float64)
|
| 211 |
+
if s.ndim == 1:
|
| 212 |
+
s = s[:, None]
|
| 213 |
+
jumps = mask * mag * s
|
| 214 |
+
jumps[:, 0] = 0.0
|
| 215 |
+
return jumps
|
| 216 |
+
|
| 217 |
+
|
| 218 |
+
# ββ family builders: each returns a (n, L) float64 block ββββββββββββββββββββ
|
| 219 |
+
|
| 220 |
+
|
| 221 |
+
def _trend_seasonal_ar(rng: np.random.Generator, n: int, L: int) -> np.ndarray:
|
| 222 |
+
t = np.arange(L, dtype=np.float64)[None, :]
|
| 223 |
+
level = rng.normal(0.0, 1.0, size=(n, 1))
|
| 224 |
+
slope = rng.normal(0.0, 0.01, size=(n, 1))
|
| 225 |
+
series = level + slope * t + _seasonal(rng, n, L)
|
| 226 |
+
phi = rng.uniform(0.0, 0.85, size=n)
|
| 227 |
+
sigma = rng.uniform(0.1, 0.6, size=(n, 1))
|
| 228 |
+
innov = rng.normal(0.0, 1.0, size=(n, L)) * sigma
|
| 229 |
+
return series + _ar1_batch(innov, phi)
|
| 230 |
+
|
| 231 |
+
|
| 232 |
+
def _regime_shift(rng: np.random.Generator, n: int, L: int) -> np.ndarray:
|
| 233 |
+
# Piecewise-constant level via cumsum of sparse jumps, plus a piecewise
|
| 234 |
+
# variance regime (occasional volatility multiplier), plus mild seasonality.
|
| 235 |
+
level = np.cumsum(_sparse_jumps(rng, n, L, rate=3.0 / L, scale=2.0), axis=1)
|
| 236 |
+
log_vol = np.cumsum(_sparse_jumps(rng, n, L, rate=3.0 / L, scale=0.5), axis=1)
|
| 237 |
+
vol = np.exp(np.clip(log_vol, -3.0, 3.0)) * rng.uniform(0.1, 0.5, size=(n, 1))
|
| 238 |
+
noise = rng.normal(0.0, 1.0, size=(n, L)) * vol
|
| 239 |
+
seas = _seasonal(rng, n, L, k_max=2) * rng.uniform(0.0, 1.0, size=(n, 1))
|
| 240 |
+
return level + seas + noise
|
| 241 |
+
|
| 242 |
+
|
| 243 |
+
def _multiplicative(rng: np.random.Generator, n: int, L: int) -> np.ndarray:
|
| 244 |
+
t = np.arange(L, dtype=np.float64)[None, :]
|
| 245 |
+
growth = rng.normal(0.0, 0.003, size=(n, 1))
|
| 246 |
+
base_level = np.exp(growth * t + rng.normal(0.0, 0.3, size=(n, 1))) # positive, drifting
|
| 247 |
+
amp = rng.uniform(0.1, 0.6, size=(n, 1))
|
| 248 |
+
seas = 1.0 + amp * np.sin(
|
| 249 |
+
2.0 * np.pi * t / rng.choice([7.0, 12.0, 24.0, 52.0], size=n)[:, None]
|
| 250 |
+
+ rng.uniform(0.0, 2 * np.pi, size=(n, 1))
|
| 251 |
+
)
|
| 252 |
+
noise = 1.0 + rng.normal(0.0, 1.0, size=(n, L)) * rng.uniform(0.02, 0.15, size=(n, 1))
|
| 253 |
+
scale = rng.uniform(1.0, 50.0, size=(n, 1))
|
| 254 |
+
return scale * base_level * np.clip(seas, 0.05, None) * np.clip(noise, 0.05, None)
|
| 255 |
+
|
| 256 |
+
|
| 257 |
+
def _ar2(rng: np.random.Generator, n: int, L: int) -> np.ndarray:
|
| 258 |
+
# Draw partial autocorrelations in (-1, 1) and map to AR(2) coeffs via
|
| 259 |
+
# Levinson-Durbin, which guarantees stationarity. Bias p1 high for
|
| 260 |
+
# persistent (sometimes near-unit-root) series.
|
| 261 |
+
p1 = rng.uniform(0.3, 0.98, size=n)
|
| 262 |
+
p2 = rng.uniform(-0.6, 0.6, size=n)
|
| 263 |
+
a2 = p2
|
| 264 |
+
a1 = p1 * (1.0 - p2)
|
| 265 |
+
sigma = rng.uniform(0.2, 0.8, size=(n, 1))
|
| 266 |
+
innov = rng.normal(0.0, 1.0, size=(n, L)) * sigma
|
| 267 |
+
x = _ar2_batch(innov, a1, a2)
|
| 268 |
+
drift = rng.normal(0.0, 0.005, size=(n, 1)) * np.arange(L, dtype=np.float64)[None, :]
|
| 269 |
+
return x + drift
|
| 270 |
+
|
| 271 |
+
|
| 272 |
+
def _integrated(rng: np.random.Generator, n: int, L: int) -> np.ndarray:
|
| 273 |
+
order2 = rng.random(n) < 0.35
|
| 274 |
+
drift = rng.normal(0.0, 0.02, size=(n, 1))
|
| 275 |
+
sigma = rng.uniform(0.2, 1.0, size=(n, 1))
|
| 276 |
+
steps = rng.normal(0.0, 1.0, size=(n, L)) * sigma + drift
|
| 277 |
+
walk = np.cumsum(steps, axis=1)
|
| 278 |
+
walk2 = np.cumsum(walk, axis=1)
|
| 279 |
+
o2 = order2[:, None]
|
| 280 |
+
# I(2) grows fast; damp it so it shares scale with the I(1) branch.
|
| 281 |
+
return np.where(o2, walk2 / max(L, 1) ** 0.5, walk)
|
| 282 |
+
|
| 283 |
+
|
| 284 |
+
def _threshold_ar(rng: np.random.Generator, n: int, L: int) -> np.ndarray:
|
| 285 |
+
# SETAR(2): coefficient flips with the sign of the previous value β a simple
|
| 286 |
+
# nonlinear recurrence that produces asymmetric, regime-switching dynamics.
|
| 287 |
+
phi_hi = rng.uniform(0.3, 0.9, size=n)
|
| 288 |
+
phi_lo = rng.uniform(-0.9, 0.3, size=n)
|
| 289 |
+
const_hi = rng.normal(0.0, 0.3, size=n)
|
| 290 |
+
const_lo = rng.normal(0.0, 0.3, size=n)
|
| 291 |
+
sigma = rng.uniform(0.2, 0.7, size=(n, 1))
|
| 292 |
+
innov = rng.normal(0.0, 1.0, size=(n, L)) * sigma
|
| 293 |
+
x = np.empty((n, L), dtype=np.float64)
|
| 294 |
+
x[:, 0] = innov[:, 0]
|
| 295 |
+
for t in range(1, L):
|
| 296 |
+
prev = x[:, t - 1]
|
| 297 |
+
hi = prev >= 0.0
|
| 298 |
+
phi = np.where(hi, phi_hi, phi_lo)
|
| 299 |
+
const = np.where(hi, const_hi, const_lo)
|
| 300 |
+
x[:, t] = np.clip(const + phi * prev + innov[:, t], -1e6, 1e6)
|
| 301 |
+
return x
|
| 302 |
+
|
| 303 |
+
|
| 304 |
+
def _chaotic(rng: np.random.Generator, n: int, L: int) -> np.ndarray:
|
| 305 |
+
# Bounded chaotic maps: logistic x_{t+1}=r x(1-x) with rβ[3.6,4.0], and the
|
| 306 |
+
# sine map r sin(pi x). Both stay in [0,1]; standardise afterwards. A random
|
| 307 |
+
# observation length as a "sampling rate" adds variety across series.
|
| 308 |
+
use_sine = rng.random(n) < 0.5
|
| 309 |
+
r_log = rng.uniform(3.6, 4.0, size=n)
|
| 310 |
+
r_sin = rng.uniform(0.85, 1.0, size=n)
|
| 311 |
+
x0 = rng.uniform(0.05, 0.95, size=n)
|
| 312 |
+
x = np.empty((n, L), dtype=np.float64)
|
| 313 |
+
cur = x0.copy()
|
| 314 |
+
x[:, 0] = cur
|
| 315 |
+
for t in range(1, L):
|
| 316 |
+
nxt_log = r_log * cur * (1.0 - cur)
|
| 317 |
+
nxt_sin = r_sin * np.sin(np.pi * cur)
|
| 318 |
+
cur = np.where(use_sine, nxt_sin, nxt_log)
|
| 319 |
+
cur = np.clip(cur, 0.0, 1.0)
|
| 320 |
+
x[:, t] = cur
|
| 321 |
+
return x
|
| 322 |
+
|
| 323 |
+
|
| 324 |
+
def _rff_gp(rng: np.random.Generator, n: int, L: int, K: int = 48) -> np.ndarray:
|
| 325 |
+
# Random Fourier features approximate a stationary (RBF-like) GP sample:
|
| 326 |
+
# f(t) = sqrt(2/K) * sum_k cos(w_k t + b_k), w_k ~ N(0, 1/lengthscale^2).
|
| 327 |
+
# Loop over the K features (K iterations, vectorised over n and L) so peak
|
| 328 |
+
# memory stays (n, L), never (n, K, L).
|
| 329 |
+
t = np.arange(L, dtype=np.float64)[None, :]
|
| 330 |
+
lengthscale = rng.uniform(20.0, 200.0, size=(n, 1))
|
| 331 |
+
acc = np.zeros((n, L), dtype=np.float64)
|
| 332 |
+
for _ in range(K):
|
| 333 |
+
w = rng.normal(0.0, 1.0, size=(n, 1)) / lengthscale
|
| 334 |
+
b = rng.uniform(0.0, 2.0 * np.pi, size=(n, 1))
|
| 335 |
+
acc += np.cos(w * t + b)
|
| 336 |
+
return np.sqrt(2.0 / K) * acc
|
| 337 |
+
|
| 338 |
+
|
| 339 |
+
def _intermittent(rng: np.random.Generator, n: int, L: int) -> np.ndarray:
|
| 340 |
+
# Zero-inflated demand: sparse positive spikes on a low baseline. Common in
|
| 341 |
+
# retail/logistics and absent from the reference generator.
|
| 342 |
+
p = rng.uniform(0.05, 0.4, size=(n, 1))
|
| 343 |
+
occur = (rng.random((n, L)) < p).astype(np.float64)
|
| 344 |
+
magnitude = rng.gamma(shape=2.0, scale=1.0, size=(n, L)) * rng.uniform(1.0, 10.0, size=(n, 1))
|
| 345 |
+
baseline = rng.uniform(0.0, 0.5, size=(n, 1))
|
| 346 |
+
return baseline + occur * magnitude
|
| 347 |
+
|
| 348 |
+
|
| 349 |
+
def _pulse_outlier(rng: np.random.Generator, n: int, L: int) -> np.ndarray:
|
| 350 |
+
# A smooth base with sparse additive pulses (outliers) and occasional flat
|
| 351 |
+
# (held-constant) gaps β the messy structure real series carry.
|
| 352 |
+
base = _rff_gp(rng, n, L, K=24) * rng.uniform(0.5, 2.0, size=(n, 1))
|
| 353 |
+
base += _seasonal(rng, n, L, k_max=1) * rng.uniform(0.0, 1.0, size=(n, 1))
|
| 354 |
+
pulses = _sparse_jumps(rng, n, L, rate=5.0 / L, scale=rng.uniform(3.0, 8.0, size=n))
|
| 355 |
+
series = base + pulses
|
| 356 |
+
# Occasional flat gaps: hold the value across a short random run.
|
| 357 |
+
hold = rng.random((n, L)) < (2.0 / L)
|
| 358 |
+
hold[:, 0] = False
|
| 359 |
+
for t in range(1, L):
|
| 360 |
+
m = hold[:, t]
|
| 361 |
+
series[m, t] = series[m, t - 1]
|
| 362 |
+
return series
|
| 363 |
+
|
| 364 |
+
|
| 365 |
+
# ββ final safety gate βββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 366 |
+
|
| 367 |
+
|
| 368 |
+
def _sanitize(block: np.ndarray) -> np.ndarray:
|
| 369 |
+
"""Guarantee the contract: finite float64, no NaN/inf, bounded magnitude.
|
| 370 |
+
|
| 371 |
+
The trainer's ``check_series`` rejects any non-finite value, which would
|
| 372 |
+
fail the whole run β so this is the hard backstop after every family
|
| 373 |
+
builder. Replaces non-finite values and clips to a generous bound.
|
| 374 |
+
"""
|
| 375 |
+
x = np.asarray(block, dtype=np.float64)
|
| 376 |
+
x = np.nan_to_num(x, nan=0.0, posinf=1e6, neginf=-1e6)
|
| 377 |
+
return np.clip(x, -1e6, 1e6)
|
requirements.txt
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Hash-locked, allowlisted deps for the generator. The generator is numpy-only
|
| 2 |
+
# by design, so this is the whole dependency surface. The trainer only
|
| 3 |
+
# FORMAT-checks these lines (pkg==ver + a 64-hex sha256) β it does not reinstall
|
| 4 |
+
# them; the sandbox image already ships the allowlisted stack (numpy, scipy,
|
| 5 |
+
# torch, β¦). So the placeholder zero-hash below is accepted, exactly as the
|
| 6 |
+
# shipped reference generators use it. Pinning real wheel hashes is optional
|
| 7 |
+
# hygiene: pip hash $(pip download numpy==1.26.4 -d /tmp/x)
|
| 8 |
+
numpy==1.26.4 --hash=sha256:0000000000000000000000000000000000000000000000000000000000000000
|