README.md

---
license: other
license_name: lowdown-labs-lovely-license-1.0
license_link: LICENSE
tags:
  - fela
  - fourier-neural-operator
  - fno
  - cpu
  - on-device
  - edge
  - time-series
  - forecasting
  - energy
  - electricity
library_name: transformers
pipeline_tag: time-series-forecasting
---

# DISCLAIMER

This model is a research preview. It is released by Lowdown Labs in the interest of advancing
public science and demonstrating that a Fourier Neural Operator can forecast on a plain CPU with
a constant memory footprint. The dataset behind this
one (UCI electricity load) is CC BY 4.0 and could permit commercial use with attribution; see the
license notes below before deploying.

# FELA-TS edge: On device electricity load forecaster

FELA-TS edge reads a window of recent load history and forecasts the next 96 steps ahead. It is
tiny, about 1.76M parameters, and runs on a plain CPU with no GPU. Its working memory does not grow with how long the stream has been running, so you can drop
it into a meter or a building controller and let it forecast a live feed for years without the
memory climbing due to the model expanding its usage.

The shipped checkpoint is trained on the standard 321 channel electricity load benchmark, but the
architecture underneath is a generic multi channel forecaster for long horizons.

# What goes in, what comes out

- Input: one history window of shape `(1, 512, 321)`: 512 past time steps across 321 load
  channels, per channel standardized on the training statistics. RevIN handles the instance
  normalization inside the model, so you pass the standardized history and the model manages the
  per window mean and scale itself.
- Output: a forecast of shape `(1, 96, 321)`: the next 96 steps for all 321 channels, returned in
  the original (denormalized) units.
- In plain terms: "here are the last 512 hours of load for these meters" goes in, and "here is the
  next 96 hours for each of them" comes out.

The edge/on device unit runs a single load stream at a time (one channel), which is the
sub millisecond, constant memory path described under Performance.

# Why we built it this way

The sequence mixer is a Fourier Neural Operator, a filter the model learns and then applies in the
frequency domain. Electricity load runs on strong daily and weekly cycles, and those cycles are
what a frequency domain filter reads best, so the method fits the signal. The model is pure FNO
with a patch embedding, and it normalizes each history window internally (a step called RevIN) so
you do not have to hand tune it. It is pure FNO, with none of the gated memory or attention layers
that some sibling FELA models add.

Because it carries no attention and no KV cache, the working memory stays small and fixed no matter
how much history has streamed through it. The live single stream path keeps its history in a 512
sample ring buffer, about 2 KB, and one forward pass runs on roughly 126 KB of activations. That
flat footprint is the whole point at the edge: a full history attention model would keep growing its
memory until it ran out, while this one does not move.

# Performance

Speed and footprint, measured on CPU (single core, single load stream).

| Format | Size on disk | Notes |
|---|---|---|
| fp32 | 9.96 MB | full model weights |
| int8 (PyTorch/ONNX) | 7.29 MB | deployable edge unit |
| TFLite float16 | 4.9 MB | smallest deployable export |

- Full 321 channel forecast: median 165.7 ms on one CPU core.
- Single channel edge unit forecast: median 0.863 ms on one CPU core.
- int8 ONNX single stream update: about 0.56 ms per update (roughly 1,700 updates per second) on
  one core.
- Live working RAM (constant, independent of history length): a 2 KB ring buffer holds the entire
  retained history, plus about 126 KB of activations, for a working set under 200 KB excluding
  weights.

Constant memory is verified: the O(1) ring buffer streaming path was checked against recomputing
the forecast on the explicit window and matched exactly (mean absolute difference 0.0 over 1,800
streamed steps). 

The model fits comfortably on a Raspberry Pi 4/5 or Pi Zero 2 W (Cortex-A class,
ONNX path); it probably could not fit a small microcontroller (the 5 to 7 MB model is larger than typical
MCU flash) without explicitly targeting those constraints, so the realistic edge target is a Cortex-A single board computer, not a Cortex-M part.

# Accuracy

Protocol: UCI ElectricityLoadDiagrams20112014 ("electricity" / ECL benchmark), the standard
Informer chronological 70/10/20 train/validation/test split, lookback L = 512, forecast horizon
H = 96, over all 321 channels. Metric is mean squared error (MSE) and mean absolute error (MAE) on
the z normalized test windows, the standard long horizon ECL protocol.

| Benchmark | Metric | This model | Baseline (named) |
|---|---|---|---|
| Electricity, horizon 96 | MSE | 0.1325 | PatchTST ~0.129 |
| Electricity, horizon 96 | MAE | 0.2233 | iTransformer MSE ~0.148, DLinear MSE ~0.140 |

The card MSE/MAE of 0.1325 / 0.2233 reproduced at 0.1344 / 0.2254 on a fresh held out evaluation
(1.76M parameters). PatchTST leads on accuracy by a small margin at this
horizon; iTransformer and DLinear are behind. However, sub millisecond single stream forecasts on one CPU core with a flat, sub 200 KB working set are provably attainable, vs a full history attention model whose memory climbs with the length of the stream.

# How to run it

The short version:

```python
import torch
from modeling import load_model, forecast

m = load_model("/path/to/weights_dir")   # a dir with model.safetensors + config.json
# x: a (1, 512, 321) history window, per channel standardized on training stats
y = forecast(m, x)                        # -> (1, 96, 321) forecast in original units
```

`load_model` also accepts the `model.safetensors` path directly or a Hugging Face repo id. The fp32
weights ship as `model.safetensors`; the loader builds the architecture from `config.json` and
loads the state dict.

# Training data and license

- Dataset: UCI ElectricityLoadDiagrams20112014 ("electricity" / ECL), the 321 client hourly
  variant used across the Informer / Autoformer / PatchTST time series literature (26,304 hourly
  timesteps x 321 client load streams). Source: UCI Machine Learning Repository, dataset 321
  (DOI 10.24432/C58C86). The training split, windowing, and the audited train window count are
  reproduced in `train.py` (`--smoke` rebuilds the split and asserts 17,805 train windows).
- License: Creative Commons Attribution 4.0 International (CC BY 4.0), as listed on the UCI dataset
  page. Commercial use is ALLOWED, contingent only on attribution and preserving the license terms.
  This is the one dataset in the FELA industrial family with an unambiguous, commercially usable
  license. Attribution: cite the UCI ElectricityLoadDiagrams20112014 dataset (DOI 10.24432/C58C86).

# Intended use, limitations, and safety

What it is for: the forecasting core inside an edge or on premises energy product, running on a CPU
or a single board computer. Fit includes live load forecasting on a meter or gateway, building and
microgrid controllers, and any long running multi channel forecast where the memory must stay flat
while history accumulates.

What it is not for: this is a forecasting model, not a control system or a guarantee. It was
trained and evaluated only on the UCI electricity benchmark; generalization to other load profiles,
sampling rates, sensors, and populations is not established and must be validated before any
operational use. Accuracy is at parity with, not ahead of, the strongest published transformer
forecaster (PatchTST leads by a small margin); the model wins on footprint and constant memory, not
on top line accuracy. Continuous stream benchmarks beyond the verified streaming equals batch check
(for example multi day real time factor on a specific physical board) are not yet measured.

# Model family

This is part of the FELA family from Lowdown Labs: one Fourier Neural Operator architecture across
many modalities, all CPU native and subquadratic. Sibling repos share no weights, so none carries a
`base_model` link.

# License

Released under the Lowdown Labs Lovely License 1.0 (CC BY-NC 4.0 plus Hippocratic License 3.0). See LICENSE. For most LL models, a commercial license may be available; contact Lowdown Labs.