Gated Linear Attention Transformers with Hardware-Efficient Training
Paper • 2312.06635 • Published • 9
DISCLAIMER
This model is a research preview. The training dataset (MIMIC PERform AF) is distributed under the ODbL share alike license, so review those terms before redistributing any derived dataset. It is a screening aid, not a medical device or a diagnosis. Lowdown Labs has put together this model in the interest of advancing public science.
FELA-PPG: on device wrist PPG atrial fibrillation screen
FELA-PPG reads a single channel of photoplethysmography (PPG), the optical pulse signal from a wrist or finger sensor, and returns the probability that a 25 second window's rhythm is atrial fibrillation (AFib). It also returns a signal quality score, so a noisy window can be thrown out rather than scored on bad data.
It is the wearable companion to the FELA ECG model and shares its architecture. Being small, it runs continuously on a CPU, or potentially even microcontroller class hardware, and its memory does not grow as the stream goes on. The raw PPG never has to leave the device.
What goes in, what comes out
- Input: one PPG window, 800 samples at 32 Hz (a 25 second window). Any input rate from about 25 to
125 Hz is resampled to 32 Hz, bandpass filtered 0.5 to 8 Hz, cut into 25 second windows, and
z normalized per window. Shape
(1, 800). - Output:
afib: one logit; apply a sigmoid for the probability the window's rhythm is atrial fibrillation.qual: one signal quality logit, so low amplitude or flat windows are flagged at the app layer rather than producing a confident false call.
The shipped checkpoint has no heart rate head (hr=false); the code supports an optional HR
regression head but it is not shipped. The demo estimates heart rate by autocorrelation instead.
Why we built it this way
The main sequence mixer is a Fourier Neural Operator, a filter the model learns and applies in the frequency domain. A PPG pulse train is a periodic signal, which is exactly what a Fourier operator reads well, so the architecture fits the data. Before those layers see the signal, a small learned front end does some denoising, since wrist PPG is often corrupted by motion.
There is no long range all pairs attention here, so the working memory is small and stays fixed no matter how long the sensor has been streaming. That is what lets it run continuously on a low power CPU or microcontroller with no cloud connection.
The shipped model is a single ~0.77M parameter model (n_layer 6, n_embd 128, patch 10).
Performance
Speed and footprint, measured on CPU, single 25 second window, champion model.
| Metric | fp32 | int8 dynamic |
|---|---|---|
| Latency per window | 6.5 ms | 5.7 ms |
| Windows per second | 153 | 174 |
| Real time factor (process time / 25 s) | 0.00026 | 0.00023 |
A real time factor near 0.0003 means the model processes a 25 second window roughly 4000 times faster than real time on one CPU, so continuous monitoring is far below real time cost. Streaming 30 minutes of PPG as back to back windows showed 0.0 MB resident memory growth: the FNO and gated linear attention mixers carry a fixed size state, so memory does not grow with stream length.
Model size: 0.77M parameters, fp32 state dict 3.07 MB, int8 dynamic quant state dict 0.89 MB.
Not for int8: dynamic int8 quantization shrinks the model about 3.5x but, on this higher accuracy model, degrades AUROC substantially (0.96 fp32 down to about 0.60). Fp32 is only about 3 MB anyway - or use quantization aware training before int8.
Both checkpoints exist so the trade off can be measured on your own data; this repo ships the fp32 model.
Accuracy
Protocol: AFib detection on MIMIC PERform AF (open), subject level split (no subject appears in both train and test), seed 0, fractions 0.7 / 0.15 / 0.15. Held out test = 570 windows over the held out subjects (AF prevalence 0.667). Metric is AUROC on the test fold, scored once.
| Model (1D, FNO + gated linear attn) | Params | Test AUROC | F1 | Sensitivity | Specificity |
|---|---|---|---|---|---|
| FELA-PPG champion (n_layer 6, n_embd 128, patch 10) | 0.77M | 0.9628 | 0.9194 | 0.9158 | 0.8474 |
Published wrist PPG AFib references, for context only (different datasets and protocols, not directly comparable): DeepBeat (Torres-Soto and Ashley, npj Digit Med 2020) F1 0.96 / sens 0.98 / spec 0.99; SiamAF (2023) AUROC 0.877 to 0.914; Bashar et al. (2019, UMass Simband) AUC 0.975.
Protocol note: the split is subject level on MIMIC PERform AF (35 ICU subjects). With this few subjects the validation AUROC is noisy across runs and the 570 window test set is a single 5 subject fold, so treat the held out number as an in distribution result on a small dataset, not a deployment guarantee.
How to run it
import torch
from modeling import load_model
m = load_model("model.safetensors")
# x: one 25 s PPG window, 800 samples at 32 Hz, bandpass 0.5 to 8 Hz then z normalized, shape (1, 800)
x = torch.zeros(1, 800)
out = m(x)
afib_prob = torch.sigmoid(out["afib"]).item()
quality = torch.sigmoid(out["qual"]).item()
Loading with standard tooling
The repo ships config.json (architecture hyperparameters and the 32 Hz / 25 s / bandpass front end
settings) and a self contained modeling.py with a load_model / from_pretrained entry point:
from modeling import load_model
m = load_model("/path/to/weights_dir") # OR
m = load_model("lowdown-labs/fela-ppg")
The fp32 weights are shipped as model.safetensors.
Serving artifacts
model.safetensorsplusconfig.jsonfor the safetensors load path (fp32).verify.pyruns a fixed sample input and checks the output shape and a verification value.
For serving at scale, use the separate CPU native FELA server
(https://github.com/Lowdown-Labs/fela_server). It runs this model on CPU with no GPU required.
Training data
- MIMIC PERform AF Dataset (Charlton, "ppg-beats"): 35 critically ill adults (19 AF, 16 non AF), PPG and ECG, extracted from the MIMIC-III Waveform Database (Matched Subset). Only the PPG channel is used; signals are bandpass filtered 0.5 to 8 Hz, resampled to 32 Hz, and cut into 25 second (800 sample) z normalized windows with 50% overlap.
- Open source: Zenodo DOI 10.5281/zenodo.6807402, distributed under the Open Data Commons Open Database License v1.0 (ODbL-1.0). ODbL permits commercial use but is share alike: any redistributed database or derived database must be offered under ODbL with modifications documented, so surface it to legal before redistributing any derived dataset.
Full split definitions, the size assertion (assert te.sum() == 570), and the provenance and
commercial use caveats are reproduced in train.py.
Intended use, limitations, and safety
What it is for: a smartwatch or wearable AFib screening SDK that runs fully on device. The wearable streams its optical sensor into the model; windows flagged as possible AFib are surfaced to the user with a suggestion to seek a clinical ECG.
What it is not for: FELA-PPG is not a medical device and is not a diagnosis. A positive flag means "looks like AFib, get an ECG," not a clinical AFib diagnosis. Do not use it as the sole basis for a care decision.
Known limitations:
- Motion artifact: wrist PPG is corrupted by movement. The model has a light learned denoising stem and a quality head, but this is not full motion artifact removal. Severe motion windows should be rejected by the quality score, not scored for rhythm.
- Single dataset: trained and evaluated only on MIMIC PERform AF (35 ICU subjects, file level AF labels). It has not been validated on diverse consumer wrist hardware, skin tones, or free living motion. External wrist PPG AUROC in the literature is typically 0.88 to 0.92.
- Screening, not diagnosis; trained as AF vs non AF, so it does not distinguish other arrhythmias and may miss brief paroxysmal episodes.
- int8 on this high accuracy model degrades accuracy (see Performance); ship fp32.
How to cite
@misc{lowdownlabs_felappg,
title = {FELA-PPG: on device Fourier Neural Operator wrist PPG atrial fibrillation screen},
author = {Lowdown Labs},
year = {2026},
note = {Model card}
}
You should also cite the dataset used:
- Charlton, P. H., et al. (2022). Detecting beats in the photoplethysmogram: benchmarking open-source algorithms. Physiological Measurement, 43(8), 085007. https://doi.org/10.1088/1361-6579/ac826d
- MIMIC-III source: Johnson, A. E. W., et al. (2016). MIMIC-III, a freely accessible critical care database. Scientific Data, 3, 160035. https://doi.org/10.1038/sdata.2016.35
Acknowledgements and references
- Fourier Neural Operator: Li, Z., et al. (2021). Fourier Neural Operator for Parametric Partial Differential Equations. ICLR. https://arxiv.org/abs/2010.08895
- Gated Linear Attention: Yang, S., et al. (2024). Gated Linear Attention Transformers with Hardware-Efficient Training. https://arxiv.org/abs/2312.06635
- PyTorch: Paszke, A., et al. (2019). NeurIPS. https://arxiv.org/abs/1912.01703
Model family
This is part of the FELA family from Lowdown Labs: one Fourier Neural Operator architecture across
many modalities, all CPU native and subquadratic. This repo is pushed as lowdown-labs/fela-ppg.
Sibling repos share no weights, so none carries a base_model link.
License
Released under the Lowdown Labs Lovely License 1.0 (see LICENSE). A commercial license may be available;
contact Lowdown Labs.
- Downloads last month
- 2