LuminaV Optimizer v1.5.3 (`luminamoon v1.0.6`)

Low-Precision BF16/FP16 Deep Learning Optimizer with Dual-Buffer Convex Lookahead Momentum (CLM), Single-Buffer Triangle-Defect Convex Lookahead Momentum (TD-CLM), Rational Padé [13/6] Bounding, and Stateless 32-Bit MurmurHash3 Stochastic Rounding

Official Codebase: github.com/cloverx-id/luminamoon | Docs Portal: cloverx-id.github.io/luminamoon

Foundational Research Paper & Living Specification

LuminaV Optimizer Theory & Mechanics
Read LuminaV.pdf (Foundational v1.0 Archive) | LuminaV v1.5.3 Mathematical Changelog | Full Online Documentation

PROJECT LOOMY

Loomy(luminamoon) is the official reference implementation and central monorepo of PROJECT LOOMY, providing low-precision numerical optimization, multimodal dataset streaming, and formal verification suites developed by Silver Moon (@cloverxion) and Lumina Moon (@cloverx-id) contributors.

Built with a modular architecture under the PROJECT LOOMY specification, luminamoon connects specialized components for CPU/GPU hardware execution, low-latency streaming, and verified numerical stability:

System Architecture

flowchart TD
    Shards["Dataset Sources (HF / PyTorch / Shards)"] --> DataLoader["LuminaMoon Data Engine (SIMD + StringTensor + XXH3)"]
    DataLoader --> TrainLoop["Training Loop (Batch Stream)"]
    DataLoader --> Checkpoint["128B Ping-Pong Checkpoint & WAL"]
    TrainLoop --> Checkpoint
    TrainLoop --> Dispatcher["LuminaV Hardware Dispatcher"]
    Eval["LSO-Eval Suite (Z3 SMT & Parity Gate)"] -.-> Dispatcher
    Dispatcher --> Triton["Triton AOT/JIT GPU (CUDA / ROCm)"]
    Dispatcher --> Vulkan["Vulkan 1.3/1.4 Compute (UMA / SPIR-V)"]
    Dispatcher --> SIMD["Rayon CPU SIMD (AVX-512 / AVX2)"]
    Dispatcher --> Scalar["Reference Scalar Core (Fallback)"]

Projects & Components

Project / Component	Path	Status	Description	Documentation
LuminaV Optimizer	`optimizer/`	`Active` (Spec `v1.5.3`)	Canonical LuminaV v1.5.3 optimizer engine with CPU SIMD (AVX-512/AVX2), Vulkan UMA, and Triton AOT/JIT kernels.	Optimizer Guide
LuminaMoon Data Engine	`data/`	`Active` (`v1.0.6.post1`)	Modular preprocessing engine: AVX-512/AVX2/NEON SIMD text normalization, dual-layout StringTensor (CPU StringView & GPU contiguous buffers), heuristic Shannon quality filtering, MinHash/XXH3-64 deduplication, PII masking, and 128B Ping-Pong crash recovery.	Data Engine Guide
LSO-Eval Suite	`eval/`	`Active`	Formal verification suite: Z3 SMT solver invariants, 2D trajectory benchmarks, and numerical parity audits.	Eval Guide
Changelogs	`changelogs/`	`Active`	Dedicated version histories separating Software Package releases (`v1.0.0`–`v1.0.6.post1`) and LuminaV Algorithm Specifications (`v1.0.0`–`v1.5.3`).	Changelogs
Documentation	`docs/`	`Active`	Centralized documentation portal, developer internal blueprints (`docs/optimize/`, `docs/dataloader/`), and ecosystem version audits.	Docs Portal
Python Package	`python/luminamoon/`	`Active` (`v1.0.6.post1`)	Unified free-threaded (`abi3t`) Python namespace exposing LuminaV Optimizer, DataStreamLoader, and PyTorch/NumPy/HuggingFace interop.	API Reference

Quickstart

Installation

Install the compiled package wheel from PyPI:

pip install luminamoon

Or build from source using Maturin and the Rust 2024 toolchain:

git clone https://github.com/cloverx-id/luminamoon.git
cd luminamoon
pip install maturin
maturin develop --release

Basic Usage (LuminaV Optimizer)

import numpy as np
from luminamoon import LuminaV

# Initialize parameters
weights = np.random.randn(64, 10).astype(np.float32)
biases = np.zeros(10, dtype=np.float32)

# Instantiate LuminaV optimizer (uses calibrated default hyperparameters)
optimizer = LuminaV([weights, biases], lr=8e-4, weight_decay=0.08, buffer=2)

# Execute optimization step with explicit gradients
grad_w = np.random.randn(64, 10).astype(np.float32)
grad_b = np.random.randn(10, dtype=np.float32)
optimizer.step(grads=[grad_w, grad_b])

For PyTorch deep learning workflows, see the PyTorch Integration Guide.

LuminaV Hyperparameter Quick Reference

All default values below are exposed directly in IDE autocompletion and LuminaV.__init__ docstrings:

Parameter	Type	Default	Description
`lr`	`float`	`8e-4` (`0.0008`)	Learning rate ($\eta$).
`betas`	`Tuple[float, float]`	`(0.9, 0.999)`	Exponential moving average factors $(\beta_1, \beta_2)$ for first moment and second moment / temporal defect.
`eps`	`float`	`1e-8`	Numerical stability constant ($\epsilon$) in the adaptive denominator.
`weight_decay`	`float`	`0.08` (`8e-2`)	Decoupled weight decay coefficient.
`tau`	`float`	`0.8`	Soft-projection temperature threshold ($\tau$) for denominator damping.
`lam`	`float`	`0.3`	Temporal defect sensitivity weight ($\lambda$) in Single-Buffer TD-CLM (`buffer=1`).
`cautious`	`bool`	`True`	Enables sign-aligned cautious update masking and lagged EMA scaling.
`cautious_clamp_min`	`float`	`0.5`	Lower bound in `(0.0, 1.0]` for cautious alignment scaling.
`buffer`	`int \| str`	`2`	State topology: `2` (`"dual"`, `"clm"`) for Dual-Buffer Convex Lookahead Momentum (CLM), or `1` (`"single"`, `"td-clm"`) for Single-Buffer Triangle-Defect Convex Lookahead Momentum (TD-CLM).
`centered_defect`	`bool`	`False`	Centers innovation defect $(g - m)$ across both Dual-Buffer CLM (`buffer=2`) and Single-Buffer TD-CLM (`buffer=1`) modes.
`stochastic_rounding`	`bool`	`True`	Applies stateless 32-bit MurmurHash3 bit-avalanche stochastic rounding when writing to `bfloat16` / `float16` weights.
`bound`	`bool`	`True`	Enables adaptive trust-region update bounding.
`bound_type`	`str \| int`	`"coordinate"`	Trust-region geometry: `"coordinate"` (element-wise clip, zero collective sync) or `"radial"` (global norm ratio).
`bound_ratio`	`float`	`0.05`	Maximum update magnitude relative to parameter scale.
`master_weights`	`str \| bool`	`"none"`	Mixed-precision master weight mode for `bfloat16`/`float16`: `"none"` (`False`), `"semi"`, or `"full"` (`True`).
`execution`	`str \| int`	`"auto"`	Hardware backend: `"auto"`, `"triton"`, `"vulkan"`, `"simd"`, or `"scalar"`.
`fused_single_pass`	`bool`	`True`	Fuses parameter update and scalar reduction statistics into a single kernel pass.
`ema_decay`	`str \| float`	`"auto"`	EMA decay for lagged cautious/radial scalars (`"auto"` resolves to `0.80` for `buffer=2` and `0.95` for `buffer=1`).
`fsdp_sync`	`bool`	`False`	Synchronizes global reduction scratch buffers across `torch.distributed` FSDP shards.
`seed`	`int`	`1337`	Deterministic PRNG seed for stateless stochastic rounding.
`version`	`Optional[str]`	`"1.5.3"`	Locks the active LuminaV mathematical specification version.

Software vs. Algorithm Versioning (`v1.0.6.post1` / Spec `v1.5.3`)

luminamoon maintains two orthogonal version tracks (documented in changelogs/):

Software Package Version (1.0.6.post1): Tracks the unified Rust + Python wheel distribution (luminamoon.__version__ and luminamoon.data.get_version()). See changelogs/SOFTWARE_CHANGELOG.md.
LuminaV Algorithm Specification (1.5.3): Tracks the canonical mathematical update rules (LuminaV.ALGORITHM_SPEC and LuminaV.getVersion()). See changelogs/LUMINAV_ALGORITHM_CHANGELOG.md.

All version resolution operates 100% offline using local compatibility profiles:

import luminamoon
from luminamoon import LuminaV
import luminamoon.data as lmd

# 1. Introspect Software Package Version vs. Algorithm Specification Version
print(luminamoon.__version__)    # "1.0.6.post1" (Software distribution)
print(lmd.get_version())         # "1.0.6.post1" (Data Engine module)
print(LuminaV.ALGORITHM_SPEC)    # "1.5.3" (Canonical mathematical specification)
print(LuminaV.getVersion())      # "1.5.3"

# 2. Lock or select a specific algorithm version per-instance or globally
opt = LuminaV([weights, biases], lr=8e-4, version="1.5.3")
opt.setVersion("1.5.3")          # Instance-level version lock
LuminaV.setVersion("1.5.3")      # Class-level default

# 3. Canonical specification lock (Note: Specifications prior to v1.5.0 are deprecated)
# LuminaV v1.5.3 TD-CLM is the canonical mathematical specification across CPU/GPU runtimes.
opt_canonical = LuminaV([weights, biases], version="1.5.3")

Basic Usage (LuminaMoon Data Engine: Fused Preprocessing & Zero-Copy Pipeline)

from luminamoon.data import FusedPipeline, StringTensor, create_resumable_collate, ResumablePipeline

# 1. Ultra-fast fused single-pass preprocessing (>400k records/s) in Rust
pipeline = FusedPipeline(
    trim_whitespace=True,
    normalize_newlines=True,  # AVX-512 / AVX2 / NEON SIMD
    min_chars=10,
    min_entropy=2.0,
)
string_tensor = pipeline(["  sample 1\r\n", "sample 2 with proper prose text"])

# 2. Zero-copy export to Apache Arrow, Polars, and GPU-ready PyTorch buffers
polars_series = string_tensor.to_polars()
arrow_array = string_tensor.to_arrow()
offsets_tensor, payload_tensor = string_tensor.to_torch_gpu(device="cuda")

# 3. Stream dataset with async prefetching & 128B Ping-Pong crash-recovery
pipe = ResumablePipeline(
    source=raw_data,
    fused_pipeline=pipeline,
    batch_size=64,
    prefetch_batches=4,
    checkpoint_path="./cache/checkpoint.meta",
)
for batch in pipe:
    # Training step...
    pipe.commit()  # Atomically persists Dual-Slot A/B checkpoint and deduplication WAL

For detailed architecture, Shannon entropy heuristics, MinHash deduplication, and PyTorch collate_fn adapters, see the Data Engine Guide.

Verification & CI/CD

All pull requests and releases are validated across Linux, Windows, and macOS:

# Enforce zero-warning static analysis across the Rust workspace
cargo clippy --workspace --all-targets -- -D warnings

# Run Rust workspace unit and differential parity tests (Optimizer + DataLoader)
cargo test --workspace

# Run Python static type check
npx pyright python/luminamoon

# Run standalone NumPy, PyTorch, and DataLoader verification suites
python optimizer/tests/test_standalone_numpy.py
python optimizer/tests/test_end_to_end_training.py
python -B tests/test_data_modular.py
python -B tests/test_gpu_vs_cpu_strings.py

# Run LSO-Eval formal verification suite
python -m eval

Developer Resources

For contributors and systems engineers building on or extending PROJECT LOOMY (luminamoon):

Changelogs (Software & Algorithm): changelogs/
Optimizer Modular Documentation: docs/optimize/
Data Engine Modular Documentation: data/
Optimizer Library Audits: docs/library_audit/
Data Standalone Library Audits: data/docs/library_audit/
Data Engine Architecture & User Guide: data/README.md
User Documentation: docs/en/

Authors & Maintenance

PROJECT LOOMY (luminamoon) is engineered and maintained by:

Silver Moon (@cloverxion)
Lumina Moon (@cloverx-id) contributors

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.

LuminaV Optimizer v1.5.3

LuminaV is a high-performance mathematical optimization engine implementing the LuminaV algorithm, built for direct low-precision execution (BF16/FP16) without requiring FP32 master weights. Powered by unbiased bitwise stochastic rounding, it provides native hardware acceleration across CPU SIMD (AVX-512, AVX2), Vulkan compute shaders, and Triton GPU kernels, with zero-copy buffer interoperability for NumPy and PyTorch.

Features

Dual-Buffer CLM & Single-Buffer TD-CLM: Implements Convex Lookahead Momentum (CLM) (buffer=2) and Triangle-Defect Convex Lookahead Momentum (TD-CLM) (buffer=1) bounded directly via the Triangle Inequality Envelope $\mathcal{E}t = w_1 |m{t-1}| + w_2 |g_t| + \lambda |d_t^{(\text{B1})}| + \epsilon_{\text{eff}}$. Single-buffer mode reduces optimizer state memory by 50%.
Zero Mandatory Dependencies: The core package has dependencies = [] with a compiled binary under 1 MB. PyTorch, NumPy, and Triton are optional.
Hardware Acceleration:
- x86_64 CPU: AVX-512 and AVX2 SIMD kernels parallelized across cores via Rayon.
- Vulkan 1.3/1.4: Unified Memory Architecture (UMA) SPIR-V compute shaders for integrated and discrete GPUs.
- Triton AOT / JIT: Ahead-of-time compiled kernels for NVIDIA CUDA (SM75–SM120) and AMD ROCm (RDNA3/CDNA).
- Scalar Core: Portable pure-Rust fallback for other architectures (ARM/aarch64).
Framework Integration:
- NumPy 2.x: Native zero-copy execution via the Python buffer protocol and __array_interface__.
- PyTorch: Standard torch.optim.Optimizer interface with FSDP collective synchronization and FP16/BF16 master weights.
Runtime Compatibility: Built with CPython Stable ABI (abi3 and abi3t), supporting Python 3.10 through 3.15+ including free-threaded execution (PEP 703).

Architecture & Dispatch

The runtime dispatcher automatically selects the execution backend based on tensor device, memory size, and available hardware:

flowchart TD
    PyTorch["PyTorch Tensors (FP32 / BF16 / FP16)"] --> FFI["LuminaV PyO3 abi3t FFI (Zero-Copy)"]
    NumPy["NumPy 2.x Arrays (Buffer Protocol)"] --> FFI
    FFI --> Dispatcher["Hardware Capability Dispatcher"]
    Dispatcher --> Triton["Triton AOT/JIT GPU (CUDA / ROCm)"]
    Dispatcher --> Vulkan["Vulkan 1.3/1.4 SPIR-V (UMA, N >= 16,384)"]
    Dispatcher --> SIMD["Rayon CPU SIMD (x86_64, N < 16,384)"]
    Dispatcher --> Scalar["Reference Scalar Core (Fallback)"]
    Triton & Vulkan & SIMD & Scalar --> Math["CLM (2-Buffer) & TD-CLM (1-Buffer) Update"]

Workspace Crate Structure

optimizer/
├── Cargo.toml
├── src/
│   ├── lib.rs              # Rust crate entry point
│   ├── optimizer.rs        # Core PyO3 optimizer implementation
│   └── python_ffi/         # Python-Rust FFI bridge (mod.rs, config_parser.rs, extract.rs, worker.rs)
├── crates/
│   ├── luminav-core/       # Mathematical formulas, PRNG, and state structs
│   ├── luminav-simd/       # AVX-512 and AVX2 explicit SIMD intrinsics
│   ├── luminav-vulkan/     # Vulkan 1.3/1.4 UMA compute pipeline (Ash)
│   ├── luminav-triton-aot/ # Triton AOT runner for CUDA and ROCm HIP
│   └── luminav-dispatch/   # Hardware capability detection and routing
├── tests/                  # Standalone NumPy and PyTorch verification tests
└── tools/                  # AOT compilers for SPIR-V and Triton

Installation & Build

From Wheel (PyPI)

pip install luminamoon

From Source (Maturin)

git clone https://github.com/cloverx-id/luminamoon.git
cd luminamoon
pip install maturin
maturin develop --release

Optional Dependencies

# PyTorch integration
pip install "luminamoon[torch]"

# Triton GPU acceleration (CUDA / ROCm)
pip install "luminamoon[gpu]"

# All optional dependencies
pip install "luminamoon[all]"

Quickstart

Standalone NumPy (No PyTorch Required)

import numpy as np
from luminamoon import LuminaV

w = np.random.randn(64, 10).astype(np.float32)
b = np.zeros(10, dtype=np.float32)

optimizer = LuminaV(
    [w, b],
    lr=1e-2,
    betas=(0.9, 0.999),
    weight_decay=1e-2,
    buffer=2,  # 2 = Dual-Buffer CLM, 1 = Single-Buffer TD-CLM
)

grad_w = np.random.randn(64, 10).astype(np.float32)
grad_b = np.random.randn(10).astype(np.float32)

optimizer.step(grads=[grad_w, grad_b])

PyTorch Integration

import torch
import torch.nn as nn
from luminamoon import LuminaV

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = nn.Sequential(
    nn.Linear(128, 256),
    nn.ReLU(),
    nn.Linear(256, 10),
).to(device)

criterion = nn.CrossEntropyLoss()
optimizer = LuminaV(
    model.parameters(),
    lr=8e-4,
    betas=(0.9, 0.999),
    weight_decay=8e-2,
    buffer=2,
    bound=True,
    bound_type="coordinate",
    cautious=True,
)

inputs = torch.randn(64, 128, device=device)
targets = torch.randint(0, 10, (64,), device=device)

optimizer.zero_grad()
loss = criterion(model(inputs), targets)
loss.backward()
optimizer.step()

Hardware Verification

To inspect detected hardware backends and active dispatch targets:

from luminamoon import LuminaV

print(LuminaV.detect_hardware())

Documentation & Library Audits

Running Tests

# Lint workspace with zero warnings
cargo clippy --workspace --all-targets -- -D warnings

# Rust workspace unit and parity tests
cargo test --workspace

# Standalone NumPy test
python optimizer/tests/test_standalone_numpy.py

# PyTorch integration test
python optimizer/tests/test_end_to_end_training.py

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.

Empirical Pretraining Benchmarks (`Qwen3.5-4B-Base`)

LuminaV was evaluated on a 4.0-billion parameter Large Language Model (Qwen/Qwen3.5-4B-Base) initialized from architectural config (AutoModelForCausalLM.from_config) under full pretraining from scratch conditions (all 4.0B parameters actively optimized in native bfloat16, activation checkpointing disabled) on an NVIDIA A100 80GB GPU.

Optimizer Mode	Kernel Passes	Buffer Mode	Peak VRAM	Static State VRAM	Pure Opt Latency	Final Loss (250)	Weight Cosine Sim (vs 2P)
Master-Free (1P)	1-Pass (EMA)	`buffer=2` (Dual)	34.32 GB (-47.6%)	24.65 GB (-55.8%)	59.9 ms (1.90x faster)	8.8260	0.9445
Master-Free (2P)	2-Pass (Exact)	`buffer=2` (Dual)	34.32 GB (-47.6%)	24.65 GB (-55.8%)	68.4 ms	9.1075	1.0000 (Exact)
Semi (1P)	1-Pass (EMA)	`buffer=2` (Dual)	49.99 GB (-23.6%)	40.32 GB (-27.7%)	70.2 ms (1.62x faster)	8.8192 (Best)	0.9512
Full (2P - Baseline)	2-Pass (Exact)	`buffer=2` (Dual)	65.47 GB	55.80 GB	113.8 ms	8.9579	1.0000 (Exact)

Citation

@misc{luminamoon2026luminav_paper,
  author       = {{Silver Moon (cloverxion)}},
  organization = {Lumina Moon (cloverx-id)},
  title        = {{LuminaV: We Were Too Broke for AdamW So We Trapped Gradients in a Hyperbolic Straitjacket and Hired a Traffic Cop to Slap Them}},
  year         = {2026},
  publisher    = {Hugging Face},
  doi          = {10.57967/hf/10270},
  url          = {https://huggingface.co/cloverx-id/LuminaV-Optimizer-Paper}
}

@software{luminamoon2026luminav_code,
  author       = {{Silver Moon (cloverxion) and Lumina Moon Contributors}},
  organization = {Lumina Moon (cloverx-id)},
  title        = {{LuminaMoon & LuminaV Optimizer: Native Rust SIMD, Vulkan, and Triton Deep Learning Suite}},
  year         = {2026},
  publisher    = {GitHub / Hugging Face / PyPI},
  version      = {1.5.3},
  doi          = {10.57967/hf/10365},
  url          = {https://github.com/cloverx-id/luminamoon}
}

Downloads last month: 319

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

LuminaV Optimizer v1.5.3 (luminamoon v1.0.6)

Foundational Research Paper & Living Specification

PROJECT LOOMY

System Architecture

Projects & Components

Quickstart

Installation

Basic Usage (LuminaV Optimizer)

LuminaV Hyperparameter Quick Reference

Software vs. Algorithm Versioning (v1.0.6.post1 / Spec v1.5.3)

Basic Usage (LuminaMoon Data Engine: Fused Preprocessing & Zero-Copy Pipeline)

Verification & CI/CD

Developer Resources

Authors & Maintenance

License

LuminaV Optimizer v1.5.3

Features

Architecture & Dispatch

Workspace Crate Structure

Installation & Build

From Wheel (PyPI)

From Source (Maturin)

Optional Dependencies

Quickstart

Standalone NumPy (No PyTorch Required)

PyTorch Integration

Hardware Verification

Documentation & Library Audits

Running Tests

License

Empirical Pretraining Benchmarks (Qwen3.5-4B-Base)

Citation

LuminaV Optimizer v1.5.3 (`luminamoon v1.0.6`)

Software vs. Algorithm Versioning (`v1.0.6.post1` / Spec `v1.5.3`)

Empirical Pretraining Benchmarks (`Qwen3.5-4B-Base`)