Popbones/LiveMem-RL-mlx-q8-backbone

LiveMem-RL MLX — Q8 backbone

This is a derived experimental Q8 backbone weight-only quantization of the official chen-l/LiveMem-RL checkpoint, intended for testing with the standalone MLX runtime. It is not an official upstream release.

Experimental artifact for testing. Quantized quality is not approved or certified by this publication. Compatibility checks alone do not establish retained memory quality, production readiness or latency targets.

Source and identity

• Original model: chen-l/LiveMem-RL.
• Pinned original revision: 3402dcace8766f9f4e0c8f80928c1795cc073f39.
• Original BF16 weight SHA-256: 1a431577034b0ce43535d20fde78df765a19b992ad0130cdd7826e63fe5f13f0.
• Base-model lineage: Qwen3-4B-Instruct-2507.
• Runtime: LiveMem MLX v0.1.0.
• This repository weight SHA-256: c1a564c1e4ce7220a99caa836583812eff34319d178814d19bc77e69d6091e39.

Precision

Only attention q/k/v/o and MLP gate/up/down linear weights use 8-bit affine quantization, group size 64. Embeddings, memory-branch weights, convolution, normalization and gates remain BF16; recurrent matrices remain FP32. This is not an all-8-bit model.

The underlying model has approximately 6.35 billion parameters, including the memory branch. The weight file is 9.30 GB. Weight storage is not total runtime memory.

Run locally on Apple Silicon

Use Python 3.11 or later and an Apple Silicon Mac with Metal support.

python -m pip install "git+https://github.com/popbones/livemem-mlx.git@v0.1.0"
livemem-mlx generate --model Popbones/LiveMem-RL-mlx-q8-backbone --prompt "Hello" --max-tokens 64

from livemem_mlx import load, generate

loaded = load("Popbones/LiveMem-RL-mlx-q8-backbone")
result = generate(loaded, [{"role": "user", "content": "Hello"}], max_tokens=64)
print(result.text)

After download, pin the repository commit with --revision or load(..., revision=...) for reproducibility. Default loading follows this repository main branch. Local paths, offline loading and --cache-dir are also supported.

These files require livemem_mlx.load; this custom architecture is not registered as a built-in Transformers AutoModel or mlx-lm model. No remote Python implementation is included or executed. Hub hosting does not provide a hosted inference endpoint.

Validation and limits

The corresponding checkpoint passed four standalone compatibility cases against its pre-extraction MLX runtime: Aurora, Zephyr, fresh-session Aurora replay, and a 2,283-token prompt with physical KV eviction followed by a simple arithmetic query. Generated token IDs, retained KV positions and saved prefill logits matched exactly within this precision variant.

Peak MLX allocation was 9.33 GiB on an Apple M1 Pro / 32 GiB for that 2,048-token KV-budget test. It is not an 8K/32K serving benchmark, total system RAM guarantee or broad quality evaluation. Full-model CUDA/vLLM parity, general long-term-memory quality and long-running stability are not established.

See the compatibility report. This publication does not add a new benchmark result. Each fresh generation starts with separate state; state is not automatically retained across independent requests.

License and attribution

The model distribution retains the upstream Apache-2.0 license. See LICENSE and NOTICE. The weights were modified by the quantization described above; the publication step preserved those quantized weight bytes. Source, precision settings, validated runtime and file hashes are recorded in manifest.json. Transformer auto_map metadata was removed to avoid advertising absent remote Python files. The original model and tokenizer assets remain credited to their authors.

LiveMem: Maintaining Memory State Continuity in Long-Running LLM Inference — Zhichen Liu, Ruihan Sun, Hengjie Yang, Zipeng Wu, Zhaohan Chen, Xiaofan Zhang and Yang Xu (2026), arXiv:2608.02515.

This independent mirror/conversion is not endorsed by the LiveMem authors, Qwen or Alibaba.