README.md

metadata

license: apache-2.0
base_model: Nanbeige/Nanbeige4.2-3B
base_model_relation: quantized
pipeline_tag: text-generation
library_name: coreai
tags:
  - coreai
  - apple-silicon
  - recurrent-transformer
  - text-generation
  - quantized
  - int8

Nanbeige4.2-3B — Core AI INT8

INT8 weight-quantized Core AI LanguageBundle converted from Nanbeige/Nanbeige4.2-3B, pinned to source revision 5ff54fb7ed86ce8e216d78bff5417ab9981de3d4.

This is a Core AI .aimodel bundle for Apple silicon. It is not a Transformers, GGUF, or MLX checkpoint.

Ported by Vadim Smirnov (@ukint-vs). Source, reproducible conversion recipe, parity gates, and the optimization report are tracked in john-rocky/coreai-model-zoo#5.

Format

Property	Value
Published variant	INT8 only
Body weights	int8, per-block 32 (int8hu)
Untied LM head	symmetric absmax int8, per-block 32 (--head-sym)
Activations / KV cache	fp16
Decode input	static sequence length 1 (--static-ids)
Bundle size	approximately 4.59 GiB
Verified context	4,096 tokens

The int4 and mixed int4/int8 candidates failed the reasoning quality gate and are intentionally not published.

Architecture

Nanbeige4.2 is a looped Llama: 22 physical transformer blocks execute twice, with RMS normalization after each pass and independent KV history. The bundle therefore has 44 executed/cache layers while storing and quantizing one copy of the 22 physical blocks.

Bundle

The complete LanguageBundle is at:

gpu-pipelined/
  nanbeige4_2_3b_decode_int8hu_block32_sym_s1/
    metadata.json
    nanbeige4_2_3b_decode_int8hu_block32_sym_s1.aimodel/
    tokenizer/

The embedded tokenizer includes the vendor chat_template.jinja with thinking enabled and disabled modes. The .aimodel producer is coreai-core 1.0.0b2.

Verification

• Official checkpoint vs Core AI authoring: float32 full and cached logits pass rtol=1e-4, atol=1e-4; identical 32-token greedy continuation.
• Quantized Core AI engine: token-exact vs fp32 on 24-token Paris, 16-token freezing-point, and 64-token decimal-reasoning gates.
• M4 Max Release benchmark, prompt 128 / generation 256 / 3 runs: 47.37 prefill / 46.35 decode tok/s.
• 4,096-token boundary: 29.83 prefill / 32.80 decode tok/s, 9.17 GiB peak RSS, zero swaps.
• iPhone h18p device acceptance is pending; no iPhone performance claim is made.

License

The source model and converted weights are distributed under Apache-2.0. Conversion code in coreai-model-zoo is BSD-3-Clause.