| --- |
| license: apache-2.0 |
| base_model: Nanbeige/Nanbeige4.2-3B |
| base_model_relation: quantized |
| pipeline_tag: text-generation |
| library_name: coreai |
| tags: |
| - coreai |
| - apple-silicon |
| - recurrent-transformer |
| - text-generation |
| - quantized |
| - int8 |
| --- |
| |
| # Nanbeige4.2-3B — Core AI INT8 |
|
|
| **INT8 weight-quantized Core AI LanguageBundle** converted from [`Nanbeige/Nanbeige4.2-3B`](https://huggingface.co/Nanbeige/Nanbeige4.2-3B), pinned to source revision [`5ff54fb7ed86ce8e216d78bff5417ab9981de3d4`](https://huggingface.co/Nanbeige/Nanbeige4.2-3B/tree/5ff54fb7ed86ce8e216d78bff5417ab9981de3d4). |
|
|
| This is a Core AI `.aimodel` bundle for Apple silicon. It is not a Transformers, GGUF, or MLX checkpoint. |
|
|
| Ported by [Vadim Smirnov (@ukint-vs)](https://github.com/ukint-vs). Source, reproducible conversion recipe, parity gates, and the optimization report are tracked in [john-rocky/coreai-model-zoo#5](https://github.com/john-rocky/coreai-model-zoo/issues/5). |
|
|
| ## Format |
|
|
| | Property | Value | |
| |---|---| |
| | Published variant | **INT8 only** | |
| | Body weights | int8, per-block 32 (`int8hu`) | |
| | Untied LM head | symmetric absmax int8, per-block 32 (`--head-sym`) | |
| | Activations / KV cache | fp16 | |
| | Decode input | static sequence length 1 (`--static-ids`) | |
| | Bundle size | approximately 4.59 GiB | |
| | Verified context | 4,096 tokens | |
|
|
| The int4 and mixed int4/int8 candidates failed the reasoning quality gate and are intentionally not published. |
|
|
| ## Architecture |
|
|
| Nanbeige4.2 is a looped Llama: **22 physical transformer blocks execute twice**, with RMS normalization after each pass and independent KV history. The bundle therefore has **44 executed/cache layers while storing and quantizing one copy of the 22 physical blocks**. |
|
|
| ## Bundle |
|
|
| The complete LanguageBundle is at: |
|
|
| ```text |
| gpu-pipelined/ |
| nanbeige4_2_3b_decode_int8hu_block32_sym_s1/ |
| metadata.json |
| nanbeige4_2_3b_decode_int8hu_block32_sym_s1.aimodel/ |
| tokenizer/ |
| ``` |
|
|
| The embedded tokenizer includes the vendor `chat_template.jinja` with thinking enabled and disabled modes. The `.aimodel` producer is `coreai-core 1.0.0b2`. |
|
|
| ## Verification |
|
|
| - Official checkpoint vs Core AI authoring: float32 full and cached logits pass `rtol=1e-4`, `atol=1e-4`; identical 32-token greedy continuation. |
| - Quantized Core AI engine: token-exact vs fp32 on 24-token Paris, 16-token freezing-point, and 64-token decimal-reasoning gates. |
| - M4 Max Release benchmark, prompt 128 / generation 256 / 3 runs: **47.37 prefill / 46.35 decode tok/s**. |
| - 4,096-token boundary: **29.83 prefill / 32.80 decode tok/s**, 9.17 GiB peak RSS, zero swaps. |
| - iPhone `h18p` device acceptance is pending; no iPhone performance claim is made. |
|
|
| ## License |
|
|
| The source model and converted weights are distributed under Apache-2.0. Conversion code in `coreai-model-zoo` is BSD-3-Clause. |
|
|
