File size: 2,804 Bytes
de31098 a55c228 de31098 a55c228 de31098 a55c228 de31098 a55c228 de31098 a55c228 de31098 a55c228 de31098 a55c228 de31098 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | ---
license: apache-2.0
base_model: Nanbeige/Nanbeige4.2-3B
base_model_relation: quantized
pipeline_tag: text-generation
library_name: coreai
tags:
- coreai
- apple-silicon
- recurrent-transformer
- text-generation
- quantized
- int8
---
# Nanbeige4.2-3B — Core AI INT8
**INT8 weight-quantized Core AI LanguageBundle** converted from [`Nanbeige/Nanbeige4.2-3B`](https://huggingface.co/Nanbeige/Nanbeige4.2-3B), pinned to source revision [`5ff54fb7ed86ce8e216d78bff5417ab9981de3d4`](https://huggingface.co/Nanbeige/Nanbeige4.2-3B/tree/5ff54fb7ed86ce8e216d78bff5417ab9981de3d4).
This is a Core AI `.aimodel` bundle for Apple silicon. It is not a Transformers, GGUF, or MLX checkpoint.
Ported by [Vadim Smirnov (@ukint-vs)](https://github.com/ukint-vs). Source, reproducible conversion recipe, parity gates, and the optimization report are tracked in [john-rocky/coreai-model-zoo#5](https://github.com/john-rocky/coreai-model-zoo/issues/5).
## Format
| Property | Value |
|---|---|
| Published variant | **INT8 only** |
| Body weights | int8, per-block 32 (`int8hu`) |
| Untied LM head | symmetric absmax int8, per-block 32 (`--head-sym`) |
| Activations / KV cache | fp16 |
| Decode input | static sequence length 1 (`--static-ids`) |
| Bundle size | approximately 4.59 GiB |
| Verified context | 4,096 tokens |
The int4 and mixed int4/int8 candidates failed the reasoning quality gate and are intentionally not published.
## Architecture
Nanbeige4.2 is a looped Llama: **22 physical transformer blocks execute twice**, with RMS normalization after each pass and independent KV history. The bundle therefore has **44 executed/cache layers while storing and quantizing one copy of the 22 physical blocks**.
## Bundle
The complete LanguageBundle is at:
```text
gpu-pipelined/
nanbeige4_2_3b_decode_int8hu_block32_sym_s1/
metadata.json
nanbeige4_2_3b_decode_int8hu_block32_sym_s1.aimodel/
tokenizer/
```
The embedded tokenizer includes the vendor `chat_template.jinja` with thinking enabled and disabled modes. The `.aimodel` producer is `coreai-core 1.0.0b2`.
## Verification
- Official checkpoint vs Core AI authoring: float32 full and cached logits pass `rtol=1e-4`, `atol=1e-4`; identical 32-token greedy continuation.
- Quantized Core AI engine: token-exact vs fp32 on 24-token Paris, 16-token freezing-point, and 64-token decimal-reasoning gates.
- M4 Max Release benchmark, prompt 128 / generation 256 / 3 runs: **47.37 prefill / 46.35 decode tok/s**.
- 4,096-token boundary: **29.83 prefill / 32.80 decode tok/s**, 9.17 GiB peak RSS, zero swaps.
- iPhone `h18p` device acceptance is pending; no iPhone performance claim is made.
## License
The source model and converted weights are distributed under Apache-2.0. Conversion code in `coreai-model-zoo` is BSD-3-Clause.
|