Newton Foundation Family
Collection
Models we trained from scratch on Apple Silicon. The in-house foundation tier. • 3 items • Updated
AXE Newton Core (1.2B, preview)
A frontier-track foundation model, trained from scratch by AXE AI on Apple Silicon. Own tokenizer, own architecture, own weights. No base model anywhere in its lineage.
Most "owned" models are fine-tunes of someone else's open weights, and a 90-second forensic check gives them away at the tokenizer. This one is different, and the same check proves it: a 32,768-vocab ByteLevel BPE with zero Qwen, ChatML, or Alibaba tokens, paired with an architecture that matches no published checkpoint. A model with its own tokenizer and its own architecture has no base to trace back to. That is what from scratch actually means, and it is the line almost no independent lab can honestly draw.
Architecture
Modern decoder, built clean:
- 1.19B parameters. 24 layers, 2048 hidden, 16 attention heads, 2 KV heads (GQA).
- RoPE positional encoding, SwiGLU activations.
- Own 32,768 ByteLevel BPE tokenizer.
- Trained from random initialization with MLX on Apple Silicon, on AXE's own corpus.
Honest status
This is a preview checkpoint, step 112k of a planned 500k, best validation loss 2.93. It is roughly a fifth of the way through training. It is genuinely ours and deliberately early. Do not expect finished-model quality yet. We publish the checkpoint and its real numbers, and the quality climbs as training completes and as we distill from our larger teacher. Every gain ships with the eval that earned it.
Why this exists
AXE AI runs two model tracks. One is a strong, honestly-branded derivative that serves production today. This is the other one: a foundation model owned base to top, owing nothing to anyone. The model with no asterisk. We would rather publish a real from-scratch preview, honestly, than a polished derivative dressed up as something it is not. The honesty is the point, and it is the moat.
License
Apache-2.0. From-scratch weights, nothing inherited.
AXE Technologies. The Newton family: foundation models (Newton) and specialist operators (AXe Operators).
Model size
1B params
Tensor type
F32
·
Hardware compatibility
Log In to add your hardware
Quantized