Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,58 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
license: apache-2.0
|
| 5 |
+
tags:
|
| 6 |
+
- causal-lm
|
| 7 |
+
- christian
|
| 8 |
+
- pytorch
|
| 9 |
+
- sentencepiece
|
| 10 |
+
- reasoning
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# North Star 1
|
| 14 |
+
|
| 15 |
+
**North Star 1** is a balanced, knowledgeable language model (198M parameters, 24 layers) built from scratch with a Christian worldview. It gives thorough, well-reasoned answers across theology, science, coding, math, history, and general knowledge.
|
| 16 |
+
|
| 17 |
+
> *"Your word is a lamp to my feet and a light to my path."* — Psalm 119:105
|
| 18 |
+
|
| 19 |
+
## About
|
| 20 |
+
|
| 21 |
+
North Star 1 is the flagship balanced model in the North Star family. It was grown from a smaller 16-layer base by layer duplication and then fine-tuned with supervised instruction data. It reasons carefully and answers directly.
|
| 22 |
+
|
| 23 |
+
This model was trained entirely from scratch — no foundation model was used. The tokenizer is SentencePiece (32k vocab).
|
| 24 |
+
|
| 25 |
+
## Christian Worldview
|
| 26 |
+
|
| 27 |
+
These models are built on the conviction that Jesus Christ is Lord, that Scripture is the inspired and authoritative Word of God, and that faith and reason illuminate each other. We believe AI should serve truth and people — not replace human relationship with God.
|
| 28 |
+
|
| 29 |
+
> *"Jesus said to him, 'I am the way, and the truth, and the life. No one comes to the Father except through me.'"* — John 14:6
|
| 30 |
+
|
| 31 |
+
The Gospel is the good news that Jesus Christ died for our sins, was buried, and rose again on the third day. All who repent and trust in Him receive forgiveness and eternal life as a free gift — not by works, but by grace through faith (Ephesians 2:8-9).
|
| 32 |
+
|
| 33 |
+
If you do not know Jesus, we invite you to read the Gospel of John and ask God to reveal Himself to you.
|
| 34 |
+
|
| 35 |
+
## Model Details
|
| 36 |
+
|
| 37 |
+
| Property | Value |
|
| 38 |
+
|---|---|
|
| 39 |
+
| Parameters | 198M |
|
| 40 |
+
| Layers | 24 |
|
| 41 |
+
| d_model | 768 |
|
| 42 |
+
| Heads | 12 (3 KV) |
|
| 43 |
+
| Context | 512 tokens |
|
| 44 |
+
| Vocabulary | 32,000 (SentencePiece) |
|
| 45 |
+
| Architecture | GQA Transformer + SwiGLU + RoPE + RMSNorm |
|
| 46 |
+
|
| 47 |
+
## Usage
|
| 48 |
+
|
| 49 |
+
```python
|
| 50 |
+
import torch
|
| 51 |
+
ckpt = torch.load("northstar1.pt", map_location="cpu", weights_only=False)
|
| 52 |
+
cfg = ckpt["cfg"]
|
| 53 |
+
# Build model from cfg, load ckpt["model"] state dict
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
## Tokenizer
|
| 57 |
+
|
| 58 |
+
Uses SentencePiece (`tokenizer.model`). Available in the [shared tokenizer repo](https://huggingface.co/arthu1/north-tokenizer).
|