Sage 1B

A custom 1.286 billion parameter language model built entirely from scratch — no base models, no fine-tuning, no dependencies on existing LLM frameworks.

Architecture

Key Features

• Built from scratch — Custom PyTorch implementation. Not a derivative of any existing model.
• BPE Tokenizer — Trained a 50,000-token BPE tokenizer on the TinyStories dataset.
• Modern Architecture — SwiGLU activations, Rotary Position Embeddings (RoPE), RMSNorm.
• Open Source — MIT license. Weights, training code, and inference code are all available.
• GGUF Format — Available for use with llama.cpp, Ollama, and other GGUF-compatible runners.

Usage

With Hugging Face Hub

from huggingface_hub import hf_hub_download
import torch, json
from tokenizers import Tokenizer

config_path = hf_hub_download('itriedcoding/Sage-1B', 'config.json')
tokenizer_path = hf_hub_download('itriedcoding/Sage-1B', 'tokenizer.json')
weights_path = hf_hub_download('itriedcoding/Sage-1B', 'pytorch_model_state.bin')

cfg = json.load(open(config_path))
tok = Tokenizer.from_file(tokenizer_path)

With GGUF (llama.cpp)

wget https://huggingface.co/itriedcoding/Sage-1B/resolve/main/sage-1b-f16.gguf
./main -m sage-1b-f16.gguf -p "Once upon a time" -n 50

Web Interface

Chat with the model at: https://sage-ai.vercel.app/chat

API

curl -X POST https://sage-ai.vercel.app/api/v1/chat \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{"message": "Tell me a story"}'

Training

The model was trained on the TinyStories dataset — a synthetic dataset of short stories designed for training compact language models. Training was performed on CPU with limited resources, making this a proof-of-concept for building LLMs from scratch without GPU access.

Files

License

MIT