ThoxMicro-1bit-16M

From-Scratch THOX Ternary Model — a THOX model trained from scratch: a BitNet b1.58 ternary decoder (16M parameters, 86.6% of weights ternary) trained on TinyStories

GGUF: Q8_0, TQ2_0 · Ollama Modelfile

Find your model: THOX Model Factory · Collection: Research & From-Scratch Models

Your AI. Your Data. Your Rules.™

Summary

ThoxMicro-1bit-16M is a THOX model trained from scratch: a BitNet b1.58 ternary decoder (16M parameters, 86.6% of weights ternary) trained on TinyStories. It is the deeper sibling of ThoxMicro-1bit-9M, an edge research model, and the base for the THOX 16M device-role models. Neither the 9M nor the 16M supersedes the other.

Highlights

• ThoxMicro-1bit-16M is a THOX model trained from scratch: a BitNet b1.58 ternary decoder (16M parameters, 86.6% of weights ternary) trained on TinyStories.
• bash hf download Thox-ai/ThoxMicro-1bit-16M --include '*.gguf' --local-dir ./ThoxMicro-1bit-16M llama-cli -m ./ThoxMicro-1bit-16M/thoxmicro-1bit-16m.TQ2_0.gguf -p "Hello" Ternary files need a recent llama.cpp build; Ollama and LM Studio load only the Q8_0 file.

Specifications

Base model	See card body
Parameters	See card body
Formats	GGUF: Q8_0, TQ2_0 · Ollama Modelfile
License	Apache-2.0
This repository contains	GGUF build(s) in this repo: thoxmicro-1bit-16m.Q8_0.gguf, thoxmicro-1bit-16m.TQ2_0.gguf

Files

File	Size	Purpose
thoxmicro-1bit-16m.Q8_0.gguf	17,033,504 B	GGUF model build
thoxmicro-1bit-16m.TQ2_0.gguf	8,030,496 B	GGUF model build
.gitattributes	1,650 B	Repository file
Modelfile	806 B	Ollama model definition

Architecture

Field	Value
Architecture	BitNet b1.58 ternary decoder
Layers	16
Hidden size	256
Attention heads	8
KV heads	8
FFN / intermediate	768
Vocab	8,192 (own byte-level BPE)
Max context	512
Tied embeddings	yes

Run it

hf download Thox-ai/ThoxMicro-1bit-16M --include '*.gguf' --local-dir ./ThoxMicro-1bit-16M
llama-cli -m ./ThoxMicro-1bit-16M/thoxmicro-1bit-16m.TQ2_0.gguf -p "Hello"

Ternary files need a recent llama.cpp build; Ollama and LM Studio load only the Q8_0 file.

Platforms

Platform / device	Run path
Hugging Face	Download the builds listed under Files from this repo (see Run it)

Intended use

• On-device and edge text generation within the THOX stack, and research on ternary-weight models.
• Not a safety-aligned public assistant unless deployed behind THOX guardrails.
• Out of scope: medical, legal, financial, safety-critical, or other high-stakes decisions without human review.

Training data and procedure

Trained by THOX from scratch as part of the THOX small-model research line. The corpus and procedure are as described in this card at release; no external base model was used.

Evaluation

No THOX GATE49 result is published for this model.

Limitations

• A 16M research model: story completion only, no instruction following or factual reliability.
• Behaviour and bias reflect the training data; quality outside short English story text is low.
• Quantization trades accuracy for size.

Provenance

Published by THOX.ai as Thox-ai/ThoxMicro-1bit-16M. THOX from-scratch research line; no upstream base model. This card was conformed to THOX Model Card Standard v1.0 on 2026-10-31 (frontmatter, tags, header, files, and section structure); the technical content is as published at release.

License

Apache License 2.0. This model was trained by THOX from scratch and is released under the Apache License 2.0. Training data note: the training corpus is TinyStories, distributed under CDLA-Sharing-1.0; that license governs the dataset itself.

THOX.ai — Your AI. Your Data. Your Rules.™

Citation

@misc{thoxai2026thoxmicro1bit16m,
      title={ThoxMicro-1bit-16M},
      author={THOX.ai},
      year={2026},
      url={https://huggingface.co/Thox-ai/ThoxMicro-1bit-16M}
}