---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
tags:
- pkai
- text-generation
- base-model
- safetensors
---

# PKaI Nano 1

PKaI Nano 1 is a 120M-class base language model from PowderKeg
Intelligence. It is a compact LLaMA-style decoder model trained from
scratch with a Mistral tokenizer and released as a PKaI-native artifact.

This repository contains PKaI-native weights and metadata, not a drop-in
`transformers.AutoModelForCausalLM` package.

## Files

- `model.safetensors`: PKaI base model weights.
- `config.json`: PKaI model architecture configuration.
- `tokenizer.json`: PKaI tokenizer metadata.
- `tokenizer.model`: SentencePiece tokenizer model from `mistralai/Mistral-7B-v0.1`.
- `THIRD_PARTY_NOTICES.txt`: tokenizer and training-data provenance notices.
- `LICENSE`: Apache License, Version 2.0.

## Architecture

- Parameters: 112,680,448
- Vocabulary size: 32,000
- Context length: 512
- Layers: 14
- Attention heads: 12
- KV heads: 4
- Embedding size: 768
- Tied embeddings: yes
- QK normalization: yes

## Training Data

Publicly disclosed training data sources include:

- `HuggingFaceFW/fineweb-edu`, released under the Open Data Commons Attribution
  License (ODC-By) v1.0 and subject to Common Crawl Terms of Use as noted by the
  dataset card.
- `HuggingFaceTB/cosmopedia`, released under the Apache License, Version 2.0.

See `THIRD_PARTY_NOTICES.txt` for source URLs, citations, and attribution notes.

## License

Copyright 2026 PowderKeg Intelligence LLC.

The PKaI Nano 1 model artifact is released under the Apache License, Version
2.0. The bundled tokenizer and training-data sources have their own provenance
and notices listed in `THIRD_PARTY_NOTICES.txt`.

## Limitations

This is a small base model and has not been instruction-tuned or safety-tuned.
It may produce inaccurate, unsafe, biased, or otherwise unsuitable text. Users
are responsible for evaluating fitness, safety, and legal compliance for their
own use cases.