| --- |
| license: apache-2.0 |
| language: |
| - en |
| pipeline_tag: text-generation |
| tags: |
| - pkai |
| - text-generation |
| - base-model |
| - safetensors |
| --- |
| |
| # PKaI Nano 1 |
|
|
| PKaI Nano 1 is a 120M-class base language model from PowderKeg |
| Intelligence. It is a compact LLaMA-style decoder model trained from |
| scratch with a Mistral tokenizer and released as a PKaI-native artifact. |
|
|
| This repository contains PKaI-native weights and metadata, not a drop-in |
| `transformers.AutoModelForCausalLM` package. |
|
|
| ## Files |
|
|
| - `model.safetensors`: PKaI base model weights. |
| - `config.json`: PKaI model architecture configuration. |
| - `tokenizer.json`: PKaI tokenizer metadata. |
| - `tokenizer.model`: SentencePiece tokenizer model from `mistralai/Mistral-7B-v0.1`. |
| - `THIRD_PARTY_NOTICES.txt`: tokenizer and training-data provenance notices. |
| - `LICENSE`: Apache License, Version 2.0. |
|
|
| ## Architecture |
|
|
| - Parameters: 112,680,448 |
| - Vocabulary size: 32,000 |
| - Context length: 512 |
| - Layers: 14 |
| - Attention heads: 12 |
| - KV heads: 4 |
| - Embedding size: 768 |
| - Tied embeddings: yes |
| - QK normalization: yes |
|
|
| ## Training Data |
|
|
| Publicly disclosed training data sources include: |
|
|
| - `HuggingFaceFW/fineweb-edu`, released under the Open Data Commons Attribution |
| License (ODC-By) v1.0 and subject to Common Crawl Terms of Use as noted by the |
| dataset card. |
| - `HuggingFaceTB/cosmopedia`, released under the Apache License, Version 2.0. |
|
|
| See `THIRD_PARTY_NOTICES.txt` for source URLs, citations, and attribution notes. |
|
|
| ## License |
|
|
| Copyright 2026 PowderKeg Intelligence LLC. |
|
|
| The PKaI Nano 1 model artifact is released under the Apache License, Version |
| 2.0. The bundled tokenizer and training-data sources have their own provenance |
| and notices listed in `THIRD_PARTY_NOTICES.txt`. |
|
|
| ## Limitations |
|
|
| This is a small base model and has not been instruction-tuned or safety-tuned. |
| It may produce inaccurate, unsafe, biased, or otherwise unsuitable text. Users |
| are responsible for evaluating fitness, safety, and legal compliance for their |
| own use cases. |
|
|
