| --- |
| library_name: transformers |
| license: apache-2.0 |
| pipeline_tag: text-generation |
| tags: |
| - custom-code |
| - nova |
| --- |
| |
| # Nova v3 |
|
|
| Nova v3 is a decoder-only causal language model with **148.8 million parameters**. This repository provides the Transformers checkpoint, tokenizer, and custom model code required to load the model. |
|
|
| ## Model specifications |
|
|
| | Specification | Value | |
| |---|---:| |
| | Parameters | 148.8M | |
| | Vocabulary | 32,768 tokens | |
| | Hidden size | 640 | |
| | Transformer layers | 26 | |
| | Attention | 10 query heads, 5 key/value heads | |
| | Maximum context | 2,048 tokens | |
| | Checkpoint step | 3,500 | |
|
|
| The model supports key/value caching during generation. Its custom implementation supports Transformers 4.57 and 5.x. Review the custom modeling files before loading with `trust_remote_code=True`. |
|
|
| ## Load with Transformers |
|
|
| Install PyTorch and Transformers: |
|
|
| ```bash |
| pip install torch transformers |
| ``` |
|
|
| Load the tokenizer and model: |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_id = "plasmova/nova-v3" |
| tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) |
| model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True) |
| |
| system_prompt = ( |
| "You are Nova, a helpful assistant. Answer the user's request directly and concisely. Do not invent facts." |
| ) |
| prompt = ( |
| f"<|system|>{system_prompt}<|/system|>" |
| "<|user|>Explain why the sky appears blue in one sentence.<|/user|>" |
| "<|assistant|>" |
| ) |
| inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False) |
| output = model.generate( |
| **inputs, |
| max_new_tokens=160, |
| do_sample=True, |
| temperature=0.2, |
| top_p=0.85, |
| top_k=12, |
| repetition_penalty=1.12, |
| no_repeat_ngram_size=3, |
| eos_token_id=[ |
| tokenizer.eos_token_id, |
| tokenizer.convert_tokens_to_ids("<|/assistant|>"), |
| ], |
| ) |
| print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)) |
| ``` |
|
|
| Recommended starting settings are temperature **0.2**, top-p **0.9**, top-k **40**, and repetition penalty **1.08**. These focused settings are a starting point; adjust them for your task. The local inference script applies this concise system instruction by default. Recommended generation defaults are temperature **0.2**, top-p **0.85**, top-k **12**, repetition penalty **1.12**, no-repeat n-gram size **3**, and up to **160 new tokens**. Keep the total prompt and generated text within the 2,048-token context. The example runs on CPU; for GPU inference, move the model and inputs to a supported device and dtype. |
|
|
| ## Evaluation |
|
|
| The following zero-shot results use the first 500 examples from each task with `lm-evaluation-harness` 0.4.13 on CPU. These are limited samples, not full validation-set results. |
|
|
| | Benchmark | Samples | Accuracy | Normalized accuracy | |
| |---|---:|---:|---:| |
| | HellaSwag | 500 | 37.4% | 43.6% | |
| | ARC Easy | 500 | 51.0% | 48.2% | |
| | ARC Challenge | 500 | 21.8% | 25.0% | |
| | WinoGrande | 500 | 52.6% | 52.2% | |
| | OpenBookQA | 500 | 18.8% | 30.0% | |
|
|
| ### Comparison with SmolLM2-135M |
|
|
| In a matched zero-shot run using 500 examples per task, Nova v3 scored 43.6% normalized accuracy on HellaSwag and 52.2% on WinoGrande, compared with 44.2% and 50.6% for SmolLM2-135M. The WinoGrande difference is small relative to sampling uncertainty; treat both results as early directional measurements, not established performance claims. The comparison details and per-run metrics are in [benchmark_comparison.json](benchmark_comparison.json). |
|
|
| On a separate 100-question GSM8K sample with five examples in context, Nova v3 and SmolLM2-135M-Instruct each answered 4 questions correctly (4.0% exact match). This small sample does not establish equivalence. Details are in [gsm8k_comparison.json](gsm8k_comparison.json). |
|
|
| ## Training and license |
|
|
| The v3 training code specifies pretraining on a mixture of FineWeb-Edu, DCLM, Cosmopedia-v2, FineMath, Wikipedia, and CodeParrot, followed by SmolTalk fine-tuning. Dataset contents remain subject to their respective upstream terms. |
|
|
| The model weights and accompanying custom model code are released under the Apache License 2.0. See [LICENSE](LICENSE). |
|
|
| ## llama.cpp and mobile apps |
|
|
| The validated F16 GGUF export is [`nova-v3-f16.gguf`](https://huggingface.co/plasmova/nova-v3/blob/main/nova-v3-f16.gguf) (about 341 MB). It uses the `qwen3` GGUF architecture metadata needed for Nova v3's per-head query/key normalization. It was checked with llama.cpp b8882, including the version pinned by Pocket AI Lab; all 289 tensors load and CPU generation completes. |
|
|
| In Pocket AI Lab, import `plasmova/nova-v3` and select `nova-v3-f16.gguf`. If you downloaded an earlier file, remove that local copy and download the current version. |
|
|
| Pocket AI Lab adds its own system instruction to every chat. An overly specific instruction can steer this small model toward an unrelated task. In the app's **System Prompt** setting, use the same short, general instruction shown in the Transformers example above: |
|
|
| ```text |
| You are Nova, a helpful assistant. Answer the user's request directly and concisely. Do not invent facts. |
| ``` |
|
|
| The GGUF chat template formats messages; it does not hardcode a greeting or a response. |
|
|
