| # NOVA-1 model card |
|
|
| | Field | Value | |
| |---|---| |
| | **Model name** | NOVA-1 | |
| | **Developer** | NOVA AI | |
| | **Version** | 1.0 | |
| | **Model type** | Decoder-only causal language model, LoRA fine-tune of an instruction-tuned base | |
| | **Base model** | [`Qwen/Qwen2.5-0.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) (© 2024 Alibaba Cloud, Apache 2.0) | |
| | **Optional base** | `Qwen/Qwen2.5-1.5B-Instruct` for the larger variant | |
| | **Languages** | Hindi (Devanagari), English, Hinglish (romanised Hindi + English) | |
| | **Parameters** | ~0.49B base + LoRA adapter (rank 16) | |
| | **Context length** | 32,768 tokens for the base; training used 512–768-token sequences | |
| | **Licence** | Apache 2.0 for code and dataset; base weights Apache 2.0; see `LICENSE`, `NOTICE` | |
| | **Repository** | this Git repository | |
|
|
| ## Architecture (inherited from the base) |
|
|
| Qwen2.5-0.5B-Instruct is a transformer with 24 layers, grouped-query attention |
| (14 query heads, 2 key/value heads), RoPE positional encoding, SwiGLU MLP, |
| RMSNorm and tied input/output embeddings. NOVA-1 does not change the architecture; |
| it adds trainable low-rank matrices (LoRA) to the linear layers. |
|
|
| ## Training status — read this first |
|
|
| NOVA-1's **training pipeline is complete and verified**, but the adapter weights |
| are not produced inside this repository checkout. The environment used to build |
| it has no GPU and no access to huggingface.co, so downloading the 0.5B base |
| model or training it there was impossible. Nothing in this repo pretends |
| otherwise. |
|
|
| To create the actual v1.0 weights: |
|
|
| 1. Open `notebooks/NOVA1_Finetune_Kaggle_Colab.ipynb` on Kaggle or Colab. |
| 2. Enable a free T4 GPU and internet. |
| 3. Run all cells. The notebook trains the adapter, verifies the files, evaluates |
| it against the base model, and writes `reports/EVALUATION_REPORT.md`. |
| 4. Fill the "Evaluation" and "Training run" sections below with those measured |
| numbers (the notebook prints a template you can copy). |
|
|
| Until that run happens, the only measured numbers in this repository are the |
| **pipeline smoke test** on a 1.7M random model (`reports/smoke/`). They are not |
| model results and must never be quoted as NOVA-1 performance. |
|
|
| ## Intended use |
|
|
| NOVA-1 is intended as a small, inspectable assistant for: |
|
|
| - everyday English/Hindi/Hinglish conversation and study help; |
| - school-level explanations in science, technology and general knowledge; |
| - step-by-step arithmetic and simple word problems; |
| - simple logical reasoning; |
| - short programming explanations and Python snippets; |
| - summarising short texts and helping with creative writing; |
| - education and research into small multilingual fine-tunes. |
|
|
| ## Out of scope / not recommended |
|
|
| - medical, legal, financial or safety-critical advice; |
| - anything requiring up-to-date news or internet access (the model has none); |
| - high-stakes decisions, identity verification or surveillance; |
| - long-document reasoning, advanced mathematics, competitive programming; |
| - languages other than Hindi/English beyond what the base model offers; |
| - generating harmful, illegal or private content. |
|
|
| ## Training data |
|
|
| A self-authored dataset of **831 records** (711 train / 60 validation / 60 test), |
| JSONL, three languages, with grouped splits so translations never leak: |
|
|
| | Category | Records | |
| |---|---| |
| | Mathematics | 439 | |
| | Reasoning | 102 | |
| | Science | 63 | |
| | Programming | 57 | |
| | Conversation | 31 | |
| | General knowledge | 30 | |
| | English grammar | 21 | |
| | Technology | 20 | |
| | History | 18 | |
| | Identity | 16 | |
| | Safety / refusal | 16 | |
| | Geography | 12 | |
| | Health | 6 | |
|
|
| Six records are multi-turn conversations (a `history` field), so the model also |
| learns to answer follow-up questions. The upgrade added 112 curated records: |
| history, geography, English grammar, refusal behaviour and programs in Java, C, |
| C++, Kotlin, SQL, JavaScript, HTML and CSS. |
| | Health | 6 | |
|
|
| Language mix: English 286, Hindi 251, Hinglish 182. Maths answers are computed |
| by Python; code samples were written for this project; no copyrighted books, |
| scraped websites or third-party instruction datasets were used. Full details and |
| known gaps: [`DATASET_GUIDE.md`](DATASET_GUIDE.md). |
|
|
| ## Training procedure |
|
|
| | Item | Value | |
| |---|---| |
| | Objective | supervised fine-tuning, next-token prediction | |
| | Loss masking | prompt + system tokens masked (`-100`), loss on assistant answer + EOS | |
| | Method | LoRA (optionally QLoRA with 4-bit NF4) | |
| | LoRA rank / alpha / dropout | 16 / 32 / 0.05 | |
| | Target modules | `q, k, v, o, gate, up, down` projections | |
| | Epochs | 3 | |
| | Learning rate | 2e-4 (0.5B) / 1.5e-4 (1.5B) | |
| | Batch × accumulation | 4 × 4 on T4 (0.5B); 1 × 16 with 4-bit (1.5B) | |
| | Sequence length | 512–768 | |
| | Precision | bf16/fp16 on GPU, fp32 on CPU | |
| | Seed | 42 | |
| | Trainable parameters | reported by `run_info.json` (typically ~1–3% of the model) | |
| | Hardware | free Kaggle/Colab T4 (16 GB) | |
|
|
| Reproduce with `python3 src/nova/train/train_lora.py --config configs/train_lora_0_5b.yaml` |
| or the notebook. Exact versions, dataset hashes and timings are stored in |
| `run_info.json` next to the adapter. |
|
|
| ## Evaluation |
|
|
| Protocol (implemented in `src/nova/eval/evaluate.py`): |
|
|
| - validation cross-entropy on assistant tokens and perplexity; |
| - 50–60 held-out prompts stratified by category and language; |
| - final-number exact match for numeric items; |
| - script/language consistency of the reply; |
| - unigram token-F1 against the reference (rough overlap, not correctness); |
| - empty-response rate, length, distinct-2, latency; |
| - raw predictions saved so every claim can be checked. |
|
|
| ### Results |
|
|
| **Real NOVA-1 results: not yet measured.** Run the notebook and paste its report |
| here. The report is generated automatically at `reports/EVALUATION_REPORT.md`. |
|
|
| **Pipeline smoke test (not a model result):** a 1.7M-parameter randomly |
| initialised Qwen2 model, 40 CPU steps, same code path: |
|
|
| | Metric | Base tiny model | After 40 steps | |
| |---|---|---| |
| | Validation loss | 7.6586 | 7.2681 | |
| | Perplexity | 2118.87 | 1433.78 | |
| | Language consistency | 0% | 0% | |
| | Numeric exact match | 0% | 0% | |
|
|
| Random models cannot follow instructions, so quality metrics stay at zero. The |
| numbers only prove that training, saving, reloading and measurement work. |
|
|
| ### Known evaluation weaknesses |
|
|
| - The test set is small and self-built; it is not MMLU/GSM8K/Indic benchmarks. |
| - Token-F1 rewards copying words and punishes valid paraphrases. |
| - Numeric exact match ignores whether the reasoning was correct. |
| - No human preference study has been done. |
| - Base-model scores on the same split are required for a fair comparison; the |
| notebook produces both. |
|
|
| ## Limitations |
|
|
| - **Small capacity.** 0.5B parameters cannot store broad factual knowledge or |
| perform reliable multi-step reasoning. Expect confident mistakes. |
| - **Maths style bias.** 60% of the data is arithmetic, so the model may answer |
| non-maths questions with unnecessary steps. |
| - **Hindi/Hinglish quality** depends on the base model and the limited dataset; |
| spelling and dialect variation are not covered. |
| - **No browsing, no tools, no persistent memory.** It sees only the current |
| conversation passed by the application. |
| - **Knowledge cutoff** is inherited from the base model and is not extended by |
| fine-tuning. |
| - **Prompt sensitivity.** Small models drift when the system prompt or chat |
| template changes; keep the NOVA-1 template used in training. |
| - **Safety is not guaranteed.** Refusal examples exist, but a 0.5B model can be |
| talked into unsafe or wrong outputs. Add application-level moderation for |
| public deployments. |
|
|
| ## Bias, safety and privacy |
|
|
| The dataset is small and India-centric in its general-knowledge examples; it |
| does not represent global perspectives. Health answers are general information, |
| not medical advice. No personal data or user conversations were used. The model |
| can reproduce biases present in the base model's pretraining data. For public |
| use, add content filtering, abuse logging (with consent), rate limits and a clear |
| disclaimer that answers can be wrong. |
|
|
| ## Licence and attribution |
|
|
| - Base model: Qwen2.5-0.5B-Instruct, © 2024 Alibaba Cloud, Apache 2.0. |
| - Fine-tuning, dataset, code and documentation: © 2026 NOVA AI, Apache 2.0. |
| - Modifications are listed in [`NOTICE`](NOTICE). |
| - NOVA AI is not affiliated with or endorsed by Alibaba Cloud or the Qwen team. |
| - Redistribution must include `LICENSE`, `NOTICE` and this model card. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{nova1_2026, |
| title = {NOVA-1: a small Hindi-English-Hinglish instruction-tuned language model}, |
| author = {NOVA AI}, |
| year = {2026}, |
| howpublished = {Fine-tune of Qwen2.5-0.5B-Instruct}, |
| note = {Base model by the Qwen team, Alibaba Cloud, Apache 2.0} |
| } |
| ``` |
|
|
| ## Contact |
|
|
| Open an issue in this repository for questions, bug reports or dataset |
| corrections. |
|
|
