---
language:
- en
- hi
tags:
- llama-cpp
- gguf
- qwen
- text-generation
- fine-tuned
- instruction-following
- rag
- AI-assistant
license: apache-2.0
datasets:
- custom-nexai
pipeline_tag: text-generation
model_creator: Anoopsingh53
model_type: qwen2
---

# 🚀 NexAI-v1 (7B Instruct - Q4_K_M GGUF)

Official repository of **NexAI-v1**, an enterprise-grade instruction-following model fine-tuned using QLoRA (SFT) on Alibaba's robust **Qwen2.5-7B-Instruct** architecture, and quantized into high-performance **Q4_K_M GGUF** format.

NexAI-v1 is engineered specifically for fast, localized CPU inference, retrieval-augmented generation (RAG) tasks, and secure, privacy-preserving local assistant applications.

---

## 📊 Model Summary

| Attribute | Details |
| :--- | :--- |
| **Developed by** | Anoop Singh |
| **Model Type** | Causal Language Model (Transformer-based) |
| **Base Architecture** | Qwen2.5 (Dense) |
| **Parameter Count** | 7.61 Billion |
| **Quantization Format** | GGUF (Q4_K_M - 4.91 Bits Per Weight) |
| **File Size** | 4.46 GB |
| **Supported Languages** | English, Hindi (Hinglish supported natively) |
| **Context Length** | 32,768 Tokens |
| **License** | Apache 2.0 |

---

## 🌟 Key Capabilities & Features

* 🧠 **Instruction Following:** Fine-tuned to respond precisely to system prompts, system instructions, and multi-turn chat templates.
* 🔍 **RAG Optimized:** Enhanced comprehension for feeding external context documents (PDFs, text, web scrapes) without losing tracking accuracy.
* 🤖 **Stealth & Safety:** Integrated system prompts designed to prevent bot detection and bypass standard security filter constraints.
* ⚡ **Ultra-low Resource Execution:** Fits comfortably within 8GB System RAM, executing at high token-generation speeds on consumer-grade laptops.

---

## 🛠️ Installation & Quickstart

### 1. Python Integration (via `llama-cpp-python`)
For programmatic use cases, install the Python binding:
```bash
pip install llama-cpp-python
```

Initialize and query the model directly from Hugging Face:
```python
from llama_cpp import Llama

# Download and load the model locally
llm = Llama.from_pretrained(
    repo_id="Anoopsingh53/nexai-v1",
    filename="nexai-v1-Q4_K_M.gguf",
    n_ctx=2048,
    n_threads=4
)

# ChatML Format Prompt
prompt = '<|im_start|>system\n' \
         'You are NexAI, a helpful, intelligent assistant.\n' \
         '<|im_end|>\n' \
         '<|im_start|>user\n' \
         'Write a python function to check if a number is prime.\n' \
         '<|im_end|>\n' \
         '<|im_start|>assistant\n'

response = llm(
    prompt,
    max_tokens=256,
    stop=["<|im_end|>"],
    echo=False
)

print(response['choices'][0]['text'])
```

### 2. Desktop deployment (via `llama.cpp` CLI)
Download the `.gguf` file manually and run:
```bash
./llama-cli \
  -m nexai-v1-Q4_K_M.gguf \
  -p "<|im_start|>system\nYou are NexAI, a helpful assistant.<|im_end|>\n<|im_start|>user\nHello!\n<|im_end|><|im_start|>assistant\n" \
  -n 128 \
  -c 2048
```

---

## ⚙️ Training Details (QLoRA)

NexAI-v1 was trained using parameter-efficient fine-tuning (PEFT) on Google Colab hardware.

* **Optimizer:** AdamW (8-bit)
* **Learning Rate:** 2e-4
* **Precision:** mixed 16-bit / 4-bit SFT
* **LoRA Target Modules:** `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
* **LoRA Rank (R):** 64 | **LoRA Alpha:** 16
* **Training Epochs:** 1 Epoch over specialized multi-turn data

---

## ⚠️ Intended Use & Limitations

* **Intended Use:** Personal assistance, local coding assistant, offline document RAG, and educational demonstrations.
* **Limitations:** Like all language models, NexAI-v1 may occasionally hallucinate or generate inaccurate facts. It is not intended for mission-critical medical, financial, or legal advice without human-in-the-loop validation.
* **Biases:** The model's outputs are heavily influenced by the pre-training data and instructions. Ensure proper filtering if deploying in customer-facing public products.

---

## 📄 License & Terms
This model is released under the **Apache 2.0 License**. Qwen2.5 base weights are governed by Alibaba's original terms of use.