| --- |
| library_name: transformers |
| license: apache-2.0 |
| base_model: |
| - Neura-Tech-AI/Nexa-AI-4B-Instruct |
| - Qwen/Qwen3-4B-Instruct-2507 |
| - Qwen/Qwen3-4B-Thinking-2507 |
| pipeline_tag: text-generation |
| language: |
| - en |
| - hi |
| - zh |
| - sa |
| - kn |
| - ur |
| - mr |
| new_version: Neura-Tech-AI/Nexa-AI-4x4B-Instruct |
| tags: |
| - Neura Tech AI |
| - Lumina AI |
| - Nexa AI |
| - instruct |
| - llm |
| - transformer |
| - qwen |
| - mixture-of-experts |
| - moe |
| - multilingual |
| - 12B |
| - Nexa-AI-4x4B-Instruct |
| - safetensors |
| - conversational |
| --- |
| |
| # Nexa-AI-4x4B-Instruct |
|
|
| > A collaborative open-source large language model featuring an advanced Mixture of Experts (MoE) architecture, developed by **Neura Tech AI** and **Lumina AI**. |
|
|
| ## Overview |
|
|
| **Nexa-AI-4x4B-Instruct** is an instruction-tuned, state-of-the-art Mixture of Experts (MoE) model built using the framework and foundations of the **Qwen3** architecture family. Featuring a dedicated setup of **4 routing experts each scaled at 4B parameters**, it provides exceptional computation balancing and advanced contextual intelligence. |
|
|
| This project is jointly developed by: |
| - **Neura Tech AI** |
| - **Lumina AI** |
|
|
| Nexa AI focuses on delivering a capable multilingual AI assistant with strong performance in: |
| - High-efficiency compute routing (MoE) |
| - Complex logical reasoning & thinking loops |
| - Advanced coding assistance & software engineering tasks |
| - Mathematics & multi-step academic problem solving |
| - Tool calling, automation, & autonomous AI agents |
| - Deep multilingual understanding (including English, Hindi, Chinese, and more) |
|
|
| ## Model Architecture & Details |
|
|
| Unlike standard dense models, this variant leverages a modern sparse MoE setup that maps optimized parameter routing to specialized neural layers dynamically during runtime. |
|
|
| - **Model Name:** Nexa-AI-4x4B-Instruct |
| - **Base Architecture:** Transformer Decoder (Sparse Mixture of Experts) |
| - **Parameters:** ~16B Total Parameters |
| - **Total Experts:** 4 Specialists |
| - **Expert Size:** ~4 Billion parameters per expert |
| - **Active Parameters:** ~4B parameters routed per token (Dynamic Routing) |
| - **Context Length:** 262,144 Tokens |
| - **License:** Apache-2.0 |
|
|
| ## Developers |
|
|
| **Project:** Nexa AI |
| **Developed by:** |
| - Neura Tech AI |
| - Lumina AI |
|
|
| ## Base Model Acknowledgment |
| We sincerely thank the Qwen Team for releasing the foundational Qwen3 model family under the Apache 2.0 License, which served as the structural baseline for this unified architectural development. |
|
|
| ## Features |
|
|
| - **Sparse MoE Architecture:** Dynamic token routing through 4 separate specialized expert spaces for optimized intelligence per parameter. |
| - **Advanced Thinking & Alignment:** Highly tuned instruction-following wrapper optimized for structured responses. |
| - **High-Quality Code Generation:** Native compatibility with complex data tracking, script generations, and error handling. |
| - **Vast Multi-turn Context:** Massive 262k context space allows keeping track of long documentation or terminal outputs seamlessly. |
| - **Native Tool Integration:** Solid structure built for external API calls, JSON function routing, and systematic agentic pipelines. |
|
|
| ## Performance |
|
|
| | Benchmark | GPT-4.1-nano-2025-04-14 | Qwen3-30B-A3B Non-Thinking | Qwen3-4B Non-Thinking | Nexa-AI-4x4B-Instruct | |
| |--- | --- | --- | --- | --- | |
| | **Knowledge** | | | | |
| | MMLU-Pro | 62.8 | 69.1 | 58.0 | **69.6** | |
| | MMLU-Redux | 80.2 | 84.1 | 77.3 | **84.2** | |
| | GPQA | 50.3 | 54.8 | 41.7 | **62.0** | |
| | SuperGPQA | 32.2 | 42.2 | 32.0 | **42.8** | |
| | **Reasoning** | | | | |
| | AIME25 | 22.7 | 21.6 | 19.1 | **47.4** | |
| | HMMT25 | 9.7 | 12.0 | 12.1 | **31.0** | |
| | ZebraLogic | 14.8 | 33.2 | 35.2 | **80.2** | |
| | LiveBench 20241125 | 41.5 | 59.4 | 48.4 | **63.0** | |
| | **Coding** | | | | |
| | LiveCodeBench v6 (25.02-25.05) | 31.5 | 29.0 | 26.4 | **35.1** | |
| | MultiPL-E | **76.3** | 74.6 | 66.6 | 76.2 | |
| | Aider-Polyglot | 9.8 | **24.4** | 13.8 | 12.9 | |
| | **Alignment** | | | | |
| | IFEval | 74.5 | 83.5 | 81.2 | **83.8** | |
| | Arena-Hard v2* | 15.9 | 24.8 | 9.5 | **43.4** | |
| | Creative Writing v3 | 72.7 | 68.1 | 53.6 | **83.5** | |
| | WritingBench | 66.9 | 72.2 | 68.5 | **83.4** | |
| | **Agent** | | | | |
| | BFCL-v3 | 53.0 | 58.6 | 57.6 | **61.9** | |
| | TAU1-Retail | 23.5 | 38.3 | 24.3 | **48.7** | |
| | TAU1-Airline | 14.0 | 18.0 | 16.0 | **32.0** | |
| | TAU2-Retail | - | 31.6 | 28.1 | **40.4** | |
| | TAU2-Airline | - | 18.0 | 12.0 | **24.0** | |
| | TAU2-Telecom | - | **18.4** | 17.5 | 13.2 | |
| | **Multilingualism** | | | | |
| | MultiIF | 60.7 | 70.8 | 61.3 | **79.1** | |
| | MMLU-ProX | 56.2 | **65.1** | 49.6 | 61.6 | |
| | INCLUDE | 58.6 | **67.8** | 53.8 | 60.1 | |
| | PolyMATH | 15.6 | 23.3 | 16.6 | **31.1** | |
|
|
| \*: For reproducibility, we report the win rates evaluated by GPT-4.1. |
| |
| --- |
| |
| # © 2026 Neura Tech AI & Lumina AI. All rights reserved. |
