File size: 6,615 Bytes

---
language:
- tr
- en
- de
- es
- fr
- ru
- zh
- ja
- ko
license: mit
tags:
- turkish
- türkiye
- ai
- lamapi
- next-codex
- coder
- codex
- text-generation
- open-source
- 30b
- moe
- mixture-of-experts
- code-generation
- coding
- llm
- transformer
- artificial-intelligence
pipeline_tag: text-generation
datasets:
- mlabonne/FineTome-100k
- google/code_x_glue_ct_code_to_text
- bigcode/the-stack-v2
- neulab/agent-data-collection
- openai/gsm8k
- princeton-nlp/SWE-bench_Verified
- microsoft/orca-math-word-problems-200k
- qwedsacf/competition_math
- hotpotqa/hotpot_qa
- wics/strategy-qa
- glaiveai/glaive-function-calling-v2
- Anthropic/hh-rlhf
- ccdv/cnn_dailymail
- allenai/ai2_arc
- allenai/sciq
- google-research-datasets/mbpp
- openai/openai_humaneval
- allenai/openbookqa
- baber/piqa
- allenai/winogrande
- Rowan/hellaswag
- allenai/social_i_qa
- databricks/databricks-dolly-15k
- truthfulqa/truthful_qa
- HuggingFaceH4/ultrachat_200k
- OpenAssistant/oasst1
- iamtarun/python_code_instructions_18k_alpaca
- nickrosh/Evol-Instruct-Code-80k-v1
- arcee-ai/agent-data
- GreenerPastures/All-Your-Base-Full
- FreedomIntelligence/Socratic
- qihoo360/Light-R1-SFTData
- dongguanting/ARPO-SFT-54K
library_name: transformers
---

![30bcoder](https://cdn-uploads.huggingface.co/production/uploads/67d46bc5fe6ad6f6511d6f44/uolLKxUkpOc_eQYIysPs_.png)

# 💻 Next-Codex (L846MoE)

### Code your future with our models.

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
[![Architecture: MoE](https://img.shields.io/badge/Architecture-MoE-violet.svg)]()
[![HuggingFace](https://img.shields.io/badge/🤗-Lamapi/next--codex-orange.svg)](https://huggingface.co/Lamapi/next-codex)

---

## 📖 Overview

**Next-Codex** is a high-performance, specialized **Mixture-of-Experts (MoE)** Large Language Model designed specifically for code generation, debugging, and software engineering tasks.

Unlike traditional dense models, **Next-Codex** utilizes a sparse architecture with **30 Billion total parameters**, but only activates **3 Billion parameters per token**. This unique design allows it to deliver the deep reasoning capabilities of a massive model while maintaining the ultra-low latency and inference cost of a lightweight 3B model. It is fine-tuned on a massive corpus of code across 20+ programming languages, making it the most efficient coding assistant in its class.

---

## ⚡ Highlights

- 🇹🇷 **Türkiye’s First Specialized MoE Coding Model:** Designed for speed and precision.
- 🚀 **Hyper-Efficient Inference:** Runs with **3B active parameters**, enabling deployment on consumer GPUs (e.g., RTX 3090/4090).
- 💻 **SOTA Coding Performance:** Surpasses Claude Sonnet 4 and rivals o3-High in Python & JavaScript benchmarks.
- 🌍 **Polyglot Programming:** Master-level proficiency in Python, JS/TS, Rust, Go, C++, SQL, and Swift.
- 🧠 **Context-Aware Debugging:** Excellent at understanding large codebases and suggesting architectural improvements.
- 🏢 **Production Ready:** Optimized for autocomplete, unit test generation, and docstring creation.

---

## 📊 Benchmark Performance (Coding & Logic)

**Next-Codex** achieves state-of-the-art results among open-weights coding models, balancing extreme efficiency with high accuracy.

Benchmarks are being conducted...
---

## 🚀 Installation & Usage

**Note:** Due to the MoE architecture, this model is memory efficient. You can run it comfortably on 24GB VRAM GPUs (4-bit quantization highly recommended for lower VRAM).

```
!pip install unsloth transformers
```

```python
from unsloth import FastLanguageModel

# Load the MoE Model
model, tokenizer = FastLanguageModel.from_pretrained(
    "Lamapi/next-codex",
    load_in_4bit = True, # Optimized for 24GB VRAM
)

messages = [
    {"role": "system", "content": "You are Next-Codex, an expert software engineer and AI coding assistant."},
    {"role" : "user", "content" : "Write a highly optimized Rust function to calculate the Fibonacci sequence using memoization."}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize = False,
    add_generation_prompt = True
)

from transformers import TextStreamer
_ = model.generate(
    **tokenizer(text, return_tensors = "pt").to("cuda"),
    max_new_tokens = 2048,
    temperature = 0.2, # Lower temperature for code precision
    top_p = 0.95,
    streamer = TextStreamer(tokenizer, skip_prompt = True),
)
```

---

## 🧩 Key Features

| Feature | Description |
| :--- | :--- |
| 🔀 **Smart Routing (MoE)** | Dynamically routes tokens to the best "expert" layers, activating only 3B params for speed. |
| 🛠️ **Full-Stack Mastery** | Trained on frontend (React, Vue), backend (Django, Spring), and systems (C, Rust) code. |
| 🇹🇷 **Code Support** | Exceptional ability to understand Turkish variable names and comments in legacy codebases. |
| 🐞 **Deep Debugging** | Analyzes stack traces and logic errors to provide instant fixes. |
| 📝 **Docstring & Testing** | Automatically generates Javadoc, PyDoc, and Unit Tests (Pytest/Jest). |
| 🔒 **Secure Coding** | Aligned to avoid common vulnerabilities (SQLi, XSS) in generated code. |

---

## 📐 Model Specifications

| Specification | Details |
| :--- | :--- |
| **Architecture** | Mixture of Experts (MoE) Transformer |
| **Total Parameters** | 30 Billion |
| **Active Parameters** | 3 Billion (per token) |
| **Context Window** | 32k Tokens |
| **Experts** | 8 Experts (Top-2 Routing) |
| **Training Data** | 1T+ Tokens of Code (The Stack v2, GitHub, Synthetic) |
| **Quantization** | GGUF, AWQ, GPTQ supported |

---

## 🎯 Ideal Use Cases

* **IDE Autocomplete Plugins** — Low latency makes it perfect for "Copilot" style completions.
* **Legacy Code Refactoring** — Converting outdated code to modern standards (e.g., Java 8 to Java 21).
* **SQL Generation** — Text-to-SQL for complex data analytics.
* **Turkish/English Development** — Teams working in bilingual environments.
* **Algorithm Optimization** — Reducing time complexity of existing functions.

---

## 📄 License

Licensed under the **MIT License** — free for commercial and non-commercial use.

---

## 📞 Contact & Support

* 📧 **Email:** [lamapicontact@gmail.com](mailto:lamapicontact@gmail.com)
* 🤗 **HuggingFace:** [Lamapi](https://huggingface.co/Lamapi)

---

> **Next-Codex** — Smart as a giant, fast as a lightweight. The future of coding is MoE.

[![Follow on HuggingFace](https://img.shields.io/badge/Follow-HuggingFace-yellow?logo=huggingface)](https://huggingface.co/Lamapi)