|
|
--- |
|
|
language: |
|
|
- tr |
|
|
- en |
|
|
- de |
|
|
- es |
|
|
- fr |
|
|
- ru |
|
|
- zh |
|
|
- ja |
|
|
- ko |
|
|
license: mit |
|
|
tags: |
|
|
- turkish |
|
|
- türkiye |
|
|
- ai |
|
|
- lamapi |
|
|
- next-codex |
|
|
- coder |
|
|
- codex |
|
|
- text-generation |
|
|
- open-source |
|
|
- 30b |
|
|
- moe |
|
|
- mixture-of-experts |
|
|
- code-generation |
|
|
- coding |
|
|
- llm |
|
|
- transformer |
|
|
- artificial-intelligence |
|
|
pipeline_tag: text-generation |
|
|
datasets: |
|
|
- mlabonne/FineTome-100k |
|
|
- google/code_x_glue_ct_code_to_text |
|
|
- bigcode/the-stack-v2 |
|
|
- neulab/agent-data-collection |
|
|
- openai/gsm8k |
|
|
- princeton-nlp/SWE-bench_Verified |
|
|
- microsoft/orca-math-word-problems-200k |
|
|
- qwedsacf/competition_math |
|
|
- hotpotqa/hotpot_qa |
|
|
- wics/strategy-qa |
|
|
- glaiveai/glaive-function-calling-v2 |
|
|
- Anthropic/hh-rlhf |
|
|
- ccdv/cnn_dailymail |
|
|
- allenai/ai2_arc |
|
|
- allenai/sciq |
|
|
- google-research-datasets/mbpp |
|
|
- openai/openai_humaneval |
|
|
- allenai/openbookqa |
|
|
- baber/piqa |
|
|
- allenai/winogrande |
|
|
- Rowan/hellaswag |
|
|
- allenai/social_i_qa |
|
|
- databricks/databricks-dolly-15k |
|
|
- truthfulqa/truthful_qa |
|
|
- HuggingFaceH4/ultrachat_200k |
|
|
- OpenAssistant/oasst1 |
|
|
- iamtarun/python_code_instructions_18k_alpaca |
|
|
- nickrosh/Evol-Instruct-Code-80k-v1 |
|
|
- arcee-ai/agent-data |
|
|
- GreenerPastures/All-Your-Base-Full |
|
|
- FreedomIntelligence/Socratic |
|
|
- qihoo360/Light-R1-SFTData |
|
|
- dongguanting/ARPO-SFT-54K |
|
|
library_name: transformers |
|
|
--- |
|
|
|
|
|
 |
|
|
|
|
|
# 💻 Next-Codex (L846MoE) |
|
|
|
|
|
### Code your future with our models. |
|
|
|
|
|
[](https://opensource.org/licenses/MIT) |
|
|
[]() |
|
|
[](https://huggingface.co/Lamapi/next-codex) |
|
|
|
|
|
--- |
|
|
|
|
|
## 📖 Overview |
|
|
|
|
|
**Next-Codex** is a high-performance, specialized **Mixture-of-Experts (MoE)** Large Language Model designed specifically for code generation, debugging, and software engineering tasks. |
|
|
|
|
|
Unlike traditional dense models, **Next-Codex** utilizes a sparse architecture with **30 Billion total parameters**, but only activates **3 Billion parameters per token**. This unique design allows it to deliver the deep reasoning capabilities of a massive model while maintaining the ultra-low latency and inference cost of a lightweight 3B model. It is fine-tuned on a massive corpus of code across 20+ programming languages, making it the most efficient coding assistant in its class. |
|
|
|
|
|
--- |
|
|
|
|
|
## ⚡ Highlights |
|
|
|
|
|
- 🇹🇷 **Türkiye’s First Specialized MoE Coding Model:** Designed for speed and precision. |
|
|
- 🚀 **Hyper-Efficient Inference:** Runs with **3B active parameters**, enabling deployment on consumer GPUs (e.g., RTX 3090/4090). |
|
|
- 💻 **SOTA Coding Performance:** Surpasses Claude Sonnet 4 and rivals o3-High in Python & JavaScript benchmarks. |
|
|
- 🌍 **Polyglot Programming:** Master-level proficiency in Python, JS/TS, Rust, Go, C++, SQL, and Swift. |
|
|
- 🧠 **Context-Aware Debugging:** Excellent at understanding large codebases and suggesting architectural improvements. |
|
|
- 🏢 **Production Ready:** Optimized for autocomplete, unit test generation, and docstring creation. |
|
|
|
|
|
--- |
|
|
|
|
|
## 📊 Benchmark Performance (Coding & Logic) |
|
|
|
|
|
**Next-Codex** achieves state-of-the-art results among open-weights coding models, balancing extreme efficiency with high accuracy. |
|
|
|
|
|
Benchmarks are being conducted... |
|
|
--- |
|
|
|
|
|
## 🚀 Installation & Usage |
|
|
|
|
|
**Note:** Due to the MoE architecture, this model is memory efficient. You can run it comfortably on 24GB VRAM GPUs (4-bit quantization highly recommended for lower VRAM). |
|
|
|
|
|
``` |
|
|
!pip install unsloth transformers |
|
|
``` |
|
|
|
|
|
```python |
|
|
from unsloth import FastLanguageModel |
|
|
|
|
|
# Load the MoE Model |
|
|
model, tokenizer = FastLanguageModel.from_pretrained( |
|
|
"Lamapi/next-codex", |
|
|
load_in_4bit = True, # Optimized for 24GB VRAM |
|
|
) |
|
|
|
|
|
messages = [ |
|
|
{"role": "system", "content": "You are Next-Codex, an expert software engineer and AI coding assistant."}, |
|
|
{"role" : "user", "content" : "Write a highly optimized Rust function to calculate the Fibonacci sequence using memoization."} |
|
|
] |
|
|
|
|
|
text = tokenizer.apply_chat_template( |
|
|
messages, |
|
|
tokenize = False, |
|
|
add_generation_prompt = True |
|
|
) |
|
|
|
|
|
from transformers import TextStreamer |
|
|
_ = model.generate( |
|
|
**tokenizer(text, return_tensors = "pt").to("cuda"), |
|
|
max_new_tokens = 2048, |
|
|
temperature = 0.2, # Lower temperature for code precision |
|
|
top_p = 0.95, |
|
|
streamer = TextStreamer(tokenizer, skip_prompt = True), |
|
|
) |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## 🧩 Key Features |
|
|
|
|
|
| Feature | Description | |
|
|
| :--- | :--- | |
|
|
| 🔀 **Smart Routing (MoE)** | Dynamically routes tokens to the best "expert" layers, activating only 3B params for speed. | |
|
|
| 🛠️ **Full-Stack Mastery** | Trained on frontend (React, Vue), backend (Django, Spring), and systems (C, Rust) code. | |
|
|
| 🇹🇷 **Code Support** | Exceptional ability to understand Turkish variable names and comments in legacy codebases. | |
|
|
| 🐞 **Deep Debugging** | Analyzes stack traces and logic errors to provide instant fixes. | |
|
|
| 📝 **Docstring & Testing** | Automatically generates Javadoc, PyDoc, and Unit Tests (Pytest/Jest). | |
|
|
| 🔒 **Secure Coding** | Aligned to avoid common vulnerabilities (SQLi, XSS) in generated code. | |
|
|
|
|
|
--- |
|
|
|
|
|
## 📐 Model Specifications |
|
|
|
|
|
| Specification | Details | |
|
|
| :--- | :--- | |
|
|
| **Architecture** | Mixture of Experts (MoE) Transformer | |
|
|
| **Total Parameters** | 30 Billion | |
|
|
| **Active Parameters** | 3 Billion (per token) | |
|
|
| **Context Window** | 32k Tokens | |
|
|
| **Experts** | 8 Experts (Top-2 Routing) | |
|
|
| **Training Data** | 1T+ Tokens of Code (The Stack v2, GitHub, Synthetic) | |
|
|
| **Quantization** | GGUF, AWQ, GPTQ supported | |
|
|
|
|
|
--- |
|
|
|
|
|
## 🎯 Ideal Use Cases |
|
|
|
|
|
* **IDE Autocomplete Plugins** — Low latency makes it perfect for "Copilot" style completions. |
|
|
* **Legacy Code Refactoring** — Converting outdated code to modern standards (e.g., Java 8 to Java 21). |
|
|
* **SQL Generation** — Text-to-SQL for complex data analytics. |
|
|
* **Turkish/English Development** — Teams working in bilingual environments. |
|
|
* **Algorithm Optimization** — Reducing time complexity of existing functions. |
|
|
|
|
|
--- |
|
|
|
|
|
## 📄 License |
|
|
|
|
|
Licensed under the **MIT License** — free for commercial and non-commercial use. |
|
|
|
|
|
--- |
|
|
|
|
|
## 📞 Contact & Support |
|
|
|
|
|
* 📧 **Email:** [lamapicontact@gmail.com](mailto:lamapicontact@gmail.com) |
|
|
* 🤗 **HuggingFace:** [Lamapi](https://huggingface.co/Lamapi) |
|
|
|
|
|
--- |
|
|
|
|
|
> **Next-Codex** — Smart as a giant, fast as a lightweight. The future of coding is MoE. |
|
|
|
|
|
[](https://huggingface.co/Lamapi) |
