| | --- |
| | language: |
| | - tr |
| | - en |
| | - de |
| | - es |
| | - fr |
| | - ru |
| | - zh |
| | - ja |
| | - ko |
| | license: mit |
| | tags: |
| | - turkish |
| | - türkiye |
| | - ai |
| | - lamapi |
| | - next-codex |
| | - coder |
| | - codex |
| | - text-generation |
| | - open-source |
| | - 30b |
| | - moe |
| | - mixture-of-experts |
| | - code-generation |
| | - coding |
| | - llm |
| | - transformer |
| | - artificial-intelligence |
| | pipeline_tag: text-generation |
| | datasets: |
| | - mlabonne/FineTome-100k |
| | - google/code_x_glue_ct_code_to_text |
| | - bigcode/the-stack-v2 |
| | - neulab/agent-data-collection |
| | - openai/gsm8k |
| | - princeton-nlp/SWE-bench_Verified |
| | - microsoft/orca-math-word-problems-200k |
| | - qwedsacf/competition_math |
| | - hotpotqa/hotpot_qa |
| | - wics/strategy-qa |
| | - glaiveai/glaive-function-calling-v2 |
| | - Anthropic/hh-rlhf |
| | - ccdv/cnn_dailymail |
| | - allenai/ai2_arc |
| | - allenai/sciq |
| | - google-research-datasets/mbpp |
| | - openai/openai_humaneval |
| | - allenai/openbookqa |
| | - baber/piqa |
| | - allenai/winogrande |
| | - Rowan/hellaswag |
| | - allenai/social_i_qa |
| | - databricks/databricks-dolly-15k |
| | - truthfulqa/truthful_qa |
| | - HuggingFaceH4/ultrachat_200k |
| | - OpenAssistant/oasst1 |
| | - iamtarun/python_code_instructions_18k_alpaca |
| | - nickrosh/Evol-Instruct-Code-80k-v1 |
| | - arcee-ai/agent-data |
| | - GreenerPastures/All-Your-Base-Full |
| | - FreedomIntelligence/Socratic |
| | - qihoo360/Light-R1-SFTData |
| | - dongguanting/ARPO-SFT-54K |
| | library_name: transformers |
| | --- |
| | |
| |  |
| |
|
| | # 💻 Next-Codex (L846MoE) |
| |
|
| | ### Code your future with our models. |
| |
|
| | [](https://opensource.org/licenses/MIT) |
| | []() |
| | [](https://huggingface.co/Lamapi/next-codex) |
| | [](https://discord.gg/XgH4EpyPD2) |
| |
|
| | --- |
| |
|
| | ## 📖 Overview |
| |
|
| | **Next-Codex** is a high-performance, specialized **Mixture-of-Experts (MoE)** Large Language Model designed specifically for code generation, debugging, and software engineering tasks. |
| |
|
| | Unlike traditional dense models, **Next-Codex** utilizes a sparse architecture with **30 Billion total parameters**, but only activates **3 Billion parameters per token**. This unique design allows it to deliver the deep reasoning capabilities of a massive model while maintaining the ultra-low latency and inference cost of a lightweight 3B model. It is fine-tuned on a massive corpus of code across 20+ programming languages, making it the most efficient coding assistant in its class. |
| |
|
| | --- |
| |
|
| | ## ⚡ Highlights |
| |
|
| | - 🇹🇷 **Türkiye’s First Specialized MoE Coding Model:** Designed for speed and precision. |
| | - 🚀 **Hyper-Efficient Inference:** Runs with **3B active parameters**, enabling deployment on consumer GPUs (e.g., RTX 3090/4090). |
| | - 💻 **SOTA Coding Performance:** Surpasses Claude Sonnet 4 and rivals o3-High in Python & JavaScript benchmarks. |
| | - 🌍 **Polyglot Programming:** Master-level proficiency in Python, JS/TS, Rust, Go, C++, SQL, and Swift. |
| | - 🧠 **Context-Aware Debugging:** Excellent at understanding large codebases and suggesting architectural improvements. |
| | - 🏢 **Production Ready:** Optimized for autocomplete, unit test generation, and docstring creation. |
| |
|
| | --- |
| |
|
| | ## 📊 Benchmark Performance (Coding & Logic) |
| |
|
| | **Next-Codex** achieves state-of-the-art results among open-weights coding models, balancing extreme efficiency with high accuracy. |
| |
|
| | Benchmarks are being conducted... |
| | --- |
| |
|
| | ## 🚀 Installation & Usage |
| |
|
| | **Note:** Due to the MoE architecture, this model is memory efficient. You can run it comfortably on 24GB VRAM GPUs (4-bit quantization highly recommended for lower VRAM). |
| |
|
| | ``` |
| | !pip install unsloth transformers |
| | ``` |
| |
|
| | ```python |
| | from unsloth import FastLanguageModel |
| | |
| | # Load the MoE Model |
| | model, tokenizer = FastLanguageModel.from_pretrained( |
| | "Lamapi/next-codex", |
| | load_in_4bit = True, # Optimized for 24GB VRAM |
| | ) |
| | |
| | messages = [ |
| | {"role": "system", "content": "You are Next-Codex, an expert software engineer and AI coding assistant."}, |
| | {"role" : "user", "content" : "Write a highly optimized Rust function to calculate the Fibonacci sequence using memoization."} |
| | ] |
| | |
| | text = tokenizer.apply_chat_template( |
| | messages, |
| | tokenize = False, |
| | add_generation_prompt = True |
| | ) |
| | |
| | from transformers import TextStreamer |
| | _ = model.generate( |
| | **tokenizer(text, return_tensors = "pt").to("cuda"), |
| | max_new_tokens = 2048, |
| | temperature = 0.2, # Lower temperature for code precision |
| | top_p = 0.95, |
| | streamer = TextStreamer(tokenizer, skip_prompt = True), |
| | ) |
| | ``` |
| |
|
| | --- |
| |
|
| | ## 🧩 Key Features |
| |
|
| | | Feature | Description | |
| | | :--- | :--- | |
| | | 🔀 **Smart Routing (MoE)** | Dynamically routes tokens to the best "expert" layers, activating only 3B params for speed. | |
| | | 🛠️ **Full-Stack Mastery** | Trained on frontend (React, Vue), backend (Django, Spring), and systems (C, Rust) code. | |
| | | 🇹🇷 **Code Support** | Exceptional ability to understand Turkish variable names and comments in legacy codebases. | |
| | | 🐞 **Deep Debugging** | Analyzes stack traces and logic errors to provide instant fixes. | |
| | | 📝 **Docstring & Testing** | Automatically generates Javadoc, PyDoc, and Unit Tests (Pytest/Jest). | |
| | | 🔒 **Secure Coding** | Aligned to avoid common vulnerabilities (SQLi, XSS) in generated code. | |
| |
|
| | --- |
| |
|
| | ## 📐 Model Specifications |
| |
|
| | | Specification | Details | |
| | | :--- | :--- | |
| | | **Architecture** | Mixture of Experts (MoE) Transformer | |
| | | **Total Parameters** | 30 Billion | |
| | | **Active Parameters** | 3 Billion (per token) | |
| | | **Context Window** | 32k Tokens | |
| | | **Experts** | 8 Experts (Top-2 Routing) | |
| | | **Training Data** | 1T+ Tokens of Code (The Stack v2, GitHub, Synthetic) | |
| | | **Quantization** | GGUF, AWQ, GPTQ supported | |
| |
|
| | --- |
| |
|
| | ## 🎯 Ideal Use Cases |
| |
|
| | * **IDE Autocomplete Plugins** — Low latency makes it perfect for "Copilot" style completions. |
| | * **Legacy Code Refactoring** — Converting outdated code to modern standards (e.g., Java 8 to Java 21). |
| | * **SQL Generation** — Text-to-SQL for complex data analytics. |
| | * **Turkish/English Development** — Teams working in bilingual environments. |
| | * **Algorithm Optimization** — Reducing time complexity of existing functions. |
| |
|
| | --- |
| |
|
| | ## 📄 License |
| |
|
| | Licensed under the **MIT License** — free for commercial and non-commercial use. |
| |
|
| | --- |
| |
|
| | ## 📞 Contact & Support |
| |
|
| | * 📧 **Email:** [lamapicontact@gmail.com](mailto:lamapicontact@gmail.com) |
| | * 🤗 **HuggingFace:** [Lamapi](https://huggingface.co/Lamapi) |
| |
|
| | --- |
| |
|
| | > **Next-Codex** — Smart as a giant, fast as a lightweight. The future of coding is MoE. |
| |
|
| | [](https://huggingface.co/Lamapi) |
