README.md

---
base_model: Jackrong/Qwopus3.6-27B-Coder
tags:
- text-generation-inference
- transformers
- unsloth
- qwen3_6
- reasoning
- chain-of-thought
- lora
- sft
- agent
- tool-use
- function-calling
- coder
- mlx
license: apache-2.0
language:
- en
- zh
- es
- ru
- ja
pipeline_tag: text-generation
datasets:
- Jackrong/Claude-opus-4.6-TraceInversion-9000x
- Jackrong/Claude-opus-4.7-TraceInversion-5000x
- lambda/hermes-agent-reasoning-traces
---

# Qwopus 3.6 27B Coder (6-bit MLX Quantization)

This repository hosts a high-performance 6-bit MLX quantization of `Jackrong/Qwopus3.6-27B-Coder`, converted using `mlx-lm` (v0.31.2).

Optimized specifically for Apple Silicon (M-series chips), this model balances the exceptional coding intelligence of the 27B parameter Qwopus architecture with the memory and speed efficiency required for local, low-latency deployment. 

## Key Features

* **TurboQuant Accelerated:** Leverages advanced MLX kernel optimizations for accelerated matrix multiplication, ensuring faster token generation rates during local execution, making the kv cache at the models whopping full 262k context just around ~3-4gb from ~180gb
* **~50% Memory Reduction:** Cuts VRAM usage nearly in half compared to the base 16-bit model, opening up local execution on mid-tier unified memory configurations without spilling into system swap.
* **Near-Lossless Precision:** Grouped 6-bit quantization maintains the original model's structural code generation capabilities, logic, and syntax proficiency with negligible degradation.
* **Apple Silicon Native:** Designed from the ground up for unified memory architectures, leveraging direct hardware acceleration via the MLX framework.

---

## Installation

Ensure you have the latest version of the MLX language model library installed:
```
pip install mlx-lm
```

## Quick Start

You can load and run inference with this model locally using the following Python script. It automatically detects and applies the correct chat template for structured instruction-following.

```
from mlx_lm import load, generate

# Load the optimized 6-bit model and its tokenizer
model, tokenizer = load("shreyan35/Qwopus3.6-27B-Coder-mlx-6Bit")

# Define your programming task or prompt
prompt = "Write an optimized Python function to find the longest palindromic substring."

# Apply the model's native chat template if available
if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )

# Generate the response with real-time streaming/verbosity
response = generate(model, tokenizer, prompt=prompt, verbose=True)
```
# Acknowledgements

* **Base Model:** Credit to `Jackrong` for the original [Qwopus3.6-27B-Coder](https://huggingface.co/Jackrong/Qwopus3.6-27B-Coder) architecture.
* **Infrastructure:** The Apple Machine Learning Research team for the ongoing development of the [MLX framework](https://github.com/ml-explore/mlx).