README.md

---
base_model: Qwen/Qwen3-Coder-Next
library_name: mlx
pipeline_tag: text-generation
license: apache-2.0
tags:
  - mlx
  - gguf
  - quantized
  - lora
  - qwen3-next
  - orchestrator
  - agent
---

# bankai-v1

`bankai-v1` is a one-step agent orchestration model based on
[`Qwen/Qwen3-Coder-Next`](https://huggingface.co/Qwen/Qwen3-Coder-Next), trained
to perform one-step agent orchestration. Given a task, it selects one specialist
worker and emits a complete delegated instruction as strict JSON.

This repository includes the original MLX LoRA, a llama.cpp GGUF LoRA, a fully
fused 4-bit MLX model, and a fully fused Q4_K_M GGUF model. The adapter was
trained against
[`mlx-community/Qwen3-Coder-Next-4bit`](https://huggingface.co/mlx-community/Qwen3-Coder-Next-4bit).
The upstream model is an 80B MoE with approximately 3B active parameters.

## Model Details

| Property | Value |
|---|---|
| Model type | One-step agent orchestration model and adapters |
| Upstream architecture | Qwen3-Next hybrid Gated DeltaNet / attention MoE |
| Total base parameters | 79,674,391,296 (79.674B) |
| Active parameters per token | Approximately 3B |
| LoRA trainable parameters | 2,857,000 (2.857M) |
| Trainable share | Approximately 0.004% of base parameters |
| Base precision used for training | MLX 4-bit quantization |
| Adapter precision | Safetensors LoRA weights |
| Native base context | 262,144 tokens |
| Fine-tuning sequence length | 2,048 tokens |
| Languages | Primarily English; base model is multilingual |
| License | Apache-2.0 |
| Framework | MLX LM 0.31.3 |

Qwen3-Coder-Next has 48 layers, 512 routed experts, ten activated routed
experts per token, and one shared expert. `bankai-v1` adapts only attention and
gated-delta projections in the final 16 layers; it does not add LoRA matrices
to the routed experts.

## Repository Contents

| Path | Type | Size | Standalone |
|---|---|---:|---|
| `adapters.safetensors` | Original MLX LoRA | 11,439,496 bytes | No |
| `adapter_config.json` | MLX LoRA configuration | 1.3 KB | No |
| `gguf/bankai-v1-f16.gguf` | llama.cpp F16 LoRA | 6,116,192 bytes | No |
| `gguf/bankai-v1-Q4_K_M.gguf` | Fused Q4_K_M GGUF | 48,528,320,544 bytes | Yes |
| `mlx/` | Fused 4-bit MLX model, nine shards | Approximately 44.9 GB | Yes |
| `bankai-system.txt` | Exact training-time system prompt | 1 KB | N/A |
| `training_config.yaml` | Reproducible training configuration | 746 bytes | N/A |
| `evaluation.json` | Per-example behavioral evaluation | 30 KB | N/A |

## Output Contract

```json
{
  "action": "call_agent",
  "agent": "software_engineer",
  "model": "auto",
  "instruction": "A complete execution-ready instruction for the worker",
  "context_refs": ["user_request"],
  "expected_output": "completed_task_with_evidence",
  "budget": {"max_tokens": 4096}
}
```

Supported worker labels:

- `software_engineer`
- `technical_writer`
- `marketing_strategist`
- `marketing_copywriter`
- `sales_specialist`
- `customer_support_specialist`
- `data_analyst`
- `product_designer`
- `financial_analyst`
- `compliance_specialist`

## Usage

Download the exact system prompt:

```bash
hf download ApplauseLab/bankai-v1 bankai-system.txt --local-dir bankai-v1
```

### Fused MLX Model

The fused MLX model does not require an adapter:

```bash
hf download ApplauseLab/bankai-v1 \
  --include "mlx/*" \
  --local-dir bankai-v1

mlx_lm.generate \
  --model bankai-v1/mlx \
  --system-prompt "$(cat bankai-v1/bankai-system.txt)" \
  --prompt "Design a GDPR-compliant CRM integration for a real-estate business." \
  --max-tokens 1024 \
  --temp 0
```

### MLX Adapter

Install MLX LM on Apple Silicon:

```bash
pip install "mlx-lm>=0.31.3" "huggingface-hub>=1.17"
hf download ApplauseLab/bankai-v1 \
  --include "adapter*" \
  --local-dir bankai-v1-adapter
```

Generate with the adapter:

```bash
mlx_lm.generate \
  --model mlx-community/Qwen3-Coder-Next-4bit \
  --adapter-path bankai-v1-adapter \
  --system-prompt "$(cat bankai-system.txt)" \
  --prompt "Design and test a GDPR-compliant CRM integration for a real-estate business." \
  --max-tokens 1024 \
  --temp 0
```

For best consistency, use the included `bankai-system.txt`.

### Fused GGUF / llama.cpp

The fused GGUF is standalone and does not require `--lora`:

```bash
hf download ApplauseLab/bankai-v1 \
  gguf/bankai-v1-Q4_K_M.gguf \
  --local-dir bankai-v1

llama-cli \
  --model bankai-v1/gguf/bankai-v1-Q4_K_M.gguf \
  --system-prompt-file bankai-v1/bankai-system.txt \
  --prompt "Design a GDPR-compliant CRM integration for a real-estate business." \
  --temp 0 \
  --n-predict 1024 \
  --single-turn
```

### GGUF Adapter

`bankai-v1-f16.gguf` is the same LoRA adapter converted for llama.cpp. It is
not a standalone 80B model; load it alongside a compatible
Qwen3-Coder-Next GGUF base:

```bash
llama-cli \
  --model Qwen3-Coder-Next-Q4_K_M.gguf \
  --lora bankai-v1/gguf/bankai-v1-f16.gguf \
  --system-prompt-file bankai-v1/bankai-system.txt \
  --prompt "Design a GDPR-compliant CRM integration for a real-estate business." \
  --temp 0 \
  --n-predict 1024
```

The GGUF adapter was converted through PEFT tensor orientation with
`lora_alpha=128`, preserving the MLX effective scale of 16 at rank 8.

Use the included `bankai-system.txt` for the exact worker vocabulary and JSON
contract seen during fine-tuning. Shorter prompts may produce valid routing in
a different schema.

## Fusion Details

The fused MLX model was produced by merging the LoRA into
`mlx-community/Qwen3-Coder-Next-4bit` and saving the resulting quantized MLX
weights.

The standalone GGUF starts from the single-file Unsloth Q4_K_M conversion of
Qwen3-Coder-Next. Untouched tensors are copied bit-for-bit. The 64 targeted
attention and gated-delta matrices in layers 32-47 are dequantized to F32, the
LoRA delta is applied at effective scale 16, and each matrix is requantized to
its original Q4_K_M mixture type. The resulting GGUF contains 843 tensors.

## Training

- Training base: `mlx-community/Qwen3-Coder-Next-4bit`
- Upstream base: `Qwen/Qwen3-Coder-Next`
- Method: MLX QLoRA
- Trainable parameters: 2.857M, approximately 0.004%
- Adapted blocks: final 16 of 48
- Adapted modules: full-attention Q/K/V/O projections and gated-delta input/output projections
- LoRA rank: 8
- LoRA scale: 16
- Optimizer: AdamW
- Learning rate: `1e-5`
- Updates: 612
- Gradient accumulation: 2
- Maximum sequence length: 2048
- Training tokens: 355,178
- Peak training memory: 74.4 GB on an Apple M4 Max

The source consisted of 1,080 synthetic task-to-delegated-prompt examples
across ten task categories and twenty industries. Grouped splitting held out
complete industries and complete `(category, subtask)` groups:

- Train: 612
- Validation: 224
- Test: 244

Only assistant tokens contributed to the loss.

## Evaluation

- Final sampled validation loss: 1.606
- Held-out test loss over 32 batches: 1.795
- Held-out test perplexity: 6.018
- Behavioral evaluation: one held-out example from each of ten categories
- Strict JSON validity: 10/10
- Runtime schema validity: 10/10
- Worker-routing accuracy: 10/10
- Complete contract: 10/10

The behavioral sample is small and measures formatting and category routing,
not end-to-end quality of worker execution.

Both fused formats received inference smoke tests with the full system prompt.
The fused MLX output matched the JSON contract. The fused GGUF output also
matched the contract, selected `compliance_specialist` for a held-out
real-estate GDPR/CRM/accounting request, and generated at approximately 64
tokens/second on an Apple M4 Max using Metal offload.

## Limitations

This first version learns only single-step worker selection and prompt
delegation. It does not execute workers, consume observations, choose among
specific model providers, verify results, retry failures, call agents in
parallel, optimize cost, or perform reinforcement-learned multi-turn
orchestration.

Training examples are synthetic and stylistically concentrated. Outputs may be
overly verbose and domain-specific legal, financial, medical, or compliance
instructions require qualified human review.

## Intended Use

Use `bankai-v1` as a routing and prompt-delegation component in an agent runtime
that validates its JSON, resolves the selected worker, enforces budgets, and
records real worker observations. Suitable initial domains include software
delivery, documentation, marketing, customer support, analysis, product
design, finance, and compliance workflows.

Do not treat generated delegated instructions as professional legal,
regulatory, medical, accounting, or investment advice. The adapter should not
be given authority to execute high-impact actions without runtime controls and
human review.

## Training Data Notes

The source is a synthetic Cartesian task grid rather than organic production
traces. Inputs combine 54 task types with 20 industries. Responses are
execution-ready delegated prompts averaging roughly 345 words. A grouped split
was used instead of a random split to reduce leakage from repeated industry and
subtask templates. No private user conversations were used.

The source artifact has a documented lineage discrepancy: records
`task-0881` through `task-0900` differ from their current shard copies. Training
used the checksummed final `training_set.jsonl` artifact.

## Integrity

| File | SHA-256 |
|---|---|
| `adapters.safetensors` | `66df3ea1d9a9f0b1c3e07b1620a33dfb0b3bf36b6ea4b1e52fe2fc1bd8da67d9` |
| `gguf/bankai-v1-f16.gguf` | `184b7855166c6c97fe366b4412de51cf98474dfc5df7caa95bda09f83c80e84d` |
| `gguf/bankai-v1-Q4_K_M.gguf` | `75f8200b83756e6f32565b4579c9bb07aa6aa591fca380bc3f85af1138102296` |

The fused GGUF also passed llama.cpp per-tensor hashing with whole-file XXH64
`c64819733a364e4d`.