Macaron-V1-Tall-L2-Coding-Fused-BF16

L2 Coding LoRA fused into Macaron-V1-Tall · BF16 · 14 shards · ~64.5 GB

This is a fused variant of Macaron-V1-Tall where the L2 Coding specialist LoRA adapter (rank 64) has been permanently baked into the base model weights. It is a standalone checkpoint — no separate adapter files or PEFT wrappers are required.

Built on top of Qwen3.6-35B-A3B, Macaron-V1-Tall uses a Mixture of LoRA (MoL) architecture with four specialist adapters. This fusion takes only the L2 Coding specialist and merges it directly into the base weights via W_fused = W_base + (B @ A).

---

🔍 What Was Fused

Component	Status
Base model	Macaron-V1-Tall (Qwen3.6-35B-A3B, BF16) ✅
L2 Coding LoRA (rank 64)	Fused via W_fused = W_base + (B @ A) ✅
L0 Chat LoRA	❌ Not included (remains an adapter)
L1 Agent LoRA	❌ Not included (remains an adapter)
L3 GenUI LoRA	❌ Not included (remains an adapter)
MTP module	❌ Removed (mtp_num_hidden_layers = 0)

Verification Details

Fusion was confirmed by inspecting model.safetensors.index.json:

• No LoRA keys: The weight map contains only standard base model keys (linear_attn.*, self_attn.*, mlp.switch_mlp.*, mlp.shared_expert.*, mlp.gate.*). There are zero lora_A, lora_B, or adapter_* keys — confirming all L2 LoRA weights were absorbed into the base.
• Parameter count: total_parameters = 34,660,608,768 (~34.7B) — matches the Macaron-V1-Tall base exactly, since a rank-64 LoRA adds negligible parameters that are now embedded in the base weights.
• MTP stripped: mtp_num_hidden_layers = 0 confirms the Multi-Token Prediction module was removed as documented in the fusion plan.
• Sharding: 14 safetensors files totaling ~64.5 GB at BF16 precision.

---

🏗️ Architecture Summary

Field	Value
Base model	Macaron-V1-Tall (Qwen3.6-35B-A3B)
Fusion method	Linear combination: W_fused = W_base + (B @ A)
LoRA rank	64
LoRA target	L2 Coding specialist
Context length	262K
Layers	40
Hidden size	2048
Attention heads	16
KV heads	2
Experts	256 total, 8 per token
Precision	BF16
MTP layers	0 (removed)
Total parameters	~34.7B

What the L2 Coding LoRA Adds

The L2 Coding specialist was trained for:

• Repository-level coding — understanding project context, navigating codebases
• SWE tasks — bug fixing, feature implementation, refactoring
• Terminal use — shell commands, git operations, build/test workflows
• Code-native workflows — tool calling, code generation, documentation

By fusing L2 directly into the base weights, you get a model that always applies the coding-specialist transformations without needing runtime LoRA switching or adapter loading.

---

🚀 Usage

Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "petergilani/Macaron-V1-Tall-L2-Coding-Fused-BF16"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model.eval()

messages = [
    {"role": "user", "content": "Write a Python function that performs a binary search on a sorted list and returns the index of the target."}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

generated_ids = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.2,
)
print(tokenizer.decode(generated_ids[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Converting to MLX

This PyTorch BF16 checkpoint can be converted to MLX format for Apple Silicon inference:

python -m mlx_lm.convert \
    --torch-sharded-path petergilani/Macaron-V1-Tall-L2-Coding-Fused-BF16 \
    --mlx-path mlx-model/

Then run inference:

from mlx_lm import load, generate

model, tokenizer = load("mlx-model/")

prompt = "Implement a simple REST API with FastAPI that returns 'Hello, World!'"
output = generate(model, tokenizer, prompt=prompt, verbose=True)

---

📜 License

This fused model inherits the MIT License from the original Macaron-V1-Tall repository.

Users should also respect any requirements inherited from the Qwen3.6-35B-A3B base model.

---

📚 Citation

If you use this model in your work, please cite the original Macaron-V1 paper:

@misc{mindlab2026macaronv1,
  author = {{Mind Lab}},
  title = {Introducing Macaron-V1},
  year = {2026},
  howpublished = {Mind Lab: A Lab for Experiential Intelligence},
  note = {https://macaron.im/mindlab/research/introducing-macaron-v1}
}

And the base model:

@misc{qwen3.6,
  author = {Qwen Team},
  title = {Qwen3.6-35B-A3B},
  year = {2026},
  publisher = {Alibaba},
  url = {https://huggingface.co/Qwen/Qwen3.6-35B-A3B}
}

---

🔗 Related Links

• Original model: mindlab-research/Macaron-V1-Tall
• Base architecture: Qwen/Qwen3.6-35B-A3B
• Macaron Artifacts: github.com/MindLab-Research/macaron-artifacts
• MoL Serving Harness: github.com/MindLab-Research/Mixture-of-LoRA-Harness
• Blog: Introducing Macaron-V1
• Hosted API: Macaron API Platform