README.md

---
base_model: meta-llama/Llama-3.3-70B-Instruct-Reference
library_name: peft
license: other
tags:
  - lora
  - peft
  - adapter
  - adaption
pipeline_tag: text-generation
datasets:
- jay2219/quantum-circuit-routing
---

# ⚛️ Q-Route-70B: Quantum Hardware Compiler Model Card

**Q-Route-70B** is a domain-adapted frontier AI compiler model fine-tuned on `Llama-3.3-70B-Instruct` via the **Adaption Labs AutoScientist** platform. It compiles abstract OpenQASM 2.0 quantum circuits to strictly adhere to physical target QPU connectivity graphs (Line, Ring, Star, Grid, and IBM Heavy-Hex hardware topologies).
---

## Model Training

A LORA adapter for `meta-llama/Llama-3.3-70B-Instruct-Reference`. This model was trained with SFT using [Adaption](https://adaptionlabs.ai)'s AutoScientist on the quantum_circuit_routing dataset.


![Training metrics](training-metrics.png)

### AutoScientist Config

```json
{
  "job_id": "72858b84-6e51-44fa-b887-56fd54517a8a",
  "training_experiment_id": "dcc1dae2-3122-4888-99a8-3c296afd1020",
  "original_model_name": "meta-llama/Llama-3.3-70B-Instruct-Reference",
  "trained_model_name": "adaption_quantum_circuit_routing",
  "training_method": "sft",
  "training_type": "lora",
  "data_format": "chat",
  "hyperparams": {
    "lora": "true",
    "lora_r": 64,
    "n_evals": 5,
    "n_epochs": 2,
    "batch_size": "max",
    "lora_alpha": 128,
    "lora_dropout": 0,
    "min_lr_ratio": 0.1,
    "warmup_ratio": 0.05,
    "weight_decay": 0.05,
    "learning_rate": 0.0001,
    "max_grad_norm": 1,
    "base_model_size": "70B",
    "train_on_inputs": "false",
    "training_method": "sft",
    "lr_scheduler_type": "cosine",
    "scheduler_num_cycles": 0.5,
    "lora_trainable_modules": "all-linear"
  }
}
```

## Training Data

The model was trained on 14,583 rows of adapted data with the following domain distribution: code (79%), science (21%), technology (0%).

## Model Evaluation

The model was evaluated on an in-distribution held-out test set as well as a broader domain-specific test set to measure generalization.


![Win rates](win-rates.png)

| Domain | Win rate vs. base model |
| --- | --- |
| code | 50% |


---

## Model Details

- **Model Name:** `Q-Route-70B`
- **Model Type:** Fine-Tuned PEFT/LoRA Causal Language Model
- **Base Model:** `meta-llama/Llama-3.3-70B-Instruct-Reference` (70B parameters)
- **Language/Domain:** OpenQASM 2.0 Spatial Graph Compiler Code

---

## Performance & Benchmark Summary (Q-Route-100 Eval)

Evaluated on the **Q-Route-100 Benchmark** (100 novel circuit/topology pairs):

| Model | Pass@1 Topology Compliance | Syntax Validity | Algorithmic Equivalence | Avg. SWAP Gates |
| :--- | :---: | :---: | :---: | :---: |
| **Q-Route-70B (Ours)** | **100.0%** | **100.0%** | **100.0%** | **50.93** |
| **Qwen2.5-Coder-32B-Instruct** | 30.0% | 99.0% | 99.0% | 19.24 |
| **Mistral-7B-v0.1** | 21.0% | 71.0% | 71.0% | 0.67 |
| **Llama-3.3-70B-Instruct** | 14.0% | 99.0% | 99.0% | 83.46 |
| **DeepSeek-R1-Distill-Llama-70B** | 6.0% | 7.0% | 7.0% | 0.17 |
| **DeepSeek-R1-Full** | 5.0% | 5.0% | 5.0% | 0.25 |

---

## How to Get Started with the Model

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model_id = "meta-llama/Meta-Llama-3.3-70B-Instruct-Reference"
adapter_id = "jay2219/Q-Route-70B"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)

system_prompt = "You are Q-Route, an enterprise-grade deterministic Quantum Hardware Compiler. Translate abstract OpenQASM 2.0 to hardware-compliant QASM."
user_prompt = """### Physical Hardware Specification:
- Target Topology: Star-15
- Valid Physical Edges (Coupling Map): [[0, 1], [0, 2], [0, 3], [0, 4], [0, 5], [0, 6], [0, 7], [0, 8], [0, 9], [0, 10], [0, 11], [0, 12], [0, 13], [0, 14]]

### Abstract Input OpenQASM 2.0:
```qasm
OPENQASM 2.0;
include "qelib1.inc";
qreg q[15];
creg c[15];
cx q[1], q[5];
measure q -> c;
```
### Compiled Hardware-Compliant OpenQASM 2.0:
```
prompt = f"<|im_start|>system\n{system_prompt}<|im_end|>\n<|im_start|>user\n{user_prompt}<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=False)

compiled_qasm = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(compiled_qasm)
```

---

## Training Hyperparameters

- **LoRA Rank ($r$):** 64
- **LoRA Alpha ($\alpha$):** 128
- **Dropout:** 0.0
- **Target Modules:** `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
- **Training Dataset:** ~15,000 Qiskit ground-truth pairs across 5 hardware topology families.
- **Platform:** Adaption Labs AutoScientist platform.