| | --- |
| | base_model: |
| | - Qwen/Qwen3-4B-Instruct-2507 |
| | datasets: |
| | - Benyucong/graph-data-quantum-rl |
| | language: |
| | - en |
| | library_name: transformers |
| | license: apache-2.0 |
| | metrics: |
| | - code_eval |
| | pipeline_tag: text-generation |
| | tags: |
| | - agent |
| | - code |
| | - QASM |
| | - quantum |
| | --- |
| | |
| | # QUASAR: Quantum Assembly Code Generation with Tool-Augmented RL |
| |
|
| | [](https://huggingface.co/papers/2510.00967) [](https://github.com/benyucong/QUASAR) [](https://huggingface.co/datasets/Benyucong/graph-data-quantum-rl) |
| |
|
| | ## Model Summary |
| |
|
| | **QUASAR** is a 4B-parameter model fine-tuned from **Qwen3-4B-Instruct-2507** using a two-stage process: supervised fine-tuning (SFT) followed by agentic reinforcement learning (RL) with tool-augmented feedback. |
| |
|
| | The model is designed to **generate OpenQASM 3.0 quantum circuits** for optimization problems such as **QAOA** and **VQE**, achieving **high syntactic validity and semantic fidelity**. |
| |
|
| | - **Framework:** Agentic RL with external quantum simulator verification |
| | - **Reward:** Hierarchical 4-level reward (syntax, distribution alignment, expectation value, optimization progress) |
| | - **Primary Domain:** Quantum circuit generation and quantum optimization algorithm design |
| |
|
| | --- |
| |
|
| | ## Model Details |
| |
|
| | - **Model type:** LLM fine-tuned with reinforcement learning |
| | - **Languages:** English |
| | - **License:** Apache-2.0 |
| | - **Base model:** Qwen/Qwen3-4B-Instruct-2507 |
| |
|
| | --- |
| |
|
| | ## Uses |
| |
|
| | ### Direct Use |
| | - Generate OpenQASM 3.0 code from natural language descriptions |
| | - Design ansatz circuits for quantum optimization tasks (QAOA, VQE) |
| |
|
| | ### Downstream Use |
| | - Integration into quantum compilers |
| | - Research on LLM-guided quantum algorithm design |
| |
|
| | --- |
| |
|
| | ## Bias, Risks, and Limitations |
| |
|
| | - May produce a valid QASM that is semantically weak if prompts are ambiguous |
| | - Tailored primarily to **graph-based quantum optimization problems** |
| | - Evaluated mainly in simulation; hardware generalization remains untested |
| |
|
| | **Recommendation:** Always verify generated circuits with independent quantum simulators or compilers before deployment. |
| |
|
| | --- |
| |
|
| | ## How to Get Started |
| |
|
| | ```python |
| | from transformers import AutoModelForCausalLM, AutoTokenizer |
| | |
| | model = AutoModelForCausalLM.from_pretrained("Benyucong/rl_quantum_4b") |
| | tokenizer = AutoTokenizer.from_pretrained("Benyucong/rl_quantum_4b") |
| | |
| | prompt = """Design a QASM 3.0 quantum circuit with 3 qubits and 3 layers to solve the vertex_cover \ |
| | given the graph: {"directed": false, "multigraph": false, "graph": {}, "nodes": [{"id": 0}, {"id": 1}, {"id": 2}], \ |
| | "edges": [{"source": 0, "target": 1}, {"source": 0, "target": 2}, {"source": 1, "target": 2}]}. \ |
| | Provide valid QASM 3.0 code with optimal parameters.""" |
| | |
| | inputs = tokenizer(prompt, return_tensors="pt") |
| | outputs = model.generate(**inputs, max_new_tokens=1024) |
| | print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| | ``` |
| |
|
| | --- |
| |
|
| | ## Training Details |
| |
|
| | ### Training Data |
| | - Dataset: [Benyucong/graph-data-quantum-rl](https://huggingface.co/datasets/Benyucong/graph-data-quantum-rl) |
| | - Contains QASM 3.0 circuits, Hamiltonians, eigenvalues, and parameterized circuits for **12 quantum optimization problems** |
| |
|
| | ### Training Setup |
| | - **Stage 1:** Supervised fine-tuning (SFT), the model is available [here](https://huggingface.co/Benyucong/sft_quantum_circuit_gen_4B). |
| | - **Stage 2:** Reinforcement learning with GRPO and hierarchical reward |
| |
|
| | ### Hyperparameters |
| | - **Batch size:** 128 |
| | - **Rollouts:** 16 per prompt (temperature = 0.7, top-p = 0.8) |
| | - **Precision:** bf16 mixed precision |
| | - **GPUs:** 16 × H100-64GB (FSDP enabled) |
| | - **Training time:** ~48 hours |
| |
|
| | --- |
| |
|
| | ## Evaluation |
| |
|
| | ### Metrics (Please check our paper for details) |
| | - **SCR:** Syntactic Correctness Ratio |
| | - **SREV:** Successful Rate of Expectation Value |
| | - **RE:** Relative Entropy (distributional alignment) |
| | - **HQCR:** High-Quality Circuit Ratio |
| |
|
| | ### Results (QUASAR vs Baselines) |
| |
|
| | | Method | Pass@1 SCR ↑ | Pass@1 SREV ↑ | Pass@1 RE ↓ | Pass@1 HQCR ↑ | Pass@10 SCR ↑ | Pass@10 SREV ↑ | Pass@10 RE ↓ | Pass@10 HQCR ↑ | |
| | |---------------------|--------------|---------------|-------------|---------------|---------------|----------------|--------------|----------------| |
| | | DeepSeek-V3 | 94.83% | 12.24% | 19.20 | 10.00% | 98.97% | 26.38% | 16.39 | 16.38% | |
| | | GPT-5 | 87.07% | 10.00% | 19.94 | 6.90% | 90.52% | 27.07% | 11.57 | 16.55% | |
| | | GPT-4o | 87.93% | 9.83% | 19.42 | 6.38% | 88.79% | 18.62% | 14.08 | 12.07% | |
| | | **Qwen3-4B SFT** | 97.41% | 18.97% | 12.74 | 15.17% | 99.65% | 31.55% | 10.81 | 23.62% | |
| | | Cold Start GRPO | 84.48% | 19.84% | 14.32 | 12.41% | 95.17% | 27.59% | 11.38 | 18.96% | |
| | | **QUASAR (ours)** | **99.31%** | **22.41%** | **11.61** | **17.24%** | **100%** | **33.10%** | **8.48** | **27.24%** | |
| |
|
| | --- |
| |
|
| | ## Environmental Impact |
| | - **Hardware Type:** NVIDIA H100 (16×, 64GB) |
| | - **Training Hours:** ~48 |
| |
|
| | --- |
| |
|
| | ## Technical Specifications |
| |
|
| | - **Architecture:** Qwen3-4B-Instruct-2507 |
| | - **Fine-tuning:** SFT + RL (GRPO) |
| | - **Reward Design:** Syntax validity, distributional alignment (JS distance), expectation-value matching, optimization-progress efficiency |
| | - **Frameworks:** PyTorch, vLLM, Qiskit, OpenQASM |
| |
|
| | --- |
| |
|
| | ## Citation |
| | ```bibtex |
| | @misc{yu2025quasarquantumassemblycode, |
| | title={QUASAR: Quantum Assembly Code Generation Using Tool-Augmented LLMs via Agentic RL}, |
| | author={Cong Yu and Valter Uotila and Shilong Deng and Qingyuan Wu and Tuo Shi and Songlin Jiang and Lei You and Bo Zhao}, |
| | year={2025}, |
| | eprint={2510.00967}, |
| | archivePrefix={arXiv}, |
| | primaryClass={cs.AI}, |
| | url={https://arxiv.org/abs/2510.00967}, |
| | } |
| | ``` |
