---
license: mit
---


## SparQLe – Speech Queries to Text via Instruction‑Tuned LLM ⚡

**What it does:**
SparQLe (Speech Routing to Query LLMs) enables direct speech-to-text understanding by aligning self‑supervised speech representations (e.g., HuBERT-like features) with instruction‑tuned Large Language Models (LLMs). This is achieved using a lightweight *modality adapter*, bridging the modalities without retraining the whole LLM. ([Moonlight][1])

**Key strengths:**

* **Preserves semantic content** of spoken input in the produced text
* **Efficiently leverages frozen SSL models**, avoiding heavy ASR backbones like Whisper
* **Modular design** with a query‑former (Q‑former) adapter and LLM backend

**Architecture:**

1. **Speech encoder** (SSL) transforms raw input into latent features.
2. **Modality adapter / Q‑former** aligns these with the LLM’s text embedding space.
3. **Instruction‑tuned LLM** processes the adapted input to generate semantic text.


## Citation

If you use SparQLe in your research, please cite:

```bibtex
@misc{djanibekov2025sparqlespeechqueriestext,
      title={SparQLe: Speech Queries to Text Translation Through LLMs},
      author={Amirbek Djanibekov and Hanan Aldarmaki},
      year={2025},
      eprint={2502.09284},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.09284},
}
```

📄 Read the full paper on arXiv: [https://arxiv.org/abs/2502.09284](https://arxiv.org/abs/2502.09284)

---

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

---

## Acknowledgments

- This work builds upon [fairseq](https://github.com/facebookresearch/fairseq) 💙
- The Qformer architecture is inspired by [BLIP-2](https://github.com/salesforce/BLIP-2) ✨