---
license: mit
datasets:
- jg-eno/msmarco-v5.1-Qwen-Embeddings
- microsoft/ms_marco
language:
- en
base_model:
- Qwen/Qwen3-0.6B
- Qwen/Qwen3-Embedding-0.6B
pipeline_tag: feature-extraction
library_name: transformers
tags:
- embedding-inversion
- text-generation-inference
- peft
- lora
---

# Qwen Embedding Inverter

Reconstructs natural-language text from a sentence embedding alone — no access to the original text at inference time.

A set of per-position MLPs maps a single 1024-dim sentence embedding (from `Qwen3-Embedding-0.6B`) into a sequence of soft prefix tokens, which condition a LoRA-adapted `Qwen3-0.6B` decoder to regenerate semantically equivalent text.

**Code**: [Semantic-Embedding-Reconstruction](https://github.com/jg-eno/Semantic-Embedding-Reconstruction)

## Training data

Trained on passages from [`microsoft/ms_marco`](https://huggingface.co/datasets/microsoft/ms_marco) (v1.1 / v2.1), pre-encoded into sentence and token embeddings and published as [`jg-eno/msmarco-v5.1-Qwen-Embeddings`](https://huggingface.co/datasets/jg-eno/msmarco-v5.1-Qwen-Embeddings).

## Intended use & limitations

- Built as a research artifact for studying how much information a single dense sentence embedding retains, and whether that information is recoverable as text — relevant to embedding-inversion / privacy-leakage research on embedding-based retrieval systems.
- Only tested on short MS MARCO-style passages (≤128 tokens). Reconstruction quality on out-of-domain or much longer text is unverified.
- This is **not** a general-purpose text generator. The decoder only produces coherent output when conditioned on a valid prefix from the paired MLPs — it is not meant to be used as a standalone causal LM.
- Reconstructions approximate the semantic content of the original text; they are not guaranteed to recover exact wording.