metadata
license: mit
datasets:
- jg-eno/msmarco-v5.1-Qwen-Embeddings
- microsoft/ms_marco
language:
- en
base_model:
- Qwen/Qwen3-0.6B
- Qwen/Qwen3-Embedding-0.6B
pipeline_tag: feature-extraction
library_name: transformers
tags:
- embedding-inversion
- text-generation-inference
- peft
- lora
Qwen Embedding Inverter
Reconstructs natural-language text from a sentence embedding alone — no access to the original text at inference time.
A set of per-position MLPs maps a single 1024-dim sentence embedding (from Qwen3-Embedding-0.6B) into a sequence of soft prefix tokens, which condition a LoRA-adapted Qwen3-0.6B decoder to regenerate semantically equivalent text.
Code: Semantic-Embedding-Reconstruction
Training data
Trained on passages from microsoft/ms_marco (v1.1 / v2.1), pre-encoded into sentence and token embeddings and published as jg-eno/msmarco-v5.1-Qwen-Embeddings.
Intended use & limitations
- Built as a research artifact for studying how much information a single dense sentence embedding retains, and whether that information is recoverable as text — relevant to embedding-inversion / privacy-leakage research on embedding-based retrieval systems.
- Only tested on short MS MARCO-style passages (≤128 tokens). Reconstruction quality on out-of-domain or much longer text is unverified.
- This is not a general-purpose text generator. The decoder only produces coherent output when conditioned on a valid prefix from the paired MLPs — it is not meant to be used as a standalone causal LM.
- Reconstructions approximate the semantic content of the original text; they are not guaranteed to recover exact wording.