| --- |
| license: mit |
| datasets: |
| - jg-eno/msmarco-v5.1-Qwen-Embeddings |
| - microsoft/ms_marco |
| language: |
| - en |
| base_model: |
| - Qwen/Qwen3-0.6B |
| - Qwen/Qwen3-Embedding-0.6B |
| pipeline_tag: feature-extraction |
| library_name: transformers |
| tags: |
| - embedding-inversion |
| - text-generation-inference |
| - peft |
| - lora |
| --- |
| |
| # Qwen Embedding Inverter |
|
|
| Reconstructs natural-language text from a sentence embedding alone — no access to the original text at inference time. |
|
|
| A set of per-position MLPs maps a single 1024-dim sentence embedding (from `Qwen3-Embedding-0.6B`) into a sequence of soft prefix tokens, which condition a LoRA-adapted `Qwen3-0.6B` decoder to regenerate semantically equivalent text. |
|
|
| **Code**: [Semantic-Embedding-Reconstruction](https://github.com/jg-eno/Semantic-Embedding-Reconstruction) |
|
|
| ## Training data |
|
|
| Trained on passages from [`microsoft/ms_marco`](https://huggingface.co/datasets/microsoft/ms_marco) (v1.1 / v2.1), pre-encoded into sentence and token embeddings and published as [`jg-eno/msmarco-v5.1-Qwen-Embeddings`](https://huggingface.co/datasets/jg-eno/msmarco-v5.1-Qwen-Embeddings). |
|
|
| ## Intended use & limitations |
|
|
| - Built as a research artifact for studying how much information a single dense sentence embedding retains, and whether that information is recoverable as text — relevant to embedding-inversion / privacy-leakage research on embedding-based retrieval systems. |
| - Only tested on short MS MARCO-style passages (≤128 tokens). Reconstruction quality on out-of-domain or much longer text is unverified. |
| - This is **not** a general-purpose text generator. The decoder only produces coherent output when conditioned on a valid prefix from the paired MLPs — it is not meant to be used as a standalone causal LM. |
| - Reconstructions approximate the semantic content of the original text; they are not guaranteed to recover exact wording. |
|
|
