# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("jg-eno/ReLoDer_v2", dtype="auto")Quick Links
Qwen Embedding Inverter
Reconstructs natural-language text from a sentence embedding alone — no access to the original text at inference time.
A set of per-position MLPs maps a single 1024-dim sentence embedding (from Qwen3-Embedding-0.6B) into a sequence of soft prefix tokens, which condition a LoRA-adapted Qwen3-0.6B decoder to regenerate semantically equivalent text.
Code: Semantic-Embedding-Reconstruction
Training data
Trained on passages from microsoft/ms_marco (v1.1 / v2.1), pre-encoded into sentence and token embeddings and published as jg-eno/msmarco-v5.1-Qwen-Embeddings.
Intended use & limitations
- Built as a research artifact for studying how much information a single dense sentence embedding retains, and whether that information is recoverable as text — relevant to embedding-inversion / privacy-leakage research on embedding-based retrieval systems.
- Only tested on short MS MARCO-style passages (≤128 tokens). Reconstruction quality on out-of-domain or much longer text is unverified.
- This is not a general-purpose text generator. The decoder only produces coherent output when conditioned on a valid prefix from the paired MLPs — it is not meant to be used as a standalone causal LM.
- Reconstructions approximate the semantic content of the original text; they are not guaranteed to recover exact wording.
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="jg-eno/ReLoDer_v2")