README.md

---
license: apple-amlr
base_model:
- mistralai/Mistral-7B-Instruct-v0.2
tags:
- rag
- compression
- retrieval
- end-to-end
- generation
---

# CLaRa-7B-E2E (Compression-16 & 128)

The **CLaRa-7B-E2E** model is our fully end-to-end unified RAG model, jointly optimizing retrieval and generation with 16× and 128x document compression.

**Training recipe:** End-to-end finetuning with differentiable top-k retrieval and a unified language-modeling objective.  
**Benchmarks:** Strong retrieval-augmented QA performance under aggressive compression.

---

## More details and usage examples:

Paper: [CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning](https://arxiv.org/abs/2511.18659)  
GitHub: https://github.com/apple/ml-clara

---

## Example Usage (End-to-End Inference)

```python
from transformers import AutoModel

unirag = AutoModel.from_pretrained(
    "/mnt/ceph_rbd/model/CLaRa-7B-E2E/compression-16",
    trust_remote_code=True
).to("cuda")

# Example documents and question
documents = [[
    "Weldenia is a monotypic genus of flowering plant in the family Commelinaceae...",
] * 20]

questions = [
    "Which genus of plant grows originally in Mexico and Guatemala, Phylica or Weldenia?"
]

# End-to-end usage (retrieval + generation)
# The effective top-k is controlled by `generation_top_k` in config.json.
out = unirag.generate_from_questions(
    questions=questions,
    documents=documents,
    max_new_tokens=64
)

print("Generated answer", out)