anonymoussubmission111 commited on
Commit
eafd7b6
·
verified ·
1 Parent(s): a60f707

Add root README for anonymous submission

Browse files
Files changed (1) hide show
  1. README.md +52 -0
README.md ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - lora
4
+ - peft
5
+ - transformers
6
+ - retrieval
7
+ - embeddings
8
+ license: apache-2.0
9
+ ---
10
+
11
+ # Checkpoints for "Improving Long-Context Retrieval with Multi-Prefix Embedding"
12
+
13
+ This repository contains model checkpoints and pre-computed embeddings for the anonymous submission *Improving Long-Context Retrieval with Multi-Prefix Embedding*.
14
+
15
+ ## Repository Structure
16
+
17
+ ```
18
+ models/
19
+ fixed-64-epoch1/ # Ablation: fixed 64-token prefix length
20
+ maxp-train-epoch1/ # Baseline: MaxP trained model
21
+ nochunk-epoch1/ # Baseline: single-vector (no chunking)
22
+ prand-32to1024-epoch1/ # Proposed: random prefix lengths (32-1024 tokens)
23
+
24
+ encode/
25
+ browsecomp-plus/ # Pre-computed embeddings - BrowseComp-Plus
26
+ longembed/ # Pre-computed embeddings - LongEmbed (2WikiMQA,
27
+ | # NarrativeQA, QMSum, SummScreenFD)
28
+ mldr-en/ # Pre-computed embeddings - MLDR (English)
29
+ ```
30
+
31
+ Each model folder contains a LoRA adapter (rank 16, alpha 64) fine-tuned from `Qwen/Qwen3-Embedding-0.6B`
32
+ for feature extraction, along with tokenizer files and a `checkpoint-625/` subfolder with the
33
+ intermediate checkpoint at the end of epoch 1 (including optimizer state).
34
+
35
+ ## Usage
36
+
37
+ Load a LoRA adapter with [PEFT](https://github.com/huggingface/peft):
38
+
39
+ ```python
40
+ from peft import PeftModel
41
+ from transformers import AutoModel
42
+
43
+ base = AutoModel.from_pretrained("Qwen/Qwen3-Embedding-0.6B")
44
+ model = PeftModel.from_pretrained(
45
+ base,
46
+ "anonymoussubmission111/mpe-checkpoints",
47
+ subfolder="models/prand-32to1024-epoch1",
48
+ )
49
+ ```
50
+
51
+ Pre-computed embeddings in `encode/` are stored as `.pkl` files (pickled numpy arrays)
52
+ and can be loaded directly to reproduce retrieval results without re-encoding.