README.md

metadata

license: cc-by-nc-4.0
language:
  - en
tags:
  - query-decomposition
  - retrieval
  - rag
  - baselines

Query Decomposition Baselines

复杂问题「查询分割 / 查询分解」的 4 个 baseline 方法，代码 + 权重一体，git clone 后 （自备基座 Llama）即可直接以 Python 函数调用运行。中文完整说明见仓库内 **使用说明.md**。

Four query-decomposition baselines that split a complex question into sub-questions. This repo ships both the code and the weights, so you can run them directly.

#	method	call	needs
①	supervised (LoRA Llama)	supervised_decompose(q)	base Llama-3.1-8B-Instruct (bring your own) + the LoRA in weights/
②	unsupervised (XLM)	unsupervised_decompose(q)	the XLM ckpt in weights/ + GPU
③	icl (Socratic, gpt-4o-mini)	icl_decompose(q)	an API key, or plug your own llm_fn
④	icl+feedback (SearChain)	service (see 使用说明.md §4)	API key + a ColBERT index

①②③ are plain in-process Python functions — no servers, no ports, no HTTP.

What's inside

decompose.py              # unified entry: query -> list[str] sub-queries
qd_baselines/             # in-process implementations (①②③), lazy-loaded
methods/  examples/  start/  reference/
weights/
  decom_baseline_strategyqa_lora/   # ① supervised LoRA adapter (~120MB)
  xlm_unsup_decomp.pth              # ② XLM checkpoint (~3.1GB)

The code auto-discovers the weights under weights/, so after cloning you only need to provide the base Llama (gated, not redistributed here) for method ①.

Quick start

git lfs install
git clone https://huggingface.co/Veblen34/Query-decompose-baselines
cd Query-decompose-baselines
pip install -r requirements.txt

# ② unsupervised — weights auto-found in weights/, just needs a GPU:
python -c "from decompose import unsupervised_decompose; \
print(unsupervised_decompose('which magazine was started first arthur\\'s magazine or first for women ?'))"

# ① supervised — also provide the gated base model:
export RAGQA_LLAMA_MODEL=/path/to/Llama-3.1-8B-Instruct   # from meta-llama on the Hub
python -c "from decompose import supervised_decompose; \
print(supervised_decompose('Which team did the player with the most NBA championships play for?'))"

# ③ icl — set an API key (or pass your own llm_fn to stay fully local):
export CHATANYWHERE_API_KEY=...   # gpt-4o-mini
python -c "from decompose import icl_decompose; \
print(icl_decompose('Who directed the highest-grossing film of 1997?'))"

Batch over your own file (one query per line):

python examples/demo_decompose.py --method icl --input examples/sample_queries.txt --output out.jsonl

Weights

• weights/decom_baseline_strategyqa_lora/ — the supervised query-decomposition LoRA adapter (rank 64, targets q/k/v/o proj). Requires the base meta-llama/Llama-3.1-8B-Instruct, which is gated and NOT included — obtain it yourself from the Meta repo and set RAGQA_LLAMA_MODEL.
• weights/xlm_unsup_decomp.pth — the XLM seq2seq unsupervised-decomposition checkpoint (self-contained; dictionary embedded). Derived from Facebook UnsupervisedDecomposition/XLM; provided for research reproduction.

License / notes

Set to cc-by-nc-4.0 (the most restrictive component: the XLM-derived checkpoint is non-commercial). The LoRA adapter is our own trained artifact. The base Llama model is subject to Meta's own license and is not redistributed here. Adjust the license field if your usage differs.

Full Chinese usage guide: 使用说明.md.