wyangw/xbridge-llama2qwen

XBridge: Latent Enrichment Bridge (Llama-3.1-8B → Qwen2.5-7B)

Pretrained bridge weights for XBridge, a decode-free communication protocol for heterogeneous multi-agent LLM systems. A frozen sender LLM processes context and transmits knowledge to a frozen receiver LLM of a different architecture, with no text exchanged.

This checkpoint is the Latent Enrichment Bridge (LEB) module for the Llama-3.1-8B-Instruct (sender) → Qwen2.5-7B-Instruct (receiver) pair, the main result reported in the paper. Both base models remain frozen; only these cross-attention modules are trained.

• 4 gated cross-attention modules, inserted after receiver layers {6, 13, 20, 27} (0-indexed, out of 28)
• ~264M trainable parameters (3.8% of the receiver)
• Trained on a balanced 587-sample set (see paper for details)

Code, training/eval scripts, and the LAM (Lexical Anchor Mapping) component are at: https://github.com/WooseongYang/XBridge

Usage

pip install huggingface_hub
huggingface-cli download wyangw/xbridge-llama2qwen best_model.pt --local-dir checkpoints/bridge_c_only_balanced_fixed

Then evaluate following Option A in the repo README:

python scripts/eval_llama2qwen_c_only_all_tasks.py \
    --bridge_ckpt checkpoints/bridge_c_only_balanced_fixed/best_model.pt \
    --tasks hotpotqa

Files

• best_model.pt — model state dict + training metadata (step, epoch, val_ntp)
  • embedded architecture config (arch_config)
• arch_config.json — architecture config in plain JSON, for reference