necrashter/interchangeable-token-embeddings

Interchangeable Token Embeddings

This repository contains the models for the ICML 2025 paper, Interchangeable Token Embeddings for Extendable Vocabulary and Alpha-Equivalence.

Links:

1. arXiv
2. Project Page
3. Code

Models

Figure 3a (LTL):

Model	Path
Proposed Method	models/ltl-5/d005-rop-bn1-fn1-ada1-s42
Baseline	models/adacos/nondyn-rope
Full-Vocabulary	models/10ap/adacos-nondyn
Alpha-Renaming	models/ltl-5/s010-rop-bn1-fn1-ada1-s46

Figure 3b (Propositional Logic):

Model	Path
Proposed Method	models-prop/5ap/d020-rop-bn1-fn1-ada1-s44
Baseline	models-prop/5ap/0000-rop-bn0-fn1-ada1-s42
Full-Vocabulary	models-prop/10ap/0000-rop-bn0-fn1-ada1-s43
Alpha-Renaming	models-prop/5ap/s010-rop-bn1-fn1-ada1-s44

Figure 5a (Ablation - LTL):

Model	Path
w/o Adacos	models/ltl-5/d005-rop-bn1-fn1-ada0-s42
w/o $f_{{fn}}$ & AdaCos	models/ltl-5/d005-rop-bn1-fn0-ada0-s42
w/o $f_{{bn}}$	models/ltl-5/d005-rop-bn0-fn1-ada1-s42

Figure 5b (Ablation - Propositional Logic):

Model	Path
w/o Adacos	models-prop/5ap/d020-rop-bn1-fn1-ada0-s43
w/o $f_{{fn}}$ & AdaCos	models-prop/5ap/d020-rop-bn1-fn0-ada0-s44
w/o $f_{{bn}}$	models-prop/5ap/d020-rop-bn0-fn1-ada1-s42

Perturbation Experiments

Table 2 and 7 (LTL):

Model	Path
Normal Baseline	models/adacos/nondyn-rope
Perturbed Baseline	models/adacos/nondyn-rope-perturbed
Perturbed Alpha-Renaming	models/adacos/s005-rope-perturbed-s42
Perturbed Proposed	models/adacos/diagbor4-rope-perturbed
Limited Baseline	models/adacos/nondyn-rope-limited
Limited Alpha-Renaming	models/adacos/s005-rope-limited-s42
Limited Proposed	models/adacos/diagbor4-rope-limited

Table 8 (Propositional Logic):

Model	Path
Normal Baseline	models-prop/5ap/0000-rop-bn0-fn1-ada1-s42
Perturbed Baseline	models-prop/5ap/base-rop-bn0-fn1-ada1-s44
Perturbed Alpha-Renaming	models-prop/5ap/s005-rop-bn1-fn1-ada1-s43
Perturbed Proposed	models-prop/5ap/d020-rop-bn1-fn1-ada1-s42