File size: 2,524 Bytes
00ac9ee 2d0e92d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | ---
license: apache-2.0
---
# Interchangeable Token Embeddings
This repository contains the models for the ICML 2025 paper, [Interchangeable Token Embeddings for Extendable Vocabulary and Alpha-Equivalence](https://arxiv.org/abs/2410.17161).
Links:
1. [arXiv](https://arxiv.org/abs/2410.17161)
2. [Project Page](https://necrashter.github.io/interchangeable-token-embeddings/)
3. [Code](https://github.com/necrashter/interchangeable-token-embeddings)
## Models
Figure 3a (LTL):
| Model | Path |
| ----- | ---- |
| Proposed Method | `models/ltl-5/d005-rop-bn1-fn1-ada1-s42` |
| Baseline | `models/adacos/nondyn-rope` |
| Full-Vocabulary | `models/10ap/adacos-nondyn` |
| Alpha-Renaming | `models/ltl-5/s010-rop-bn1-fn1-ada1-s46` |
Figure 3b (Propositional Logic):
| Model | Path |
| ----- | ---- |
| Proposed Method | `models-prop/5ap/d020-rop-bn1-fn1-ada1-s44` |
| Baseline | `models-prop/5ap/0000-rop-bn0-fn1-ada1-s42` |
| Full-Vocabulary | `models-prop/10ap/0000-rop-bn0-fn1-ada1-s43` |
| Alpha-Renaming | `models-prop/5ap/s010-rop-bn1-fn1-ada1-s44` |
Figure 5a (Ablation - LTL):
| Model | Path |
| ----- | ---- |
| w/o Adacos | `models/ltl-5/d005-rop-bn1-fn1-ada0-s42` |
| w/o $f_{{fn}}$ & AdaCos | `models/ltl-5/d005-rop-bn1-fn0-ada0-s42` |
| w/o $f_{{bn}}$ | `models/ltl-5/d005-rop-bn0-fn1-ada1-s42` |
Figure 5b (Ablation - Propositional Logic):
| Model | Path |
| ----- | ---- |
| w/o Adacos | `models-prop/5ap/d020-rop-bn1-fn1-ada0-s43` |
| w/o $f_{{fn}}$ & AdaCos | `models-prop/5ap/d020-rop-bn1-fn0-ada0-s44` |
| w/o $f_{{bn}}$ | `models-prop/5ap/d020-rop-bn0-fn1-ada1-s42` |
### Perturbation Experiments
Table 2 and 7 (LTL):
| Model | Path |
| ----- | ---- |
| Normal Baseline | `models/adacos/nondyn-rope` |
| Perturbed Baseline | `models/adacos/nondyn-rope-perturbed` |
| Perturbed Alpha-Renaming | `models/adacos/s005-rope-perturbed-s42` |
| Perturbed Proposed | `models/adacos/diagbor4-rope-perturbed` |
| Limited Baseline | `models/adacos/nondyn-rope-limited` |
| Limited Alpha-Renaming | `models/adacos/s005-rope-limited-s42` |
| Limited Proposed | `models/adacos/diagbor4-rope-limited` |
Table 8 (Propositional Logic):
| Model | Path |
| ----- | ---- |
| Normal Baseline | `models-prop/5ap/0000-rop-bn0-fn1-ada1-s42` |
| Perturbed Baseline | `models-prop/5ap/base-rop-bn0-fn1-ada1-s44` |
| Perturbed Alpha-Renaming | `models-prop/5ap/s005-rop-bn1-fn1-ada1-s43` |
| Perturbed Proposed | `models-prop/5ap/d020-rop-bn1-fn1-ada1-s42` |
|