| | ---
|
| | license: apache-2.0
|
| | ---
|
| |
|
| | # Interchangeable Token Embeddings
|
| |
|
| | This repository contains the models for the ICML 2025 paper, [Interchangeable Token Embeddings for Extendable Vocabulary and Alpha-Equivalence](https://arxiv.org/abs/2410.17161).
|
| |
|
| | Links:
|
| | 1. [arXiv](https://arxiv.org/abs/2410.17161)
|
| | 2. [Project Page](https://necrashter.github.io/interchangeable-token-embeddings/)
|
| | 3. [Code](https://github.com/necrashter/interchangeable-token-embeddings)
|
| |
|
| | ## Models
|
| |
|
| | Figure 3a (LTL):
|
| | | Model | Path |
|
| | | ----- | ---- |
|
| | | Proposed Method | `models/ltl-5/d005-rop-bn1-fn1-ada1-s42` |
|
| | | Baseline | `models/adacos/nondyn-rope` |
|
| | | Full-Vocabulary | `models/10ap/adacos-nondyn` |
|
| | | Alpha-Renaming | `models/ltl-5/s010-rop-bn1-fn1-ada1-s46` |
|
| |
|
| | Figure 3b (Propositional Logic):
|
| | | Model | Path |
|
| | | ----- | ---- |
|
| | | Proposed Method | `models-prop/5ap/d020-rop-bn1-fn1-ada1-s44` |
|
| | | Baseline | `models-prop/5ap/0000-rop-bn0-fn1-ada1-s42` |
|
| | | Full-Vocabulary | `models-prop/10ap/0000-rop-bn0-fn1-ada1-s43` |
|
| | | Alpha-Renaming | `models-prop/5ap/s010-rop-bn1-fn1-ada1-s44` |
|
| |
|
| | Figure 5a (Ablation - LTL):
|
| | | Model | Path |
|
| | | ----- | ---- |
|
| | | w/o Adacos | `models/ltl-5/d005-rop-bn1-fn1-ada0-s42` |
|
| | | w/o $f_{{fn}}$ & AdaCos | `models/ltl-5/d005-rop-bn1-fn0-ada0-s42` |
|
| | | w/o $f_{{bn}}$ | `models/ltl-5/d005-rop-bn0-fn1-ada1-s42` |
|
| |
|
| | Figure 5b (Ablation - Propositional Logic):
|
| | | Model | Path |
|
| | | ----- | ---- |
|
| | | w/o Adacos | `models-prop/5ap/d020-rop-bn1-fn1-ada0-s43` |
|
| | | w/o $f_{{fn}}$ & AdaCos | `models-prop/5ap/d020-rop-bn1-fn0-ada0-s44` |
|
| | | w/o $f_{{bn}}$ | `models-prop/5ap/d020-rop-bn0-fn1-ada1-s42` |
|
| |
|
| | ### Perturbation Experiments
|
| |
|
| | Table 2 and 7 (LTL):
|
| | | Model | Path |
|
| | | ----- | ---- |
|
| | | Normal Baseline | `models/adacos/nondyn-rope` |
|
| | | Perturbed Baseline | `models/adacos/nondyn-rope-perturbed` |
|
| | | Perturbed Alpha-Renaming | `models/adacos/s005-rope-perturbed-s42` |
|
| | | Perturbed Proposed | `models/adacos/diagbor4-rope-perturbed` |
|
| | | Limited Baseline | `models/adacos/nondyn-rope-limited` |
|
| | | Limited Alpha-Renaming | `models/adacos/s005-rope-limited-s42` |
|
| | | Limited Proposed | `models/adacos/diagbor4-rope-limited` |
|
| |
|
| | Table 8 (Propositional Logic):
|
| | | Model | Path |
|
| | | ----- | ---- |
|
| | | Normal Baseline | `models-prop/5ap/0000-rop-bn0-fn1-ada1-s42` |
|
| | | Perturbed Baseline | `models-prop/5ap/base-rop-bn0-fn1-ada1-s44` |
|
| | | Perturbed Alpha-Renaming | `models-prop/5ap/s005-rop-bn1-fn1-ada1-s43` |
|
| | | Perturbed Proposed | `models-prop/5ap/d020-rop-bn1-fn1-ada1-s42` |
|
| |
|
