File size: 707 Bytes
4ca4e4c | 1 2 3 4 5 6 7 | ## Expressivity-Efficiency Tradeoffs for Hybrid Sequence Models
This repository includes all of the experiments relevant to the associated paper.
- `micro`: A collection of old experiments with pytorch transformers and mamba. Not currently used, but might be useful as a reference.
- `micro_hf`: A collection of experiments for micro-scale experiments, using huggingface models. `run.ipynb` will execute the experiments, and `process.ipynb` can be used to analyze the results. Each individual run does not take long to run on an RTX 4090.
- `mini`: A collection of larger experiments, mostly ported from the ones in the Repeat After Me paper. Added to this are hybrid models implemented in huggingface. |