metadata
license: apache-2.0
language:
- en
base_model:
- state-spaces/mamba2-130m
pipeline_tag: question-answering
Single-Pass Scanner
This repository contains model checkpoint for Single-Pass Scanner
The model architecture is built upon mamba, and is trained from mamba2-130m
Usage
We highly recommend creating a new conda environment first:
conda create -n mamba_retriever python=3.10.14
conda activate mamba_retriever
Then, run the following in your terminal:
git clone https://github.com/state-spaces/mamba.git
conda install cudatoolkit==11.8 -c nvidia
pip install -r requirements.txt
pip3 install torch==2.1.1 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install accelerate -U
cd mamba
pip install .
Next, download and install the following two files from https://github.com/state-spaces/mamba/releases and https://github.com/Dao-AILab/causal-conv1d/releases:
mamba_ssm-2.2.2+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
causal_conv1d-1.4.0+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
You can install them using
pip install mamba_ssm-2.2.2+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
pip install causal_conv1d-1.4.0+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
Evaluation
All evaluation code and details are available at Single-Pass Scanner Github