feat-add-configs (#18)

fix: handle window_size passed as list

fix-glu-mlp (#17)

feat: for converting v2, added lines to save model weights and print config

Porting v2 models to flash attention (#15)

feat: updated activation checkpointing (#14)

fix: use staticmethod istead of classmethod

feat: added comment

feat: added docstrings

feat: Allow LoRA to be merged into weights (#12)

feat: added README

removed num_tasks from config

feat: removed tokenizer

feat: removed task type embeddings

fix: remove cleaving (#13)

fix: added trust_remote_code to tokenizer

fix: fixed from_bert method

fix: fix LoRA implementation

feat: cleave off layers from encoder (#11)

feat: only apply select_task_for_layer if task has changed

feat: make num of loras part of the config

feat: make main parameters trainable

fix BertForMaskedLM

feat: added separate BertForMaskedLM class

Update tokenizer.py

feat-add-constant-for-task-type-ids (#10)

feat: added return_dict

fix: fixed syntax error in LoRA

feat: add current_task to forward

feat: use property in LoRA parametrization

feat: added LoRA copyright notice

feat: use property instead of setter

clean up embeddings.py (#7)

feat: return from_bert for from_pretrained

support-fast-tokenizer (#6)

feat: made from_bert work

feat: choose flash attention heuristically if not set explicitly

feat: select first LoRA upon initialization

feat: formatting and type hints

support-multiple-task-ids (#5)

fix: use proper initilization for embedding layer

fix: fixed typo

feat: added LoRA

feat: assert return_dict

fix: same assertions in other models

fix: assert is None for other kwargs too

feat: added head_mask

added classifier dropout

fix: formatting

fix: formatting