Mixtral multipack (#928)

fixing prompt template of chatml by removal of linebreak (#922)

support for mamba (#915)

chore: clarify Readme on sharegpt system role

fix(tokenizer): handle fast tokenizer properly for bos/eos (#914)

Pin flash-attn to 2.3.3 (#919)

feat: add check for quantized model (#913)

Support device_map=sequential & max_memory config parameters (#903)

Feat(wandb): Refactor to be more flexible (#767)

feature: loss watchdog for terminating training runs that are failing (#899)

Remove learning rate scheduler in deepspeed config to avoid conflict (#909)

fix for qwen w lora (#906)

ensure merged model matches the training dtype (#902)

Determine FSDP/deepspeed settings on device select. (#883)

fix: remove FA for qwen examples (#900)

update datasets version to cut down the warnings due to pyarrow arg change (#897)

Feat: Add Qwen (#894)

fix: warning should not show if eval_batch_size not provided (#896)

Feat: Add warmup_ratio (#893)

chore(doc): Add info on changing role in sharegpt (#886)

fix: revert local dir dataset load (#878)

Install from git url (#874)

Phi update 202311 (#876)

don't train if eval split is too small (#873)

try #2: pin hf transformers and accelerate to latest release, don't reinstall pytorch (#867)

Feat: Add dataset loading from S3, GCS (#765)

allow overriding of model_config parameters from the YML (#853)

add e2e tests for checking functionality of resume from checkpoint (#865)

make docker command more robust (#861)

lint fix that didn't get caught by linter (#866)

Update data.py for signature generation (#851)

Docs: add instructions to 1-click launching on public clouds (#862)

multipack len should use max, not min (#863)

adds llama and mistral dropout support (#858)

various bugfixes (#856)

chore(doc): Separate section on runpod (#860)

feat(doc): add more info on train_on_split (#855)

include the suffix modified string in ascii art (#852)

cleanup the old multipack dataloader (#841)

Pin optimum package (#838)

don't compile deepspeed or bitsandbytes from source (#837)

multipack w batch sampler (#795)

use temp_dir kwarg instead

missing dunder-init

chore: lint

make sure to cleanup tmp output_dir for e2e tests

use accelerate logging for zero/main loggin only

cleanup verbosity a bit

add deepspeed-kernels dependency for deepspeed>=0.12.0 (#827)

Feat: Added Gradio support (#812)