feat: vmap optimizer (#166)

feat: scan layers + gradient checkpointing (#161)

Merge branch 'main' of https://github.com/borisdayma/dalle-mini into main

feat: better multi-node support (#158)

feat(text): support emojis (#154)

fix: smelu

fix: sinkformer

fix: support smelu

feat: allow relative position (#156)

feat: sinkhorn in lse mode (#155)

fix: sinkformer gradient

feat(model): allow bias (#152)

feat: add sinkformer + custom final ln + pre-ln (#151)

feat: placeholders for more config

feat: force final ln in encoder

feat: allow more configurations

fix: DeepNet doesn't scale weights of embedding/output layers (#150)

feat: remove unecessary LN

feat: add cogview

fix(textnormalizer): consider utf8 on windows (#148)

feat: implement transformer variants (#144)

feat(data): super conditioning (#141)

feat: support pod (#139)

feat: handle gradient checkpointing

feat: load from bucket

feat: reduce artifact space + offset step

feat: restore weights on CPU

fix: position embedding for generate method

fix: typo

fix: load from checkpoint

feat(modeling): simplify abstract_init

feat(train) - handle multiple nodes (#130)

feat: handle model parallel

fix: style

style(tokenizer): remove unused variables

feat: use fast tokenizer

feat(train): improve pjit speed

fix(train): consider correct batch size

feat(train): distributed_shampoo with pjit

style: unsused import

feat(model): clean way to load on cpu

feat(train): no batch dimension with pjit

feat(train): progress on pjit

feat: use_artifact if run existing

Load from wandb artifact (#121)

Style (isort).

Tokenizer, config, model can be loaded from wandb.

feat(data): support accumulation in non-streaming

feat: custom gradient accumulation

Change import order again.