Checkpoint at step 20

Checkpoint at step 20

addd num_small_experts config access sparsemoeblock

remove num_small_expert overwrite

Checkpoint at step 20

remove strategies

Checkpoint at step 20

Checkpoint at step 160

Checkpoint at step 20

try patch hook

corrected change to 64

add num_small_experts to config

add max small expert to config

Revert "expert usage stats"

expert usage stats

Revert "18k checkpoint"

attempts to fix more

fix config

dataclass config

all OlMoeConfig to MyOlmoeConfig

olmoe to myolmoe in modelingcode

add expert logging

Checkpoint at step 12000

progress 20k

Checkpoint at step 2000

delete checkpoints all epochs to 3

change to constant strategy

increment

dynamic mask for logits

load balacing fix

attempt new distribution experts

changes

name

aaux to torch tensor

huge fixes

fix small experts loss calculation for gradient

reorder small experts

modify batch and fix tensor issue

ratio to divisor set to 16

fix import

overhaul

reset modeling file

config shenanigans

changes without config

import name

initial stuff

remove old routing stuff

init no routing