layer id init

corrected depth routing

2 new depth routing types

add different routing types

set to multinomial top k

change back to topk

multinomial2

4multi

multinomial8

ex1

2 ex

4exp

8exp

16exp

32 exp

experts per token to 64

fix imports

safetensors index

change name casings

all thing

classname

remove docstrings

fix issues

total repo overhaul

move olmoe out of folder

delete local harness and remove imports

additions