Initialize rope embeddings properly for the entropy model (#72)

Make apex logs less noisy (#60)

using apex rmsnorm (#57)

disable reshard after forward (#56)

Fix init and repro (#48)

Changes for training entropy model and correcting attention in local models (#25)

Initial commit