File size: 930 Bytes
6a285d1 | 1 2 3 4 5 6 7 8 9 10 | === flame38m_dense_local (flags: DENSE=1 grain 1) ===
[actprobe] saved /workspace/FLAME-MoE/results/phase0/runs/flame38m_dense_local/act_log.pt: MoE layers=[], attn layers=[0, 1, 2, 3, 4, 5, 6, 7, 8], temporal=False
=== flame38m_g1_temporal (flags: grain 1) ===
[actprobe] saved /workspace/FLAME-MoE/results/phase0/runs/flame38m_g1_temporal/act_log.pt: MoE layers=[1, 2, 3, 4, 5, 6, 7, 8], attn layers=[0, 1, 2, 3, 4, 5, 6, 7, 8], temporal=True
=== flame38m_g3_moe (flags: MOE_FULL=1 grain 3) ===
[actprobe] saved /workspace/FLAME-MoE/results/phase0/runs/flame38m_g3_moe/act_log.pt: MoE layers=[1, 2, 3, 4, 5, 6, 7, 8], attn layers=[0, 1, 2, 3, 4, 5, 6, 7, 8], temporal=False
=== flame38m_g3_temporal (flags: grain 3) ===
[actprobe] saved /workspace/FLAME-MoE/results/phase0/runs/flame38m_g3_temporal/act_log.pt: MoE layers=[1, 2, 3, 4, 5, 6, 7, 8], attn layers=[0, 1, 2, 3, 4, 5, 6, 7, 8], temporal=True
ALL 38M ACTPROBES DONE
|