Drop-Upcycling
updated
9B • Updated • 34
9B • Updated • 40
19B • Updated • 34
9B • Updated • 39
9B • Updated • 36
0.4B • Updated • 37
0.4B • Updated • 34
0.4B • Updated • 41
0.4B • Updated • 45
9B • Updated • 57
19B • Updated • 34
0.4B • Updated • 38
9B • Updated • 35
0.4B • Updated • 34
• 1
2B • Updated • 35
0.2B • Updated • 43
4B • Updated • 35
14B • Updated • 39
llm-jp/Dense-btx-code-expert-152M
0.2B • Updated • 39
• 1
llm-jp/Dense-btx-english-expert-1.5B
2B • Updated • 37
llm-jp/Dense-btx-code-expert-1.5B
2B • Updated • 41
• 1
llm-jp/Dense-btx-japanese-expert-1.5B
2B • Updated • 31
• 1
llm-jp/Dense-btx-english-expert-152M
0.2B • Updated • 31
llm-jp/Dense-btx-japanese-expert-152M
0.2B • Updated • 33
Drop-Upcycling: Training Sparse Mixture of Experts with Partial
Re-initialization
Paper
• 2502.19261
• Published • 6
Text Generation
• 73B • Updated • 41
llm-jp/llm-jp-3-8x13b-instruct3
Text Generation
• 73B • Updated • 138
• 8
llm-jp/llm-jp-3-8x1.8b-instruct3
Text Generation
• 9B • Updated • 50
• 4
Text Generation
• 9B • Updated • 102
llm-jp/llm-jp-3-8x13b-instruct2
Text Generation
• 73B • Updated • 138
llm-jp/llm-jp-3-8x1.8b-instruct2
Text Generation
• 9B • Updated • 44
llm-jp/llm-jp-3.1-8x13b-instruct4
Text Generation
• 73B • Updated • 150
• 4
Text Generation
• 73B • Updated • 41