wayyresearch
/

aetheris

Text Generation

state-space-model

mixture-of-experts

Mixture of Experts

knowledge-distillation

hybrid-architecture

Model card Files Files and versions

4.53 GB

Ctrl+K

Ctrl+K

1 contributor

History: 34 commits

rcgalbo's picture

Upload final Stage 2 best checkpoint (loss=2.7305, 20K steps)

d420efe verified about 2 months ago

.gitattributes

1.52 kB
initial commit about 2 months ago
README.md

1.89 kB
Stage 2 checkpoint: [Step 18500/20000] loss=3.1250 about 2 months ago
stage1_checkpoint.pt
Detected Pickle imports (4)
- "torch.BFloat16Storage",
- "collections.OrderedDict",
- "torch._utils._rebuild_tensor_v2",
- "torch.FloatStorage"
What is a pickle import?
1.64 GB
xet

Stage 1 checkpoint: [Step 50/20000] loss=7.7500 about 2 months ago
stage1_metadata.json

414 Bytes
Stage 1 checkpoint: [Step 50/20000] loss=7.7500 about 2 months ago
stage2_best.pt
Detected Pickle imports (3)
- "collections.OrderedDict",
- "torch.BFloat16Storage",
- "torch._utils._rebuild_tensor_v2"
What is a pickle import?
1.44 GB
xet

Upload final Stage 2 best checkpoint (loss=2.7305, 20K steps) about 2 months ago
stage2_checkpoint.pt
Detected Pickle imports (3)
- "collections.OrderedDict",
- "torch._utils._rebuild_tensor_v2",
- "torch.BFloat16Storage"
What is a pickle import?
1.44 GB
xet

Stage 2 checkpoint: [Step 18500/20000] loss=3.1250 about 2 months ago
stage2_metadata.json

298 Bytes
Stage 2 checkpoint: [Step 18500/20000] loss=3.1250 about 2 months ago
student_config.yaml

668 Bytes
Stage 1 initial: step 1000, loss=0.29, cka=0.60 about 2 months ago
training_config.yaml

2.74 kB
Stage 1 initial: step 1000, loss=0.29, cka=0.60 about 2 months ago