bcdennis commited on
Commit
2d02ef6
·
verified ·
1 Parent(s): 7b710e0

Upload folder using huggingface_hub

Browse files
Files changed (2) hide show
  1. README.md +16 -0
  2. model.pt +3 -0
README.md ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - from-scratch
4
+ - hierarchical-concept-diffusion
5
+ - synth
6
+ ---
7
+
8
+ # ncp arm — Hierarchical Concept-Diffusion synth run
9
+
10
+ - arm: `ncp`
11
+ - params: 50,500,737
12
+ - steps: 12000
13
+ - seq_len: 256, batch: 32
14
+ - final val LM loss: **6.1264**
15
+ - data: PleIAs/SYNTH (en subset: query+reasoning+answer, streaming, gpt2 tokenizer)
16
+ - optimizer: AdamW lr 0.0001, wd 0.1, cosine, warmup 200
model.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7fd1046077019a951dc98f326e4e7a21ace193098d4096cdb8752f1a3b13f44f
3
+ size 279650571