everythingchalna commited on
Commit
3828e1e
·
verified ·
1 Parent(s): 35547c6

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +78 -0
README.md ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - catalyst
5
+ - materials-science
6
+ - jax
7
+ - structure-generation
8
+ - constrained-decoding
9
+ datasets:
10
+ - Open-Catalyst-Project/OC20
11
+ metrics:
12
+ - generation_validity
13
+ - uniqueness
14
+ - novelty
15
+ ---
16
+
17
+ # nanocatalyst (depth=8, 25.2M params)
18
+
19
+ Minimal JAX/Flax transformer for catalyst structure generation with single-parameter depth scaling.
20
+
21
+ ## Model Details
22
+
23
+ | | |
24
+ |---|---|
25
+ | Architecture | Transformer (RMSNorm, RoPE, QK-norm, ReLU², logit softcapping, residual scalars) |
26
+ | Parameters | 25.2M |
27
+ | Depth | 8 (n_embd=512, n_layer=8, n_head=8) |
28
+ | Vocab size | 186 (WordLevel, 2-digit pair encoding) |
29
+ | Training data | 174K OC20 structures |
30
+ | Training time | 96 min on TPU v5p-8 |
31
+ | Framework | JAX / Flax |
32
+
33
+ ## Results (CuPt3 + OH, T=0.8, top_k=40, 100 samples)
34
+
35
+ | Metric | Result |
36
+ |--------|--------|
37
+ | Generation Validity | 95/100 (95.0%) |
38
+ | Uniqueness | 95/95 (100.0%) |
39
+ | Novelty | 95/95 (100.0%) |
40
+
41
+ ## Usage
42
+
43
+ ```python
44
+ from catalyst.hub import download_checkpoint
45
+ from catalyst.config import CatalystConfig
46
+ from catalyst.generate import generate_samples
47
+
48
+ # Download checkpoint
49
+ ckpt_path = download_checkpoint("everythingchalna/nanocatalyst")
50
+ config = CatalystConfig.load(ckpt_path / "config.json")
51
+
52
+ # Load params and generate (see README for full example)
53
+ ```
54
+
55
+ ## Training
56
+
57
+ Trained on 174K structures from the OC20 S2EF dataset using a TPU v5p-8 (Google TRC program). 20 epochs, WSD learning rate schedule, AdamW optimizer.
58
+
59
+ ## Files
60
+
61
+ - `config.json` — Model configuration
62
+ - `params/` — Orbax checkpoint (model parameters)
63
+ - `tokenizer.json` — HuggingFace WordLevel tokenizer
64
+ - `tokenizer_stats.json` — Tokenizer coverage statistics
65
+
66
+ ## Citation
67
+
68
+ ```bibtex
69
+ @software{nanocatalyst,
70
+ title = {nanocatalyst},
71
+ url = {https://github.com/everythingchalna/nanocatalyst},
72
+ license = {MIT}
73
+ }
74
+ ```
75
+
76
+ ## Acknowledgments
77
+
78
+ Training compute provided by the [Google TPU Research Cloud (TRC)](https://sites.research.google/trc/) program.