Upload folder using huggingface_hub

README.md

@@ -1,3 +1,156 @@
----
-license: mit
----
+---
+license: mit
+---
+
+<h1 align="center" style="font-size: 24px;">EquiformerV3:<br>Scaling Efficient, Expressive, and General SE(3)-Equivariant Graph Attention Transformers</h1> 
+
+<!--
+# **[Code](https://github.com/atomicarchitects/equiformer_v3)** | **[Paper]()**
+-->
+
+<a href="https://github.com/atomicarchitects/equiformer_v3" style="color: #1a73e8; font-weight: bold; font-size: 20px;">Code</a> | 
+<a href="https://github.com/atomicarchitects/equiformer_v3/blob/main/experimental/docs/equiformer_v3_paper.pdf" style="color: #1a73e8; font-weight: bold; font-size: 20px;">Paper</a>
+
+This repository contains the checkpoints of the work "EquiformerV3: Scaling Efficient, Expressive, and General SE(3)-Equivariant Graph Attention Transformers".
+Please refer to the [code](https://github.com/atomicarchitects/equiformer_v3) for detailed description of usage.
+
+<p align="center">
+  <img width="50%" height="50%" src="https://cdn-uploads.huggingface.co/production/uploads/64948a4a8d5ff0dd776655fe/03TPndezDyUw4FcfTBk4n.png"?
+</p>
+
+
+
+
+## Content ##
+<!-- 
+0. [OC20](#oc20)
+-->
+0. [MPtrj](#mptrj)
+0. [OMat24 → MPtrj and sAlex](#oam)
+
+
+<!--
+<h2 id="oc20">OC20</h2>
+
+<table>
+  <tr style="background-color: #f0f0f0;">
+   <td><strong>Model</strong></td>
+   <td><strong>Training data</strong></td>
+   <td><strong>Config</strong></td>
+   <td><strong>Checkpoint</strong></td>
+  </tr>
+
+  <tr>
+   <td>EquiformerV3 (91M)</td>
+   <td>OC20 S2EF-2M</td>
+   <td><a href="https://github.com/atomicarchitects/equiformer_v3/blob/main/experimental/configs/oc20/2M/equiformer_v3/experiments/base_N%408-L%406-C%40128-attn-hidden%4064-ffn%40512-envelope-num-rbf%40128_merge-layer-norm_gates2-gridmlp_use-gate-force-head_wd%401e-3-grad-clip%40100_lin-ref-e%404.yml">base.yml</a></td>
+   <td></td>
+  </tr>
+</table>
+-->
+
+
+<h2 id="mptrj">MPtrj</h2>
+<!--
+Training consists of (1) direct pre-training and (2) gradient fine-tuning initialized from (1).
+<table>
+  <tr style="background-color: #f0f0f0;">
+   <td><strong>Model</strong></td>
+   <td><strong>Training data</strong></td>
+   <td><strong>Config</strong></td>
+   <td><strong>Checkpoint</strong></td>
+  </tr>
+  <tr>
+   <td>EquiformerV3 (direct pre-training)</td>
+   <td>MPtrj</td>
+   <td>
+     <a href="https://github.com/atomicarchitects/equiformer_v3/blob/main/experimental/configs/omat24/mptrj/experiments/direct/equiformer_v3_N%407_L%404_attn-hidden%4032_rbf%4010_max-neighbors%40300_attn-grid%4014-8_ffn-grid%4014_use-gate-force-head_merge-layer-norm_epochs%4070-bs%40512-wd%401e-3-beta2%400.95_dens-p%400.5-std%400.025-r%400.5-w%4010-strict-max-r%400.75-no-stress.yml">
+       direct.yml
+     </a>
+   </td>
+   <td></td>
+  </tr>
+  <tr>
+   <td>EquiformerV3 (gradient fine-tuning)</td>
+   <td>MPtrj</td>
+   <td>
+     <a href="https://github.com/atomicarchitects/equiformer_v3/blob/main/experimental/configs/omat24/mptrj/experiments/gradient/equiformer_v3_grad-finetune_N%407_L%404_attn-hidden%4032_rbf%4010_max-neighbors%40300_attn-grid%4014-8_ffn-grid%4014_pt-reg-dens-no-stress-strict-max-r%400.75-ft-no-reg_lr%400-5e-5-epochs%4010-bs%4064x8-wd%401e-3-beta2%400.95.yml">
+       gradient.yml
+     </a>
+   </td>
+   <td></td>
+  </tr>
+</table>
+-->
+<table>
+  <tr style="background-color: #f0f0f0;">
+   <td><strong>Model</strong></td>
+   <td><strong>Training data</strong></td>
+   <td><strong>Checkpoint</strong></td>
+  </tr>
+  <tr>
+   <td>EquiformerV3</td>
+   <td>MPtrj</td>
+   <td>
+     <a href="https://huggingface.co/yilunliao/equiformer_v3/blob/main/checkpoint/mptrj_gradient.pt">
+       mptrj_gradient.pt
+     </a>
+   </td>
+  </tr>
+</table>
+
+
+<h2 id="oam">OMat24 → MPtrj and sAlex</h2>
+Training consists of (1) direct pre-training on OMat24, 
+(2) gradient fine-tuning on OMat24 initialized from (1), and
+(3) gradient fine-tuning on MPtrj and sAlex initialized from (2).
+<table>
+  <tr style="background-color: #f0f0f0;">
+   <td><strong>Model</strong></td>
+   <td><strong>Training data</strong></td>
+   <td><strong>Config</strong></td>
+   <td><strong>Checkpoint</strong></td>
+  </tr>
+  <tr>
+   <td>EquiformerV3 (direct pre-training)</td>
+   <td>OMat24</td>
+   <td>
+     <a href="https://github.com/atomicarchitects/equiformer_v3/blob/main/experimental/configs/omat24/omat24/experiments/direct/equiformer_v3_N%407_L%404_attn-hidden%4032_rbf%4064_max-neighbors%40300_attn-grid%4014-8_ffn-grid%4014_use-gate-force-head_merge-layer-norm_epochs%404-bs%40512-wd%401e-3-beta2%400.98-eps%401e-6_dens-p%400.5-std%400.025-r%400.5-0.75-w%401-no-stress-max-f%402.5_no-amp.yml">
+       omat24_direct.yml
+     </a>
+   </td>
+   <td>
+     <a href="https://huggingface.co/yilunliao/equiformer_v3/blob/main/checkpoint/omat24_direct.pt">
+       omat24_direct.pt
+     </a>
+   </td>
+  </tr>
+  <tr>
+   <td>EquiformerV3 (gradient fine-tuning)</td>
+   <td>OMat24</td>
+   <td>
+     <a href="https://github.com/atomicarchitects/equiformer_v3/blob/main/experimental/configs/omat24/omat24/experiments/gradient/equiformer_v3_grad-finetune_N%407_L%404_attn-hidden%4032_rbf%4064_max-neighbors%40300_attn-grid%4014-8_ffn-grid%4014_merge-layer-norm_lr%400-1e-4-epochs%402-bs%40512-wd%401e-3-beta2%400.98-eps%401e-6_pt-reg-dens-ft-no-reg.yml">
+       omat24_gradient.yml
+     </a>
+   </td>
+   <td>
+     <a href="https://huggingface.co/yilunliao/equiformer_v3/blob/main/checkpoint/omat24_gradient.pt">
+       omat24_gradient.pt
+     </a>
+   </td>
+  </tr>
+  <tr>
+   <td>EquiformerV3 (gradient fine-tuning)</td>
+   <td>MPtrj and sAlex</td>
+   <td>
+     <a href="https://github.com/atomicarchitects/equiformer_v3/blob/main/experimental/configs/omat24/salex_mptrj/experiments/gradient/equiformer_v3_grad-finetune_N%407_L%404_attn-hidden%4032_rbf%4064_max-neighbors%40300_attn-grid%4014-8_ffn-grid%4014_attn-eps%401e-8_lr%400-5e-5-warmup%400.1-epochs%402-mptrj-salex-ratio%408-bs%40256-wd%401e-3-beta2%400.98-eps%401e-6_pt-reg-dens-ft-no-reg-lr%401e-4.yml">
+       mptrj-salex_gradient.yml
+     </a>
+   </td>
+   <td>
+     <a href="https://huggingface.co/yilunliao/equiformer_v3/blob/main/checkpoint/omat24-mptrj-salex_gradient.pt">
+       omat24-mptrj-salex_gradient.pt
+     </a>
+   </td>
+  </tr>
+</table>

checkpoint/mptrj_gradient.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59c6c23573a3b05b347662f473209d1bb1ccb5b85f6624a8f87072e7c397addf
+size 485354773

checkpoint/omat24-mptrj-salex_gradient.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:429ccded98163122e7ba588d78e2441653f37f3e091e106c432807fe373c8f98
+size 486243861

checkpoint/omat24_direct.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd874ab2698a391a03a5083b8729c286c676d76ca584437ff5e550696609a96d
+size 559867247

checkpoint/omat24_gradient.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c4b9139c1a14094ed724c3addd584ce07afaf0032e1ec77490d2c60ed8b8a2b
+size 486243797