Upload TFBertForPreTraining

README.md

@@ -33,14 +33,14 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- optimizer: {'name': 'Adam', 'learning_rate': 1e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
+- optimizer: {'name': 'Adam', 'learning_rate': 1e-04, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
 - training_precision: float32
 
 ### Training results
 
 | Train Loss | Epoch |
 |:----------:|:-----:|
-| 10.2428    | 0     |
+| 9.1922     | 0     |
 | nan        | 1     |
 | nan        | 2     |
 | nan        | 3     |

tf_model.h5

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8e63c377320d94e13c5266931288e63a364c08c1ed10c336a2bc8742496f80db
+oid sha256:f02a366153e5da2af0c2f922c1bf2e89280e08524a027e521c1bae758fee12c3
 size 526681688