Model save

README.md

@@ -2,36 +2,22 @@
 library_name: transformers
 tags:
 - generated_from_trainer
-datasets:
-- HuggingFaceFW/fineweb
 metrics:
 - accuracy
 model-index:
 - name: T5LA
-  results:
-  - task:
-      name: Causal Language Modeling
-      type: text-generation
-    dataset:
-      name: HuggingFaceFW/fineweb sample-10BT
-      type: HuggingFaceFW/fineweb
-      args: sample-10BT
-    metrics:
-    - name: Accuracy
-      type: accuracy
-      value: 0.0322300343763811
+  results: []
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/uoy/llm_training/runs/pzcq293g)
 # T5LA
 
-This model is a fine-tuned version of [](https://huggingface.co/) on the HuggingFaceFW/fineweb sample-10BT dataset.
+This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 5.5470
 - Accuracy: 0.0322
+- Loss: 5.5470
 
 ## Model description
 

config.json

@@ -16,6 +16,7 @@
   "is_gated_act": false,
   "layer_norm_epsilon": 1e-06,
   "lookahead_size": 2,
+  "lookahead_type": "la",
   "model_type": "t5la",
   "num_decoder_layers": 6,
   "num_heads": 8,

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:250fa1af03ffaa8bda0e2278749102dd8574803e5d42d069bf5be28611ad9412
+oid sha256:3b0f480ede6b17d1fc31d60ba9c13c539402fa821bdb403c1b911e315bfca608
 size 5432