Model save

README.md

@@ -0,0 +1,77 @@
+---
+library_name: transformers
+license: mit
+base_model: OnurDemircioglu/OmniGPT-355M
+tags:
+- generated_from_trainer
+model-index:
+- name: OmniGPT-355M-Instruct
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# OmniGPT-355M-Instruct
+
+This model is a fine-tuned version of [OnurDemircioglu/OmniGPT-355M](https://huggingface.co/OnurDemircioglu/OmniGPT-355M) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 3.1367
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 1e-05
+- train_batch_size: 2
+- eval_batch_size: 8
+- seed: 42
+- gradient_accumulation_steps: 8
+- total_train_batch_size: 16
+- optimizer: Use OptimizerNames.ADAFACTOR and the args are:
+No additional optimizer arguments
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 100
+- num_epochs: 1
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 3.4186        | 0.0651 | 200  | 3.3400          |
+| 3.3925        | 0.1302 | 400  | 3.2980          |
+| 3.3676        | 0.1952 | 600  | 3.2674          |
+| 3.4006        | 0.2603 | 800  | 3.2426          |
+| 3.2497        | 0.3254 | 1000 | 3.2226          |
+| 3.3650        | 0.3905 | 1200 | 3.2083          |
+| 3.2403        | 0.4555 | 1400 | 3.1940          |
+| 3.2321        | 0.5206 | 1600 | 3.1802          |
+| 3.2777        | 0.5857 | 1800 | 3.1701          |
+| 3.3052        | 0.6508 | 2000 | 3.1608          |
+| 3.2324        | 0.7159 | 2200 | 3.1531          |
+| 3.2948        | 0.7809 | 2400 | 3.1465          |
+| 3.1677        | 0.8460 | 2600 | 3.1420          |
+| 3.2207        | 0.9111 | 2800 | 3.1384          |
+| 3.2370        | 0.9762 | 3000 | 3.1367          |
+
+
+### Framework versions
+
+- Transformers 5.0.0
+- Pytorch 2.10.0+cu128
+- Datasets 4.8.3
+- Tokenizers 0.22.2

tokenizer_config.json

@@ -0,0 +1,12 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "is_local": false,
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}