End of training

README.md

@@ -0,0 +1,90 @@
+---
+license: mit
+library_name: peft
+tags:
+- generated_from_trainer
+base_model: TheBloke/zephyr-7B-alpha-GPTQ
+model-index:
+- name: zephyr-chatbot
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# zephyr-chatbot
+
+This model is a fine-tuned version of [TheBloke/zephyr-7B-alpha-GPTQ](https://huggingface.co/TheBloke/zephyr-7B-alpha-GPTQ) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.8794
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: gptq
+- bits: 4
+- tokenizer: None
+- dataset: None
+- group_size: 128
+- damp_percent: 0.1
+- desc_act: True
+- sym: True
+- true_sequential: True
+- use_cuda_fp16: False
+- model_seqlen: 4095
+- block_name_to_quantize: model.layers
+- module_name_preceding_first_block: ['model.embed_tokens']
+- batch_size: 1
+- pad_token_id: None
+- use_exllama: False
+- max_input_length: None
+- exllama_config: {'version': <ExllamaVersion.ONE: 1>}
+- cache_block_outputs: True
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0002
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- training_steps: 1000
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 0.9364        | 1.0   | 121  | 0.9402          |
+| 0.8289        | 2.0   | 242  | 0.8849          |
+| 0.7249        | 3.0   | 363  | 0.8540          |
+| 0.6322        | 4.0   | 484  | 0.8387          |
+| 0.5337        | 5.0   | 605  | 0.8475          |
+| 0.4523        | 6.0   | 726  | 0.8586          |
+| 0.405         | 7.0   | 847  | 0.8698          |
+| 0.3965        | 8.0   | 968  | 0.8794          |
+| 0.3859        | 8.26  | 1000 | 0.8794          |
+
+
+### Framework versions
+
+- PEFT 0.7.0
+- Transformers 4.36.0.dev0
+- Pytorch 2.1.1+cu121
+- Datasets 2.15.0
+- Tokenizers 0.15.0

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5b3d0301ff43271c7843c35cdf5b29430e5e23bab47646dd500c123001db0e85
+oid sha256:4e7ce423ff9917ced524db65c4387b96c528f2f4b500e742429461c4b08f8681
 size 27280152

emissions.csv

@@ -0,0 +1,2 @@
+timestamp,project_name,run_id,duration,emissions,emissions_rate,cpu_power,gpu_power,ram_power,cpu_energy,gpu_energy,ram_energy,energy_consumed,country_name,country_iso_code,region,cloud_provider,cloud_region,os,python_version,codecarbon_version,cpu_count,cpu_model,gpu_count,gpu_model,longitude,latitude,ram_total_size,tracking_mode,on_cloud,pue
+2023-12-10T23:58:16,codecarbon,463d7c05-7786-4d02-a093-f13933f14fc2,12112.945874452591,0.32955738331212314,2.72070383809105e-05,42.5,128.371,73.71764945983887,0.14299199634525528,0.5018511333381926,0.2479411869527795,0.8927843166362271,United States,USA,virginia,,,Linux-5.15.0-67-generic-x86_64-with-glibc2.29,3.8.10,2.2.3,30,Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz,1,1 x NVIDIA A10,-77.539,39.018,196.5803985595703,machine,N,1.0

runs/Dec10_20-36-19_141-148-31-151/events.out.tfevents.1702240581.141-148-31-151.310144.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:df03a2a39f3fd14c266e7bd31c82cd4c91f5b8b44cb4b55bbb9adf59759c1dbb
-size 10125
+oid sha256:2e2b70bc39729dfbdd725848c9a9b5272c1c3075775b0602692d721c58ced9c8
+size 10907