wage-predictor

README.md

@@ -0,0 +1,90 @@
+---
+library_name: transformers
+license: apache-2.0
+base_model: distilbert/distilbert-base-uncased
+tags:
+- generated_from_trainer
+model-index:
+- name: wage_prediction
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# wage_prediction
+
+This model is a fine-tuned version of [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.4480
+- Mse: -0.4480
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 16
+- eval_batch_size: 16
+- seed: 42
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: cosine
+- num_epochs: 32
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Mse      |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|
+| No log        | 1.0   | 18   | 32.6746         | -32.6746 |
+| No log        | 2.0   | 36   | 17.4924         | -17.4924 |
+| No log        | 3.0   | 54   | 9.4001          | -9.4001  |
+| No log        | 4.0   | 72   | 4.4795          | -4.4795  |
+| No log        | 5.0   | 90   | 1.6010          | -1.6010  |
+| No log        | 6.0   | 108  | 0.4903          | -0.4903  |
+| No log        | 7.0   | 126  | 0.4478          | -0.4478  |
+| No log        | 8.0   | 144  | 0.4468          | -0.4468  |
+| No log        | 9.0   | 162  | 0.4755          | -0.4755  |
+| No log        | 10.0  | 180  | 0.4231          | -0.4231  |
+| No log        | 11.0  | 198  | 0.3998          | -0.3998  |
+| No log        | 12.0  | 216  | 0.4729          | -0.4729  |
+| No log        | 13.0  | 234  | 0.4091          | -0.4091  |
+| No log        | 14.0  | 252  | 0.3988          | -0.3988  |
+| No log        | 15.0  | 270  | 0.4114          | -0.4114  |
+| No log        | 16.0  | 288  | 0.4927          | -0.4927  |
+| No log        | 17.0  | 306  | 0.4816          | -0.4816  |
+| No log        | 18.0  | 324  | 0.4277          | -0.4277  |
+| No log        | 19.0  | 342  | 0.4626          | -0.4626  |
+| No log        | 20.0  | 360  | 0.4846          | -0.4846  |
+| No log        | 21.0  | 378  | 0.4476          | -0.4476  |
+| No log        | 22.0  | 396  | 0.4320          | -0.4320  |
+| No log        | 23.0  | 414  | 0.4600          | -0.4600  |
+| No log        | 24.0  | 432  | 0.4463          | -0.4463  |
+| No log        | 25.0  | 450  | 0.4599          | -0.4599  |
+| No log        | 26.0  | 468  | 0.4634          | -0.4634  |
+| No log        | 27.0  | 486  | 0.4711          | -0.4711  |
+| 4.2194        | 28.0  | 504  | 0.4542          | -0.4542  |
+| 4.2194        | 29.0  | 522  | 0.4465          | -0.4465  |
+| 4.2194        | 30.0  | 540  | 0.4453          | -0.4453  |
+| 4.2194        | 31.0  | 558  | 0.4477          | -0.4477  |
+| 4.2194        | 32.0  | 576  | 0.4480          | -0.4480  |
+
+
+### Framework versions
+
+- Transformers 4.52.4
+- Pytorch 2.6.0+cu124
+- Datasets 2.14.4
+- Tokenizers 0.21.1

config.json

@@ -0,0 +1,30 @@
+{
+  "activation": "gelu",
+  "architectures": [
+    "DistilBertForSequenceClassification"
+  ],
+  "attention_dropout": 0.1,
+  "dim": 768,
+  "dropout": 0.1,
+  "hidden_dim": 3072,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "max_position_embeddings": 512,
+  "model_type": "distilbert",
+  "n_heads": 12,
+  "n_layers": 6,
+  "pad_token_id": 0,
+  "problem_type": "regression",
+  "qa_dropout": 0.1,
+  "seq_classif_dropout": 0.2,
+  "sinusoidal_pos_embds": false,
+  "tie_weights_": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.52.4",
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59ef10b188c8f8a073b6007d50aa4dbebf57c1ec858ca2021c16d24587470732
+size 267829484

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,56 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "DistilBertTokenizer",
+  "unk_token": "[UNK]"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:68e49f6404837de506d440722310ce4f73c1fe5ff5f59b592555c3f0905726b9
+size 5304