Push model using huggingface_hub.

1_Pooling/config.json

@@ -1,10 +1,10 @@
-{
-    "word_embedding_dimension": 768,
-    "pooling_mode_cls_token": true,
-    "pooling_mode_mean_tokens": false,
-    "pooling_mode_max_tokens": false,
-    "pooling_mode_mean_sqrt_len_tokens": false,
-    "pooling_mode_weightedmean_tokens": false,
-    "pooling_mode_lasttoken": false,
-    "include_prompt": true
+{
+    "word_embedding_dimension": 384,
+    "pooling_mode_cls_token": false,
+    "pooling_mode_mean_tokens": true,
+    "pooling_mode_max_tokens": false,
+    "pooling_mode_mean_sqrt_len_tokens": false,
+    "pooling_mode_weightedmean_tokens": false,
+    "pooling_mode_lasttoken": false,
+    "include_prompt": true
 }

README.md

@@ -1,228 +1,130 @@
----
-tags:
-- setfit
-- sentence-transformers
-- text-classification
-- generated_from_setfit_trainer
-widget:
-- text: Great to hear that this basic (on the face of it) functionality is being implemented,
-    it needs to be done thanks.
-- text: Greg, I totally agree with you. Regretting moving from Reckon Desktop. Xero
-    has created twice as much work for me, without the basic functionality that even
-    Open Source packages have. We have one customer that may have 100 different site
-    addresses in one year, and Xero is creating a nightmare.
-- text: Agree with Benjamin on 16.08.22 - this is such a basic feature. Time to move
-    a batch of clients back to Sage - hugely embarrassing and a complete waste of
-    time.
-- text: Game changer for our construction projects. Invoicing for different sites
-    is a breeze now.
-- text: "Personally can't believe this functionality hasn't been created given the\
-    \ amount of businesses that have multiple shipments locations for the same business.\r\
-    \n\r\ncustomers want a statement for all stores combined and to manually type\
-    \ in an address each invoice just show PO's can be matched and paid is silly.\
-    \ Given Xero generally improves efficiency though automation rather than manual\
-    \ entry"
-metrics:
-- accuracy
-pipeline_tag: text-classification
-library_name: setfit
-inference: true
-base_model: BAAI/bge-base-en-v1.5
-model-index:
-- name: SetFit with BAAI/bge-base-en-v1.5
-  results:
-  - task:
-      type: text-classification
-      name: Text Classification
-    dataset:
-      name: Unknown
-      type: unknown
-      split: test
-    metrics:
-    - type: accuracy
-      value: 0.87
-      name: Accuracy
----
-
-# SetFit with BAAI/bge-base-en-v1.5
-
-This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
-
-The model has been trained using an efficient few-shot learning technique that involves:
-
-1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
-2. Training a classification head with features from the fine-tuned Sentence Transformer.
-
-## Model Details
-
-### Model Description
-- **Model Type:** SetFit
-- **Sentence Transformer body:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5)
-- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
-- **Maximum Sequence Length:** 512 tokens
-- **Number of Classes:** 5 classes
-<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
-<!-- - **Language:** Unknown -->
-<!-- - **License:** Unknown -->
-
-### Model Sources
-
-- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
-- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
-- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
-
-### Model Labels
-| Label | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
-|:------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| 3     | <ul><li>"Glad we don't have to leave for Quickbooks now, this update should solve the issues we were having."</li><li>'This is a solid improvement. Being able to add a secondary address for delivery without editing the main contact is useful.'</li><li>'Good that future company returns can be done, but surely it isnt that hard to build one for trusts and partnerships too?'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
-| 0     | <ul><li>'How many more years do you require to have this issue resolved? you may end up loosing clients to competitors over this issue. This issue become more crucial as time goes on.'</li><li>'I asked for this "Improvement" over 10 years ago in this thread (or a previous one). I don\'t believe anyone at Xero actually takes any notice of this forum. its just here so you can vent and think that something will get done. Xero is far too busy developing  "New" features to entice new clients rather than supporting existing clients who they have already "Captured"'</li><li>'Too little too late for me Xero. The last price rise was the last straw for me - 80% increase over 3 years. Encouraged me to look at the opposition and realise that I could get significantly more features for half the price. Look back over this issue - a clear indication of how a significant feature gap receives so little attention from Xero. Meanwhile the big news is that the main menu is going to be fiddled with yet again. Really! Too much resource spent rearranging deck chairs I would suggest. Bye.'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
-| 1     | <ul><li>"I'd like very much to trial early payment discounts with my clients, but despite using some pretty sophisticated accounting software - by the way, that's your product Xero - I can't add this simple thing to an invoice. Wow! \r\n\r\nIf this functionality hasn't been implemented in 12 years, there must be a reason."</li><li>"@Scott Osburne you technically CAN [CAPS] tab through the fields in the payment pop-up, but to change the date, you have to press space or enter when the date field is highlighted, then tab several times to get to the grid of dates (with the mouse cursor NOT [CAPS] over the grid because if it's over the grid then tab exits the grid), use the arrow keys to find the one you want, then press enter or space (if you press tab it goes to the next field instead of changing the date) when you have the right one highlighted, AND [CAPS] THEN [CAPS] YOU [CAPS] HAVE [CAPS] TO SHIFT [CAPS]-TAB [CAPS] BACK [CAPS] FOUR [CAPS] TIMES [CAPS] TO THE [CAPS] ACCOUNT [CAPS] FIELD [CAPS] because the focus has dissipated and both tab and shift-tab move the focus to the Add button instead of the focus moving in a logical order (WCAG [CAPS] success criterion 2.4.3 failure).\r\n\r\nIf you could enter the date instead of having to tab and arrow etc it would be significantly fewer keypresses, even without the focus order failure."</li><li>'Xero you are bleeding customers due to increased fees. That you aren’t prioritising upgrades that are standard features for your competitors is shocking. Provide a timetable for beta and full deployment schedule for each idea so that we can actually see it working its way through the apps dev pipeline!! [EMPHASIS [CAPS]]'</li></ul> |
-| 4     | <ul><li>'Great to hear that this basic (on the face of it) functionality is being implemented, it needs to be done thanks.'</li><li>'Thank you Xero! This worked today!! [EMPHASIS [CAPS]] What a relief to have this feature back. Now that item codes are pulled in for billable expenses, the new invoicing actually represents an improvement for my purposes. Thank you for addressing this request.'</li><li>'Hey Kelly, thank you for the updates. We appreciate you!\r\n\r\nIs there a quarter on your roadmap for when this feature has been allocated to?\r\n\r\nAlso, just checking if this also means that the address selected for an invoice will persist with that invoice, even after a contact adds a new address in the future?'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
-| 2     | <ul><li>"Another way Xero could design this is to have multiple sub contacts under a main contact. For example McDonald's is the main contact but then you have McDonald's - Brisbane CBD [CAPS] as a sub contact. Just a thought.. [ELONGATED [CAPS]]"</li><li>'Agreed with all. Please add an option to opt out of email notification. Thank you!'</li><li>"We are a school using Xero we invoice annually and parents can pay monthly so it is really necessary for the online payment method to be linked to statements as well. Doesn't make sense that this is still unavailable."</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
-
-## Evaluation
-
-### Metrics
-| Label   | Accuracy |
-|:--------|:---------|
-| **all** | 0.87     |
-
-## Uses
-
-### Direct Use for Inference
-
-First install the SetFit library:
-
-```bash
-pip install setfit
-```
-
-Then you can load this model and run inference.
-
-```python
-from setfit import SetFitModel
-
-# Download from the 🤗 Hub
-model = SetFitModel.from_pretrained("setfit_model_id")
-# Run inference
-preds = model("Game changer for our construction projects. Invoicing for different sites is a breeze now.")
-```
-
-<!--
-### Downstream Use
-
-*List how someone could finetune this model on their own dataset.*
--->
-
-<!--
-### Out-of-Scope Use
-
-*List how the model may foreseeably be misused and address what users ought not to do with the model.*
--->
-
-<!--
-## Bias, Risks and Limitations
-
-*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
--->
-
-<!--
-### Recommendations
-
-*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
--->
-
-## Training Details
-
-### Training Set Metrics
-| Training set | Min | Median | Max |
-|:-------------|:----|:-------|:----|
-| Word count   | 1   | 50.106 | 701 |
-
-| Label | Training Sample Count |
-|:------|:----------------------|
-| 0     | 100                   |
-| 1     | 100                   |
-| 2     | 100                   |
-| 3     | 100                   |
-| 4     | 100                   |
-
-### Training Hyperparameters
-- batch_size: (16, 16)
-- num_epochs: (1, 16)
-- max_steps: -1
-- sampling_strategy: oversampling
-- num_iterations: 10
-- body_learning_rate: (2e-05, 1e-05)
-- head_learning_rate: 0.01
-- loss: CosineSimilarityLoss
-- distance_metric: cosine_distance
-- margin: 0.25
-- end_to_end: False
-- use_amp: False
-- warmup_proportion: 0.1
-- l2_weight: 0.01
-- seed: 42
-- eval_max_steps: -1
-- load_best_model_at_end: True
-
-### Training Results
-| Epoch  | Step | Training Loss | Validation Loss |
-|:------:|:----:|:-------------:|:---------------:|
-| 0.0016 | 1    | 0.2036        | -               |
-| 0.08   | 50   | 0.2392        | -               |
-| 0.16   | 100  | 0.2148        | -               |
-| 0.24   | 150  | 0.1638        | -               |
-| 0.32   | 200  | 0.1284        | -               |
-| 0.4    | 250  | 0.1046        | -               |
-| 0.48   | 300  | 0.0822        | -               |
-| 0.56   | 350  | 0.07          | -               |
-| 0.64   | 400  | 0.0457        | -               |
-| 0.72   | 450  | 0.0328        | -               |
-| 0.8    | 500  | 0.0213        | -               |
-| 0.88   | 550  | 0.0137        | -               |
-| 0.96   | 600  | 0.0139        | -               |
-| 1.0    | 625  | -             | 0.1038          |
-
-### Framework Versions
-- Python: 3.11.9
-- SetFit: 1.1.3
-- Sentence Transformers: 5.2.0
-- Transformers: 4.57.3
-- PyTorch: 2.7.1+cu118
-- Datasets: 4.4.2
-- Tokenizers: 0.22.2
-
-## Citation
-
-### BibTeX
-```bibtex
-@article{https://doi.org/10.48550/arxiv.2209.11055,
-    doi = {10.48550/ARXIV.2209.11055},
-    url = {https://arxiv.org/abs/2209.11055},
-    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
-    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
-    title = {Efficient Few-Shot Learning Without Prompts},
-    publisher = {arXiv},
-    year = {2022},
-    copyright = {Creative Commons Attribution 4.0 International}
-}
-```
-
-<!--
-## Glossary
-
-*Clearly define terms in order to be accessible across audiences.*
--->
-
-<!--
-## Model Card Authors
-
-*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
--->
-
-<!--
-## Model Card Contact
-
-*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+---
+tags:
+- setfit
+- sentence-transformers
+- text-classification
+- generated_from_setfit_trainer
+widget: []
+metrics:
+- accuracy
+pipeline_tag: text-classification
+library_name: setfit
+inference: true
+---
+
+# SetFit
+
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
+
+The model has been trained using an efficient few-shot learning technique that involves:
+
+1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
+2. Training a classification head with features from the fine-tuned Sentence Transformer.
+
+## Model Details
+
+### Model Description
+- **Model Type:** SetFit
+<!-- - **Sentence Transformer:** [Unknown](https://huggingface.co/unknown) -->
+- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
+- **Maximum Sequence Length:** 256 tokens
+- **Number of Classes:** 5 classes
+<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
+- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
+- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
+
+## Uses
+
+### Direct Use for Inference
+
+First install the SetFit library:
+
+```bash
+pip install setfit
+```
+
+Then you can load this model and run inference.
+
+```python
+from setfit import SetFitModel
+
+# Download from the 🤗 Hub
+model = SetFitModel.from_pretrained("setfit_model_id")
+# Run inference
+preds = model("I loved the spiderman movie!")
+```
+
+<!--
+### Downstream Use
+
+*List how someone could finetune this model on their own dataset.*
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Framework Versions
+- Python: 3.11.14
+- SetFit: 1.1.3
+- Sentence Transformers: 5.2.0
+- Transformers: 4.57.5
+- PyTorch: 2.9.1
+- Datasets: 4.4.2
+- Tokenizers: 0.22.2
+
+## Citation
+
+### BibTeX
+```bibtex
+@article{https://doi.org/10.48550/arxiv.2209.11055,
+    doi = {10.48550/ARXIV.2209.11055},
+    url = {https://arxiv.org/abs/2209.11055},
+    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
+    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
+    title = {Efficient Few-Shot Learning Without Prompts},
+    publisher = {arXiv},
+    year = {2022},
+    copyright = {Creative Commons Attribution 4.0 International}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
 -->

config.json

@@ -1,31 +1,25 @@
-{
-  "architectures": [
-    "BertModel"
-  ],
-  "attention_probs_dropout_prob": 0.1,
-  "classifier_dropout": null,
-  "dtype": "float32",
-  "gradient_checkpointing": false,
-  "hidden_act": "gelu",
-  "hidden_dropout_prob": 0.1,
-  "hidden_size": 768,
-  "id2label": {
-    "0": "LABEL_0"
-  },
-  "initializer_range": 0.02,
-  "intermediate_size": 3072,
-  "label2id": {
-    "LABEL_0": 0
-  },
-  "layer_norm_eps": 1e-12,
-  "max_position_embeddings": 512,
-  "model_type": "bert",
-  "num_attention_heads": 12,
-  "num_hidden_layers": 12,
-  "pad_token_id": 0,
-  "position_embedding_type": "absolute",
-  "transformers_version": "4.57.3",
-  "type_vocab_size": 2,
-  "use_cache": true,
-  "vocab_size": 30522
-}
+{
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 6,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "transformers_version": "4.57.5",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

config_sentence_transformers.json

@@ -1,14 +1,14 @@
-{
-  "__version__": {
-    "sentence_transformers": "5.2.0",
-    "transformers": "4.57.3",
-    "pytorch": "2.7.1+cu118"
-  },
-  "model_type": "SentenceTransformer",
-  "prompts": {
-    "query": "",
-    "document": ""
-  },
-  "default_prompt_name": null,
-  "similarity_fn_name": "cosine"
+{
+  "__version__": {
+    "sentence_transformers": "5.2.0",
+    "transformers": "4.57.5",
+    "pytorch": "2.9.1"
+  },
+  "model_type": "SentenceTransformer",
+  "prompts": {
+    "query": "",
+    "document": ""
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
 }

config_setfit.json

@@ -1,10 +1,10 @@
-{
-  "normalize_embeddings": false,
-  "labels": [
-    0,
-    1,
-    2,
-    3,
-    4
-  ]
+{
+  "labels": [
+    0,
+    1,
+    2,
+    3,
+    4
+  ],
+  "normalize_embeddings": false
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5b85322091c0722e761ff13c31f84507915fee2f9a19c8a6280aa1c6691601c6
-size 437951328
+oid sha256:18a663e92fec7aa0b54431fa5c10a1481be64df6b7f98defb81887c939f13c28
+size 90864192

model_head.pkl

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8359dda036c661b2ec116a3d4c3dbed0e415d49b60a97c3e83ba1549aa6438bc
-size 31647
+oid sha256:c3e83f7cb8466031df74b40f5ca1f8cc3e3d217bf29f4ff1b64b512bc01c5e0a
+size 16287

modules.json

@@ -1,20 +1,20 @@
-[
-  {
-    "idx": 0,
-    "name": "0",
-    "path": "",
-    "type": "sentence_transformers.models.Transformer"
-  },
-  {
-    "idx": 1,
-    "name": "1",
-    "path": "1_Pooling",
-    "type": "sentence_transformers.models.Pooling"
-  },
-  {
-    "idx": 2,
-    "name": "2",
-    "path": "2_Normalize",
-    "type": "sentence_transformers.models.Normalize"
-  }
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
 ]

sentence_bert_config.json

@@ -1,4 +1,4 @@
-{
-    "max_seq_length": 512,
-    "do_lower_case": true
+{
+    "max_seq_length": 256,
+    "do_lower_case": false
 }

special_tokens_map.json

@@ -1,37 +1,37 @@
-{
-  "cls_token": {
-    "content": "[CLS]",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "mask_token": {
-    "content": "[MASK]",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "pad_token": {
-    "content": "[PAD]",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "sep_token": {
-    "content": "[SEP]",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "unk_token": {
-    "content": "[UNK]",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  }
-}
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -2,7 +2,7 @@
   "version": "1.0",
   "truncation": {
     "direction": "Right",
-    "max_length": 512,
+    "max_length": 256,
     "strategy": "LongestFirst",
     "stride": 0
   },

tokenizer_config.json

@@ -1,58 +1,65 @@
-{
-  "added_tokens_decoder": {
-    "0": {
-      "content": "[PAD]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "100": {
-      "content": "[UNK]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "101": {
-      "content": "[CLS]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "102": {
-      "content": "[SEP]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "103": {
-      "content": "[MASK]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    }
-  },
-  "clean_up_tokenization_spaces": true,
-  "cls_token": "[CLS]",
-  "do_basic_tokenize": true,
-  "do_lower_case": true,
-  "extra_special_tokens": {},
-  "mask_token": "[MASK]",
-  "model_max_length": 512,
-  "never_split": null,
-  "pad_token": "[PAD]",
-  "sep_token": "[SEP]",
-  "strip_accents": null,
-  "tokenize_chinese_chars": true,
-  "tokenizer_class": "BertTokenizer",
-  "unk_token": "[UNK]"
-}
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_length": 128,
+  "model_max_length": 256,
+  "never_split": null,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}