| --- |
| license: mit |
| base_model: microsoft/deberta-v3-base |
| tags: |
| - generated_from_trainer |
| metrics: |
| - precision |
| - recall |
| - f1 |
| - accuracy |
| model-index: |
| - name: deberta-v3-base-pii-en |
| results: [] |
| pipeline_tag: token-classification |
| widget: |
| - text: My name is Yoni Go and I live in Israel. My phone number is 054-1234567 |
| inference: |
| parameters: |
| aggregation_strategy: first |
| --- |
| |
| <!-- This model card has been generated automatically according to the information the Trainer had access to. You |
| should probably proofread and complete it, then remove this comment. --> |
|
|
| # Export of deberta-v3-base-pii-en |
|
|
| This is an onnx-converted version of yonigo/deberta-v3-base-pii-en. This is a test because onnx exports from [here](https://huggingface.co/spaces/onnx-community/convert-to-onnx) and [there](https://huggingface.co/spaces/onnx/export) produce non-working models. |
|
|
| [IMPORTANT] I learned that deberta-V3 is not supported by default for ONNX conversion, apparently because it uses disentangled attention, which calculates relative position embeddings on the fly. When this graph is exported to ONNX and run inside a browser's WASM engine, certain operators can fail silently. In a nutshell, converting a DeBERTa v3 model to ONNX requires a bit more work than for other models like RoBERTa. |
|
|
| # deberta-v3-base-pii-en |
|
|
| This model is a fine-tuned version of [microsoft/deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) on English samples from [ai4privacy/pii-masking-300k](https://huggingface.co/datasets/ai4privacy/pii-masking-300k). |
|
|
| Usage: |
| ```python |
| from transformers import pipeline |
| |
| pipe = pipeline("token-classification", model="yonigo/deberta-v3-base-pii-en", aggregation_strategy="first") |
| pipe("My name is Yoni Go and I live in Israel. My phone number is 054-1234567") |
| ``` |
|
|
| training code [git](https://github.com/yonigottesman/pii-model) |
|
|
|
|
| ### Training results |
|
|
| | Training Loss | Epoch | Step | Validation Loss | Bod F1 | Building F1 | Cardissuer F1 | City F1 | Country F1 | Date F1 | Driverlicense F1 | Email F1 | Geocoord F1 | Givenname1 F1 | Givenname2 F1 | Idcard F1 | Ip F1 | Lastname1 F1 | Lastname2 F1 | Lastname3 F1 | Pass F1 | Passport F1 | Postcode F1 | Secaddress F1 | Sex F1 | Socialnumber F1 | State F1 | Street F1 | Tel F1 | Time F1 | Title F1 | Username F1 | Precision | Recall | F1 | Accuracy | |
| |:-------------:|:-------:|:-----:|:---------------:|:------:|:-----------:|:-------------:|:-------:|:----------:|:-------:|:----------------:|:--------:|:-----------:|:-------------:|:-------------:|:---------:|:------:|:------------:|:------------:|:------------:|:-------:|:-----------:|:-----------:|:-------------:|:------:|:---------------:|:--------:|:---------:|:------:|:-------:|:--------:|:-----------:|:---------:|:------:|:------:|:--------:| |
| | 0.2437 | 1.0695 | 1000 | 0.1168 | 0.9421 | 0.8791 | 0.0 | 0.8847 | 0.8841 | 0.8507 | 0.8617 | 0.9746 | 0.7903 | 0.5186 | 0.0 | 0.7928 | 0.9609 | 0.5720 | 0.0 | 0.0 | 0.9128 | 0.7991 | 0.8952 | 0.7145 | 0.8960 | 0.8583 | 0.8807 | 0.8816 | 0.9170 | 0.9390 | 0.7071 | 0.8946 | 0.8053 | 0.8619 | 0.8326 | 0.9736 | |
| | 0.0841 | 2.1390 | 2000 | 0.0731 | 0.9605 | 0.9633 | 0.0 | 0.9526 | 0.9399 | 0.8957 | 0.9035 | 0.9819 | 0.9245 | 0.7832 | 0.5095 | 0.9001 | 0.9664 | 0.6905 | 0.3578 | 0.0 | 0.9417 | 0.8973 | 0.9628 | 0.9651 | 0.9592 | 0.9111 | 0.9699 | 0.9631 | 0.9419 | 0.9631 | 0.9382 | 0.9388 | 0.8858 | 0.9315 | 0.9081 | 0.9826 | |
| | 0.0592 | 3.2086 | 3000 | 0.0544 | 0.9675 | 0.9787 | 0.0 | 0.9630 | 0.9524 | 0.9192 | 0.9337 | 0.9844 | 0.9457 | 0.8391 | 0.7142 | 0.9139 | 0.9862 | 0.7777 | 0.5887 | 0.2644 | 0.9573 | 0.9166 | 0.9743 | 0.9682 | 0.9680 | 0.9437 | 0.9787 | 0.9420 | 0.9698 | 0.9674 | 0.9516 | 0.9491 | 0.9168 | 0.9492 | 0.9327 | 0.9874 | |
| | 0.0436 | 4.2781 | 4000 | 0.0488 | 0.9673 | 0.9821 | 0.0 | 0.9679 | 0.9709 | 0.9187 | 0.9487 | 0.9836 | 0.9722 | 0.8580 | 0.7335 | 0.9322 | 0.9912 | 0.7998 | 0.6667 | 0.5722 | 0.9432 | 0.9371 | 0.9791 | 0.9778 | 0.9705 | 0.9548 | 0.9831 | 0.9701 | 0.9680 | 0.9673 | 0.9543 | 0.9584 | 0.9331 | 0.9529 | 0.9429 | 0.9888 | |
| | 0.037 | 5.3476 | 5000 | 0.0518 | 0.9653 | 0.9811 | 0.0 | 0.9671 | 0.9660 | 0.9052 | 0.9392 | 0.9859 | 0.9745 | 0.8469 | 0.7616 | 0.9225 | 0.9873 | 0.8108 | 0.7059 | 0.6450 | 0.9578 | 0.9437 | 0.9772 | 0.9774 | 0.9715 | 0.9511 | 0.9827 | 0.9645 | 0.9681 | 0.9639 | 0.9617 | 0.9556 | 0.9283 | 0.9574 | 0.9426 | 0.9883 | |
| | 0.028 | 6.4171 | 6000 | 0.0488 | 0.9624 | 0.9842 | 0.0 | 0.9709 | 0.9732 | 0.9112 | 0.9437 | 0.9869 | 0.9767 | 0.8614 | 0.7818 | 0.9322 | 0.9860 | 0.8266 | 0.7344 | 0.7080 | 0.9518 | 0.9509 | 0.9797 | 0.9802 | 0.9768 | 0.9564 | 0.9831 | 0.9756 | 0.9717 | 0.9714 | 0.9610 | 0.9537 | 0.9383 | 0.9577 | 0.9479 | 0.9891 | |
| | 0.0238 | 7.4866 | 7000 | 0.0483 | 0.9625 | 0.9844 | 0.0 | 0.9705 | 0.9732 | 0.9144 | 0.9360 | 0.9804 | 0.9814 | 0.8654 | 0.7707 | 0.9328 | 0.9885 | 0.8234 | 0.7253 | 0.6873 | 0.9504 | 0.9372 | 0.9787 | 0.9750 | 0.9753 | 0.9523 | 0.9848 | 0.9755 | 0.9717 | 0.9730 | 0.9677 | 0.9567 | 0.9379 | 0.9554 | 0.9466 | 0.9893 | |
| | 0.0197 | 8.5561 | 8000 | 0.0517 | 0.9651 | 0.9857 | 0.0 | 0.9735 | 0.9735 | 0.9100 | 0.9579 | 0.9858 | 0.9679 | 0.8630 | 0.7748 | 0.9375 | 0.9858 | 0.8229 | 0.7259 | 0.6764 | 0.9496 | 0.9571 | 0.9800 | 0.9780 | 0.9753 | 0.9543 | 0.9829 | 0.9769 | 0.9763 | 0.9725 | 0.9637 | 0.9628 | 0.9409 | 0.9579 | 0.9493 | 0.9892 | |
| | 0.0164 | 9.6257 | 9000 | 0.0536 | 0.9642 | 0.9859 | 0.0 | 0.9707 | 0.9628 | 0.9175 | 0.9533 | 0.9857 | 0.9814 | 0.8617 | 0.7674 | 0.9377 | 0.9869 | 0.8193 | 0.7331 | 0.7110 | 0.9471 | 0.9535 | 0.9818 | 0.9756 | 0.9758 | 0.9602 | 0.9829 | 0.9746 | 0.9762 | 0.9710 | 0.9631 | 0.9596 | 0.9396 | 0.9581 | 0.9488 | 0.9893 | |
| | 0.0148 | 10.6952 | 10000 | 0.0545 | 0.9676 | 0.9849 | 0.0 | 0.9728 | 0.9741 | 0.9351 | 0.9563 | 0.9833 | 0.9791 | 0.8693 | 0.7877 | 0.9351 | 0.9863 | 0.8294 | 0.7536 | 0.7332 | 0.9609 | 0.9523 | 0.9808 | 0.9809 | 0.9775 | 0.9514 | 0.9837 | 0.9791 | 0.9713 | 0.9707 | 0.9624 | 0.9602 | 0.9435 | 0.9588 | 0.9511 | 0.9897 | |
| | 0.0115 | 11.7647 | 11000 | 0.0546 | 0.9661 | 0.9849 | 0.0 | 0.9757 | 0.9661 | 0.9133 | 0.9579 | 0.9800 | 0.9769 | 0.8661 | 0.7935 | 0.9439 | 0.9894 | 0.8292 | 0.7485 | 0.7126 | 0.9513 | 0.9607 | 0.9793 | 0.9815 | 0.9770 | 0.9581 | 0.9851 | 0.9803 | 0.9711 | 0.9645 | 0.9672 | 0.9588 | 0.9413 | 0.9597 | 0.9504 | 0.9896 | |
| | 0.0101 | 12.8342 | 12000 | 0.0573 | 0.9634 | 0.9861 | 0.0 | 0.9742 | 0.9693 | 0.9234 | 0.9574 | 0.9850 | 0.9837 | 0.8602 | 0.7854 | 0.9391 | 0.9898 | 0.8220 | 0.7470 | 0.7056 | 0.9515 | 0.9586 | 0.9834 | 0.9803 | 0.9787 | 0.9617 | 0.9841 | 0.9773 | 0.9753 | 0.9691 | 0.9649 | 0.9594 | 0.9459 | 0.9560 | 0.9509 | 0.9898 | |
| | 0.0084 | 13.9037 | 13000 | 0.0597 | 0.9657 | 0.9861 | 0.0 | 0.9761 | 0.9733 | 0.9136 | 0.9542 | 0.9828 | 0.9813 | 0.8672 | 0.7989 | 0.9418 | 0.9889 | 0.8326 | 0.7458 | 0.7409 | 0.9556 | 0.9573 | 0.9815 | 0.9797 | 0.9772 | 0.9616 | 0.9866 | 0.9810 | 0.9784 | 0.9644 | 0.9658 | 0.9609 | 0.9467 | 0.9568 | 0.9517 | 0.9897 | |
| | 0.0065 | 14.9733 | 14000 | 0.0621 | 0.9684 | 0.9859 | 0.0 | 0.9726 | 0.9741 | 0.9277 | 0.9539 | 0.9789 | 0.9814 | 0.8696 | 0.7879 | 0.9348 | 0.9868 | 0.8368 | 0.7542 | 0.7456 | 0.9487 | 0.9543 | 0.9805 | 0.9809 | 0.9780 | 0.9582 | 0.9863 | 0.9801 | 0.9763 | 0.9716 | 0.9629 | 0.9580 | 0.9439 | 0.9590 | 0.9514 | 0.9896 | |
| | 0.0059 | 16.0428 | 15000 | 0.0613 | 0.9679 | 0.9874 | 0.0 | 0.9770 | 0.9694 | 0.9347 | 0.9621 | 0.9786 | 0.9791 | 0.8723 | 0.7857 | 0.9403 | 0.9891 | 0.8414 | 0.7594 | 0.7371 | 0.9508 | 0.9595 | 0.9813 | 0.9797 | 0.9775 | 0.9562 | 0.9856 | 0.9790 | 0.9805 | 0.9725 | 0.9677 | 0.9554 | 0.9457 | 0.9609 | 0.9532 | 0.9901 | |
| | 0.005 | 17.1123 | 16000 | 0.0639 | 0.9693 | 0.9839 | 0.0 | 0.9781 | 0.9735 | 0.9264 | 0.9631 | 0.9827 | 0.9791 | 0.8731 | 0.7996 | 0.9437 | 0.9869 | 0.8406 | 0.7714 | 0.7593 | 0.9547 | 0.9559 | 0.9813 | 0.9809 | 0.9782 | 0.9502 | 0.9849 | 0.9810 | 0.9795 | 0.9731 | 0.9654 | 0.9617 | 0.9460 | 0.9616 | 0.9537 | 0.9901 | |
| | 0.0038 | 18.1818 | 17000 | 0.0651 | 0.9681 | 0.9869 | 0.0 | 0.9785 | 0.9747 | 0.9311 | 0.9606 | 0.9831 | 0.9837 | 0.8749 | 0.7899 | 0.9366 | 0.9889 | 0.8331 | 0.7520 | 0.7230 | 0.9582 | 0.9596 | 0.9805 | 0.9802 | 0.9784 | 0.9609 | 0.9858 | 0.9805 | 0.9800 | 0.9756 | 0.9663 | 0.9614 | 0.9494 | 0.9586 | 0.9540 | 0.9902 | |
| | 0.0035 | 19.2513 | 18000 | 0.0716 | 0.9661 | 0.9857 | 0.0 | 0.9791 | 0.9715 | 0.9319 | 0.9607 | 0.9829 | 0.9791 | 0.8707 | 0.8026 | 0.9385 | 0.9859 | 0.8354 | 0.7557 | 0.7374 | 0.9564 | 0.9580 | 0.9795 | 0.9803 | 0.9767 | 0.9563 | 0.9871 | 0.9823 | 0.9750 | 0.9745 | 0.9654 | 0.9574 | 0.9450 | 0.9610 | 0.9529 | 0.9896 | |
| | 0.0023 | 20.3209 | 19000 | 0.0682 | 0.9686 | 0.9857 | 0.0 | 0.9789 | 0.9755 | 0.9310 | 0.9621 | 0.9850 | 0.9837 | 0.8777 | 0.7974 | 0.9430 | 0.9880 | 0.8424 | 0.7600 | 0.7545 | 0.9566 | 0.9628 | 0.9813 | 0.9773 | 0.9765 | 0.9620 | 0.9863 | 0.9813 | 0.9743 | 0.9742 | 0.9660 | 0.9567 | 0.9474 | 0.9622 | 0.9548 | 0.9901 | |
| | 0.002 | 21.3904 | 20000 | 0.0727 | 0.9696 | 0.9857 | 0.0 | 0.9759 | 0.9742 | 0.9315 | 0.9636 | 0.9814 | 0.9814 | 0.8791 | 0.8011 | 0.9427 | 0.9898 | 0.8383 | 0.7556 | 0.7419 | 0.9588 | 0.9575 | 0.9826 | 0.9756 | 0.9756 | 0.9519 | 0.9853 | 0.9802 | 0.9733 | 0.9749 | 0.9618 | 0.9569 | 0.9459 | 0.9614 | 0.9536 | 0.9900 | |
| | 0.002 | 22.4599 | 21000 | 0.0756 | 0.9690 | 0.9859 | 0.0 | 0.9770 | 0.9752 | 0.9225 | 0.9626 | 0.9829 | 0.9814 | 0.8734 | 0.7850 | 0.9417 | 0.9878 | 0.8312 | 0.7560 | 0.7405 | 0.9570 | 0.9591 | 0.9805 | 0.9814 | 0.9768 | 0.9614 | 0.9858 | 0.9795 | 0.9758 | 0.9700 | 0.9643 | 0.9596 | 0.9452 | 0.9610 | 0.9530 | 0.9898 | |
| | 0.0014 | 23.5294 | 22000 | 0.0746 | 0.9694 | 0.9874 | 0.0 | 0.9779 | 0.9749 | 0.9309 | 0.9653 | 0.9869 | 0.9814 | 0.8739 | 0.8019 | 0.9439 | 0.9884 | 0.8365 | 0.7684 | 0.7559 | 0.9570 | 0.9587 | 0.9808 | 0.9797 | 0.9777 | 0.9570 | 0.9861 | 0.9807 | 0.9758 | 0.9732 | 0.9658 | 0.9625 | 0.9476 | 0.9621 | 0.9548 | 0.9902 | |
| | 0.0012 | 24.5989 | 23000 | 0.0762 | 0.9696 | 0.9864 | 1.0 | 0.9784 | 0.9761 | 0.9389 | 0.9666 | 0.9844 | 0.9814 | 0.8718 | 0.7860 | 0.9440 | 0.9880 | 0.8296 | 0.7611 | 0.7513 | 0.9579 | 0.9621 | 0.9826 | 0.9797 | 0.9777 | 0.9584 | 0.9856 | 0.9810 | 0.9757 | 0.9754 | 0.9673 | 0.9615 | 0.9484 | 0.9614 | 0.9548 | 0.9902 | |
| | 0.0011 | 25.6684 | 24000 | 0.0744 | 0.9698 | 0.9862 | 0.0 | 0.9783 | 0.9785 | 0.9353 | 0.9666 | 0.9832 | 0.9837 | 0.8775 | 0.8007 | 0.9454 | 0.9878 | 0.8417 | 0.7705 | 0.7680 | 0.9592 | 0.9629 | 0.9821 | 0.9785 | 0.9785 | 0.9625 | 0.9858 | 0.9831 | 0.9779 | 0.9756 | 0.9688 | 0.9615 | 0.9502 | 0.9626 | 0.9564 | 0.9904 | |
| | 0.001 | 26.7380 | 25000 | 0.0750 | 0.9702 | 0.9869 | 1.0 | 0.9803 | 0.9752 | 0.9335 | 0.9666 | 0.9831 | 0.9791 | 0.8836 | 0.8048 | 0.9451 | 0.9871 | 0.8435 | 0.7724 | 0.7708 | 0.9589 | 0.9624 | 0.9821 | 0.9774 | 0.9782 | 0.9605 | 0.9871 | 0.9828 | 0.9760 | 0.9762 | 0.9653 | 0.9611 | 0.9499 | 0.9630 | 0.9564 | 0.9904 | |
| | 0.0009 | 27.8075 | 26000 | 0.0764 | 0.9695 | 0.9877 | 1.0 | 0.9798 | 0.9767 | 0.9379 | 0.9647 | 0.9825 | 0.9746 | 0.8781 | 0.7989 | 0.9463 | 0.9878 | 0.8417 | 0.7605 | 0.7708 | 0.9595 | 0.9626 | 0.9826 | 0.9780 | 0.9782 | 0.9609 | 0.9881 | 0.9810 | 0.974 | 0.9761 | 0.9663 | 0.9606 | 0.9493 | 0.9627 | 0.9560 | 0.9903 | |
| | 0.0008 | 28.8770 | 27000 | 0.0767 | 0.9699 | 0.9867 | 1.0 | 0.9788 | 0.9773 | 0.9356 | 0.9654 | 0.9844 | 0.9746 | 0.8798 | 0.7989 | 0.9441 | 0.9880 | 0.8411 | 0.7651 | 0.7676 | 0.9603 | 0.9627 | 0.9815 | 0.9797 | 0.9782 | 0.9592 | 0.9881 | 0.9815 | 0.9765 | 0.9767 | 0.9666 | 0.9604 | 0.9496 | 0.9626 | 0.9561 | 0.9904 | |
| | 0.0008 | 29.9465 | 28000 | 0.0765 | 0.9707 | 0.9869 | 1.0 | 0.9785 | 0.9773 | 0.9381 | 0.9671 | 0.9835 | 0.9746 | 0.8823 | 0.7963 | 0.9426 | 0.9857 | 0.8426 | 0.7621 | 0.7708 | 0.9610 | 0.9621 | 0.9815 | 0.9791 | 0.9782 | 0.9590 | 0.9878 | 0.9815 | 0.9760 | 0.9770 | 0.9668 | 0.9606 | 0.9501 | 0.9623 | 0.9562 | 0.9904 | |
| | 0.0009 | 31.0160 | 29000 | 0.0768 | 0.9705 | 0.9867 | 1.0 | 0.9781 | 0.9773 | 0.9376 | 0.9641 | 0.9852 | 0.9769 | 0.8812 | 0.7989 | 0.9441 | 0.9875 | 0.8435 | 0.7673 | 0.7676 | 0.9610 | 0.9623 | 0.9820 | 0.9791 | 0.9782 | 0.9617 | 0.9871 | 0.9818 | 0.9767 | 0.9767 | 0.9668 | 0.9608 | 0.9504 | 0.9626 | 0.9565 | 0.9904 | |
| | 0.0007 | 32.0856 | 30000 | 0.0767 | 0.9705 | 0.9869 | 1.0 | 0.9781 | 0.9773 | 0.9374 | 0.9645 | 0.9850 | 0.9769 | 0.8810 | 0.7996 | 0.9443 | 0.9873 | 0.8433 | 0.7641 | 0.7696 | 0.9603 | 0.9619 | 0.9820 | 0.9791 | 0.9782 | 0.9615 | 0.9878 | 0.9815 | 0.9767 | 0.9762 | 0.9668 | 0.9606 | 0.9504 | 0.9625 | 0.9564 | 0.9904 | |
|
|
|
|
| ### Framework versions |
|
|
| - Transformers 4.41.2 |
| - Pytorch 2.3.1+cu121 |
| - Datasets 2.20.0 |
| - Tokenizers 0.19.1 |
