Hugging Face's logo

MarinaTA
/
FeverCodeChallenge

Text Classification
Transformers
PyTorch
English
deberta-v2
Model card Files
xet
Community
1
MarinaTA commited on
Commit
87128cb
·
1 Parent(s): a5bfb8c

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +134 -0
README.md ADDED
@@ -0,0 +1,134 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ ---
4
+
5
+ # FeverCodeChallenge
6
+ This model is a fine-tuned version of [microsoft/deberta-v3-small](https://huggingface.co/microsoft/deberta-v3-small) on a a simplified version of [Amazon 2018](https://jmcauley.ucsd.edu/data/amazon/), only containing products and their descriptions.
7
+
8
+ # Model description
9
+ SequenceClassification to predict amazon product's main category (22 categories):
10
+ ```json
11
+ {0: 'All Electronics',
12
+ 1: 'Amazon Fashion',
13
+ 2: 'Amazon Home',
14
+ 3: 'Arts, Crafts & Sewing',
15
+ 4: 'Automotive',
16
+ 5: 'Books',
17
+ 6: 'Camera & Photo',
18
+ 7: 'Cell Phones & Accessories',
19
+ 8: 'Computers',
20
+ 9: 'Digital Music',
21
+ 10: 'Grocery',
22
+ 11: 'Health & Personal Care',
23
+ 12: 'Home Audio & Theater',
24
+ 13: 'Industrial & Scientific',
25
+ 14: 'Movies & TV',
26
+ 15: 'Musical Instruments',
27
+ 16: 'Office Products',
28
+ 17: 'Pet Supplies',
29
+ 18: 'Sports & Outdoors',
30
+ 19: 'Tools & Home Improvement',
31
+ 20: 'Toys & Games',
32
+ 21: 'Video Games'}
33
+
34
+ ```
35
+
36
+
37
+ # Data
38
+
39
+ Example of a product in the dataset
40
+ ```json
41
+ {
42
+ "also_buy": ["B071WSK6R8", "B006K8N5WQ", "B01ASDJLX0", "B00658TPYI"],
43
+ "also_view": [],
44
+ "asin": "B00N31IGPO",
45
+ "brand": "Speed Dealer Customs",
46
+ "category": ["Automotive", "Replacement Parts", "Shocks, Struts & Suspension", "Tie Rod Ends & Parts", "Tie Rod Ends"],
47
+ "description": ["Universal heim joint tie rod weld in tube adapter bung. Made in the USA by Speed Dealer Customs. Tube adapter measurements are as in the title, please contact us about any questions you
48
+ may have."],
49
+ "feature": ["Completely CNC machined 1045 Steel", "Single RH Tube Adapter", "Thread: 3/4-16", "O.D.: 1-1/4", "Fits 1-1/4\" tube with .120\" wall thickness"],
50
+ "image": [],
51
+ "price": "",
52
+ "title": "3/4-16 RH Weld In Threaded Heim Joint Tube Adapter Bung for 1-1/4" Dia by .120 Wall Tube",
53
+ "main_cat": "Automotive"
54
+ }
55
+ ```
56
+
57
+ ## Fields used
58
+ * [Used for the split] **also_buy/also_view**: IDs of related products
59
+ * **description**: description of the product
60
+ * **feature**: bullet point format features of the product
61
+ * **title**: name of the product
62
+ * [label] **main_cat**: main category of the product
63
+
64
+ ## Split of the data
65
+ | | # Samples |
66
+ |------------|-----------|
67
+ | Train | 317662 |
68
+ | Validation | 53890 |
69
+ | Test | 54716 |
70
+
71
+
72
+ # Evaluation results
73
+ | TEST | precision | recall | f1-score | support |
74
+ |--------------|-----------|--------|----------|---------|
75
+ | 0 | 0.56 | 0.36 | 0.44 | 5327 |
76
+ | 1 | 0.80 | 0.84 | 0.82 | 1595 |
77
+ | 2 | 0.79 | 0.70 | 0.74 | 2224 |
78
+ | 3 | 0.79 | 0.79 | 0.79 | 1190 |
79
+ | 4 | 0.88 | 0.92 | 0.90 | 2632 |
80
+ | 5 | 0.98 | 0.97 | 0.98 | 4775 |
81
+ | 6 | 0.72 | 0.77 | 0.74 | 1024 |
82
+ | 7 | 0.61 | 0.68 | 0.64 | 1111 |
83
+ | 8 | 0.77 | 0.81 | 0.79 | 9765 |
84
+ | 9 | 0.94 | 0.94 | 0.94 | 840 |
85
+ | 10 | 0.93 | 0.99 | 0.96 | 1639 |
86
+ | 11 | 0.61 | 0.47 | 0.53 | 539 |
87
+ | 12 | 0.58 | 0.60 | 0.59 | 3802 |
88
+ | 13 | 0.74 | 0.82 | 0.78 | 2476 |
89
+ | 14 | 0.84 | 0.93 | 0.88 | 813 |
90
+ | 15 | 0.84 | 0.87 | 0.86 | 3004 |
91
+ | 16 | 0.68 | 0.63 | 0.66 | 2031 |
92
+ | 17 | 0.89 | 0.88 | 0.89 | 577 |
93
+ | 18 | 0.70 | 0.69 | 0.69 | 1813 |
94
+ | 19 | 0.70 | 0.85 | 0.77 | 3840 |
95
+ | 20 | 0.88 | 0.91 | 0.89 | 3253 |
96
+ | 21 | 0.73 | 0.78 | 0.76 | 446 |
97
+ | accuracy | | | 0.77 | 54716 |
98
+ | macro avg | 0.77 | 0.78 | 0.77 | 54716 |
99
+ | weighted avg | 0.76 | 0.77 | 0.76 | 54716 |
100
+
101
+ | VALIDATION | precision | recall | f1-score | support |
102
+ |--------------|-----------|--------|----------|---------|
103
+ | 0 | 0.52 | 0.23 | 0.32 | 1034 |
104
+ | 1 | 0.79 | 0.84 | 0.82 | 1747 |
105
+ | 2 | 0.79 | 0.70 | 0.74 | 2273 |
106
+ | 3 | 0.85 | 0.85 | 0.85 | 2982 |
107
+ | 4 | 0.89 | 0.92 | 0.91 | 2236 |
108
+ | 5 | 0.97 | 0.98 | 0.97 | 2893 |
109
+ | 6 | 0.86 | 0.73 | 0.79 | 1335 |
110
+ | 7 | 0.70 | 0.76 | 0.73 | 837 |
111
+ | 8 | 0.74 | 0.71 | 0.73 | 790 |
112
+ | 9 | 0.92 | 0.97 | 0.95 | 839 |
113
+ | 10 | 0.96 | 0.99 | 0.97 | 13182 |
114
+ | 11 | 0.47 | 0.25 | 0.33 | 907 |
115
+ | 12 | 0.56 | 0.64 | 0.59 | 965 |
116
+ | 13 | 0.82 | 0.84 | 0.83 | 2780 |
117
+ | 14 | 0.94 | 0.92 | 0.93 | 1245 |
118
+ | 15 | 0.85 | 0.91 | 0.88 | 930 |
119
+ | 16 | 0.82 | 0.87 | 0.85 | 3226 |
120
+ | 17 | 0.95 | 0.96 | 0.96 | 2633 |
121
+ | 18 | 0.73 | 0.69 | 0.71 | 2518 |
122
+ | 19 | 0.67 | 0.80 | 0.73 | 2303 |
123
+ | 20 | 0.91 | 0.90 | 0.91 | 6032 |
124
+ | 21 | 0.70 | 0.90 | 0.79 | 203 |
125
+ | accuracy | | | 0.86 | 53890 |
126
+ | macro avg | 0.79 | 0.79 | 0.78 | 53890 |
127
+ | weighted avg | 0.86 | 0.86 | 0.86 | 53890 |
128
+
129
+ # Training results
130
+ | train_runtime | train_samples_per_second | train_steps_per_second | train_loss | epoch |
131
+ |---------------|--------------------------|------------------------|--------------------|-------|
132
+ | 22408.2701 | 21.94 | 2.743 | 0.6738368626461785 | 1 |
133
+
134
+