File size: 5,852 Bytes
87128cb e275322 87128cb ab962b7 87128cb ab962b7 87128cb ab962b7 87128cb d769a74 87128cb dd1a5c8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 | ---
license: apache-2.0
language:
- en
metrics:
- accuracy
- precision
- recall
pipeline_tag: text-classification
---
# FeverCodeChallenge
This model is a fine-tuned version of [microsoft/deberta-v3-small](https://huggingface.co/microsoft/deberta-v3-small) on a a simplified version of [Amazon 2018](https://jmcauley.ucsd.edu/data/amazon/), only containing products and their descriptions.
# Model description
SequenceClassification to predict amazon product's main category (22 categories):
```json
{0: 'All Electronics',
1: 'Amazon Fashion',
2: 'Amazon Home',
3: 'Arts, Crafts & Sewing',
4: 'Automotive',
5: 'Books',
6: 'Camera & Photo',
7: 'Cell Phones & Accessories',
8: 'Computers',
9: 'Digital Music',
10: 'Grocery',
11: 'Health & Personal Care',
12: 'Home Audio & Theater',
13: 'Industrial & Scientific',
14: 'Movies & TV',
15: 'Musical Instruments',
16: 'Office Products',
17: 'Pet Supplies',
18: 'Sports & Outdoors',
19: 'Tools & Home Improvement',
20: 'Toys & Games',
21: 'Video Games'}
```
# Data
Example of a product in the dataset
```json
{
"also_buy": ["B071WSK6R8", "B006K8N5WQ", "B01ASDJLX0", "B00658TPYI"],
"also_view": [],
"asin": "B00N31IGPO",
"brand": "Speed Dealer Customs",
"category": ["Automotive", "Replacement Parts", "Shocks, Struts & Suspension", "Tie Rod Ends & Parts", "Tie Rod Ends"],
"description": ["Universal heim joint tie rod weld in tube adapter bung. Made in the USA by Speed Dealer Customs. Tube adapter measurements are as in the title, please contact us about any questions you
may have."],
"feature": ["Completely CNC machined 1045 Steel", "Single RH Tube Adapter", "Thread: 3/4-16", "O.D.: 1-1/4", "Fits 1-1/4\" tube with .120\" wall thickness"],
"image": [],
"price": "",
"title": "3/4-16 RH Weld In Threaded Heim Joint Tube Adapter Bung for 1-1/4" Dia by .120 Wall Tube",
"main_cat": "Automotive"
}
```
## Fields used
* [Used for the split] **also_buy/also_view**: IDs of related products
* **description**: description of the product
* **feature**: bullet point format features of the product
* **title**: name of the product
* [label] **main_cat**: main category of the product
## Split of the data
| | # Samples |
|------------|-----------|
| Train | 317662 |
| Validation | 53890 |
| Test | 54716 |
# Evaluation results
| TEST | precision | recall | f1-score | support |
|--------------|-----------|--------|----------|---------|
| 0 | 0.56 | 0.42 | 0.48 | 5327 |
| 1 | 0.81 | 0.86 | 0.83 | 1595 |
| 2 | 0.75 | 0.76 | 0.76 | 2224 |
| 3 | 0.80 | 0.82 | 0.81 | 1190 |
| 4 | 0.93 | 0.92 | 0.93 | 2632 |
| 5 | 0.99 | 0.97 | 0.98 | 4775 |
| 6 | 0.74 | 0.80 | 0.77 | 1024 |
| 7 | 0.71 | 0.64 | 0.67 | 1111 |
| 8 | 0.79 | 0.80 | 0.80 | 9765 |
| 9 | 0.94 | 0.93 | 0.94 | 840 |
| 10 | 0.94 | 0.98 | 0.96 | 1639 |
| 11 | 0.62 | 0.52 | 0.56 | 539 |
| 12 | 0.57 | 0.74 | 0.64 | 3802 |
| 13 | 0.79 | 0.84 | 0.81 | 2476 |
| 14 | 0.83 | 0.94 | 0.88 | 813 |
| 15 | 0.88 | 0.87 | 0.87 | 3004 |
| 16 | 0.76 | 0.61 | 0.68 | 2031 |
| 17 | 0.88 | 0.88 | 0.88 | 577 |
| 18 | 0.73 | 0.71 | 0.72 | 1813 |
| 19 | 0.79 | 0.85 | 0.82 | 3840 |
| 20 | 0.89 | 0.91 | 0.90 | 3253 |
| 21 | 0.69 | 0.75 | 0.72 | 446 |
| accuracy | | | 0.79 | 54716 |
| macro avg | 0.79 | 0.80 | 0.79 | 54716 |
| weighted avg | 0.79 | 0.79 | 0.79 | 54716 |
| VALIDATION | precision | recall | f1-score | support |
|--------------|-----------|--------|----------|---------|
| 0 | 0.55 | 0.32 | 0.40 | 1034 |
| 1 | 0.79 | 0.85 | 0.82 | 1747 |
| 2 | 0.75 | 0.78 | 0.76 | 2273 |
| 3 | 0.84 | 0.88 | 0.86 | 2982 |
| 4 | 0.93 | 0.92 | 0.93 | 2236 |
| 5 | 0.97 | 0.98 | 0.97 | 2893 |
| 6 | 0.88 | 0.76 | 0.81 | 1335 |
| 7 | 0.77 | 0.74 | 0.75 | 837 |
| 8 | 0.76 | 0.73 | 0.74 | 790 |
| 9 | 0.95 | 0.96 | 0.95 | 839 |
| 10 | 0.96 | 0.98 | 0.97 | 13182 |
| 11 | 0.50 | 0.30 | 0.37 | 907 |
| 12 | 0.55 | 0.74 | 0.64 | 965 |
| 13 | 0.83 | 0.86 | 0.85 | 2780 |
| 14 | 0.93 | 0.94 | 0.93 | 1245 |
| 15 | 0.89 | 0.92 | 0.91 | 930 |
| 16 | 0.87 | 0.85 | 0.86 | 3226 |
| 17 | 0.96 | 0.97 | 0.96 | 2633 |
| 18 | 0.75 | 0.71 | 0.73 | 2518 |
| 19 | 0.74 | 0.81 | 0.77 | 2303 |
| 20 | 0.92 | 0.91 | 0.92 | 6032 |
| 21 | 0.72 | 0.89 | 0.79 | 203 |
| accuracy | | | 0.87 | 53890 |
| macro avg | 0.81 | 0.81 | 0.81 | 53890 |
| weighted avg | 0.87 | 0.87 | 0.87 | 53890 |
# Training results
| train_runtime | train_samples_per_second | train_steps_per_second | eval_loss | epoch |
|---------------|--------------------------|------------------------|--------------------|-------|
| 48601.2302 | 13.072 | 1.634 | 0.5335464077893132 | 2 | |