File size: 3,676 Bytes
08b0a3e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 |
---
base_model: x2bee/ModernBert_MLM_kotoken_v01
model-index:
- name: plateer_classifier_ModernBERT_v01
results: []
---
# plateer_classifier_ModernBERT_v01
This model is a fine-tuned version of [x2bee/ModernBert_MLM_kotoken_v01](https://huggingface.co/x2bee/ModernBert_MLM_kotoken_v01) on [x2bee/plateer_category_data](https://huggingface.co/datasets/x2bee/plateer_category_data). <br>
It achieves the following results on the evaluation set:
- Loss: 0.3379
#### Example Use
```python
import joblib;
from huggingface_hub import hf_hub_download;
from peft import PeftModel, PeftConfig;
from transformers import AutoTokenizer, TextClassificationPipeline, AutoModelForSequenceClassification;
from huggingface_hub import HfApi, login
# need hgf token for accessing X2BEE repo.
with open('./api_key/HGF_TOKEN.txt', 'r') as hgf:
login(token=hgf.read())
api = HfApi()
repo_id = "x2bee/plateer_classifier_ModernBERT_v01"
data_id = "x2bee/plateer_category_data"
# Load Config, Tokenizer, Label_Encoder
tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="last-checkpoint")
label_encoder_file = hf_hub_download(repo_id=data_id, repo_type="dataset", filename="label_encoder.joblib")
label_encoder = joblib.load(label_encoder_file)
# Load Model
model = AutoModelForSequenceClassification.from_pretrained(repo_id, subfolder="last-checkpoint")
import torch
class TextClassificationPipeline(TextClassificationPipeline):
def __call__(self, inputs, top_k=5, **kwargs):
inputs = self.tokenizer(inputs, return_tensors="pt", truncation=True, padding=True, max_length=512, **kwargs)
inputs = {k: v.to(self.model.device) for k, v in inputs.items()}
with torch.no_grad():
outputs = self.model(**inputs)
probs = torch.nn.functional.softmax(outputs.logits, dim=-1)
scores, indices = torch.topk(probs, top_k, dim=-1)
results = []
for batch_idx in range(indices.shape[0]):
batch_results = []
for score, idx in zip(scores[batch_idx], indices[batch_idx]):
temp_list = []
label = self.model.config.id2label[idx.item()]
label = int(label.split("_")[1])
temp_list.append(label)
predicted_class = label_encoder.inverse_transform(temp_list)[0]
batch_results.append({
"label": label,
"label_decode": predicted_class,
"score": score.item(),
})
results.append(batch_results)
return results
classifier_model = TextClassificationPipeline(tokenizer=tokenizer, model=model)
def plateer_classifier(text, top_k=3):
result = classifier_model(text, top_k=top_k)
return result
# run
result = plateer_classifier("겨울 등산에서 사용할 옷")[0]
print(result)
# result
-----------Category-----------
{'label': 2, 'label_decode': '기능성의류', 'score': 0.9214227795600891}
{'label': 8, 'label_decode': '스포츠', 'score': 0.07054771482944489}
{'label': 15, 'label_decode': '패션/의류/잡화', 'score': 0.0036312134470790625}
```
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-4
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 10000
- num_epochs: 3
### Framework versions
- Transformers 4.48 |