JonusNattapong commited on
Commit
bf014e8
·
verified ·
1 Parent(s): f0a5d8a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +55 -402
README.md CHANGED
@@ -114,7 +114,7 @@ datasets:
114
  - ZombitX64/Sentiment-Benchmark
115
  ---
116
 
117
- # MultiSent-E5
118
 
119
  <div align="center">
120
  <picture>
@@ -221,7 +221,7 @@ from transformers import AutoTokenizer, AutoModelForSequenceClassification
221
  import torch
222
 
223
  # Load the model and tokenizer
224
- model_name = "ZombitX64/MultiSent-E5"
225
  tokenizer = AutoTokenizer.from_pretrained(model_name)
226
  model = AutoModelForSequenceClassification.from_pretrained(model_name)
227
 
@@ -280,8 +280,8 @@ from transformers import pipeline
280
 
281
  # Create a sentiment analysis pipeline
282
  classifier = pipeline("text-classification",
283
- model="ZombitX64/MultiSent-E5",
284
- tokenizer="ZombitX64/MultiSent-E5")
285
 
286
  # Analyze sentiment
287
  texts = [
@@ -364,148 +364,56 @@ The model showed excellent convergence with minimal overfitting:
364
  - Early convergence suggests effective transfer learning from the base model
365
 
366
  ============================================================
367
- Evaluating Model: MultiSent-E5
368
  ============================================================
369
- Accuracy: 0.849
370
- F1-Macro: 0.839
371
- F1-Weighted: 0.850
372
-
373
- === ERROR ANALYSIS FOR MultiSent-E5 ===
374
- Total Errors: 26 / 172 (15.1%)
375
-
376
- Error Types:
377
- error_type
378
- negative -> positive 7
379
- question -> neutral 7
380
- neutral -> positive 3
381
- negative -> neutral 3
382
- question -> positive 2
383
- positive -> neutral 2
384
- neutral -> negative 1
385
- neutral -> question 1
386
- Name: count, dtype: int64
387
-
388
- Low Confidence Errors (< 60%): 4
389
- High Confidence Errors (> 80%): 19
390
-
391
- === ERROR EXAMPLES ===
392
-
393
- negative -> positive:
394
- Text: 'สุดยอดไปเลย! เธอใช้เวลาทั้งวันทำงานชิ้นนี้ออกมาได้แค่นี้เองเหรอเนี่ย!'
395
- Confidence: 0.517
396
- Text: 'ไอเดียสร้างสรรค์มาก! ไม่มีใครคิดจะเสนออะไรที่ไม่มีทางเป็นไปได้แบบนี้หรอก'
397
- Confidence: 1.000
398
- Text: 'ไอเดียสร้างสรรค์มาก! ไม่มีใครคิดจะเสนออะไรที่ไม่มีทางเป็นไปได้แบบนี้หรอก'
399
- Confidence: 1.000
400
-
401
- question -> neutral:
402
- Text: 'คุณคิดว่าอย่างไรกับเรื่องนี้'
403
- Confidence: 0.999
404
- Text: 'How was your day today?'
405
- Confidence: 1.000
406
- Text: '你觉得怎么样?'
407
- Confidence: 0.999
408
-
409
- neutral -> positive:
410
- Text: 'ก็แข็งแรงอยู่นะ'
411
- Confidence: 0.727
412
- Text: 'ก็แข็งแรงอยู่นะ'
413
- Confidence: 0.727
414
- Text: 'บรรยากาศดีมาก เหมาะกับการนั่งเงียบๆ คนเดียว'
415
- Confidence: 0.723
416
-
417
- negative -> neutral:
418
- Text: 'Good day. Unfortunately, I had to walk 10 kilometers from home to school, and now I'm feeling quite ...'
419
- Confidence: 0.970
420
- Text: 'Good day. Unfortunately, I had to walk 10 kilometers from home to school, and now I'm feeling quite ...'
421
- Confidence: 0.970
422
- Text: 'ส่งของไวมาก...ถ้านับวันเป็นเดือน'
423
- Confidence: 0.999
424
-
425
- question -> positive:
426
- Text: 'ลำไยอร่อยดีสดมากและลูกใหญ่ด้วยแต่เน่าไปครึ่งนึงมั้ย'
427
- Confidence: 0.550
428
- Text: 'ลำไยอร่อยดีสดมากและลูกใหญ่ด้วยแต่เน่าไปครึ่งนึงมั้ย'
429
- Confidence: 0.550
430
-
431
- === LOW CONFIDENCE PREDICTIONS ===
432
- Total Low Confidence: 7 (4.1%)
433
-
434
- Low Confidence Examples:
435
- 'ลำไยอร่อยดีสดมากและลูกใหญ่ด้วยแต่เน่าไปครึ่งนึงมั้ย'
436
- Predicted: positive, Confidence: 0.550
437
- True: question, Correct: False
438
-
439
- 'ลำไยอร่อยดีสดมากและลูกใหญ่ด้วยแต่เน่าไปครึ่งนึงรึเปล่า'
440
- Predicted: question, Confidence: 0.521
441
- True: question, Correct: True
442
-
443
- 'สุดยอดไปเลย! เธอใช้เวลาทั้งวันทำงานชิ้นนี้ออกมาได้แค่นี้เองเหรอเนี่ย!'
444
- Predicted: positive, Confidence: 0.517
445
- True: negative, Correct: False
446
-
447
- 'เกือบดีแล้วล่ะ เหลือแค่ดีจริงๆ นิดเดียว'
448
- Predicted: neutral, Confidence: 0.546
449
- True: neutral, Correct: True
450
-
451
- 'ลำไยอร่อยดีสดมากและลูกใหญ่ด้วยแต่เน่าไปคร��่งนึงมั้ย'
452
- Predicted: positive, Confidence: 0.550
453
- True: question, Correct: False
454
-
455
-
456
- ### 📊 **สรุปผลการประเมินโมเดล: MultiSent-E5**
457
-
458
- | Metric | ค่า (Value) |
459
- | --------------------------------- | ----------- |
460
- | **Accuracy** | **84.9%** |
461
- | **F1 Macro** | 83.9% |
462
- | **F1 Weighted** | 85.0% |
463
- | **จำนวนตัวอย่างทั้งหมด** | 172 |
464
- | **จำนวนข้อผิดพลาด (Error)** | 26 |
465
- | **เปอร์เซ็นต์ความผิดพลาด** | 15.1% |
466
- | **Low Confidence Errors (<60%)** | 4 |
467
- | **High Confidence Errors (>80%)** | 19 |
468
 
469
- ---
470
-
471
- ### 🧩 **ประเภทความผิดพลาด (Error Types)**
472
-
473
- | ผิดจาก (True Label) | เป็น (Predicted Label) | จำนวนครั้ง (Count) |
474
- | ------------------- | ---------------------- | ------------------ |
475
- | negative | positive | 7 |
476
- | question | neutral | 7 |
477
- | neutral | positive | 3 |
478
- | negative | neutral | 3 |
479
- | question | positive | 2 |
480
- | positive | neutral | 2 |
481
- | neutral | negative | 1 |
482
- | neutral | question | 1 |
483
-
484
- ---
485
-
486
- ### 🔍 **ตัวอย่าง Error ที่น่าสนใจ**
487
-
488
- #### 1. **ประชด/เสียดสี** ที่ผิดเป็น Positive
489
-
490
- | ข้อความ | ทำนาย | จริง | Confidence |
491
- | ------------------------------------------------------------------------ | -------- | -------- | ---------- |
492
- | สุดยอดไปเลย! เธอใช้เวลาทั้งวันทำงานชิ้นนี้ออกมาได้แค่นี้เองเหรอเนี่ย! | positive | negative | 0.517 |
493
- | ไอเดียสร้างสรรค์มาก! ไม่มีใครคิดจะเสนออะไรที่ไม่มีทางเป็นไปได้แบบนี้หรอก | positive | negative | 1.000 |
494
-
495
- #### 2. **คำถาม** ผิดเป็น Neutral
496
-
497
- | ข้อความ | ทำนาย | จริง | Confidence |
498
- | ---------------------------- | ------- | -------- | ---------- |
499
- | คุณคิดว่าอย่างไรกับเรื่องนี้ | neutral | question | 0.999 |
500
- | How was your day today? | neutral | question | 1.000 |
501
- | 你觉得怎么样? | neutral | question | 0.999 |
502
-
503
- #### 3. **ประโยคคลุมเครือที่ Low Confidence**
504
 
505
- | ข้อความ | ทำนาย | จริง | Confidence |
506
- | --------------------------------------------------- | -------- | -------- | ---------- |
507
- | ลำไยอร่อยดีสดมากและลูกใหญ่ด้วยแต่เน่าไปครึ่งนึงมั้ย | positive | question | 0.550 |
508
- | เกือบดีแล้วล่ะ เหลือแค่ดีจริงๆ นิดเดียว | neutral | neutral | 0.546 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
509
 
510
 
511
 
@@ -515,7 +423,7 @@ Low Confidence Examples:
515
 
516
  The model was evaluated on a carefully selected validation set with the following characteristics:
517
 
518
- * **Total Validation Samples:** 273
519
  * **Selection Method:** Stratified random sampling to maintain class distribution
520
  * **Data Quality:** Manually verified and cleaned validation samples
521
  * **Evaluation Period:** Final model checkpoint from epoch 5
@@ -532,95 +440,6 @@ The model was comprehensively evaluated using multiple metrics:
532
  - **Recall:** Per-class and overall recall scores
533
  - **Support:** Number of samples per class in validation set
534
 
535
- ### Results
536
-
537
- #### Final Test Results
538
-
539
- **Per-Class Performance:**
540
-
541
- | Class | Precision | Recall | F1-Score | Support | Performance Notes |
542
- |-------|-----------|---------|----------|---------|-------------------|
543
- | Question | N/A | N/A | N/A | 0 | No question samples in validation set |
544
- | Negative | 1.00 | 1.00 | 1.00 | 231 | Perfect classification |
545
- | Neutral | 1.00 | 0.90 | 0.95 | 10 | 1 misclassification due to small sample size |
546
- | Positive | 1.00 | 1.00 | 1.00 | 32 | Perfect classification |
547
-
548
- **Overall Performance Summary:**
549
-
550
- | Metric | Value | Interpretation |
551
- |--------|-------|----------------|
552
- | **Overall Accuracy** | 100% (273/273) | Exceptional performance |
553
- | **Macro Average F1** | 0.98 | Excellent across all represented classes |
554
- | **Weighted Average F1** | 1.00 | Perfect when weighted by class frequency |
555
- | **Total Correct Predictions** | 272/273 | Only 1 misclassification |
556
-
557
- #### Detailed Confusion Matrix Results
558
-
559
- **Classification Breakdown:**
560
- - **Negative Class:** 231/231 correctly classified (100% accuracy)
561
- - **Neutral Class:** 9/10 correctly classified (90% accuracy)
562
- - 1 neutral sample misclassified (likely as positive due to ambiguous language)
563
- - **Positive Class:** 32/32 correctly classified (100% accuracy)
564
- - **Question Class:** Not present in validation set
565
-
566
- ### Model Capabilities
567
-
568
- #### Demonstrated Strengths
569
-
570
- The model shows exceptional capability in understanding various aspects of Thai sentiment:
571
-
572
- **1. Straightforward Sentiment Classification:**
573
- - Clear positive expressions: "วันนี้อากาศดีจังเลย" (The weather is so nice today) → Positive (99.96%)
574
- - Clear negative expressions: "แย่ที่สุดเท่าที่เคยเจอมา" (The worst I've ever encountered) → Negative (99.99%)
575
- - Neutral expressions: "ก็งั้นๆ แหละ ไม่มีอะไรพิเศษ" (It's just okay, nothing special) → Neutral (99.70%)
576
-
577
- **2. Advanced Linguistic Understanding:**
578
-
579
- **Sarcasm Detection:**
580
- - "เก่งจังเลยนะ ทำผิดซ้ำได้เหมือนเดิมเป๊ะเลย"
581
- (So talented! You can make the same mistake repeatedly) → Negative (99.99%)
582
- - The model correctly identifies that "เก่งจัง" (so talented) is used sarcastically
583
-
584
- **Implicit Criticism:**
585
- - "ไอเดียสร้างสรรค์มาก! ไม่มีใครคิดจะเสนออะไรที่ไม่มีทางเป็นไปได้แบบนี้หรอก"
586
- (Very creative idea! No one would think to propose something this impossible) → Negative (99.43%)
587
- - Successfully detects negative sentiment despite seemingly positive words
588
-
589
- **3. Cultural Context Understanding:**
590
- - Thai-specific expressions and idioms
591
- - Formal vs. informal language registers
592
- - Regional variations in expression
593
-
594
- #### Performance Analysis by Text Type
595
-
596
- | Text Type | Accuracy | Confidence Range | Notes |
597
- |-----------|----------|------------------|--------|
598
- | Direct statements | 99-100% | 90-100% | Excellent performance |
599
- | Sarcastic content | 95-99% | 85-99% | Very good sarcasm detection (e.g., "เก่งจังเลยนะ ทำผิดซ้ำได้เหมือนเดิมเป๊ะเลย" → 99.98% negative) |
600
- | Implicit sentiment | 90-95% | 80-95% | Good at reading between the lines |
601
- | **Mixed sentiment** | **60-75%** | **50-60%** | **Struggles with texts containing both positive and negative aspects** |
602
- | **Question-like text** | **40-60%** | **50-60%** | **Poor question detection, often classified as other categories** |
603
- | Star ratings | 95-100% | 99%+ | Excellent (e.g., "ให้5ดาวเลย" → 99.98% positive, "ให้1ดาวเลย" → 99.49% negative) |
604
- | Formal language | 98-100% | 85-100% | Strong performance on formal text |
605
- | Colloquial language | 95-99% | 80-95% | Handles informal text well |
606
-
607
- #### Real-World Performance Issues
608
-
609
- **Low Confidence Predictions (< 60%):**
610
- Based on empirical testing, these text types frequently produce low confidence:
611
-
612
- 1. **Mixed Sentiment Examples:**
613
- - "ลำไยอร่อยดีสดมากและลูกใหญ่ด้วยแต่เน่าไปครึ่งนึ..." → Positive (55.0%) or Question (52.1%)
614
- - "เกือบดีแล้วล่ะ เหลือแค่ดีจริงๆ นิดเดียว" → Neutral (54.6%)
615
-
616
- 2. **Ambiguous Praise with Criticism:**
617
- - "สุดยอดไปเลย! เธอใช้เวลาทั้งวันทำงานชิ้นนี้ออกม..." → Positive (51.7%)
618
-
619
- **High Confidence Predictions (> 99%):**
620
- The model excels at:
621
- - Clear sarcasm: "เก่งจังเลยนะ ทำผิดซ้ำได้เหมือนเดิมเป๊ะเลย" → Negative (99.98%)
622
- - Obvious negative sentiment: "ไม่ให้ดาวเลย" → Negative (99.94%)
623
- - Simple positive expressions: "ให้5ดาวเลย" → Positive (99.98%)
624
 
625
  #### Known Limitations
626
 
@@ -731,166 +550,6 @@ The model excels at:
731
 
732
  ### Best Practices for Implementation
733
 
734
- #### Text Preprocessing
735
- ```python
736
- def preprocess_thai_text(text):
737
- """
738
- Recommended preprocessing for Thai text
739
- """
740
- # Remove excessive whitespace
741
- text = ' '.join(text.split())
742
-
743
- # Handle common Thai punctuation
744
- text = text.replace('...', ' ')
745
- text = text.replace('!!', '!')
746
-
747
- # Normalize quotation marks
748
- text = text.replace('"', '"').replace('"', '"')
749
-
750
- return text.strip()
751
- ```
752
-
753
- #### Confidence Thresholding
754
- ```python
755
- def classify_with_confidence(text, threshold=0.6):
756
- """
757
- Classification with confidence thresholding
758
- Recommended threshold: 0.6 based on empirical testing
759
- """
760
- inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
761
-
762
- with torch.no_grad():
763
- outputs = model(**inputs)
764
- predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
765
- confidence = torch.max(predictions).item()
766
- predicted_class = torch.argmax(predictions, dim=-1).item()
767
-
768
- if confidence >= threshold:
769
- return labels[predicted_class], confidence
770
- else:
771
- return "Low Confidence - Manual Review Needed", confidence
772
-
773
- # Enhanced classification with question detection fallback
774
- def enhanced_classify(text, confidence_threshold=0.6):
775
- """
776
- Enhanced classification with special handling for low confidence
777
- and potential question detection
778
- """
779
- sentiment, confidence = classify_with_confidence(text, confidence_threshold)
780
-
781
- # Special handling for low confidence predictions
782
- if confidence < confidence_threshold:
783
- # Simple question detection fallback
784
- question_indicators = ['?', 'ไหม', 'หรือ', 'ครับ', 'คะ', 'มั้ย']
785
- if any(indicator in text for indicator in question_indicators):
786
- return "Question (Detected by Rules)", confidence
787
- else:
788
- return f"Uncertain ({sentiment})", confidence
789
-
790
- return sentiment, confidence
791
-
792
- # Example usage with test cases
793
- test_texts = [
794
- "ลำไยอร่อยดีสดมากและลูกใหญ่ด้วยแต่เน่าไปครึ่งนึ...", # Mixed sentiment
795
- "สุดยอดไปเลย! เธอใช้เวลาทั้งวันทำงานชิ้นนี้ออกม...", # Low confidence positive
796
- "เก่งจังเลยนะ ทำผิดซ้ำ��ด้เหมือนเดิมเป๊ะเลย", # High confidence sarcasm
797
- ]
798
-
799
- for text in test_texts:
800
- result, conf = enhanced_classify(text)
801
- print(f"Text: {text[:50]}...")
802
- print(f"Result: {result} (Confidence: {conf:.1%})")
803
- print()
804
- ```
805
-
806
- #### Production Deployment Example
807
- ```python
808
- from fastapi import FastAPI
809
- from pydantic import BaseModel
810
- import logging
811
-
812
- app = FastAPI()
813
-
814
- class SentimentRequest(BaseModel):
815
- text: str
816
-
817
- class SentimentResponse(BaseModel):
818
- sentiment: str
819
- confidence: float
820
- warning: str = None
821
-
822
- def classify_with_warnings(text, confidence_threshold=0.6):
823
- """
824
- Production-ready classification with warnings for low confidence
825
- """
826
- inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
827
-
828
- with torch.no_grad():
829
- outputs = model(**inputs)
830
- predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
831
- confidence = torch.max(predictions).item()
832
- predicted_class = torch.argmax(predictions, dim=-1).item()
833
-
834
- sentiment = labels[predicted_class]
835
- warning = None
836
-
837
- # Add warnings based on empirical testing
838
- if confidence < confidence_threshold:
839
- warning = "Low confidence prediction - manual review recommended"
840
-
841
- if predicted_class == 0: # Question class
842
- warning = "Question classification has known accuracy issues - consider manual review"
843
-
844
- # Detect potential mixed sentiment
845
- if confidence < 0.7 and any(pos_word in text for pos_word in ['ดี', 'อร่อย', 'สวย']) and any(neg_word in text for neg_word in ['แย่', 'เน่า', 'แต่']):
846
- warning = "Possible mixed sentiment detected - consider aspect-based analysis"
847
-
848
- return sentiment, confidence, warning
849
-
850
- @app.post("/analyze-sentiment", response_model=SentimentResponse)
851
- async def analyze_sentiment(request: SentimentRequest):
852
- try:
853
- # Preprocess text
854
- text = preprocess_thai_text(request.text)
855
-
856
- # Get prediction with warnings
857
- sentiment, confidence, warning = classify_with_warnings(text)
858
-
859
- # Log low confidence predictions for monitoring
860
- if confidence < 0.6:
861
- logging.warning(f"Low confidence prediction: {text[:50]}... -> {sentiment} ({confidence:.3f})")
862
-
863
- return SentimentResponse(
864
- sentiment=sentiment,
865
- confidence=confidence,
866
- warning=warning
867
- )
868
-
869
- except Exception as e:
870
- logging.error(f"Error processing text: {str(e)}")
871
- return SentimentResponse(
872
- sentiment="Error",
873
- confidence=0.0,
874
- warning="Processing error occurred"
875
- )
876
-
877
- # Batch processing endpoint for efficiency
878
- @app.post("/analyze-batch")
879
- async def analyze_batch(texts: list[str]):
880
- """
881
- Batch processing for multiple texts
882
- """
883
- results = []
884
- for text in texts:
885
- sentiment, confidence, warning = classify_with_warnings(text)
886
- results.append({
887
- "text": text[:100] + "..." if len(text) > 100 else text,
888
- "sentiment": sentiment,
889
- "confidence": confidence,
890
- "warning": warning
891
- })
892
- return {"results": results}
893
- ```
894
 
895
  ## Citation
896
 
@@ -898,21 +557,15 @@ async def analyze_batch(texts: list[str]):
898
 
899
  **BibTeX:**
900
  ```bibtex
901
- @misc{MultiSent-E5,
902
  title={Thai-sentiment-e5: A Fine-tuned Multilingual Sentiment Analysis Model for Thai Text Classification},
903
  author={ZombitX64 and Janutsaha, Krittanut and Saengwichain, Chanyut},
904
  year={2024},
905
- url={https://huggingface.co/ZombitX64/MultiSent-E5},
906
  note={Hugging Face Model Repository}
907
  }
908
  ```
909
 
910
- **APA Style:**
911
- ZombitX64, Janutsaha, K., & Saengwichain, C. (2024). *MultiSent-E5: A Fine-tuned Multilingual Sentiment Analysis Model for Thai Text Classification*. Hugging Face. https://huggingface.co/ZombitX64/MultiSent-E5
912
-
913
- **IEEE Style:**
914
- ZombitX64, K. Janutsaha, and C. Saengwichain, "MultiSent-E5: A Fine-tuned Multilingual Sentiment Analysis Model for Thai Text Classification," Hugging Face, 2024. [Online]. Available: https://huggingface.co/ZombitX64/MultiSent-E5
915
-
916
  ### Usage in Publications
917
 
918
  If you use this model in your research or applications, please cite both this model and the base model:
@@ -939,7 +592,7 @@ If you use this model in your research or applications, please cite both this mo
939
  For questions, issues, or contributions regarding this model, please use the following channels:
940
 
941
  * **Primary Contact:** Hugging Face model repository issues and discussions
942
- * **Repository:** [https://huggingface.co/ZombitX64/MultiSent-E5](https://huggingface.co/ZombitX64/MultiSent-E5)
943
  * **Community:** Hugging Face community forums for general questions
944
 
945
  ### Collaboration Opportunities
 
114
  - ZombitX64/Sentiment-Benchmark
115
  ---
116
 
117
+ # ZombitX64-MultiSent-E5-Pro
118
 
119
  <div align="center">
120
  <picture>
 
221
  import torch
222
 
223
  # Load the model and tokenizer
224
+ model_name = "ZombitX64/MultiSent-E5-Pro"
225
  tokenizer = AutoTokenizer.from_pretrained(model_name)
226
  model = AutoModelForSequenceClassification.from_pretrained(model_name)
227
 
 
280
 
281
  # Create a sentiment analysis pipeline
282
  classifier = pipeline("text-classification",
283
+ model="ZombitX64/MultiSent-E5-Pro",
284
+ tokenizer="ZombitX64/MultiSent-E5-Pro")
285
 
286
  # Analyze sentiment
287
  texts = [
 
364
  - Early convergence suggests effective transfer learning from the base model
365
 
366
  ============================================================
367
+ Evaluating: ZombitX64/MultiSent-E5-Pro
368
  ============================================================
369
+ Loading ZombitX64/MultiSent-E5-Pro...
370
+ Predicting 2183 samples...
371
+ Predicting: 2183/2183
372
+ Accuracy: 0.846
373
+ F1-Macro: 0.846
374
+ F1-Weighted: 0.847
375
+ Avg Confidence: 0.985
376
+ Low Confidence %: 1.0%
377
+ Error Rate: 0.154
378
+
379
+ Sample Errors:
380
+ '今天的表现无可挑剔' -> neutral (conf: 1.00) [True: positive]
381
+ '这真是个天才的想法,我简直佩服得五体投地' -> positive (conf: 1.00) [True: negative]
382
+ '你真是太能干了,把事情搞成这样' -> positive (conf: 1.00) [True: negative]
383
+ '这个项目真是太成功了,成功到一塌糊涂' -> positive (conf: 1.00) [True: negative]
384
+ '这饭菜做得真是太好吃了,我一点都吃不下' -> positive (conf: 1.00) [True: negative]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
385
 
386
+ ============================================================
387
+ BEST PERFORMING MODEL: ZombitX64/MultiSent-E5-Pro
388
+ ============================================================
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
389
 
390
+ Per-Class Performance:
391
+ precision recall f1-score support
392
+ negative 0.910 0.846 0.877 661.0
393
+ neutral 0.719 0.816 0.764 517.0
394
+ positive 0.830 0.943 0.883 471.0
395
+ question 0.944 0.790 0.860 534.0
396
+
397
+ ================================================================================
398
+ COMPREHENSIVE MODEL COMPARISON REPORT
399
+ Dataset: ZombitX64/Sentiment-Benchmark
400
+ ================================================================================
401
+
402
+ Ranked by F1-Macro Score:
403
+ Model Accuracy F1-Macro F1-Weighted Avg_Confidence Low_Conf_% Error_Rate
404
+ ZombitX64/MultiSent-E5-Pro 0.8461 0.8461 0.8475 0.9853 0.9620 0.1539
405
+ ZombitX64/MultiSent-E5 0.8062 0.8062 0.8072 0.9708 1.6033 0.1938
406
+ ZombitX64/sentiment-103 0.5740 0.4987 0.5020 0.9647 2.2446 0.4260
407
+ ZombitX64/Sentiment-03 0.4828 0.4906 0.4856 0.9609 2.7485 0.5172
408
+ ZombitX64/Sentiment-02 0.4137 0.3884 0.3910 0.8151 10.0779 0.5863
409
+ ZombitX64/Thai-sentiment-e5 0.4961 0.3713 0.3704 0.9874 0.8246 0.5039
410
+ nlptown/bert-base-multilingual-uncased-sentiment 0.3587 0.2870 0.2896 0.4103 87.9066 0.6413
411
+ ZombitX64/Sentiment-01 0.2712 0.1928 0.1894 0.5085 94.5946 0.7288
412
+ SandboxBhh/sentiment-thai-text-model 0.2620 0.1807 0.1982 0.8610 20.2016 0.7380
413
+ Thaweewat/wangchanberta-hyperopt-sentiment-01 0.2336 0.1501 0.1655 0.9128 2.9776 0.7664
414
+ phoner45/wangchan-sentiment-thai-text-model 0.2203 0.1073 0.1270 0.7123 41.7316 0.7797
415
+ poom-sci/WangchanBERTa-finetuned-sentiment 0.2093 0.1061 0.1246 0.7889 14.7045 0.7907
416
+ cardiffnlp/twitter-xlm-roberta-base-sentiment 0.0944 0.0848 0.0841 0.6897 32.2492 0.9056
417
 
418
 
419
 
 
423
 
424
  The model was evaluated on a carefully selected validation set with the following characteristics:
425
 
426
+ * **Total Samples:** 2183
427
  * **Selection Method:** Stratified random sampling to maintain class distribution
428
  * **Data Quality:** Manually verified and cleaned validation samples
429
  * **Evaluation Period:** Final model checkpoint from epoch 5
 
440
  - **Recall:** Per-class and overall recall scores
441
  - **Support:** Number of samples per class in validation set
442
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
443
 
444
  #### Known Limitations
445
 
 
550
 
551
  ### Best Practices for Implementation
552
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
553
 
554
  ## Citation
555
 
 
557
 
558
  **BibTeX:**
559
  ```bibtex
560
+ @misc{MultiSent-E5-Pro,
561
  title={Thai-sentiment-e5: A Fine-tuned Multilingual Sentiment Analysis Model for Thai Text Classification},
562
  author={ZombitX64 and Janutsaha, Krittanut and Saengwichain, Chanyut},
563
  year={2024},
564
+ url={https://huggingface.co/ZombitX64/MultiSent-E5-Pro},
565
  note={Hugging Face Model Repository}
566
  }
567
  ```
568
 
 
 
 
 
 
 
569
  ### Usage in Publications
570
 
571
  If you use this model in your research or applications, please cite both this model and the base model:
 
592
  For questions, issues, or contributions regarding this model, please use the following channels:
593
 
594
  * **Primary Contact:** Hugging Face model repository issues and discussions
595
+ * **Repository:** [https://huggingface.co/ZombitX64/MultiSent-E5-Pro](https://huggingface.co/ZombitX64/MultiSent-E5-Pro)
596
  * **Community:** Hugging Face community forums for general questions
597
 
598
  ### Collaboration Opportunities