Deploy

.gitattributes

@@ -0,0 +1,4 @@
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+models/** filter=lfs diff=lfs merge=lfs -text
+*.png filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,16 @@
+.cache/
+__pycache__/
+*.pyc
+.pytest_cache/
+.mypy_cache/
+.venv/
+venv/
+env/
+
+# Training checkpoints (regenerable; large)
+artifacts/transformers/*/checkpoints/
+models/transformers/*/checkpoints/
+
+# Per-candidate transformer weights are regenerable.
+# Keep only models/best_transformer tracked with Git LFS.
+models/transformers/*/model/model.safetensors

README.md

@@ -0,0 +1,151 @@
+---
+title: Matcha Sentiment
+emoji: 🍵
+colorFrom: green
+colorTo: yellow
+sdk: gradio
+sdk_version: 5.0.1
+app_file: app.py
+python_version: "3.12"
+pinned: false
+license: mit
+---
+
+# Matcha Sentiment
+
+Sentiment analysis bahasa Indonesia untuk review Matchaya/IKUYO. Dataset dibersihkan menjadi klasifikasi biner `Negatif` dan `Positif`, lalu dibandingkan dengan baseline machine learning klasik dan fine-tuning 9 model Transformer Indonesia.
+
+![Dashboard](docs/images/dashboard_home_wide.png)
+
+## Ringkasan
+
+| Area | Hasil |
+| --- | --- |
+| Dataset final | 2028 review |
+| Label | 1014 `Negatif`, 1014 `Positif` |
+| Label dihapus | 14 `Netral` |
+| Duplikat dibuang | 219 teks |
+| Best classical | `TF-IDF + Linear SVM` |
+| Best Transformer | `indolem/indobert-base-uncased` |
+| Runtime | Docker + NVIDIA GPU |
+| Dashboard | Gradio, siap Hugging Face Spaces |
+
+> Catatan push: model Transformer terbaik disimpan di `models/best_transformer` dan ditrack lewat Git LFS. Weight kandidat di `models/transformers/*/model/model.safetensors` di-ignore karena bisa dibuat ulang dari pipeline training.
+
+## Hasil Utama
+
+### Transformer
+
+| Model | Accuracy | Precision | Recall | F1 | ROC AUC |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| `indolem/indobert-base-uncased` | 0.9951 | 0.9902 | 1.0000 | 0.9951 | 0.9998 |
+| `naufalihsan/indonesian-sbert-large` | 0.9901 | 0.9806 | 1.0000 | 0.9902 | 0.9998 |
+| `flax-community/indonesian-roberta-base` | 0.9901 | 0.9806 | 1.0000 | 0.9902 | 0.9997 |
+| `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2` | 0.9901 | 0.9806 | 1.0000 | 0.9902 | 0.9996 |
+| `indobenchmark/indobert-base-p1` | 0.9901 | 0.9806 | 1.0000 | 0.9902 | 0.9989 |
+| `ChristopherA08/IndoELECTRA` | 0.9901 | 0.9806 | 1.0000 | 0.9902 | 0.9989 |
+| `cahya/distilbert-base-indonesian` | 0.9901 | 0.9901 | 0.9901 | 0.9901 | 0.9998 |
+| `indolem/indobertweet-base-uncased` | 0.9852 | 0.9804 | 0.9901 | 0.9852 | 0.9994 |
+| `w11wo/indonesian-roberta-base-sentiment-classifier` | 0.9803 | 0.9619 | 1.0000 | 0.9806 | 1.0000 |
+
+Model terbaik sudah disimpan di:
+
+```text
+models/best_transformer
+```
+
+### Machine Learning Klasik
+
+TF-IDF dan Word2Vec diuji dengan Stratified 10-fold cross validation. Hasil lengkap:
+
+| Feature | Model | Accuracy | Precision | Recall | F1 | ROC AUC |
+| --- | --- | ---: | ---: | ---: | ---: | ---: |
+| TF-IDF | Linear SVM | 0.9684 | 0.9788 | 0.9576 | 0.9681 | 0.9951 |
+| Word2Vec | Logistic Regression | 0.9635 | 0.9663 | 0.9606 | 0.9634 | 0.9939 |
+| Word2Vec | Extra Trees | 0.9625 | 0.9653 | 0.9596 | 0.9624 | 0.9940 |
+| TF-IDF | Logistic Regression | 0.9610 | 0.9756 | 0.9458 | 0.9604 | 0.9933 |
+| Word2Vec | Linear SVM | 0.9596 | 0.9660 | 0.9527 | 0.9593 | 0.9924 |
+| Word2Vec | Random Forest | 0.9591 | 0.9632 | 0.9546 | 0.9589 | 0.9927 |
+| Word2Vec | Gradient Boosting | 0.9571 | 0.9603 | 0.9536 | 0.9570 | 0.9933 |
+| TF-IDF | Extra Trees | 0.9522 | 0.9580 | 0.9458 | 0.9519 | 0.9918 |
+| TF-IDF | Random Forest | 0.9443 | 0.9353 | 0.9546 | 0.9449 | 0.9882 |
+| TF-IDF | Gradient Boosting | 0.9147 | 0.9304 | 0.8964 | 0.9131 | 0.9735 |
+
+## Visual Evaluasi
+
+![Results Gallery](docs/images/results_gallery.png)
+
+### Dashboard
+
+| Prediksi | Visual | Kata Kunci |
+| --- | --- | --- |
+| ![Dashboard Home](docs/images/dashboard_home_wide.png) | ![Dashboard Visual](docs/images/dashboard_visual_tab.png) | ![Dashboard Keywords](docs/images/dashboard_keywords_tab.png) |
+
+### Detail Plot
+
+| Training Loss | Confusion Matrix | ROC AUC |
+| --- | --- | --- |
+| ![Training Loss](docs/images/transformer_best_training_loss.png) | ![Confusion Matrix](docs/images/transformer_best_confusion_matrix.png) | ![ROC AUC](docs/images/transformer_best_roc_auc.png) |
+
+| Top Words | Word Cloud Positif | Word Cloud Negatif |
+| --- | --- | --- |
+| ![Top Words](docs/images/top_words_tfidf.png) | ![Word Cloud Positif](docs/images/wordcloud_positif.png) | ![Word Cloud Negatif](docs/images/wordcloud_negatif.png) |
+
+## Kata Kunci Bermakna
+
+Beberapa kata yang paling membantu membaca arah sentimen:
+
+| Kata | Positif Docs | Negatif Docs | Dominan |
+| --- | ---: | ---: | --- |
+| `enak` | 173 | 10 | Positif |
+| `nyaman` | 54 | 10 | Positif |
+| `ramah` | 37 | 9 | Positif |
+| `terbaik` | 18 | 0 | Positif |
+| `mahal` | 10 | 24 | Negatif |
+| `harga` | 10 | 28 | Negatif |
+| `buruk` | 0 | 19 | Negatif |
+| `antrean` | 0 | 19 | Negatif |
+| `lama` | 1 | 16 | Negatif |
+
+File lengkapnya ada di:
+
+```text
+artifacts/classical/keyword_counts.csv
+```
+
+## Struktur Proyek
+
+```text
+.
+├── app.py
+├── Dockerfile
+├── docker-compose.yml
+├── INSTALL_DOCKER.md
+├── data/processed/matcha_sentiment_binary.csv
+├── docs/images/
+├── artifacts/
+├── models/best_transformer/
+├── models/classical/best_model.joblib
+├── scripts/
+└── src/matcha_sentiment/
+```
+
+## Quick Start
+
+```bash
+docker build -t matcha-sentiment .
+docker run --rm --gpus all -p 7860:7860 -v "${PWD}:/workspace" matcha-sentiment
+```
+
+Buka:
+
+```text
+http://localhost:7860
+```
+
+Panduan dari nol sampai deploy ada di [INSTALL_DOCKER.md](INSTALL_DOCKER.md).
+
+## Catatan
+
+Skor evaluasi sangat tinggi karena dataset masih kecil dan domainnya sempit. Model ini sudah bagus untuk demo, dashboard, dan eksperimen sentiment analysis review matcha, tetapi untuk production lintas brand atau lintas kategori sebaiknya ditambah data baru yang lebih beragam.
+# matchaSentiment

app.py

@@ -0,0 +1,183 @@
+from __future__ import annotations
+
+import json
+import os
+import sys
+from pathlib import Path
+
+import gradio as gr
+import joblib
+import numpy as np
+import pandas as pd
+import torch
+from transformers import AutoModelForSequenceClassification, AutoTokenizer
+
+sys.path.append(str(Path(__file__).resolve().parent / "src"))
+
+from matcha_sentiment.config import ARTIFACT_DIR, ID2LABEL, MODEL_DIR
+from matcha_sentiment.text import normalize_text
+
+
+ROOT = Path(__file__).resolve().parent
+FIG_DIR = ARTIFACT_DIR / "figures"
+TRANSFORMER_MODEL_PATH = Path(os.getenv("MODEL_DIR", MODEL_DIR / "best_transformer"))
+CLASSICAL_MODEL_PATH = Path(os.getenv("CLASSICAL_MODEL_PATH", MODEL_DIR / "classical" / "best_model.joblib"))
+MODEL_ID = os.getenv("MODEL_ID", "")
+
+
+class Predictor:
+    def __init__(self):
+        self.kind = "none"
+        self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+        self.tokenizer = None
+        self.model = None
+        self.classical = None
+        self.load()
+
+    def load(self) -> None:
+        model_source = None
+        if (TRANSFORMER_MODEL_PATH / "config.json").exists():
+            model_source = str(TRANSFORMER_MODEL_PATH)
+        elif MODEL_ID:
+            model_source = MODEL_ID
+
+        if model_source:
+            self.tokenizer = AutoTokenizer.from_pretrained(model_source)
+            self.model = AutoModelForSequenceClassification.from_pretrained(model_source)
+            self.model.to(self.device)
+            self.model.eval()
+            self.kind = "transformer"
+            return
+
+        if CLASSICAL_MODEL_PATH.exists():
+            self.classical = joblib.load(CLASSICAL_MODEL_PATH)
+            self.kind = "classical"
+
+    def predict(self, text: str) -> tuple[dict[str, float] | None, str]:
+        text = normalize_text(text)
+        if not text:
+            return None, "Masukkan teks review."
+
+        if self.kind == "transformer":
+            encoded = self.tokenizer(
+                text,
+                truncation=True,
+                padding=True,
+                max_length=160,
+                return_tensors="pt",
+            )
+            encoded = {key: value.to(self.device) for key, value in encoded.items()}
+            with torch.no_grad():
+                logits = self.model(**encoded).logits
+                probs = torch.softmax(logits, dim=-1).detach().cpu().numpy()[0]
+            scores = {ID2LABEL[idx]: float(probs[idx]) for idx in range(len(probs))}
+            label = ID2LABEL[int(np.argmax(probs))]
+            return scores, f"{label} - {self.kind} on {self.device.type}"
+
+        if self.kind == "classical":
+            pred = int(self.classical.predict([text])[0])
+            if hasattr(self.classical, "predict_proba"):
+                proba = self.classical.predict_proba([text])
+                if proba is not None:
+                    probs = proba[0]
+                    return {ID2LABEL[idx]: float(probs[idx]) for idx in range(len(probs))}, f"{ID2LABEL[pred]} - classical"
+            if hasattr(self.classical, "decision_function"):
+                score = self.classical.decision_function([text])
+                if score is not None:
+                    p_pos = 1.0 / (1.0 + np.exp(-float(np.ravel(score)[0])))
+                    return {"Negatif": 1.0 - p_pos, "Positif": p_pos}, f"{ID2LABEL[pred]} - classical"
+            return {ID2LABEL[pred]: 1.0}, f"{ID2LABEL[pred]} - classical"
+
+        return None, "Model belum tersedia."
+
+
+predictor = Predictor()
+
+
+def predict_review(text: str):
+    return predictor.predict(text)
+
+
+def read_json(path: Path) -> dict:
+    if not path.exists():
+        return {}
+    return json.loads(path.read_text(encoding="utf-8"))
+
+
+def read_csv(path: Path) -> pd.DataFrame:
+    if not path.exists():
+        return pd.DataFrame()
+    return pd.read_csv(path)
+
+
+def image_value(name: str):
+    path = FIG_DIR / name
+    return str(path) if path.exists() else None
+
+
+summary = read_json(ROOT / "data" / "processed" / "summary.json")
+classical_results = read_csv(ARTIFACT_DIR / "classical" / "results.csv")
+transformer_results = read_csv(ARTIFACT_DIR / "transformers" / "results.csv")
+top_words = read_csv(ARTIFACT_DIR / "classical" / "top_words_tfidf.csv")
+keyword_counts = read_csv(ARTIFACT_DIR / "classical" / "keyword_counts.csv")
+
+
+css = """
+.metric-card textarea { font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; }
+.gradio-container { max-width: 1180px !important; }
+"""
+
+
+with gr.Blocks(title="Matcha Sentiment", css=css) as demo:
+    gr.Markdown("# Matcha Sentiment")
+
+    with gr.Tab("Prediksi"):
+        with gr.Row():
+            review = gr.Textbox(
+                label="Review",
+                lines=7,
+                value="Matchanya enak, tempatnya nyaman, tapi harganya agak mahal.",
+            )
+            with gr.Column():
+                output_label = gr.Label(label="Sentimen")
+                output_text = gr.Textbox(label="Model", interactive=False)
+                submit = gr.Button("Analisis", variant="primary")
+        submit.click(predict_review, inputs=review, outputs=[output_label, output_text])
+        gr.Examples(
+            examples=[
+                ["Matchanya enak dan pelayanannya ramah."],
+                ["Harganya terlalu mahal dan rasanya biasa saja."],
+                ["Tempat nyaman, tetapi antrean lama dan staf kurang ramah."],
+            ],
+            inputs=review,
+        )
+
+    with gr.Tab("Metrik"):
+        with gr.Row():
+            gr.JSON(value=summary, label="Dataset")
+        with gr.Row():
+            gr.Dataframe(value=classical_results, label="TF-IDF dan Word2Vec 10-fold", interactive=False)
+        with gr.Row():
+            gr.Dataframe(value=transformer_results, label="Transformer", interactive=False)
+
+    with gr.Tab("Visual"):
+        with gr.Row():
+            gr.Image(value=image_value("transformer_best_training_loss.png"), label="Training loss")
+            gr.Image(value=image_value("transformer_best_confusion_matrix.png"), label="Confusion matrix transformer")
+        with gr.Row():
+            gr.Image(value=image_value("transformer_best_roc_auc.png"), label="ROC AUC transformer")
+            gr.Image(value=image_value("classical_best_confusion_matrix.png"), label="Confusion matrix klasik")
+        with gr.Row():
+            gr.Image(value=image_value("classical_best_roc_auc.png"), label="ROC AUC klasik")
+            gr.Image(value=image_value("top_words_tfidf.png"), label="Top words")
+        with gr.Row():
+            gr.Image(value=image_value("wordcloud_positif.png"), label="Word cloud positif")
+            gr.Image(value=image_value("wordcloud_negatif.png"), label="Word cloud negatif")
+
+    with gr.Tab("Kata Kunci"):
+        gr.Dataframe(value=top_words, label="Top words TF-IDF", interactive=False)
+        gr.Dataframe(value=keyword_counts, label="Keyword penting", interactive=False)
+
+
+if __name__ == "__main__":
+    demo.launch(server_name="0.0.0.0", server_port=int(os.getenv("PORT", "7860")))

artifacts/classical/fold_metrics.csv

@@ -0,0 +1,101 @@
+accuracy,precision,recall,f1,roc_auc,feature,model,fold,n_valid
+0.9605911330049262,0.979381443298969,0.9405940594059405,0.9595959595959596,0.9887400504756357,tfidf,logistic_regression,1,203
+0.9753694581280788,1.0,0.9504950495049505,0.9746192893401016,0.9933022714036109,tfidf,logistic_regression,2,203
+0.9753694581280788,0.9897959183673469,0.9603960396039604,0.9748743718592965,0.9939817511162882,tfidf,logistic_regression,3,203
+0.9458128078817734,0.96875,0.9207920792079208,0.9441624365482234,0.9888371190060182,tfidf,logistic_regression,4,203
+0.9408866995073891,0.96875,0.9117647058823529,0.9393939393939394,0.9936905455251408,tfidf,logistic_regression,5,203
+0.9556650246305419,0.9345794392523364,0.9803921568627451,0.9569377990430622,0.9921374490390216,tfidf,logistic_regression,6,203
+0.9753694581280788,0.970873786407767,0.9803921568627451,0.975609756097561,0.9968938070277615,tfidf,logistic_regression,7,203
+0.9605911330049262,0.9895833333333334,0.9313725490196079,0.9595959595959596,0.9946612308289653,tfidf,logistic_regression,8,203
+0.9504950495049505,0.9789473684210527,0.9207920792079208,0.9489795918367347,0.9940201940986177,tfidf,logistic_regression,9,202
+0.9702970297029703,0.9797979797979798,0.9603960396039604,0.97,0.9972551710616606,tfidf,logistic_regression,10,202
+0.9704433497536946,0.9896907216494846,0.9504950495049505,0.9696969696969697,0.9955348476024073,tfidf,linear_svm,1,203
+0.9852216748768473,1.0,0.9702970297029703,0.9849246231155779,0.99524364201126,tfidf,linear_svm,2,203
+0.9852216748768473,0.99,0.9801980198019802,0.9850746268656716,0.9951465734808775,tfidf,linear_svm,3,203
+0.9458128078817734,0.9591836734693877,0.9306930693069307,0.9447236180904522,0.9890312560667831,tfidf,linear_svm,4,203
+0.9458128078817734,0.9690721649484536,0.9215686274509803,0.9447236180904522,0.9958260531935546,tfidf,linear_svm,5,203
+0.9556650246305419,0.9514563106796117,0.9607843137254902,0.9560975609756097,0.9938846825859057,tfidf,linear_svm,6,203
+0.9901477832512315,0.9807692307692307,1.0,0.9902912621359223,0.9966996699669967,tfidf,linear_svm,7,203
+0.9655172413793104,0.9797979797979798,0.9509803921568627,0.9651741293532339,0.9977674238012035,tfidf,linear_svm,8,203
+0.9603960396039604,0.9894736842105263,0.9306930693069307,0.9591836734693877,0.9941182237035585,tfidf,linear_svm,9,202
+0.9801980198019802,0.9801980198019802,0.9801980198019802,0.9801980198019802,0.9985295559258895,tfidf,linear_svm,10,202
+0.9359605911330049,0.9150943396226415,0.9603960396039604,0.9371980676328503,0.9880605707629587,tfidf,random_forest,1,203
+0.9458128078817734,0.9411764705882353,0.9504950495049505,0.9458128078817734,0.9909726266744321,tfidf,random_forest,2,203
+0.9704433497536946,0.9702970297029703,0.9702970297029703,0.9702970297029703,0.9925257231605513,tfidf,random_forest,3,203
+0.9507389162561576,0.941747572815534,0.9603960396039604,0.9509803921568627,0.9841778295476606,tfidf,random_forest,4,203
+0.9408866995073891,0.9245283018867925,0.9607843137254902,0.9423076923076923,0.9881576392933411,tfidf,random_forest,5,203
+0.916256157635468,0.8828828828828829,0.9607843137254902,0.92018779342723,0.9802950883323627,tfidf,random_forest,6,203
+0.9310344827586207,0.9230769230769231,0.9411764705882353,0.9320388349514563,0.9866045428072219,tfidf,random_forest,7,203
+0.9458128078817734,0.9504950495049505,0.9411764705882353,0.9458128078817734,0.9867986798679869,tfidf,random_forest,8,203
+0.9356435643564357,0.9489795918367347,0.9207920792079208,0.9346733668341709,0.9912753651602784,tfidf,random_forest,9,202
+0.9702970297029703,0.9611650485436893,0.9801980198019802,0.9705882352941176,0.9931379276541517,tfidf,random_forest,10,202
+0.9408866995073891,0.9494949494949495,0.9306930693069307,0.94,0.9880605707629586,tfidf,extra_trees,1,203
+0.9507389162561576,0.9504950495049505,0.9504950495049505,0.9504950495049505,0.9937876140555232,tfidf,extra_trees,2,203
+0.9802955665024631,0.98989898989899,0.9702970297029703,0.98,0.9927198602213163,tfidf,extra_trees,3,203
+0.9507389162561576,0.9690721649484536,0.9306930693069307,0.9494949494949495,0.9913609007959618,tfidf,extra_trees,4,203
+0.9605911330049262,0.9795918367346939,0.9411764705882353,0.96,0.9951465734808775,tfidf,extra_trees,5,203
+0.9310344827586207,0.9150943396226415,0.9509803921568627,0.9326923076923077,0.9867016113376043,tfidf,extra_trees,6,203
+0.9507389162561576,0.9509803921568627,0.9509803921568627,0.9509803921568627,0.9919433119782566,tfidf,extra_trees,7,203
+0.9310344827586207,0.94,0.9215686274509803,0.9306930693069307,0.9895165987186955,tfidf,extra_trees,8,203
+0.9554455445544554,0.96,0.9504950495049505,0.9552238805970149,0.9940201940986177,tfidf,extra_trees,9,202
+0.9702970297029703,0.9797979797979798,0.9603960396039604,0.97,0.9942162533084992,tfidf,extra_trees,10,202
+0.896551724137931,0.9081632653061225,0.8811881188118812,0.8944723618090452,0.9764123471170647,tfidf,gradient_boosting,1,203
+0.916256157635468,0.9375,0.8910891089108911,0.9137055837563451,0.9801980198019802,tfidf,gradient_boosting,2,203
+0.9458128078817734,0.95,0.9405940594059405,0.945273631840796,0.9851970491166764,tfidf,gradient_boosting,3,203
+0.9014778325123153,0.9354838709677419,0.8613861386138614,0.8969072164948454,0.9653950689186565,tfidf,gradient_boosting,4,203
+0.9064039408866995,0.9278350515463918,0.8823529411764706,0.9045226130653267,0.9652494661230829,tfidf,gradient_boosting,5,203
+0.9014778325123153,0.8942307692307693,0.9117647058823529,0.9029126213592233,0.967676179382644,tfidf,gradient_boosting,6,203
+0.9359605911330049,0.9405940594059405,0.9313725490196079,0.9359605911330049,0.9850514463211028,tfidf,gradient_boosting,7,203
+0.9064039408866995,0.946236559139785,0.8627450980392157,0.9025641025641026,0.9619976703552708,tfidf,gradient_boosting,8,203
+0.8960396039603961,0.9166666666666666,0.8712871287128713,0.8934010152284264,0.9630428389373591,tfidf,gradient_boosting,9,202
+0.9405940594059405,0.9494949494949495,0.9306930693069307,0.94,0.9850504852465445,tfidf,gradient_boosting,10,202
+0.9704433497536946,0.9702970297029703,0.9702970297029703,0.9702970297029703,0.9916521063871093,word2vec,logistic_regression,1,203
+0.9950738916256158,1.0,0.9900990099009901,0.9950248756218906,0.9990293146961755,word2vec,logistic_regression,2,203
+0.9901477832512315,0.9900990099009901,0.9900990099009901,0.9900990099009901,0.9949524364201127,word2vec,logistic_regression,3,203
+0.9458128078817734,0.96875,0.9207920792079208,0.9441624365482234,0.9911667637351971,word2vec,logistic_regression,4,203
+0.9359605911330049,0.9405940594059405,0.9313725490196079,0.9359605911330049,0.9918462434478742,word2vec,logistic_regression,5,203
+0.9507389162561576,0.9339622641509434,0.9705882352941176,0.9519230769230769,0.9905843525529023,word2vec,logistic_regression,6,203
+0.9556650246305419,0.9428571428571428,0.9705882352941176,0.9565217391304348,0.9963113958454668,word2vec,logistic_regression,7,203
+0.9704433497536946,0.98,0.9607843137254902,0.9702970297029703,0.9915550378567268,word2vec,logistic_regression,8,203
+0.9554455445544554,0.9693877551020408,0.9405940594059405,0.9547738693467337,0.9929418684442702,word2vec,logistic_regression,9,202
+0.9653465346534653,0.97,0.9603960396039604,0.9651741293532339,0.9954906381727282,word2vec,logistic_regression,10,202
+0.9655172413793104,0.9795918367346939,0.9504950495049505,0.964824120603015,0.9859250630945448,word2vec,linear_svm,1,203
+0.9901477832512315,1.0,0.9801980198019802,0.99,0.998932246165793,word2vec,linear_svm,2,203
+0.9802955665024631,0.98989898989899,0.9702970297029703,0.98,0.9962143273150845,word2vec,linear_svm,3,203
+0.9408866995073891,0.968421052631579,0.9108910891089109,0.9387755102040817,0.9890312560667832,word2vec,linear_svm,4,203
+0.9458128078817734,0.9595959595959596,0.9313725490196079,0.945273631840796,0.9890312560667831,word2vec,linear_svm,5,203
+0.9458128078817734,0.9252336448598131,0.9705882352941176,0.9473684210526315,0.9887400504756357,word2vec,linear_svm,6,203
+0.9507389162561576,0.9339622641509434,0.9705882352941176,0.9519230769230769,0.9952436420112599,word2vec,linear_svm,7,203
+0.9655172413793104,0.9797979797979798,0.9509803921568627,0.9651741293532339,0.9964084643758493,word2vec,linear_svm,8,203
+0.9504950495049505,0.9690721649484536,0.9306930693069307,0.9494949494949495,0.9901970395059307,word2vec,linear_svm,9,202
+0.9603960396039604,0.9603960396039604,0.9603960396039604,0.9603960396039604,0.9950004901480247,word2vec,linear_svm,10,202
+0.9458128078817734,0.9411764705882353,0.9504950495049505,0.9458128078817734,0.9868957483983692,word2vec,random_forest,1,203
+0.9901477832512315,1.0,0.9801980198019802,0.99,0.9985439720442633,word2vec,random_forest,2,203
+0.9753694581280788,0.9897959183673469,0.9603960396039604,0.9748743718592965,0.9935934769947583,word2vec,random_forest,3,203
+0.9507389162561576,0.9504950495049505,0.9504950495049505,0.9504950495049505,0.9907784896136672,word2vec,random_forest,4,203
+0.9507389162561576,0.9509803921568627,0.9509803921568627,0.9509803921568627,0.9945641622985828,word2vec,random_forest,5,203
+0.9310344827586207,0.9150943396226415,0.9509803921568627,0.9326923076923077,0.9862162686856921,word2vec,random_forest,6,203
+0.9507389162561576,0.9423076923076923,0.9607843137254902,0.9514563106796117,0.9936905455251408,word2vec,random_forest,7,203
+0.9605911330049262,0.97,0.9509803921568627,0.9603960396039604,0.9953407105416425,word2vec,random_forest,8,203
+0.9603960396039604,1.0,0.9207920792079208,0.9587628865979382,0.9967650230369571,word2vec,random_forest,9,202
+0.9752475247524752,0.98,0.9702970297029703,0.9751243781094527,0.9917655131849819,word2vec,random_forest,10,202
+0.9556650246305419,0.9509803921568627,0.9603960396039604,0.9556650246305419,0.9923315860997864,word2vec,extra_trees,1,203
+0.9852216748768473,1.0,0.9702970297029703,0.9849246231155779,0.9992234517569404,word2vec,extra_trees,2,203
+0.9704433497536946,0.9797979797979798,0.9603960396039604,0.97,0.9923315860997864,word2vec,extra_trees,3,203
+0.9605911330049262,0.9696969696969697,0.9504950495049505,0.96,0.993302271403611,word2vec,extra_trees,4,203
+0.9507389162561576,0.9509803921568627,0.9509803921568627,0.9509803921568627,0.9939817511162881,word2vec,extra_trees,5,203
+0.9507389162561576,0.9259259259259259,0.9803921568627451,0.9523809523809523,0.9889341875364006,word2vec,extra_trees,6,203
+0.9605911330049262,0.9519230769230769,0.9705882352941176,0.9611650485436893,0.9947582993593477,word2vec,extra_trees,7,203
+0.9556650246305419,0.9603960396039604,0.9509803921568627,0.9556650246305419,0.9937876140555233,word2vec,extra_trees,8,203
+0.9603960396039604,0.9894736842105263,0.9306930693069307,0.9591836734693877,0.9962748750122538,word2vec,extra_trees,9,202
+0.9752475247524752,0.98,0.9702970297029703,0.9751243781094527,0.9954906381727282,word2vec,extra_trees,10,202
+0.9507389162561576,0.9504950495049505,0.9504950495049505,0.9504950495049505,0.9896136672490778,word2vec,gradient_boosting,1,203
+0.9852216748768473,0.99,0.9801980198019802,0.9850746268656716,0.9990293146961755,word2vec,gradient_boosting,2,203
+0.9605911330049262,0.9894736842105263,0.9306930693069307,0.9591836734693877,0.9947582993593478,word2vec,gradient_boosting,3,203
+0.9458128078817734,0.9411764705882353,0.9504950495049505,0.9458128078817734,0.9926227916909338,word2vec,gradient_boosting,4,203
+0.9605911330049262,0.9607843137254902,0.9607843137254902,0.9607843137254902,0.9946612308289653,word2vec,gradient_boosting,5,203
+0.9211822660098522,0.9134615384615384,0.9313725490196079,0.9223300970873787,0.9867016113376044,word2vec,gradient_boosting,6,203
+0.9507389162561576,0.9423076923076923,0.9607843137254902,0.9514563106796117,0.9947582993593477,word2vec,gradient_boosting,7,203
+0.9704433497536946,0.9705882352941176,0.9705882352941176,0.9705882352941176,0.9946612308289652,word2vec,gradient_boosting,8,203
+0.9603960396039604,0.9894736842105263,0.9306930693069307,0.9591836734693877,0.9954906381727282,word2vec,gradient_boosting,9,202
+0.9653465346534653,0.9607843137254902,0.9702970297029703,0.9655172413793104,0.993333986864033,word2vec,gradient_boosting,10,202

artifacts/classical/keyword_counts.csv

@@ -0,0 +1,25 @@
+term,positif_docs,negatif_docs,positif_rate,negatif_rate,dominant_label,lift
+terbaik,49,0,0.048323471400394474,0.0,Positif,48323472.40039447
+lezat,20,0,0.01972386587771203,0.0,Positif,19723866.87771203
+mantap,18,0,0.01775147928994083,0.0,Positif,17751480.289940827
+nikmat,9,0,0.008875739644970414,0.0,Positif,8875740.644970413
+autentik,21,1,0.020710059171597635,0.0009861932938856016,Positif,20.999979720020566
+enak,510,33,0.5029585798816568,0.03254437869822485,Positif,15.454545010396707
+bagus,91,6,0.08974358974358974,0.005917159763313609,Positif,15.166664272500403
+nyaman,208,23,0.20512820512820512,0.022682445759368838,Positif,9.043477906257104
+direkomendasikan,55,7,0.054240631163708086,0.006903353057199211,Positif,7.857141863836878
+ramah,108,16,0.10650887573964497,0.015779092702169626,Positif,6.749999635593773
+manis,41,30,0.04043392504930966,0.029585798816568046,Positif,1.3666666542733337
+pahit,23,21,0.022682445759368838,0.020710059171597635,Positif,1.0952380906394559
+biasa,18,26,0.01775147928994083,0.02564102564102564,Negatif,0.6923077043076918
+kurang,27,49,0.026627218934911243,0.048323471400394474,Negatif,0.5510204174543939
+harganya,22,43,0.021696252465483234,0.04240631163708087,Negatif,0.5116279184932393
+harga,31,62,0.03057199211045365,0.0611439842209073,Negatif,0.5000000081774192
+kecewa,2,8,0.0019723865877712033,0.007889546351084813,Negatif,0.25000009506248794
+lama,10,42,0.009861932938856016,0.04142011834319527,Negatif,0.23809525648979546
+mahal,11,47,0.010848126232741617,0.046351084812623275,Negatif,0.23404256971661347
+tidak,86,380,0.08481262327416174,0.3747534516765286,Negatif,0.22631579153819942
+murah,2,9,0.0019723865877712033,0.008875739644970414,Negatif,0.222222309851842
+menunggu,4,37,0.0039447731755424065,0.03648915187376726,Negatif,0.10810813255076633
+antrean,2,34,0.0019723865877712033,0.03353057199211045,Negatif,0.05882355748096803
+buruk,0,36,0.0,0.03550295857988166,Negatif,2.816666587330558e-08

artifacts/classical/report_tfidf_extra_trees.json

@@ -0,0 +1,27 @@
+{
+  "Negatif": {
+    "precision": 0.9464459591041869,
+    "recall": 0.9585798816568047,
+    "f1-score": 0.9524742773150416,
+    "support": 1014.0
+  },
+  "Positif": {
+    "precision": 0.958041958041958,
+    "recall": 0.9457593688362919,
+    "f1-score": 0.9518610421836228,
+    "support": 1014.0
+  },
+  "accuracy": 0.9521696252465484,
+  "macro avg": {
+    "precision": 0.9522439585730724,
+    "recall": 0.9521696252465484,
+    "f1-score": 0.9521676597493323,
+    "support": 2028.0
+  },
+  "weighted avg": {
+    "precision": 0.9522439585730724,
+    "recall": 0.9521696252465484,
+    "f1-score": 0.9521676597493323,
+    "support": 2028.0
+  }
+}

artifacts/classical/report_tfidf_gradient_boosting.json

@@ -0,0 +1,27 @@
+{
+  "Negatif": {
+    "precision": 0.9000951474785919,
+    "recall": 0.9329388560157791,
+    "f1-score": 0.9162227602905569,
+    "support": 1014.0
+  },
+  "Positif": {
+    "precision": 0.9303991811668373,
+    "recall": 0.8964497041420119,
+    "f1-score": 0.9131089904570567,
+    "support": 1014.0
+  },
+  "accuracy": 0.9146942800788954,
+  "macro avg": {
+    "precision": 0.9152471643227146,
+    "recall": 0.9146942800788955,
+    "f1-score": 0.9146658753738068,
+    "support": 2028.0
+  },
+  "weighted avg": {
+    "precision": 0.9152471643227146,
+    "recall": 0.9146942800788954,
+    "f1-score": 0.9146658753738068,
+    "support": 2028.0
+  }
+}

artifacts/classical/report_tfidf_linear_svm.json

@@ -0,0 +1,27 @@
+{
+  "Negatif": {
+    "precision": 0.9584942084942085,
+    "recall": 0.9792899408284024,
+    "f1-score": 0.968780487804878,
+    "support": 1014.0
+  },
+  "Positif": {
+    "precision": 0.9788306451612904,
+    "recall": 0.9575936883629191,
+    "f1-score": 0.9680957128614157,
+    "support": 1014.0
+  },
+  "accuracy": 0.9684418145956607,
+  "macro avg": {
+    "precision": 0.9686624268277495,
+    "recall": 0.9684418145956608,
+    "f1-score": 0.9684381003331468,
+    "support": 2028.0
+  },
+  "weighted avg": {
+    "precision": 0.9686624268277494,
+    "recall": 0.9684418145956607,
+    "f1-score": 0.9684381003331468,
+    "support": 2028.0
+  }
+}

artifacts/classical/report_tfidf_logistic_regression.json

@@ -0,0 +1,27 @@
+{
+  "Negatif": {
+    "precision": 0.9473684210526315,
+    "recall": 0.9763313609467456,
+    "f1-score": 0.9616318601262749,
+    "support": 1014.0
+  },
+  "Positif": {
+    "precision": 0.9755849440488301,
+    "recall": 0.9457593688362919,
+    "f1-score": 0.9604406609914873,
+    "support": 1014.0
+  },
+  "accuracy": 0.9610453648915187,
+  "macro avg": {
+    "precision": 0.9614766825507308,
+    "recall": 0.9610453648915187,
+    "f1-score": 0.9610362605588811,
+    "support": 2028.0
+  },
+  "weighted avg": {
+    "precision": 0.9614766825507308,
+    "recall": 0.9610453648915187,
+    "f1-score": 0.961036260558881,
+    "support": 2028.0
+  }
+}

artifacts/classical/report_tfidf_random_forest.json

@@ -0,0 +1,27 @@
+{
+  "Negatif": {
+    "precision": 0.9536757301107754,
+    "recall": 0.9339250493096647,
+    "f1-score": 0.9436970602889886,
+    "support": 1014.0
+  },
+  "Positif": {
+    "precision": 0.9352657004830918,
+    "recall": 0.9546351084812623,
+    "f1-score": 0.9448511469009273,
+    "support": 1014.0
+  },
+  "accuracy": 0.9442800788954635,
+  "macro avg": {
+    "precision": 0.9444707152969336,
+    "recall": 0.9442800788954635,
+    "f1-score": 0.944274103594958,
+    "support": 2028.0
+  },
+  "weighted avg": {
+    "precision": 0.9444707152969336,
+    "recall": 0.9442800788954635,
+    "f1-score": 0.9442741035949579,
+    "support": 2028.0
+  }
+}

artifacts/classical/report_word2vec_extra_trees.json

@@ -0,0 +1,27 @@
+{
+  "Negatif": {
+    "precision": 0.9598039215686275,
+    "recall": 0.965483234714004,
+    "f1-score": 0.9626352015732547,
+    "support": 1014.0
+  },
+  "Positif": {
+    "precision": 0.9652777777777778,
+    "recall": 0.9595660749506904,
+    "f1-score": 0.9624134520276953,
+    "support": 1014.0
+  },
+  "accuracy": 0.9625246548323472,
+  "macro avg": {
+    "precision": 0.9625408496732026,
+    "recall": 0.9625246548323472,
+    "f1-score": 0.9625243268004751,
+    "support": 2028.0
+  },
+  "weighted avg": {
+    "precision": 0.9625408496732026,
+    "recall": 0.9625246548323472,
+    "f1-score": 0.962524326800475,
+    "support": 2028.0
+  }
+}

artifacts/classical/report_word2vec_gradient_boosting.json

@@ -0,0 +1,27 @@
+{
+  "Negatif": {
+    "precision": 0.9539666993143977,
+    "recall": 0.960552268244576,
+    "f1-score": 0.9572481572481573,
+    "support": 1014.0
+  },
+  "Positif": {
+    "precision": 0.9602780536246276,
+    "recall": 0.9536489151873767,
+    "f1-score": 0.9569520039584364,
+    "support": 1014.0
+  },
+  "accuracy": 0.9571005917159763,
+  "macro avg": {
+    "precision": 0.9571223764695127,
+    "recall": 0.9571005917159763,
+    "f1-score": 0.9571000806032968,
+    "support": 2028.0
+  },
+  "weighted avg": {
+    "precision": 0.9571223764695126,
+    "recall": 0.9571005917159763,
+    "f1-score": 0.9571000806032968,
+    "support": 2028.0
+  }
+}

artifacts/classical/report_word2vec_linear_svm.json

@@ -0,0 +1,27 @@
+{
+  "Negatif": {
+    "precision": 0.953307392996109,
+    "recall": 0.9664694280078896,
+    "f1-score": 0.9598432908912831,
+    "support": 1014.0
+  },
+  "Positif": {
+    "precision": 0.966,
+    "recall": 0.9526627218934911,
+    "f1-score": 0.9592850049652433,
+    "support": 1014.0
+  },
+  "accuracy": 0.9595660749506904,
+  "macro avg": {
+    "precision": 0.9596536964980544,
+    "recall": 0.9595660749506904,
+    "f1-score": 0.9595641479282632,
+    "support": 2028.0
+  },
+  "weighted avg": {
+    "precision": 0.9596536964980545,
+    "recall": 0.9595660749506904,
+    "f1-score": 0.9595641479282633,
+    "support": 2028.0
+  }
+}

artifacts/classical/report_word2vec_logistic_regression.json

@@ -0,0 +1,27 @@
+{
+  "Negatif": {
+    "precision": 0.9607843137254902,
+    "recall": 0.9664694280078896,
+    "f1-score": 0.9636184857423795,
+    "support": 1014.0
+  },
+  "Positif": {
+    "precision": 0.9662698412698413,
+    "recall": 0.960552268244576,
+    "f1-score": 0.963402571711177,
+    "support": 1014.0
+  },
+  "accuracy": 0.9635108481262328,
+  "macro avg": {
+    "precision": 0.9635270774976658,
+    "recall": 0.9635108481262328,
+    "f1-score": 0.9635105287267782,
+    "support": 2028.0
+  },
+  "weighted avg": {
+    "precision": 0.9635270774976658,
+    "recall": 0.9635108481262328,
+    "f1-score": 0.9635105287267783,
+    "support": 2028.0
+  }
+}

artifacts/classical/report_word2vec_random_forest.json

@@ -0,0 +1,27 @@
+{
+  "Negatif": {
+    "precision": 0.9550342130987293,
+    "recall": 0.9635108481262328,
+    "f1-score": 0.9592538046146294,
+    "support": 1014.0
+  },
+  "Positif": {
+    "precision": 0.96318407960199,
+    "recall": 0.9546351084812623,
+    "f1-score": 0.9588905398712234,
+    "support": 1014.0
+  },
+  "accuracy": 0.9590729783037475,
+  "macro avg": {
+    "precision": 0.9591091463503596,
+    "recall": 0.9590729783037475,
+    "f1-score": 0.9590721722429264,
+    "support": 2028.0
+  },
+  "weighted avg": {
+    "precision": 0.9591091463503597,
+    "recall": 0.9590729783037475,
+    "f1-score": 0.9590721722429264,
+    "support": 2028.0
+  }
+}

artifacts/classical/results.csv

@@ -0,0 +1,11 @@
+accuracy,precision,recall,f1,roc_auc,feature,model,folds,n
+0.9684418145956607,0.9788306451612904,0.9575936883629191,0.9680957128614157,0.9951448945531786,tfidf,linear_svm,10,2028
+0.9635108481262328,0.9662698412698413,0.960552268244576,0.963402571711177,0.9938698458270603,word2vec,logistic_regression,10,2028
+0.9625246548323472,0.9652777777777778,0.9595660749506904,0.9624134520276953,0.9939816922065442,word2vec,extra_trees,10,2028
+0.9610453648915187,0.9755849440488301,0.9457593688362919,0.9604406609914873,0.993330065473898,tfidf,logistic_regression,10,2028
+0.9595660749506904,0.966,0.9526627218934911,0.9592850049652433,0.9924450202101543,word2vec,linear_svm,10,2028
+0.9590729783037475,0.96318407960199,0.9546351084812623,0.9588905398712234,0.9926978902855098,word2vec,random_forest,10,2028
+0.9571005917159763,0.9602780536246276,0.9536489151873767,0.9569520039584364,0.9932901898081689,word2vec,gradient_boosting,10,2028
+0.9521696252465484,0.958041958041958,0.9457593688362919,0.9518610421836228,0.9917681064699727,tfidf,extra_trees,10,2028
+0.9442800788954635,0.9352657004830918,0.9546351084812623,0.9448511469009273,0.9882347334554891,tfidf,random_forest,10,2028
+0.9146942800788954,0.9303991811668373,0.8964497041420119,0.9131089904570567,0.9734953257939148,tfidf,gradient_boosting,10,2028

artifacts/classical/top_words_tfidf.csv

@@ -0,0 +1,121 @@
+term,weight,label_name
+enak,6.779675935425394,Positif
+bagus,2.8658279629291394,Positif
+nyaman,2.8581534389813177,Positif
+tempatnya,2.627008864949178,Positif
+terbaik,2.1523917977830203,Positif
+suka,2.054488930967297,Positif
+ramah,2.038783583222844,Positif
+sangat enak,1.7820287632132352,Positif
+matcha-nya,1.7207730656764704,Positif
+enak dan,1.7081207989749454,Positif
+mantap,1.6769131021022596,Positif
+pelayanan,1.6575130843871289,Positif
+enak sekali,1.631890295781384,Positif
+braga,1.5049659927850072,Positif
+direkomendasikan,1.4197258991884931,Positif
+lezat,1.3810395926163632,Positif
+autentik,1.322057364683747,Positif
+menarik,1.271781010383556,Positif
+tidak terlalu,1.2681605237449671,Positif
+mencoba,1.24669576582905,Positif
+wajib,1.2148858565278498,Positif
+luar biasa,1.198937692181637,Positif
+keren,1.189645695081413,Positif
+feel,1.1633305392227709,Positif
+feel matcha,1.1633305392227709,Positif
+menyenangkan,1.1611621129155691,Positif
+nyaman dan,1.157465948684494,Positif
+sini,1.1161567380461428,Positif
+pelayanannya,1.1021108164083118,Positif
+pecinta,1.088950917653697,Positif
+pecinta matcha,1.088950917653697,Positif
+dicoba,1.0513872719613726,Positif
+luar,1.0491663284442232,Positif
+cocok,1.0424196999756452,Positif
+bandung,1.037843601039172,Positif
+sangat direkomendasikan,1.0357254221127778,Positif
+favorit,1.011042823809348,Positif
+terlalu manis,0.981645744103316,Positif
+terjangkau,0.973202784805922,Positif
+nikmat,0.9395737633825911,Positif
+creamy,0.9151266830322586,Positif
+yang bagus,0.9079359550103053,Positif
+suasananya,0.9069920858067819,Positif
+matchanya enak,0.8836480085517279,Positif
+di braga,0.876123980986992,Positif
+cepat,0.8748172674136202,Positif
+matcha enak,0.8709581560752803,Positif
+sangat nyaman,0.8692630220462553,Positif
+pilihan,0.8669644030165948,Positif
+good,0.8646022512513619,Positif
+matcha terbaik,0.8621637427659486,Positif
+rasanya enak,0.8468944055617108,Positif
+sangat suka,0.8307670140605237,Positif
+lumayan,0.8289733956058337,Positif
+ramah dan,0.8105082760482645,Positif
+tempatnya nyaman,0.8003474687367536,Positif
+lucu,0.7853360623739081,Positif
+suasana,0.7818980428197028,Positif
+ke sini,0.7725116187773643,Positif
+menyukai,0.7715396284164138,Positif
+tidak,-3.7154029272590687,Negatif
+banget,-1.8565624312210731,Negatif
+beli,-1.7407328065710748,Negatif
+sangat tidak,-1.5329173647435836,Negatif
+pelanggan,-1.5293160222843036,Negatif
+pesanan,-1.4286890774714087,Negatif
+kayak,-1.3870013285793157,Negatif
+bikin,-1.3699115300927542,Negatif
+area,-1.2964871224998862,Negatif
+kasir,-1.288853516329438,Negatif
+gelas,-1.22731379211243,Negatif
+buruk,-1.2158849363519226,Negatif
+terlalu,-1.1562493904459072,Negatif
+malah,-1.1352520953426055,Negatif
+tidak ada,-1.0817406575783368,Negatif
+jam,-1.0680265948999101,Negatif
+meja,-1.0396674798600292,Negatif
+air,-1.026312085826463,Negatif
+padahal,-1.0214326834781853,Negatif
+pakai,-0.9944526416621025,Negatif
+bau,-0.9530867517678938,Negatif
+gerai,-0.9488617313357083,Negatif
+mahal,-0.9450652598687478,Negatif
+staf,-0.8896046049224152,Negatif
+keras,-0.8504849297916145,Negatif
+sangat buruk,-0.8495769270692559,Negatif
+menunggu,-0.8399522169109798,Negatif
+kurang,-0.8220857045133737,Negatif
+lama,-0.8052777380257831,Negatif
+teh,-0.7908825395876438,Negatif
+sepertinya,-0.7769467947991883,Negatif
+biasa saja,-0.770782100556984,Negatif
+kotor,-0.760729009779986,Negatif
+dikasih,-0.7540166847646226,Negatif
+bubuk,-0.739718714033697,Negatif
+harga,-0.7324491727275422,Negatif
+beli matcha,-0.7321285665156619,Negatif
+antrean,-0.7282306924125145,Negatif
+masuk,-0.7234527379574038,Negatif
+sempit,-0.7183160892820316,Negatif
+cuma,-0.7102850475433332,Negatif
+hanya,-0.7021569999486995,Negatif
+tidak nyaman,-0.7018459658083473,Negatif
+minta,-0.7017103575877348,Negatif
+plastik,-0.7008926248184666,Negatif
+menit,-0.6907597159178882,Negatif
+jauh,-0.6907428639139481,Negatif
+pesan,-0.687907417251262,Negatif
+kualitas,-0.685890280204763,Negatif
+lengket,-0.6821969933381855,Negatif
+aneh,-0.6747821234672332,Negatif
+tumpah,-0.666470971743141,Negatif
+krimnya,-0.6574423085479482,Negatif
+satu,-0.6374849785170068,Negatif
+waktu,-0.6349304476706374,Negatif
+zonk,-0.6282434592606427,Negatif
+panas,-0.6201632756305473,Negatif
+ukuran,-0.6152009043724824,Negatif
+porsi,-0.6006432748736134,Negatif
+tangan,-0.600042149862903,Negatif