Spaces:
Sleeping
Sleeping
gregorio commited on
Commit ·
fc2f1a6
1
Parent(s): bb93703
docs: add planning.txt for future true MTL transition
Browse files- planning.txt +93 -0
planning.txt
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# PLAN: TRANSISI KE TRUE MULTI-TASK LEARNING (MTL)
|
| 2 |
+
======================================================
|
| 3 |
+
Dokumen ini berisi panduan teknis bagi developer untuk mengubah arsitektur kode saat ini (Single-Task/Ensemble) menjadi True Multi-Task Learning sesuai dengan Paper penelitian.
|
| 4 |
+
|
| 5 |
+
Langkah-langkah yang harus dilakukan pada 'fake_news_pipeline.ipynb':
|
| 6 |
+
|
| 7 |
+
1. UPDATE DATASET LOADER (Memasukkan 2 Label)
|
| 8 |
+
------------------------------------------------------
|
| 9 |
+
Ubah class `TransformerDataset` agar menerima dua array label (fake_news dan ai_origin) dan me-return keduanya saat iterasi.
|
| 10 |
+
|
| 11 |
+
```python
|
| 12 |
+
class MTLTransformerDataset(Dataset):
|
| 13 |
+
def __init__(self, texts, labels_fake, labels_ai, tokenizer, max_len):
|
| 14 |
+
self.texts = texts
|
| 15 |
+
self.labels_fake = labels_fake
|
| 16 |
+
self.labels_ai = labels_ai
|
| 17 |
+
# ... sisa inisialisasi
|
| 18 |
+
|
| 19 |
+
def __getitem__(self, idx):
|
| 20 |
+
# ... proses encode
|
| 21 |
+
return enc, torch.tensor(self.labels_fake[idx], dtype=torch.long), torch.tensor(self.labels_ai[idx], dtype=torch.long)
|
| 22 |
+
```
|
| 23 |
+
|
| 24 |
+
2. UPDATE ARSITEKTUR MODEL (Dua Kepala Output)
|
| 25 |
+
------------------------------------------------------
|
| 26 |
+
Ganti penggunaan `AutoModelForSequenceClassification` dengan `AutoModel` dasar, lalu buat dua layer klasifikasi terpisah.
|
| 27 |
+
|
| 28 |
+
```python
|
| 29 |
+
class MTLTransformerClassifier(nn.Module):
|
| 30 |
+
def __init__(self, model_name):
|
| 31 |
+
super().__init__()
|
| 32 |
+
from transformers import AutoModel
|
| 33 |
+
self.encoder = AutoModel.from_pretrained(model_name)
|
| 34 |
+
|
| 35 |
+
hidden_size = self.encoder.config.hidden_size
|
| 36 |
+
|
| 37 |
+
# Kepala 1: Fake/Real
|
| 38 |
+
self.veracity_head = nn.Sequential(
|
| 39 |
+
nn.Dropout(0.1),
|
| 40 |
+
nn.Linear(hidden_size, 2)
|
| 41 |
+
)
|
| 42 |
+
|
| 43 |
+
# Kepala 2: Human/AI
|
| 44 |
+
self.origin_head = nn.Sequential(
|
| 45 |
+
nn.Dropout(0.1),
|
| 46 |
+
nn.Linear(hidden_size, 2)
|
| 47 |
+
)
|
| 48 |
+
|
| 49 |
+
def forward(self, **kwargs):
|
| 50 |
+
outputs = self.encoder(**kwargs)
|
| 51 |
+
# Ambil representasi token [CLS]
|
| 52 |
+
pooled_output = outputs.last_hidden_state[:, 0, :]
|
| 53 |
+
|
| 54 |
+
veracity_logits = self.veracity_head(pooled_output)
|
| 55 |
+
origin_logits = self.origin_head(pooled_output)
|
| 56 |
+
|
| 57 |
+
return veracity_logits, origin_logits
|
| 58 |
+
```
|
| 59 |
+
|
| 60 |
+
3. UPDATE FUNGSI TRAINING (Kombinasi 2 Loss)
|
| 61 |
+
------------------------------------------------------
|
| 62 |
+
Ubah `train_transformer` agar menghitung dua Error/Loss dan menggabungkannya.
|
| 63 |
+
|
| 64 |
+
```python
|
| 65 |
+
# Di dalam loop training:
|
| 66 |
+
veracity_out, origin_out = model(**enc)
|
| 67 |
+
|
| 68 |
+
loss_fake = criterion(veracity_out, label_fake)
|
| 69 |
+
loss_ai = criterion(origin_out, label_ai)
|
| 70 |
+
|
| 71 |
+
# Gabungkan loss (bisa disesuaikan bobotnya, misal 0.5 dan 0.5)
|
| 72 |
+
total_loss = loss_fake + loss_ai
|
| 73 |
+
total_loss.backward()
|
| 74 |
+
```
|
| 75 |
+
|
| 76 |
+
4. UPDATE FUNGSI INFERENSI (Backend)
|
| 77 |
+
------------------------------------------------------
|
| 78 |
+
Setelah model selesai ditraining dan di-save (contoh: `mtl_deberta.pt`), ubah `utils.py` di backend. Hapus pemanggilan model `chatgpt-detector-roberta`.
|
| 79 |
+
|
| 80 |
+
```python
|
| 81 |
+
# Di dalam utils.py
|
| 82 |
+
veracity_logits, origin_logits = model(**inputs)
|
| 83 |
+
|
| 84 |
+
v_probs = torch.softmax(veracity_logits, dim=1)
|
| 85 |
+
o_probs = torch.softmax(origin_logits, dim=1)
|
| 86 |
+
|
| 87 |
+
# Ambil hasil dari satu model untuk dua jawaban
|
| 88 |
+
```
|
| 89 |
+
|
| 90 |
+
======================================================
|
| 91 |
+
CATATAN PENTING:
|
| 92 |
+
* Training MTL membutuhkan waktu yang lebih lama.
|
| 93 |
+
* Pastikan dataset `final_dataset.csv` memiliki kolom `ai_label` yang proporsional dan akurat sebelum memulai training MTL.
|