gregorio commited on
Commit
fc2f1a6
·
1 Parent(s): bb93703

docs: add planning.txt for future true MTL transition

Browse files
Files changed (1) hide show
  1. planning.txt +93 -0
planning.txt ADDED
@@ -0,0 +1,93 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # PLAN: TRANSISI KE TRUE MULTI-TASK LEARNING (MTL)
2
+ ======================================================
3
+ Dokumen ini berisi panduan teknis bagi developer untuk mengubah arsitektur kode saat ini (Single-Task/Ensemble) menjadi True Multi-Task Learning sesuai dengan Paper penelitian.
4
+
5
+ Langkah-langkah yang harus dilakukan pada 'fake_news_pipeline.ipynb':
6
+
7
+ 1. UPDATE DATASET LOADER (Memasukkan 2 Label)
8
+ ------------------------------------------------------
9
+ Ubah class `TransformerDataset` agar menerima dua array label (fake_news dan ai_origin) dan me-return keduanya saat iterasi.
10
+
11
+ ```python
12
+ class MTLTransformerDataset(Dataset):
13
+ def __init__(self, texts, labels_fake, labels_ai, tokenizer, max_len):
14
+ self.texts = texts
15
+ self.labels_fake = labels_fake
16
+ self.labels_ai = labels_ai
17
+ # ... sisa inisialisasi
18
+
19
+ def __getitem__(self, idx):
20
+ # ... proses encode
21
+ return enc, torch.tensor(self.labels_fake[idx], dtype=torch.long), torch.tensor(self.labels_ai[idx], dtype=torch.long)
22
+ ```
23
+
24
+ 2. UPDATE ARSITEKTUR MODEL (Dua Kepala Output)
25
+ ------------------------------------------------------
26
+ Ganti penggunaan `AutoModelForSequenceClassification` dengan `AutoModel` dasar, lalu buat dua layer klasifikasi terpisah.
27
+
28
+ ```python
29
+ class MTLTransformerClassifier(nn.Module):
30
+ def __init__(self, model_name):
31
+ super().__init__()
32
+ from transformers import AutoModel
33
+ self.encoder = AutoModel.from_pretrained(model_name)
34
+
35
+ hidden_size = self.encoder.config.hidden_size
36
+
37
+ # Kepala 1: Fake/Real
38
+ self.veracity_head = nn.Sequential(
39
+ nn.Dropout(0.1),
40
+ nn.Linear(hidden_size, 2)
41
+ )
42
+
43
+ # Kepala 2: Human/AI
44
+ self.origin_head = nn.Sequential(
45
+ nn.Dropout(0.1),
46
+ nn.Linear(hidden_size, 2)
47
+ )
48
+
49
+ def forward(self, **kwargs):
50
+ outputs = self.encoder(**kwargs)
51
+ # Ambil representasi token [CLS]
52
+ pooled_output = outputs.last_hidden_state[:, 0, :]
53
+
54
+ veracity_logits = self.veracity_head(pooled_output)
55
+ origin_logits = self.origin_head(pooled_output)
56
+
57
+ return veracity_logits, origin_logits
58
+ ```
59
+
60
+ 3. UPDATE FUNGSI TRAINING (Kombinasi 2 Loss)
61
+ ------------------------------------------------------
62
+ Ubah `train_transformer` agar menghitung dua Error/Loss dan menggabungkannya.
63
+
64
+ ```python
65
+ # Di dalam loop training:
66
+ veracity_out, origin_out = model(**enc)
67
+
68
+ loss_fake = criterion(veracity_out, label_fake)
69
+ loss_ai = criterion(origin_out, label_ai)
70
+
71
+ # Gabungkan loss (bisa disesuaikan bobotnya, misal 0.5 dan 0.5)
72
+ total_loss = loss_fake + loss_ai
73
+ total_loss.backward()
74
+ ```
75
+
76
+ 4. UPDATE FUNGSI INFERENSI (Backend)
77
+ ------------------------------------------------------
78
+ Setelah model selesai ditraining dan di-save (contoh: `mtl_deberta.pt`), ubah `utils.py` di backend. Hapus pemanggilan model `chatgpt-detector-roberta`.
79
+
80
+ ```python
81
+ # Di dalam utils.py
82
+ veracity_logits, origin_logits = model(**inputs)
83
+
84
+ v_probs = torch.softmax(veracity_logits, dim=1)
85
+ o_probs = torch.softmax(origin_logits, dim=1)
86
+
87
+ # Ambil hasil dari satu model untuk dua jawaban
88
+ ```
89
+
90
+ ======================================================
91
+ CATATAN PENTING:
92
+ * Training MTL membutuhkan waktu yang lebih lama.
93
+ * Pastikan dataset `final_dataset.csv` memiliki kolom `ai_label` yang proporsional dan akurat sebelum memulai training MTL.