Rename README.txt to README.md
Browse files- README.md +78 -0
- README.txt +0 -0
README.md
ADDED
|
@@ -0,0 +1,78 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Source: CrisisMMD (Alam et al., 2017)
|
| 2 |
+
|
| 3 |
+
Data Type: Multimodal β each sample includes:
|
| 4 |
+
|
| 5 |
+
tweet_text (social media text)
|
| 6 |
+
|
| 7 |
+
tweet_image (corresponding image from the tweet)
|
| 8 |
+
|
| 9 |
+
Total Samples Used: ~18,802(from the dataset)
|
| 10 |
+
|
| 11 |
+
Class Labels:
|
| 12 |
+
|
| 13 |
+
0 β Non-informative
|
| 14 |
+
|
| 15 |
+
1 β Informative
|
| 16 |
+
|
| 17 |
+
Collect only values where tweet_text and tweet_image are equal. (thus collected 12,743 tweets and convert it into test and train .pt files)
|
| 18 |
+
|
| 19 |
+
β
Preprocessing Done
|
| 20 |
+
Text:
|
| 21 |
+
|
| 22 |
+
Tokenized using BERT tokenizer (bert-base-uncased)
|
| 23 |
+
|
| 24 |
+
Extracted input_ids and attention_mask
|
| 25 |
+
|
| 26 |
+
Image:
|
| 27 |
+
|
| 28 |
+
Processed using ResNet-50
|
| 29 |
+
|
| 30 |
+
Extracted 2048-dimensional feature vectors
|
| 31 |
+
|
| 32 |
+
Label:
|
| 33 |
+
|
| 34 |
+
Encoded to 0 or 1 as per class
|
| 35 |
+
|
| 36 |
+
The final preprocessed dataset was saved as .pt files:
|
| 37 |
+
|
| 38 |
+
train_info.pt
|
| 39 |
+
|
| 40 |
+
test_info.pt
|
| 41 |
+
|
| 42 |
+
Each contains: input_ids, attention_mask, image_vector, and label tensors.
|
| 43 |
+
|
| 44 |
+
β
Model Architecture
|
| 45 |
+
A custom multimodal neural network combining both BERT and ResNet features:
|
| 46 |
+
|
| 47 |
+
Component Details
|
| 48 |
+
Text Encoder BERT base model (bert-base-uncased) β outputs pooler_output (768-d)
|
| 49 |
+
Image Encoder ResNet-50 pre-extracted features (2048-d)
|
| 50 |
+
Fusion Concatenation β FC layers β Softmax
|
| 51 |
+
Classifier Fully connected layers with BatchNorm, ReLU, Dropout
|
| 52 |
+
|
| 53 |
+
β
Training Setup
|
| 54 |
+
Loss Function: CrossEntropyLoss
|
| 55 |
+
|
| 56 |
+
Optimizer: AdamW
|
| 57 |
+
|
| 58 |
+
Scheduler: StepLR (Ξ³ = 0.9)
|
| 59 |
+
|
| 60 |
+
Epochs: 8
|
| 61 |
+
|
| 62 |
+
Batch Size: 16
|
| 63 |
+
|
| 64 |
+
Device: CUDA (if available)
|
| 65 |
+
|
| 66 |
+
β
Evaluation Metrics
|
| 67 |
+
Accuracy
|
| 68 |
+
|
| 69 |
+
Precision
|
| 70 |
+
|
| 71 |
+
Recall
|
| 72 |
+
|
| 73 |
+
F1 Score
|
| 74 |
+
|
| 75 |
+
β
Test Accuracy : 0.8518
|
| 76 |
+
β
Precision : 0.8289
|
| 77 |
+
β
Recall : 0.8032
|
| 78 |
+
β
F1 Score : 0.8142
|
README.txt
DELETED
|
File without changes
|