Remediation & Enhancement Plan

Every gap found during validation, with step-by-step actions to fix each one. Work through fixes in priority order — later fixes depend on earlier ones.

Critical

High

Medium

Low

Quick Reference: All Issues

#	Severity	Issue	Files Affected
1	CRITICAL	`run_training()` never defined in notebook	`.ipynb`
2	CRITICAL	Synthetic toy data — model learns nothing real	`.ipynb`, `train.py`
3	HIGH	Notebook section order is broken (§9 → §7 → §8)	`.ipynb`
4	HIGH	No meaningful evaluation metrics (only accuracy)	`.ipynb`
5	HIGH	Ablation study conclusion is invalid (all 100%)	`.ipynb`
6	MEDIUM	`train.py` only trains M3, not M1/M2	`train.py`
7	MEDIUM	Project report document missing from repo	`Documents/`
8	MEDIUM	No literature comparison or baseline results	`.ipynb`
9	LOW	`get_gradcam()` in app.py is simplified vs notebook	`app.py`
10	LOW	Gradio vs Streamlit discrepancy not documented	`README.md`

Critical Fixes

Add the Missing run_training() Function to the Notebook

Problem

Cell 16 calls run_training('M1', ...), run_training('M2', ...), and run_training('M3', ...) but this function is never defined in any visible cell. The notebook cannot be run end-to-end — an examiner will get a NameError immediately.

Why it matters: A Colab notebook is expected to be fully self-contained. Every function called must be defined above its call site.

Open Image_Forgery_Detection_Colab_1.ipynb in Google Colab.

Insert a new code cell between the build_model() cell and the ablation execution cell (between current cell-9 and cell-16).

Paste the following function into that new cell:

Python — run_training() definition

def run_training(model_type, train_ds_base, val_ds_base, n_train, n_val):
    print(f"\n{'='*55}")
    print(f"Training {model_type}")
    print(f"{'='*55}")

    train_ds = adapt_dataset_for_model(train_ds_base, model_type)
    val_ds   = adapt_dataset_for_model(val_ds_base,   model_type)

    model = build_model(model_type)
    model.compile(
        optimizer='adam',
        loss='binary_crossentropy',
        metrics=['accuracy']
    )

    steps_per_epoch  = max(1, int(np.ceil(n_train / BATCH_SIZE)))
    validation_steps = max(1, int(np.ceil(n_val   / BATCH_SIZE)))

    history = model.fit(
        train_ds,
        validation_data=val_ds,
        epochs=EPOCHS,
        steps_per_epoch=steps_per_epoch,
        validation_steps=validation_steps,
        verbose=1,
    )

    save_path = f"{model_type}_best.keras"
    model.save(save_path)
    print(f"✔ {model_type} saved → {save_path}")
    return model, history

Run all cells from top to bottom to confirm there are no errors.

Confirm the output shows three training runs (M1, M2, M3) completing with no NameError.

Replace Synthetic Toy Data with Real CASIA v2

Problem

Current dataset: Authentic = random noise (RGB 100–200) | Forged = same noise + a solid red rectangle at [50–150, 50–150]. The model learns "red rectangle = forged" and gets 100% accuracy — this has no relationship to real image forgery detection.

Why it matters: An examiner will immediately recognise that 100% accuracy on 120 synthetic noise images is not a valid result. It is the single biggest weakness in the submission.

Step 2a — Download CASIA v2

Go to Kaggle and search for "CASIA v2 image forgery" or "CASIA 2.0 dataset".

Download the dataset (~3.3 GB). It contains ~12,614 authentic images (Au_*) and ~5,123 tampered images (Tp_*).

Upload the extracted folder to your Google Drive and name it casia_v2/.

Step 2b — Mount Drive and Update Data Path

Add this cell at the top of the data section in the notebook:

Python — Mount Google Drive

from google.colab import drive
drive.mount('/content/drive')

TARGET_DIR = "/content/drive/MyDrive/casia_v2"  # adjust if needed

Comment out or remove the call to generate_robust_dataset() — synthetic data is no longer needed.

Run split_dataset(TARGET_DIR) directly on the real data path.

Step 2c — Verify the Split

Python — Verify split counts and label balance

splits = split_dataset(TARGET_DIR)
print(f"Train: {len(splits['train'])} | Val: {len(splits['val'])} | Test: {len(splits['test'])}")

for split_name, paths in splits.items():
    authentic = sum(1 for p in paths if os.path.basename(p).startswith('Au_'))
    forged    = sum(1 for p in paths if os.path.basename(p).startswith('Tp_'))
    print(f"{split_name}: {authentic} authentic, {forged} forged")

Expected Output (approximate)

Train: ~14000 | Val: ~1700 | Test: ~1700 train: ~10000 authentic, ~4000 forged

Step 2d — Handle Class Imbalance

CASIA v2 has ~2.5× more authentic than tampered images. Without class weights, the model will bias toward predicting "authentic" and appear to have high accuracy while missing most forgeries.

Python — Compute class weights

from sklearn.utils.class_weight import compute_class_weight

classes = np.unique(train_labels)
weights = compute_class_weight('balanced', classes=classes, y=train_labels)
class_weight_dict = dict(zip(classes, weights))
print("Class weights:", class_weight_dict)
# Then pass class_weight=class_weight_dict to model.fit()

Step 2e — Retrain and Save

Run training. Expect accuracy in the range 80–92% (not 100%). If above 95%, check for data leakage. If below 70%, increase EPOCHS to 10–15.

Download the trained model from Colab:

Python — Download model from Colab

from google.colab import files
files.download('M3_best.keras')

Replace the existing M3_best.keras in the repo. Git LFS will handle the large file upload automatically on the next commit.

High Priority

Reorder Notebook Sections

Problem

Section headings appear in the wrong order: §9 (Execute) appears before §7 (Explainability) and §8 (Interface). The notebook is hard to follow and looks unpolished for submission.

Open the notebook in Colab and rearrange cells into this order:

Section	Content
§1	Setup & Dependencies
§2	Synthetic Dataset Generation (mark as optional — replaced by real data)
§3	ELA Utility (`compute_ela`)
§4	Data Pipeline (CASIAParser, split, preload, make_dataset)
§5	Model Architecture (get_rgb_branch, get_ela_branch, build_model)
§6	Training Engine (`run_training` — added in Fix 1)
§7	Explainability (`get_gradcam`)
§8	Interactive Interface (Gradio demo)
§9	Execute: 3-Way Ablation Study
§10	Results & Evaluation (new — see Fix 4)

Renumber all section headings to match the table above.

Run all cells again top-to-bottom to confirm no execution errors.

Add Proper Evaluation Metrics

Problem

Only accuracy is reported. On an imbalanced dataset like CASIA v2, a model that always predicts "authentic" achieves ~71% accuracy while being completely useless. Accuracy alone is not sufficient for a forensics task.

After the training cell (§9), add a new section §10 Results & Evaluation.

Add this evaluation code to generate a classification report and confusion matrix:

Python — Classification report + confusion matrix

from sklearn.metrics import (
    confusion_matrix, classification_report,
    roc_auc_score, RocCurveDisplay
)
import matplotlib.pyplot as plt
import seaborn as sns

test_ds_m3 = adapt_dataset_for_model(
    make_dataset(test_rgb, test_ela, test_labels, repeat=False), 'M3'
)
y_pred_prob = model_m3.predict(test_ds_m3, verbose=0).flatten()
y_pred = (y_pred_prob > 0.5).astype(int)
y_true = test_labels

print("="*50)
print("M3 (Fused) — Classification Report")
print("="*50)
print(classification_report(y_true, y_pred,
      target_names=['Authentic', 'Forged']))

cm = confusion_matrix(y_true, y_pred)
fig, ax = plt.subplots(figsize=(5, 4))
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',
            xticklabels=['Authentic', 'Forged'],
            yticklabels=['Authentic', 'Forged'])
ax.set_xlabel('Predicted'); ax.set_ylabel('Actual')
ax.set_title('M3 Confusion Matrix')
plt.savefig('confusion_matrix_m3.png', dpi=150)
plt.show()

auc = roc_auc_score(y_true, y_pred_prob)
print(f"ROC-AUC Score: {auc:.4f}")
RocCurveDisplay.from_predictions(y_true, y_pred_prob)
plt.title("M3 ROC Curve")
plt.savefig('roc_curve_m3.png', dpi=150)
plt.show()

Add the model comparison table across all three variants:

Python — M1 vs M2 vs M3 metrics table

results = {}
for name, model, m_type in [("M1_RGB", model_m1, 'M1'),
                              ("M2_ELA", model_m2, 'M2'),
                              ("M3_Fused", model_m3, 'M3')]:
    ds = adapt_dataset_for_model(
        make_dataset(test_rgb, test_ela, test_labels, repeat=False), m_type)
    probs = model.predict(ds, verbose=0).flatten()
    preds = (probs > 0.5).astype(int)
    from sklearn.metrics import f1_score, precision_score, recall_score
    results[name] = {
        'Accuracy':  np.mean(preds == test_labels),
        'Precision': precision_score(test_labels, preds, zero_division=0),
        'Recall':    recall_score(test_labels, preds, zero_division=0),
        'F1':        f1_score(test_labels, preds, zero_division=0),
        'AUC':       roc_auc_score(test_labels, probs),
    }

import pandas as pd
df_results = pd.DataFrame(results).T
print("\nAblation Study Results")
print(df_results.to_string(float_format="{:.4f}".format))

Save confusion_matrix_m3.png and roc_curve_m3.png and include them in the project report.

Make the Ablation Study Meaningful

Problem

With synthetic data, M1=M2=M3=100% — the ablation proves nothing. With real CASIA v2, the three models will produce genuinely different results. Depends on Fix 2 and Fix 4 being completed first.

After real-data training, expect results similar to this pattern:

Model	Input	Expected Accuracy	Strength	Weakness
M1 (RGB)	Original image	~75–85%	Semantic inconsistencies	Misses compression artifacts
M2 (ELA)	ELA residuals	~70–80%	Compression tampering	Misses structural forgeries
M3 (Fused)	Both	~85–92%	Combines both signals	Slightly slower inference

The comparison table from Fix 4 is your ablation study table — no separate code needed.

Add a markdown cell before the results table explaining why fusion outperforms single-branch models. Use the table above as a guide.

Medium Priority

Update train.py to Match the Notebook

Problem

train.py only builds and trains M3. It is missing adapt_dataset_for_model(), run_training(), and M1/M2 variants — all of which exist in the notebook.

Add adapt_dataset_for_model() from the notebook to train.py.

Add run_training() (same as Fix 1) to train.py.

Update build_model() to accept a model_type parameter ('M1', 'M2', 'M3') matching the notebook version.

Update the if __name__ == "__main__": block to train all three models and print the ablation comparison table.

Add the Project Report to the Repository

Problem

README.md references Documents/Project_Report_Digital_Image_Forgery_Detector.docx but neither the file nor the Documents/ folder exist in the repo.

Create the Documents/ folder in the repo root.

Place the project report .docx file inside it.

Run git add Documents/ and commit.

If the report does not yet exist, remove the reference from README.md until it is ready — a broken link is worse than no link.

Add a Literature Comparison Section

Problem

The notebook does not reference any published results on CASIA v2, making it impossible for an examiner to judge whether the results are competitive.

After the results table (§10), add a markdown cell titled "Comparison with Published Baselines".

Use this template (fill in your actual results after Fix 2 is done):

Method	Accuracy	F1	Notes
Rao et al. (2016) — CNN on SRM features	82.2%	—	Single-branch
Salloum et al. (2018) — FCN	89.3%	—	Pixel-level
Our M1 (RGB only)	your result	your result	ResNet50
Our M2 (ELA only)	your result	your result	Custom CNN
Our M3 (Fused)	your result	your result	Dual-branch

Add 2–3 sentences commenting on whether M3 is competitive and why it may be higher or lower than the baselines.

Low Priority

Align get_gradcam() in app.py with the Notebook

Problem

The notebook's get_gradcam() uses a model_type parameter to pick the correct last conv layer per model variant. The app.py version only searches for conv2d named layers — if the model is updated it could silently pick the wrong layer.

In app.py, replace the current get_gradcam() with the more robust version from the notebook.

Since app.py only runs M3, hard-code the call as:

Python — Updated call in app.py

heatmap = get_gradcam(m3, input_data, model_type='M3')

Document the Gradio → Streamlit Difference in README

Problem

The notebook uses Gradio (Colab-native); the deployed app uses Streamlit (Hugging Face Spaces). This intentional difference is not explained anywhere and may confuse an examiner.

Add the following section to README.md:

Markdown — README.md addition

## Development vs Deployment UI

The Colab notebook uses **Gradio** for its interactive demo because Gradio
works natively within Colab with a public share link.
The deployed Hugging Face Space uses **Streamlit** because it is the SDK
configured in the Space settings.
Both interfaces implement identical inference logic.

Recommended Implementation Order

Work through the fixes in this order to avoid rework — later fixes depend on earlier ones.

Fix 2a–b Download CASIA v2

→

Fix 1 run_training()

→

Fix 3 Reorder notebook

→

Fix 2c–e Retrain on real data

→

Fix 4 Eval metrics

→

Fix 5 Ablation write-up

→

Fix 6 train.py parity

→

Fix 7 Project report

→

Fix 8 Literature

→

Fix 9 Grad-CAM

→

Fix 10 README

Image Forgery Detector Remediation & Enhancement Plan — PGD Student Project