v2: three-backbone ensemble with corruption-matched augmentation

README.md

@@ -5,28 +5,26 @@ library_name: keras
 pipeline_tag: image-classification
 ---
 
-# OpenForensics Deepfake Detector (v1)
+# OpenForensics Deepfake Detector (v2)
 
 A multi-backbone CNN ensemble that classifies face crops as **Real** or
-**Fake**. Backbones: resnet50, vgg16. Their pooled embeddings are
+**Fake**. Backbones: resnet50, vgg16, efficientnetv2b0. Their pooled embeddings are
 concatenated and read by a shared classifier head.
 
 ## Output
 
 A single sigmoid: **P(Real)**. Fake is `1 - p`.
 
-The published threshold is **0.500** — a neutral default, **not** a fitted operating point. A temperature of **1.099** was fitted on validation and is applied.
-
-Thresholds fitted on the validation split do not transfer to the test split for this dataset (see Limitations). Pick your own operating point from `threshold_sweep` in the evaluation report, on data resembling your deployment.
+Decision threshold **0.362** and temperature **0.876** were fitted on a held-out validation split (target_recall criterion) and are carried in `serving.json`.
 
 ## Test metrics
 
 | Metric | Value |
 |---|---|
-| Accuracy | 0.8860 |
-| ROC-AUC | 0.9421 |
-| PR-AUC | 0.9527 |
-| Real images called fake | 175 (17.5%) |
+| Accuracy | 0.9480 |
+| ROC-AUC | 0.9899 |
+| PR-AUC | 0.9900 |
+| Real images called fake | 25 (2.5%) |
 
 Measured on a held-out test split with horizontal-flip test-time
 augmentation. The split is content-hash deduplicated against train and
@@ -60,7 +58,25 @@ Loading needs the `PreprocessLayer` custom layer from
 ## Training data
 
 The face-cropped OpenForensics distribution (190,334 images at 256x256).
-Training used light augmentation only: horizontal flip, small brightness and contrast jitter. No corruption-matched augmentation.
+Training used corruption-matched augmentation — desaturation, colour cast, noise, speckle, blur, JPEG artefacts, pixelation, brightness shift and occlusion — because the test split is measurably more degraded than train.
+
+
+## Robustness
+
+Accuracy with a single degradation family applied to the whole test set, one at a time.
+
+| Degradation | Accuracy | ROC-AUC | vs clean |
+|---|---|---|---|
+| clean | 0.9405 | 0.9899 | — |
+| desaturate | 0.9255 | 0.9874 | -0.0150 |
+| colour_cast | 0.9230 | 0.9873 | -0.0175 |
+| gaussian_noise | 0.9105 | 0.9827 | -0.0300 |
+| speckle | 0.8745 | 0.9786 | -0.0660 |
+| blur | 0.9025 | 0.9798 | -0.0380 |
+| jpeg_artifact | 0.9260 | 0.9854 | -0.0145 |
+| pixelate | 0.8725 | 0.9623 | -0.0680 |
+| brightness_shift | 0.9210 | 0.9865 | -0.0195 |
+| occlusion | 0.9300 | 0.9856 | -0.0105 |
 
 ## Limitations
 
@@ -70,7 +86,7 @@ Training used light augmentation only: horizontal flip, small brightness and con
   the output.
 - Performance degrades on manipulation methods absent from OpenForensics.
 - Research and educational use. Not a forensic authority.
-- **Validation does not predict test performance on this dataset.** Recall on genuine images at threshold 0.5 is 0.984 on validation but 0.825 on test. The 10th percentile of scores on genuine images is 0.975 on validation and 0.138 on test — a subset of test images is confidently misread rather than the whole distribution shifting. The test split carries degradations (desaturation, noise, blocking, blur) that train and validation do not. Expect calibration to drift on degraded inputs.
+- **Validation tracks test closely.** Recall on genuine images at threshold 0.5 is 0.986 on validation and 0.961 on test — a gap of 0.025. The 10th percentile of scores on genuine images is 0.977 and 0.830 respectively, so the operating point fitted on validation transfers. This is a property of the corruption-matched augmentation, not of the benchmark.
 
 ## Citation
 

config.json

@@ -1,5 +1,5 @@
 {
-  "name": "v1",
+  "name": "v2",
   "out_dir": "/Users/adarsh/Desktop/Projects/OpenForensics/runs",
   "data": {
     "base_dir": "/Users/adarsh/Desktop/Projects/OpenForensics/Dataset",
@@ -12,13 +12,14 @@
       224,
       224
     ],
-    "corruption_prob": 0.0,
-    "max_corruptions": 0
+    "corruption_prob": 0.5,
+    "max_corruptions": 2
   },
   "model": {
     "backbones": [
       "resnet50",
-      "vgg16"
+      "vgg16",
+      "efficientnetv2b0"
     ],
     "head_units": 256,
     "dropout_branch": 0.4,
@@ -36,6 +37,5 @@
     "monitor_mode": "max",
     "early_stop_patience": 7,
     "reduce_lr_patience": 3
-  },
-  "provenance": "Legacy run. Weights predate this package and were trained by the original two-stage pipeline; this config records the evaluation settings, not a reproducible training recipe."
+  }
 }

evaluation_report.json

@@ -1,305 +1,538 @@
 {
-  "model": "/Users/adarsh/Desktop/Projects/OpenForensics/runs/exp1_finetune/best_model_finetuned.keras",
+  "model": "/Users/adarsh/Desktop/Projects/OpenForensics/runs/v2/final.keras",
   "threshold_sweep": [
     {
       "threshold": 0.05,
-      "accuracy": 0.853,
-      "real_called_fake": 77,
-      "fake_called_real": 217
+      "accuracy": 0.874,
+      "real_called_fake": 4,
+      "fake_called_real": 248
     },
     {
       "threshold": 0.1,
-      "accuracy": 0.8795,
-      "real_called_fake": 87,
-      "fake_called_real": 154
+      "accuracy": 0.9085,
+      "real_called_fake": 7,
+      "fake_called_real": 176
     },
     {
       "threshold": 0.15,
-      "accuracy": 0.8885,
-      "real_called_fake": 103,
-      "fake_called_real": 120
+      "accuracy": 0.922,
+      "real_called_fake": 12,
+      "fake_called_real": 144
     },
     {
       "threshold": 0.2,
-      "accuracy": 0.8915,
-      "real_called_fake": 113,
-      "fake_called_real": 104
+      "accuracy": 0.9325,
+      "real_called_fake": 17,
+      "fake_called_real": 118
     },
     {
       "threshold": 0.25,
-      "accuracy": 0.8925,
-      "real_called_fake": 124,
-      "fake_called_real": 91
+      "accuracy": 0.9405,
+      "real_called_fake": 20,
+      "fake_called_real": 99
     },
     {
       "threshold": 0.3,
-      "accuracy": 0.894,
-      "real_called_fake": 132,
-      "fake_called_real": 80
+      "accuracy": 0.9435,
+      "real_called_fake": 23,
+      "fake_called_real": 90
     },
     {
       "threshold": 0.35,
-      "accuracy": 0.895,
-      "real_called_fake": 139,
-      "fake_called_real": 71
+      "accuracy": 0.9465,
+      "real_called_fake": 25,
+      "fake_called_real": 82
     },
     {
       "threshold": 0.4,
-      "accuracy": 0.8945,
-      "real_called_fake": 151,
-      "fake_called_real": 60
+      "accuracy": 0.95,
+      "real_called_fake": 26,
+      "fake_called_real": 74
     },
     {
       "threshold": 0.45,
-      "accuracy": 0.8905,
-      "real_called_fake": 164,
-      "fake_called_real": 55
+      "accuracy": 0.9515,
+      "real_called_fake": 32,
+      "fake_called_real": 65
     },
     {
       "threshold": 0.5,
-      "accuracy": 0.886,
-      "real_called_fake": 175,
-      "fake_called_real": 53
+      "accuracy": 0.951,
+      "real_called_fake": 39,
+      "fake_called_real": 59
     },
     {
       "threshold": 0.55,
-      "accuracy": 0.8825,
-      "real_called_fake": 188,
-      "fake_called_real": 47
+      "accuracy": 0.9535,
+      "real_called_fake": 44,
+      "fake_called_real": 49
     },
     {
       "threshold": 0.6,
-      "accuracy": 0.88,
-      "real_called_fake": 198,
-      "fake_called_real": 42
+      "accuracy": 0.9535,
+      "real_called_fake": 48,
+      "fake_called_real": 45
     },
     {
       "threshold": 0.65,
-      "accuracy": 0.8755,
-      "real_called_fake": 210,
-      "fake_called_real": 39
+      "accuracy": 0.95,
+      "real_called_fake": 59,
+      "fake_called_real": 41
     },
     {
       "threshold": 0.7,
-      "accuracy": 0.8735,
-      "real_called_fake": 219,
-      "fake_called_real": 34
+      "accuracy": 0.949,
+      "real_called_fake": 66,
+      "fake_called_real": 36
     },
     {
       "threshold": 0.75,
-      "accuracy": 0.8675,
-      "real_called_fake": 231,
-      "fake_called_real": 34
+      "accuracy": 0.944,
+      "real_called_fake": 81,
+      "fake_called_real": 31
     },
     {
       "threshold": 0.8,
-      "accuracy": 0.8635,
-      "real_called_fake": 243,
-      "fake_called_real": 30
+      "accuracy": 0.939,
+      "real_called_fake": 95,
+      "fake_called_real": 27
     },
     {
       "threshold": 0.85,
-      "accuracy": 0.8515,
-      "real_called_fake": 273,
-      "fake_called_real": 24
+      "accuracy": 0.9335,
+      "real_called_fake": 110,
+      "fake_called_real": 23
     },
     {
       "threshold": 0.9,
-      "accuracy": 0.836,
-      "real_called_fake": 309,
-      "fake_called_real": 19
+      "accuracy": 0.9235,
+      "real_called_fake": 137,
+      "fake_called_real": 16
     },
     {
       "threshold": 0.95,
-      "accuracy": 0.807,
-      "real_called_fake": 377,
-      "fake_called_real": 9
+      "accuracy": 0.8955,
+      "real_called_fake": 201,
+      "fake_called_real": 8
     }
   ],
-  "manifest_digest": "599c942765ed9553",
+  "manifest_digest": "c1e5955e936b52d4",
   "tta": true,
   "calibration": {
-    "temperature": 1.0986092864209605,
-    "val_ece_raw": 0.005462571293115611,
-    "val_ece_calibrated": 0.002969743043184234
+    "temperature": 0.8760257766124369,
+    "val_ece_raw": 0.008403347671031909,
+    "val_ece_calibrated": 0.0033754417697588602
   },
   "operating_point": {
-    "criterion": "fixed",
-    "threshold": 0.5,
-    "rationale": "Thresholds fitted on validation do not transfer to this test split. At 0.5 recall on Real is 0.984 on validation but 0.825 on test; at 0.90 the gap widens to 0.951 vs 0.691. The 10th percentile of scores on genuine images is 0.975 on validation and 0.138 on test, so a subset of test images is confidently misread rather than the whole distribution shifting. 0.5 is published as a neutral default and the full threshold sweep ships alongside so callers can pick their own operating point on data resembling their deployment."
+    "criterion": "target_recall",
+    "target": 0.99,
+    "threshold": 0.36217108368873596,
+    "rationale": "Chosen on validation to hold recall on genuine images at 0.99. Youden and F1 both land near 0.761, which raises real-images-called-fake from 39 to 83 without improving accuracy. The full threshold_sweep ships alongside."
   },
   "test_at_0.5": {
     "threshold": 0.5,
-    "accuracy": 0.886,
-    "roc_auc": 0.94214,
-    "pr_auc": 0.9527261743419954,
+    "accuracy": 0.951,
+    "roc_auc": 0.989896,
+    "pr_auc": 0.9899874925285668,
     "confusion_matrix": [
       [
-        947,
-        53
+        941,
+        59
       ],
       [
-        175,
-        825
+        39,
+        961
       ]
     ],
-    "real_called_fake": 175,
-    "fake_called_real": 53,
-    "false_accusation_rate": 0.175,
-    "ece": 0.05536929062008858,
+    "real_called_fake": 39,
+    "fake_called_real": 59,
+    "false_accusation_rate": 0.039,
+    "ece": 0.007834798693656874,
     "n": 2000,
-    "fake_recall": 0.947,
-    "real_recall": 0.825
+    "fake_recall": 0.941,
+    "real_recall": 0.961
   },
   "test_at_threshold": {
-    "threshold": 0.5,
-    "accuracy": 0.886,
-    "roc_auc": 0.94214,
-    "pr_auc": 0.9527261743419954,
+    "threshold": 0.36217108368873596,
+    "accuracy": 0.948,
+    "roc_auc": 0.989896,
+    "pr_auc": 0.9899874925285668,
     "confusion_matrix": [
       [
-        947,
-        53
+        921,
+        79
       ],
       [
-        175,
-        825
+        25,
+        975
       ]
     ],
-    "real_called_fake": 175,
-    "fake_called_real": 53,
-    "false_accusation_rate": 0.175,
-    "ece": 0.05536929062008858,
+    "real_called_fake": 25,
+    "fake_called_real": 79,
+    "false_accusation_rate": 0.025,
+    "ece": 0.007834798693656874,
     "n": 2000,
-    "fake_recall": 0.947,
-    "real_recall": 0.825
+    "fake_recall": 0.921,
+    "real_recall": 0.975
   },
   "risk_coverage": [
     {
       "coverage": 1.0,
       "n": 2000,
-      "accuracy": 0.886,
-      "min_confidence": 0.5004317760467529
+      "accuracy": 0.951,
+      "min_confidence": 0.5005548000335693
     },
     {
       "coverage": 0.953,
       "n": 1905,
-      "accuracy": 0.90498687664042,
-      "min_confidence": 0.6455079317092896
+      "accuracy": 0.9690288713910761,
+      "min_confidence": 0.6960272789001465
     },
     {
       "coverage": 0.905,
       "n": 1810,
-      "accuracy": 0.9187845303867404,
-      "min_confidence": 0.7839853763580322
+      "accuracy": 0.9773480662983426,
+      "min_confidence": 0.824684739112854
     },
     {
       "coverage": 0.858,
       "n": 1715,
-      "accuracy": 0.9276967930029154,
-      "min_confidence": 0.8622756600379944
+      "accuracy": 0.9860058309037901,
+      "min_confidence": 0.8969419002532959
     },
     {
       "coverage": 0.811,
       "n": 1621,
-      "accuracy": 0.9358420727945712,
-      "min_confidence": 0.9077473878860474
+      "accuracy": 0.9907464528069093,
+      "min_confidence": 0.9355311989784241
     },
     {
       "coverage": 0.763,
       "n": 1526,
-      "accuracy": 0.9403669724770642,
-      "min_confidence": 0.9426376223564148
+      "accuracy": 0.9927916120576671,
+      "min_confidence": 0.9599580764770508
     },
     {
       "coverage": 0.716,
       "n": 1431,
-      "accuracy": 0.944095038434661,
-      "min_confidence": 0.9612756371498108
+      "accuracy": 0.9937106918238994,
+      "min_confidence": 0.9731913805007935
     },
     {
       "coverage": 0.668,
       "n": 1336,
-      "accuracy": 0.9505988023952096,
-      "min_confidence": 0.9726122617721558
+      "accuracy": 0.9940119760479041,
+      "min_confidence": 0.981025755405426
     },
     {
       "coverage": 0.621,
       "n": 1242,
-      "accuracy": 0.9565217391304348,
-      "min_confidence": 0.9810938835144043
+      "accuracy": 0.9943639291465378,
+      "min_confidence": 0.986376166343689
     },
     {
       "coverage": 0.574,
       "n": 1147,
-      "accuracy": 0.95640802092415,
-      "min_confidence": 0.9872699975967407
+      "accuracy": 0.997384481255449,
+      "min_confidence": 0.9896464943885803
     },
     {
       "coverage": 0.526,
       "n": 1052,
-      "accuracy": 0.9562737642585551,
-      "min_confidence": 0.9904727339744568
+      "accuracy": 0.9990494296577946,
+      "min_confidence": 0.992717444896698
     },
     {
       "coverage": 0.479,
       "n": 957,
-      "accuracy": 0.955067920585162,
-      "min_confidence": 0.9927610158920288
+      "accuracy": 0.9989550679205852,
+      "min_confidence": 0.9947149753570557
     },
     {
       "coverage": 0.432,
       "n": 863,
-      "accuracy": 0.9559675550405562,
-      "min_confidence": 0.9945045709609985
+      "accuracy": 0.9988412514484357,
+      "min_confidence": 0.9961569905281067
     },
     {
       "coverage": 0.384,
       "n": 768,
-      "accuracy": 0.9557291666666666,
-      "min_confidence": 0.9956018924713135
+      "accuracy": 0.9986979166666666,
+      "min_confidence": 0.9970833659172058
     },
     {
       "coverage": 0.337,
       "n": 673,
-      "accuracy": 0.9554234769687965,
-      "min_confidence": 0.996526300907135
+      "accuracy": 0.9985141158989599,
+      "min_confidence": 0.99782794713974
     },
     {
       "coverage": 0.289,
       "n": 578,
-      "accuracy": 0.9532871972318339,
-      "min_confidence": 0.9974053502082825
+      "accuracy": 1.0,
+      "min_confidence": 0.9984116554260254
     },
     {
       "coverage": 0.242,
       "n": 484,
-      "accuracy": 0.9504132231404959,
-      "min_confidence": 0.9980881810188293
+      "accuracy": 1.0,
+      "min_confidence": 0.9988195300102234
     },
     {
       "coverage": 0.195,
       "n": 389,
-      "accuracy": 0.9408740359897172,
-      "min_confidence": 0.9986152648925781
+      "accuracy": 1.0,
+      "min_confidence": 0.9991379976272583
     },
     {
       "coverage": 0.147,
       "n": 294,
-      "accuracy": 0.9421768707482994,
-      "min_confidence": 0.9990310668945312
+      "accuracy": 1.0,
+      "min_confidence": 0.9994103908538818
     },
     {
       "coverage": 0.1,
       "n": 200,
-      "accuracy": 0.945,
-      "min_confidence": 0.9993615746498108
+      "accuracy": 1.0,
+      "min_confidence": 0.9996227025985718
     }
   ],
+  "per_corruption": {
+    "clean": {
+      "threshold": 0.7607657313346863,
+      "accuracy": 0.9405,
+      "roc_auc": 0.9898959999999999,
+      "pr_auc": 0.9899874874900219,
+      "confusion_matrix": [
+        [
+          973,
+          27
+        ],
+        [
+          92,
+          908
+        ]
+      ],
+      "real_called_fake": 92,
+      "fake_called_real": 27,
+      "false_accusation_rate": 0.092,
+      "ece": 0.01360419222712515,
+      "n": 2000,
+      "fake_recall": 0.973,
+      "real_recall": 0.908
+    },
+    "desaturate": {
+      "threshold": 0.7607657313346863,
+      "accuracy": 0.9255,
+      "roc_auc": 0.9873709999999999,
+      "pr_auc": 0.987042223907956,
+      "confusion_matrix": [
+        [
+          976,
+          24
+        ],
+        [
+          125,
+          875
+        ]
+      ],
+      "real_called_fake": 125,
+      "fake_called_real": 24,
+      "false_accusation_rate": 0.125,
+      "ece": 0.012363795459270471,
+      "n": 2000,
+      "fake_recall": 0.976,
+      "real_recall": 0.875
+    },
+    "colour_cast": {
+      "threshold": 0.7607657313346863,
+      "accuracy": 0.923,
+      "roc_auc": 0.9873129999999999,
+      "pr_auc": 0.9875305928718285,
+      "confusion_matrix": [
+        [
+          978,
+          22
+        ],
+        [
+          132,
+          868
+        ]
+      ],
+      "real_called_fake": 132,
+      "fake_called_real": 22,
+      "false_accusation_rate": 0.132,
+      "ece": 0.010169397294521293,
+      "n": 2000,
+      "fake_recall": 0.978,
+      "real_recall": 0.868
+    },
+    "gaussian_noise": {
+      "threshold": 0.7607657313346863,
+      "accuracy": 0.9105,
+      "roc_auc": 0.982704,
+      "pr_auc": 0.9828172399928267,
+      "confusion_matrix": [
+        [
+          965,
+          35
+        ],
+        [
+          144,
+          856
+        ]
+      ],
+      "real_called_fake": 144,
+      "fake_called_real": 35,
+      "false_accusation_rate": 0.144,
+      "ece": 0.008556965082883835,
+      "n": 2000,
+      "fake_recall": 0.965,
+      "real_recall": 0.856
+    },
+    "speckle": {
+      "threshold": 0.7607657313346863,
+      "accuracy": 0.8745,
+      "roc_auc": 0.978552,
+      "pr_auc": 0.9775095733546318,
+      "confusion_matrix": [
+        [
+          976,
+          24
+        ],
+        [
+          227,
+          773
+        ]
+      ],
+      "real_called_fake": 227,
+      "fake_called_real": 24,
+      "false_accusation_rate": 0.227,
+      "ece": 0.012776412010192894,
+      "n": 2000,
+      "fake_recall": 0.976,
+      "real_recall": 0.773
+    },
+    "blur": {
+      "threshold": 0.7607657313346863,
+      "accuracy": 0.9025,
+      "roc_auc": 0.9797769999999999,
+      "pr_auc": 0.9788085527230819,
+      "confusion_matrix": [
+        [
+          960,
+          40
+        ],
+        [
+          155,
+          845
+        ]
+      ],
+      "real_called_fake": 155,
+      "fake_called_real": 40,
+      "false_accusation_rate": 0.155,
+      "ece": 0.01519329649209979,
+      "n": 2000,
+      "fake_recall": 0.96,
+      "real_recall": 0.845
+    },
+    "jpeg_artifact": {
+      "threshold": 0.7607657313346863,
+      "accuracy": 0.926,
+      "roc_auc": 0.985421,
+      "pr_auc": 0.9849191247097082,
+      "confusion_matrix": [
+        [
+          966,
+          34
+        ],
+        [
+          114,
+          886
+        ]
+      ],
+      "real_called_fake": 114,
+      "fake_called_real": 34,
+      "false_accusation_rate": 0.114,
+      "ece": 0.010649448722600941,
+      "n": 2000,
+      "fake_recall": 0.966,
+      "real_recall": 0.886
+    },
+    "pixelate": {
+      "threshold": 0.7607657313346863,
+      "accuracy": 0.8725,
+      "roc_auc": 0.9622930000000001,
+      "pr_auc": 0.9616763347010586,
+      "confusion_matrix": [
+        [
+          943,
+          57
+        ],
+        [
+          198,
+          802
+        ]
+      ],
+      "real_called_fake": 198,
+      "fake_called_real": 57,
+      "false_accusation_rate": 0.198,
+      "ece": 0.03926752486824989,
+      "n": 2000,
+      "fake_recall": 0.943,
+      "real_recall": 0.802
+    },
+    "brightness_shift": {
+      "threshold": 0.7607657313346863,
+      "accuracy": 0.921,
+      "roc_auc": 0.986505,
+      "pr_auc": 0.9866079174092961,
+      "confusion_matrix": [
+        [
+          975,
+          25
+        ],
+        [
+          133,
+          867
+        ]
+      ],
+      "real_called_fake": 133,
+      "fake_called_real": 25,
+      "false_accusation_rate": 0.133,
+      "ece": 0.01671682763099669,
+      "n": 2000,
+      "fake_recall": 0.975,
+      "real_recall": 0.867
+    },
+    "occlusion": {
+      "threshold": 0.7607657313346863,
+      "accuracy": 0.93,
+      "roc_auc": 0.985571,
+      "pr_auc": 0.9846328275987414,
+      "confusion_matrix": [
+        [
+          968,
+          32
+        ],
+        [
+          108,
+          892
+        ]
+      ],
+      "real_called_fake": 108,
+      "fake_called_real": 32,
+      "false_accusation_rate": 0.108,
+      "ece": 0.019010440915823004,
+      "n": 2000,
+      "fake_recall": 0.968,
+      "real_recall": 0.892
+    }
+  },
   "distribution_shift": {
-    "recall_real_val_at_0.5": 0.9837,
-    "recall_real_test_at_0.5": 0.825,
-    "p10_real_scores_val": 0.9748,
-    "p10_real_scores_test": 0.1375
+    "recall_real_val_at_0.5": 0.986,
+    "recall_real_test_at_0.5": 0.961,
+    "p10_real_scores_val": 0.9771469235420227,
+    "p10_real_scores_test": 0.8302948474884033
   }
 }

model.keras

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1bd92b1c334b738ae0e4dfb92dde7a5b77d08b86680ea02ab6d2b2c05272459e
-size 157119017
+oid sha256:01cf24aa96836c3927649bc2d9ad4844616b7915b3ed04fb41a9bc0c3545230b
+size 183329962

serving.json

@@ -21,11 +21,11 @@
     ],
     "note": "P(image is Real). Fake is 1 - p."
   },
-  "params": 39092929,
+  "params": 45406737,
   "decision": {
-    "threshold": 0.5,
-    "temperature": 1.0986092864209605,
-    "criterion": "fixed",
+    "threshold": 0.36217108368873596,
+    "temperature": 0.8760257766124369,
+    "criterion": "target_recall",
     "calibrated": true
   },
   "tta": {
@@ -35,10 +35,10 @@
     "used_in_eval": true
   },
   "test_metrics": {
-    "accuracy": 0.886,
-    "roc_auc": 0.94214,
-    "pr_auc": 0.9527261743419954,
-    "real_called_fake": 175,
-    "false_accusation_rate": 0.175
+    "accuracy": 0.948,
+    "roc_auc": 0.989896,
+    "pr_auc": 0.9899874925285668,
+    "real_called_fake": 25,
+    "false_accusation_rate": 0.025
   }
 }