Add files using upload-large-folder tool

README.md

@@ -0,0 +1,27 @@
+---
+library_name: pytorch
+tags:
+- conditional-diffusion
+- inkjet-quality-control
+- yolo
+- out-of-distribution-detection
+- thesis
+license: cc-by-4.0
+---
+
+# Inkjet Conditional Diffusion and YOLO Feature Detection
+
+This Hugging Face package contains the model/data-heavy release side for the final inkjet thesis experiments. It includes local data artifacts requested for reproducibility, available trained model weights with clear provenance, final CV score files, metrics, and figure/table artifacts.
+
+## Reported Thesis Results
+- Crop-based inkjet CDM, lambda=0, 5-fold CV: AUROC 0.8673 +/- 0.0230.
+- Non-zero separation-loss weights did not significantly improve over baseline after correction.
+- YOLOv8 feature detector: approximately 95% mAP@0.5 on the thesis feature detection task.
+
+## Contents
+- `models/`: available local YOLO/CDM weights with clear curated-package provenance.
+- `data/yolo_format_v2/`: local YOLO image/label dataset artifacts.
+- `data/crop_examples/`: crop examples used for inspection/documentation.
+- `data/metadata/`: local metadata/labels available in the source folder.
+- `results/`: final CV summaries, per-fold metrics/scores, single-split traces, figures, and tables.
+- `missing_checkpoints.md`: note for deleted final inkjet CV checkpoints.

docs/EXPERIMENTS.md

@@ -0,0 +1,210 @@
+# Inkjet CDM — Experiment Log
+
+Full chronological log of all inkjet CDM experiments.
+
+---
+
+## Model Architecture
+
+```
+CDM UNet — Multi-Head Conditioning
+  Parameters:    ~9.3M (inkjet model, smaller than CIFAR-10's 35.7M)
+  Input:         YOLO-cropped inkjet print region (bbox crop)
+  Conditioning:  4 heads — template_id, feature_id, quality_label, bbox_coords
+  Schedule:      Cosine (Nichol & Dhariwal 2021), 1000 timesteps, epsilon prediction
+  Scoring:       Algorithm 1 — difference method (pred_good_err - pred_bad_err)
+```
+
+---
+
+## Dataset Statistics
+
+```
+Total samples:   1,327 (from feature_cls.csv)
+Train split:     80% = ~1,061 samples (with BAD oversampling ×3 → effective train size larger)
+Test split:      20% = 266 samples (FIXED, seed=42, no oversampling)
+
+Test set breakdown:
+  Templates:  A=105, B=28, C=133
+  Labels:     GOOD=174, BAD=92  (imbalance: 1.89:1)
+
+Per-feature test counts:
+  angle:  27 GOOD,  3 BAD   ← WARNING: only 3 BAD — AUROC unreliable
+  dist1:  21 GOOD, 10 BAD
+  dist6:  30 GOOD,  6 BAD
+  dots:   26 GOOD, 11 BAD
+  edge1:  12 GOOD, 16 BAD
+  edge2:  18 GOOD, 28 BAD
+  edge3:  16 GOOD, 12 BAD
+  edge4:  24 GOOD,  6 BAD
+```
+
+---
+
+## Experiment 1 — Baseline CDM (λ=0)
+
+**Goal:** Establish CDM performance without separation loss.
+
+**Config:**
+```
+sep_loss_weight:  0.0
+batch_size:       128  (single forward pass, fits 32GB VRAM comfortably)
+epochs:           100
+schedule:         cosine
+num_trials:       100 (K=100 at evaluation)
+out_dir:          results/inkjet_lambda0
+```
+
+**Key fix applied:** Trainer gated separation loss computation behind `if sep_loss_weight > 0.0`
+to prevent 3× VRAM usage and OOM errors on the baseline run.
+
+**Results (K=100):**
+```
+Overall  AUROC=0.8325  Acc=0.7895  FPR@95TPR=0.8161  N=266
+```
+
+**Output files:**
+- `results/inkjet_lambda0/cdm_best.pt`
+- `results/inkjet_lambda0_k100/scores.csv`
+- `results/inkjet_lambda0.log`
+
+---
+
+## Experiment 2 — Separation Loss λ=0.02
+
+**Goal:** Apply CIFAR-10 optimal λ directly to inkjet.
+
+**Config:**
+```
+sep_loss_weight:  0.02
+batch_size:       64  (3 forward passes per step: main + good + bad)
+epochs:           100
+```
+
+**Results (K=100, re-evaluated):**
+```
+Overall  AUROC=0.8541  Acc=0.8008  FPR@95TPR=0.6609  N=266
+```
+
+*Note: First evaluation (K=50) showed 0.8581; variance ±0.005 at K=50.*
+
+**Output files:**
+- `results/inkjet_lambda0.02/cdm_best.pt`
+- `results/inkjet_lambda0.02_k100/scores.csv`
+- `results/inkjet_lambda0.02.log`
+
+---
+
+## Experiment 3 — Separation Loss λ=0.01
+
+**Goal:** Test CIFAR-10 sub-optimal value on inkjet (one step below λ=0.02).
+
+**Config:**
+```
+sep_loss_weight:  0.01
+batch_size:       64
+epochs:           100
+num_trials:       100
+```
+
+**Results (K=100, inline with training via --eval_after):**
+```
+Overall  AUROC=0.8603  Acc=0.8158  FPR@95TPR=0.6264  N=266
+```
+
+**→ Best AUROC of all λ values on inkjet.**
+
+**Output files:**
+- `results/inkjet_lambda0.01/cdm_best.pt`
+- `results/inkjet_lambda0.01/scores_final.csv`
+- `results/inkjet_lambda0.01.log`
+
+---
+
+## Experiment 4 — Separation Loss λ=0.05
+
+**Goal:** Test CIFAR-10 upper boundary of optimal zone on inkjet.
+
+**Config:**
+```
+sep_loss_weight:  0.05
+batch_size:       64
+epochs:           100
+num_trials:       100
+```
+
+**Results (K=100):**
+```
+Overall  AUROC=0.8553  Acc=0.8233  FPR@95TPR=0.5287  N=266
+```
+
+**→ Best FPR@95TPR of all λ values. Best accuracy.**
+
+**Output files:**
+- `results/inkjet_lambda0.05/cdm_best.pt`
+- `results/inkjet_lambda0.05/scores_final.csv`
+- `results/inkjet_lambda0.05.log`
+
+---
+
+## Experiment 5 — 5-Fold CV Ablation (Phase 2, FINAL)
+
+**Date:** March 1-2, 2026
+**Goal:** Rigorous evaluation with confidence intervals. All four λ values evaluated under identical conditions (batch=64 for fair comparison).
+
+**Config (all runs):**
+```
+batch_size:       64 (all runs — eliminates confound)
+epochs:           100 per fold
+seed:             42
+num_trials:       100
+n_folds:          5 (stratified on label)
+Total GPU time:   ~24h
+```
+
+**Results:**
+
+| λ | AUROC (5-fold CV) | Accuracy | FPR@95TPR |
+|---|:---:|:---:|:---:|
+| **0.0** | **0.8673 ± 0.0230** | **0.8094 ± 0.0151** | 0.5631 ± 0.1697 |
+| 0.01 | 0.8628 ± 0.0286 | 0.7928 ± 0.0291 | **0.5516 ± 0.1841** |
+| 0.02 | 0.8510 ± 0.0326 | 0.8003 ± 0.0246 | 0.6240 ± 0.1334 |
+| 0.05 | 0.8670 ± 0.0256 | 0.8071 ± 0.0241 | 0.5700 ± 0.1948 |
+
+**→ Separation loss does NOT improve over baseline on inkjet. All λ values within std.**
+
+**Output files:**
+- `results/cv_lambda0.0/cv_summary.json` (+ per-fold scores)
+- `results/cv_lambda0.01/cv_summary.json`
+- `results/cv_lambda0.02/cv_summary.json`
+- `results/cv_lambda0.05/cv_summary.json`
+
+---
+
+## Key Decisions & Rationale
+
+| Decision | Choice | Rationale |
+|----------|--------|-----------|
+| 5-fold CV for final results | 5-fold stratified | Single-split results unreliable; CV provides confidence intervals |
+| batch=64 for ALL runs | 64 | Eliminates batch-size confound; sep loss needs 3 forward passes |
+| λ values tested | 0.0, 0.01, 0.02, 0.05 | Covers CIFAR-10 optimal zone; more points not justified |
+| K=100 for final eval | K=100 | Stable estimates; CIFAR-10 K-ablation showed diminishing returns after K=25 |
+| seed=42 | 42 | Matches CIFAR-10 primary seed |
+
+---
+
+## Known Issues
+
+1. **`angle` feature**: Only 2-4 BAD samples per fold. AUROC highly variable (0.54–0.95). Report with caveat.
+2. **MC variance**: ±0.003 at K=100. Negligible compared to cross-fold variance (±0.025).
+3. **Separation loss ineffective**: The primary thesis finding for inkjet. See cross-domain comparison in RESULTS.md.
+
+---
+
+## Hardware
+
+- **GPU:** CUDA device 3 (Quadro GV100, 32 GB VRAM)
+- **Environment:** `/system/apps/studentenv/mohammed/sdm/`
+- **Training time per fold:** ~70 min (100 epochs, batch=64, λ>0)
+- **Evaluation time per fold (K=100):** ~45 seconds
+- **Total 5-fold CV ablation time:** ~24h (4 λ values × 5 folds)

docs/FIGURES.md

@@ -0,0 +1,77 @@
+# Inkjet CDM — Thesis Figure Index
+
+All figures are in: `results/figures/`
+Generated by: `scripts/plot_inkjet_results.py`
+
+---
+
+## Figure 1 — λ Ablation Curve
+**File:** `results/figures/fig1_inkjet_lambda_ablation.png`
+
+**Caption (thesis):**
+> *Separation loss weight ablation on the inkjet QC dataset. AUROC is reported across four values of λ (K=100 Monte Carlo trials). All values in the optimal zone [0.01–0.05] consistently outperform the baseline (λ=0, AUROC=0.833), confirming that the optimal zone identified on CIFAR-10 transfers directly to the industrial inkjet domain without domain-specific tuning.*
+
+**Where to use:** Chapter 5 (Inkjet Application), Section "Ablation: Separation Loss Weight"
+
+---
+
+## Figure 2 — Per-Feature AUROC Bar Chart
+**File:** `results/figures/fig2_per_feature_auroc.png`
+
+**Caption (thesis):**
+> *Per-feature AUROC comparison of the CDM baseline (λ=0) and the proposed model (λ=0.01). Green Δ labels indicate improvements of ≥0.02 AUROC. The largest gains occur on `edge3` (+0.156) and `edge4` (+0.069), both high-frequency texture features sensitive to printing artefacts.*
+
+**Where to use:** Chapter 5, Section "Per-Feature Analysis"
+
+---
+
+## Figure 3 — FPR@95TPR Bar Chart
+**File:** `results/figures/fig3_fpr_comparison.png`
+
+**Caption (thesis):**
+> *Per-feature FPR at 95% TPR for the baseline and proposed model. Lower is better. The separation loss reduces false positive rates on `dots` (−15.4pp), `edge3` (−12.5pp), `edge4` (−12.5pp), and `edge2` (−11.1pp), while slightly increasing FPR on `dist1` — a feature that is already near-perfect at baseline (AUROC=0.90) and does not benefit from additional separation signal.*
+
+**Where to use:** Chapter 5, Section "Per-Feature Analysis" or "Operational Impact"
+
+---
+
+## Figure 4 — Overall ROC Curves
+**File:** `results/figures/fig4_roc_curves.png`
+
+**Caption (thesis):**
+> *ROC curves for the CDM baseline (λ=0, AUROC=0.833) and the proposed model (λ=0.01, AUROC=0.860) on the inkjet test set (N=266, K=100 MC trials). The dotted horizontal line marks TPR=0.95, the operating point for computing FPR@95TPR. The proposed model's curve lies consistently above the baseline, indicating improved detection across all thresholds.*
+
+**Where to use:** Chapter 5, Section "Overall Results" — typically alongside or below the main results table.
+
+---
+
+## Figure 5 — Score Distributions (GOOD vs BAD)
+**File:** `results/figures/fig5_score_distributions.png`
+
+**Caption (thesis):**
+> *OOD score distributions for GOOD (blue) and BAD (orange) samples. Left: baseline (λ=0). Right: proposed (λ=0.01). Score > 0 indicates predicted BAD. Both distributions are tightly clustered near zero due to the small inkjet dataset and the per-crop image representation. The separation loss shifts the BAD distribution slightly rightward and reduces overlap with the GOOD distribution, consistent with the AUROC improvement.*
+
+**Where to use:** Chapter 5 appendix or as supporting evidence for the score-based classification mechanism.
+
+---
+
+## CIFAR-10 Figure (Chapter 4 Reference)
+**File:** `/system/user/studentwork/mohammed/2025/diffusion_classifier_ood/results/figures/separation_loss_ablation_final.png`
+
+**Caption (thesis):**
+> *Separation loss weight ablation study on CIFAR-10 (binary: airplane vs. all other classes). AUROC peaks at λ=0.02 (0.9911), establishing the optimal zone λ∈[0.01, 0.05]. This figure is the primary ablation result for Chapter 4; the inkjet ablation (Figure X above) confirms cross-domain generalisability of the optimal zone.*
+
+**Where to use:** Chapter 4 (CIFAR-10 Experiments), Section "Separation Loss Ablation"
+
+---
+
+## Figure Generation
+
+To regenerate all inkjet figures from scratch:
+```bash
+cd /system/user/studentwork/mohammed/2025/inkjet_qc/thesis_cdm_final
+conda activate /system/apps/studentenv/mohammed/sdm/
+python scripts/plot_inkjet_results.py
+```
+
+To update a single figure, edit `scripts/plot_inkjet_results.py` and call the relevant function.

docs/RESULTS.md

@@ -0,0 +1,99 @@
+# Inkjet CDM — Final Thesis Results
+
+> All results use K=100 MC trials (Algorithm 1, difference scoring).
+> Dataset: 1,327 total samples → 20% test split = **266 samples** (GOOD=174, BAD=92).
+> Evaluation seed: 42 (same train/test split for all λ values).
+> Last verified: 2026-02-26
+
+---
+
+## 1. Overall Comparison — All λ Values
+
+| λ | AUROC ↑ | Accuracy ↑ | FPR@95TPR ↓ | Δ AUROC vs baseline |
+|---|:---:|:---:|:---:|:---:|
+| 0.0 (baseline, no sep loss) | 0.8325 | 0.7895 | 0.8161 | — |
+| **0.01 (best AUROC)** | **0.8603** | **0.8158** | 0.6264 | **+0.0278** |
+| 0.02 | 0.8541 | 0.8008 | 0.6609 | +0.0216 |
+| **0.05 (best FPR)** | 0.8553 | **0.8233** | **0.5287** | +0.0228 |
+
+**Recommended thesis citation:** λ=0.01 as primary result (best AUROC), mention λ=0.05 for best operational FPR.
+
+---
+
+## 2. Per-Feature AUROC (K=100)
+
+| Feature | λ=0 | λ=0.01 | λ=0.02 | λ=0.05 | Best Δ |
+|---------|:---:|:---:|:---:|:---:|:---:|
+| angle   | 0.5556 | 0.5679 | 0.6173 | 0.5556 | +0.0617 |
+| dist1   | 0.9000 | 0.8571 | 0.9429 | 0.9143 | — |
+| dist6   | 0.8278 | 0.8111 | 0.8389 | 0.8278 | — |
+| dots    | 0.9126 | **0.9266** | 0.9126 | 0.8881 | +0.0140 |
+| edge1   | 0.7760 | 0.8177 | **0.8594** | 0.8542 | **+0.0834** |
+| edge2   | 0.7302 | 0.7242 | 0.6786 | **0.7857** | — |
+| edge3   | 0.7188 | **0.8750** | 0.7708 | 0.8229 | **+0.1562** |
+| edge4   | 0.6667 | **0.7361** | 0.6806 | 0.6597 | **+0.0694** |
+| **Overall** | **0.8325** | **0.8603** | **0.8541** | **0.8553** | **+0.0278** |
+
+> Note: `angle` has only 3 BAD samples in the test set (27 GOOD, 3 BAD) — AUROC is statistically unreliable for this feature. `edge3` shows the largest absolute gain (+15.62pp at λ=0.01).
+
+---
+
+## 3. Per-Feature FPR@95TPR (K=100) — lower is better
+
+| Feature | λ=0 | λ=0.01 | Δ |
+|---------|:---:|:---:|:---:|
+| angle   | 0.9630 | 0.9630 | 0.000 |
+| dist1   | 0.2381 | 0.4286 | +0.190 (regression) |
+| dist6   | 0.9667 | 0.9333 | **−0.033** |
+| dots    | 0.9615 | 0.8077 | **−0.154** |
+| edge1   | 0.9167 | 0.9167 | 0.000 |
+| edge2   | 0.8889 | 0.7778 | **−0.111** |
+| edge3   | 0.5625 | 0.4375 | **−0.125** |
+| edge4   | 0.5833 | 0.4583 | **−0.125** |
+| **Overall** | **0.8161** | **0.6264** | **−0.190** |
+
+> Overall FPR@95TPR drops by **19pp** at λ=0.01 — at 95% defect detection sensitivity, 19% fewer good products are falsely rejected.
+
+---
+
+## 4. Per-Template Breakdown (λ=0.01)
+
+| Template | AUROC | Acc | FPR@95TPR | N |
+|----------|:---:|:---:|:---:|:---:|
+| A | 0.7981 | 0.7048 | 0.6863 | 105 |
+| B | **0.8750** | 0.8214 | 0.4375 | 28 |
+| C | 0.8325 | 0.9023 | 0.9533 | 133 |
+
+---
+
+## 5. Stochastic Variance (MC Scoring)
+
+The OOD score is stochastic (K random timestep samples per image). Across 3 evaluations of the λ=0.02 checkpoint:
+
+| Eval run | K | AUROC |
+|----------|---|-------|
+| Run 1 | 50 | 0.8581 |
+| Run 2 | 50 | 0.8474 |
+| Run 3 | 100 | 0.8541 |
+
+**Variance ≈ ±0.005** at K=50, **±0.003** at K=100. All final results use K=100.
+
+---
+
+## 6. Training Configuration
+
+```
+model:           CDM UNet (multi-head conditioning)
+                 template + feature + quality + bbox heads
+dataset:         1,327 inkjet print samples (174 GOOD, 92 BAD → test set)
+train/test:      80/20 stratified split (seed=42)
+oversampling:    BAD samples × 3.0 (minority oversampling during training only)
+image_size:      crop-based (YOLO bbox region)
+epochs:          100
+schedule:        cosine (Nichol & Dhariwal 2021)
+optimizer:       AdamW, lr=1e-4
+sep_loss:        L = L_MSE + λ · L_sep
+scoring:         Algorithm 1 (difference method), K=100 MC trials
+batch_size:      64 (sep loss runs) / 128 (baseline λ=0)
+GPU:             CUDA device 3 (32 GB VRAM)
+```

missing_checkpoints.md

@@ -0,0 +1,5 @@
+# Missing Inkjet Checkpoints
+
+The final thesis package states that final 5-fold CV `.pt` checkpoint files were deleted to free disk space. The auditable per-fold `scores_fold*.csv`, `metrics_fold*.json`, and `cv_summary.json` files are preserved here.
+
+Available local model weights with clear curated-package provenance are copied under `models/`.

results/cv_lambda_ablation/cv_lambda0.0/cv_summary.json

@@ -0,0 +1,387 @@
+{
+  "n_folds": 5,
+  "mean_auroc": 0.8672848816651193,
+  "std_auroc": 0.023037826231008213,
+  "mean_acc": 0.809365867498936,
+  "std_acc": 0.015099091855035324,
+  "mean_fpr95": 0.5630855092684871,
+  "std_fpr95": 0.16969236725665754,
+  "fold_aurocs": [
+    0.8705022488755623,
+    0.8516366816591704,
+    0.8341920080422217,
+    0.9016041429834533,
+    0.8784893267651889
+  ],
+  "fold_accs": [
+    0.793233082706767,
+    0.7969924812030075,
+    0.8037735849056604,
+    0.8339622641509434,
+    0.8188679245283019
+  ],
+  "fold_fpr95s": [
+    0.4540229885057471,
+    0.5344827586206896,
+    0.884393063583815,
+    0.39655172413793105,
+    0.5459770114942529
+  ],
+  "sep_loss_weight": 0.0,
+  "per_feature": {
+    "angle": {
+      "mean": 0.8166153530377669,
+      "std": 0.1381097848343882
+    },
+    "dist1": {
+      "mean": 0.8865807269653423,
+      "std": 0.07294165887097198
+    },
+    "dist6": {
+      "mean": 0.9361776567856422,
+      "std": 0.0665847288956416
+    },
+    "dots": {
+      "mean": 0.955718168470726,
+      "std": 0.03530116667630583
+    },
+    "edge1": {
+      "mean": 0.7957978279030911,
+      "std": 0.13760051702537338
+    },
+    "edge2": {
+      "mean": 0.812891382868937,
+      "std": 0.030717617655154305
+    },
+    "edge3": {
+      "mean": 0.7440582092486119,
+      "std": 0.07570580644004583
+    },
+    "edge4": {
+      "mean": 0.7617353760894432,
+      "std": 0.09850442277082597
+    }
+  },
+  "fold_details": [
+    {
+      "fold": 1,
+      "overall_auroc": 0.8705022488755623,
+      "overall_acc": 0.793233082706767,
+      "overall_fpr95": 0.4540229885057471,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.96875,
+          "accuracy": 0.9411764705882353,
+          "fpr95": 0.0625,
+          "n_good": 32,
+          "n_bad": 2
+        },
+        "dist1": {
+          "auroc": 0.9545454545454546,
+          "accuracy": 0.8484848484848485,
+          "fpr95": 0.13636363636363635,
+          "n_good": 22,
+          "n_bad": 11
+        },
+        "dist6": {
+          "auroc": 1.0,
+          "accuracy": 1.0,
+          "fpr95": 0.0,
+          "n_good": 26,
+          "n_bad": 2
+        },
+        "dots": {
+          "auroc": 0.9953703703703703,
+          "accuracy": 0.9,
+          "fpr95": 0.05555555555555555,
+          "n_good": 18,
+          "n_bad": 12
+        },
+        "edge1": {
+          "auroc": 0.9,
+          "accuracy": 0.7619047619047619,
+          "fpr95": 0.2727272727272727,
+          "n_good": 11,
+          "n_bad": 10
+        },
+        "edge2": {
+          "auroc": 0.8125,
+          "accuracy": 0.7058823529411765,
+          "fpr95": 0.5,
+          "n_good": 18,
+          "n_bad": 16
+        },
+        "edge3": {
+          "auroc": 0.7225,
+          "accuracy": 0.65,
+          "fpr95": 0.65,
+          "n_good": 20,
+          "n_bad": 20
+        },
+        "edge4": {
+          "auroc": 0.6491228070175439,
+          "accuracy": 0.6521739130434783,
+          "fpr95": 0.9629629629629629,
+          "n_good": 27,
+          "n_bad": 19
+        }
+      }
+    },
+    {
+      "fold": 2,
+      "overall_auroc": 0.8516366816591704,
+      "overall_acc": 0.7969924812030075,
+      "overall_fpr95": 0.5344827586206896,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.6041666666666666,
+          "accuracy": 0.875,
+          "fpr95": 0.8611111111111112,
+          "n_good": 36,
+          "n_bad": 4
+        },
+        "dist1": {
+          "auroc": 0.8454545454545455,
+          "accuracy": 0.8888888888888888,
+          "fpr95": 0.3181818181818182,
+          "n_good": 22,
+          "n_bad": 5
+        },
+        "dist6": {
+          "auroc": 0.9415204678362572,
+          "accuracy": 0.7857142857142857,
+          "fpr95": 0.15789473684210525,
+          "n_good": 19,
+          "n_bad": 9
+        },
+        "dots": {
+          "auroc": 0.9360613810741689,
+          "accuracy": 0.9,
+          "fpr95": 0.5217391304347826,
+          "n_good": 23,
+          "n_bad": 17
+        },
+        "edge1": {
+          "auroc": 0.8222222222222222,
+          "accuracy": 0.7575757575757576,
+          "fpr95": 0.9333333333333333,
+          "n_good": 15,
+          "n_bad": 18
+        },
+        "edge2": {
+          "auroc": 0.8229166666666667,
+          "accuracy": 0.7352941176470589,
+          "fpr95": 0.6111111111111112,
+          "n_good": 18,
+          "n_bad": 16
+        },
+        "edge3": {
+          "auroc": 0.7436974789915967,
+          "accuracy": 0.6129032258064516,
+          "fpr95": 0.8235294117647058,
+          "n_good": 17,
+          "n_bad": 14
+        },
+        "edge4": {
+          "auroc": 0.7592592592592593,
+          "accuracy": 0.7878787878787878,
+          "fpr95": 0.75,
+          "n_good": 24,
+          "n_bad": 9
+        }
+      }
+    },
+    {
+      "fold": 3,
+      "overall_auroc": 0.8341920080422217,
+      "overall_acc": 0.8037735849056604,
+      "overall_fpr95": 0.884393063583815,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.7946428571428572,
+          "accuracy": 0.90625,
+          "fpr95": 0.7857142857142857,
+          "n_good": 28,
+          "n_bad": 4
+        },
+        "dist1": {
+          "auroc": 0.7721893491124261,
+          "accuracy": 0.6923076923076923,
+          "fpr95": 0.5769230769230769,
+          "n_good": 26,
+          "n_bad": 13
+        },
+        "dist6": {
+          "auroc": 0.8083333333333333,
+          "accuracy": 0.9655172413793104,
+          "fpr95": 0.9583333333333334,
+          "n_good": 24,
+          "n_bad": 5
+        },
+        "dots": {
+          "auroc": 0.9125,
+          "accuracy": 0.9411764705882353,
+          "fpr95": 0.5833333333333334,
+          "n_good": 24,
+          "n_bad": 10
+        },
+        "edge1": {
+          "auroc": 0.5321637426900585,
+          "accuracy": 0.4864864864864865,
+          "fpr95": 0.8421052631578947,
+          "n_good": 19,
+          "n_bad": 18
+        },
+        "edge2": {
+          "auroc": 0.763157894736842,
+          "accuracy": 0.7931034482758621,
+          "fpr95": 1.0,
+          "n_good": 10,
+          "n_bad": 19
+        },
+        "edge3": {
+          "auroc": 0.8875000000000001,
+          "accuracy": 0.8125,
+          "fpr95": 0.5833333333333334,
+          "n_good": 12,
+          "n_bad": 20
+        },
+        "edge4": {
+          "auroc": 0.9444444444444444,
+          "accuracy": 0.9090909090909091,
+          "fpr95": 0.16666666666666666,
+          "n_good": 30,
+          "n_bad": 3
+        }
+      }
+    },
+    {
+      "fold": 4,
+      "overall_auroc": 0.9016041429834533,
+      "overall_acc": 0.8339622641509434,
+      "overall_fpr95": 0.39655172413793105,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.9655172413793104,
+          "accuracy": 0.96875,
+          "fpr95": 0.10344827586206896,
+          "n_good": 29,
+          "n_bad": 3
+        },
+        "dist1": {
+          "auroc": 0.9714285714285714,
+          "accuracy": 0.8709677419354839,
+          "fpr95": 0.09523809523809523,
+          "n_good": 21,
+          "n_bad": 10
+        },
+        "dist6": {
+          "auroc": 0.9655172413793104,
+          "accuracy": 0.9117647058823529,
+          "fpr95": 0.1724137931034483,
+          "n_good": 29,
+          "n_bad": 5
+        },
+        "dots": {
+          "auroc": 0.9346590909090909,
+          "accuracy": 0.9302325581395349,
+          "fpr95": 0.71875,
+          "n_good": 32,
+          "n_bad": 11
+        },
+        "edge1": {
+          "auroc": 0.9111111111111111,
+          "accuracy": 0.8611111111111112,
+          "fpr95": 0.38095238095238093,
+          "n_good": 21,
+          "n_bad": 15
+        },
+        "edge2": {
+          "auroc": 0.8588235294117648,
+          "accuracy": 0.75,
+          "fpr95": 0.8,
+          "n_good": 15,
+          "n_bad": 17
+        },
+        "edge3": {
+          "auroc": 0.694078947368421,
+          "accuracy": 0.6857142857142857,
+          "fpr95": 0.9375,
+          "n_good": 16,
+          "n_bad": 19
+        },
+        "edge4": {
+          "auroc": 0.7190082644628099,
+          "accuracy": 0.5909090909090909,
+          "fpr95": 0.8181818181818182,
+          "n_good": 11,
+          "n_bad": 11
+        }
+      }
+    },
+    {
+      "fold": 5,
+      "overall_auroc": 0.8784893267651889,
+      "overall_acc": 0.8188679245283019,
+      "overall_fpr95": 0.5459770114942529,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.75,
+          "accuracy": 0.9545454545454546,
+          "fpr95": 0.5,
+          "n_good": 20,
+          "n_bad": 2
+        },
+        "dist1": {
+          "auroc": 0.8892857142857142,
+          "accuracy": 0.813953488372093,
+          "fpr95": 0.34285714285714286,
+          "n_good": 35,
+          "n_bad": 8
+        },
+        "dist6": {
+          "auroc": 0.9655172413793104,
+          "accuracy": 0.9117647058823529,
+          "fpr95": 0.1724137931034483,
+          "n_good": 29,
+          "n_bad": 5
+        },
+        "dots": {
+          "auroc": 1.0,
+          "accuracy": 1.0,
+          "fpr95": 0.0,
+          "n_good": 17,
+          "n_bad": 11
+        },
+        "edge1": {
+          "auroc": 0.8134920634920635,
+          "accuracy": 0.71875,
+          "fpr95": 0.6111111111111112,
+          "n_good": 18,
+          "n_bad": 14
+        },
+        "edge2": {
+          "auroc": 0.8070588235294118,
+          "accuracy": 0.7857142857142857,
+          "fpr95": 0.8235294117647058,
+          "n_good": 17,
+          "n_bad": 25
+        },
+        "edge3": {
+          "auroc": 0.672514619883041,
+          "accuracy": 0.6216216216216216,
+          "fpr95": 0.8947368421052632,
+          "n_good": 19,
+          "n_bad": 18
+        },
+        "edge4": {
+          "auroc": 0.7368421052631579,
+          "accuracy": 0.8518518518518519,
+          "fpr95": 0.8947368421052632,
+          "n_good": 19,
+          "n_bad": 8
+        }
+      }
+    }
+  ]
+}

results/cv_lambda_ablation/cv_lambda0.01/cv_summary.json

@@ -0,0 +1,387 @@
+{
+  "n_folds": 5,
+  "mean_auroc": 0.8627759763959727,
+  "std_auroc": 0.028624410151127697,
+  "mean_acc": 0.79279046673287,
+  "std_acc": 0.029092819923175913,
+  "mean_fpr95": 0.551617832702146,
+  "std_fpr95": 0.18413309927308624,
+  "fold_aurocs": [
+    0.8819965017491255,
+    0.8413293353323338,
+    0.8191128424227193,
+    0.8981306050271567,
+    0.8733105974485285
+  ],
+  "fold_accs": [
+    0.8045112781954887,
+    0.7481203007518797,
+    0.769811320754717,
+    0.8188679245283019,
+    0.8226415094339623
+  ],
+  "fold_fpr95s": [
+    0.4367816091954023,
+    0.5517241379310345,
+    0.9075144508670521,
+    0.4482758620689655,
+    0.41379310344827586
+  ],
+  "sep_loss_weight": 0.01,
+  "per_feature": {
+    "angle": {
+      "mean": 0.7726669403393542,
+      "std": 0.15803829769963884
+    },
+    "dist1": {
+      "mean": 0.8507593571929236,
+      "std": 0.07681005302994184
+    },
+    "dist6": {
+      "mean": 0.9422786852187942,
+      "std": 0.0737298334613098
+    },
+    "dots": {
+      "mean": 0.9655125207746691,
+      "std": 0.02466213593918774
+    },
+    "edge1": {
+      "mean": 0.8409827599301284,
+      "std": 0.13790375034456145
+    },
+    "edge2": {
+      "mean": 0.8426224630202958,
+      "std": 0.044292034640384156
+    },
+    "edge3": {
+      "mean": 0.7007929627991547,
+      "std": 0.03895889171971354
+    },
+    "edge4": {
+      "mean": 0.7528969117007395,
+      "std": 0.054273160353044664
+    }
+  },
+  "fold_details": [
+    {
+      "fold": 1,
+      "overall_auroc": 0.8819965017491255,
+      "overall_acc": 0.8045112781954887,
+      "overall_fpr95": 0.4367816091954023,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.9375,
+          "accuracy": 0.9117647058823529,
+          "fpr95": 0.09375,
+          "n_good": 32,
+          "n_bad": 2
+        },
+        "dist1": {
+          "auroc": 0.9710743801652892,
+          "accuracy": 0.8484848484848485,
+          "fpr95": 0.3181818181818182,
+          "n_good": 22,
+          "n_bad": 11
+        },
+        "dist6": {
+          "auroc": 1.0,
+          "accuracy": 0.9285714285714286,
+          "fpr95": 0.0,
+          "n_good": 26,
+          "n_bad": 2
+        },
+        "dots": {
+          "auroc": 0.9907407407407407,
+          "accuracy": 0.8666666666666667,
+          "fpr95": 0.05555555555555555,
+          "n_good": 18,
+          "n_bad": 12
+        },
+        "edge1": {
+          "auroc": 0.9090909090909091,
+          "accuracy": 0.7619047619047619,
+          "fpr95": 0.5454545454545454,
+          "n_good": 11,
+          "n_bad": 10
+        },
+        "edge2": {
+          "auroc": 0.8923611111111112,
+          "accuracy": 0.8235294117647058,
+          "fpr95": 0.3888888888888889,
+          "n_good": 18,
+          "n_bad": 16
+        },
+        "edge3": {
+          "auroc": 0.7575,
+          "accuracy": 0.725,
+          "fpr95": 0.6,
+          "n_good": 20,
+          "n_bad": 20
+        },
+        "edge4": {
+          "auroc": 0.7056530214424952,
+          "accuracy": 0.6521739130434783,
+          "fpr95": 0.7777777777777778,
+          "n_good": 27,
+          "n_bad": 19
+        }
+      }
+    },
+    {
+      "fold": 2,
+      "overall_auroc": 0.8413293353323338,
+      "overall_acc": 0.7481203007518797,
+      "overall_fpr95": 0.5517241379310345,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.576388888888889,
+          "accuracy": 0.9,
+          "fpr95": 0.9166666666666666,
+          "n_good": 36,
+          "n_bad": 4
+        },
+        "dist1": {
+          "auroc": 0.8545454545454545,
+          "accuracy": 0.7777777777777778,
+          "fpr95": 0.36363636363636365,
+          "n_good": 22,
+          "n_bad": 5
+        },
+        "dist6": {
+          "auroc": 0.9941520467836258,
+          "accuracy": 0.7857142857142857,
+          "fpr95": 0.05263157894736842,
+          "n_good": 19,
+          "n_bad": 9
+        },
+        "dots": {
+          "auroc": 0.9462915601023018,
+          "accuracy": 0.8,
+          "fpr95": 0.5652173913043478,
+          "n_good": 23,
+          "n_bad": 17
+        },
+        "edge1": {
+          "auroc": 0.9000000000000001,
+          "accuracy": 0.6666666666666666,
+          "fpr95": 0.2,
+          "n_good": 15,
+          "n_bad": 18
+        },
+        "edge2": {
+          "auroc": 0.8090277777777778,
+          "accuracy": 0.7647058823529411,
+          "fpr95": 0.6666666666666666,
+          "n_good": 18,
+          "n_bad": 16
+        },
+        "edge3": {
+          "auroc": 0.6722689075630252,
+          "accuracy": 0.5161290322580645,
+          "fpr95": 0.8823529411764706,
+          "n_good": 17,
+          "n_bad": 14
+        },
+        "edge4": {
+          "auroc": 0.6898148148148149,
+          "accuracy": 0.7272727272727273,
+          "fpr95": 0.7916666666666666,
+          "n_good": 24,
+          "n_bad": 9
+        }
+      }
+    },
+    {
+      "fold": 3,
+      "overall_auroc": 0.8191128424227193,
+      "overall_acc": 0.769811320754717,
+      "overall_fpr95": 0.9075144508670521,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.7589285714285714,
+          "accuracy": 0.90625,
+          "fpr95": 0.8928571428571429,
+          "n_good": 28,
+          "n_bad": 4
+        },
+        "dist1": {
+          "auroc": 0.7662721893491125,
+          "accuracy": 0.7435897435897436,
+          "fpr95": 0.7692307692307693,
+          "n_good": 26,
+          "n_bad": 13
+        },
+        "dist6": {
+          "auroc": 0.8,
+          "accuracy": 0.9655172413793104,
+          "fpr95": 1.0,
+          "n_good": 24,
+          "n_bad": 5
+        },
+        "dots": {
+          "auroc": 0.9416666666666668,
+          "accuracy": 0.8823529411764706,
+          "fpr95": 0.4583333333333333,
+          "n_good": 24,
+          "n_bad": 10
+        },
+        "edge1": {
+          "auroc": 0.5672514619883041,
+          "accuracy": 0.5135135135135135,
+          "fpr95": 0.8947368421052632,
+          "n_good": 19,
+          "n_bad": 18
+        },
+        "edge2": {
+          "auroc": 0.7736842105263158,
+          "accuracy": 0.7586206896551724,
+          "fpr95": 1.0,
+          "n_good": 10,
+          "n_bad": 19
+        },
+        "edge3": {
+          "auroc": 0.6458333333333334,
+          "accuracy": 0.53125,
+          "fpr95": 0.8333333333333334,
+          "n_good": 12,
+          "n_bad": 20
+        },
+        "edge4": {
+          "auroc": 0.8111111111111111,
+          "accuracy": 0.9090909090909091,
+          "fpr95": 0.4666666666666667,
+          "n_good": 30,
+          "n_bad": 3
+        }
+      }
+    },
+    {
+      "fold": 4,
+      "overall_auroc": 0.8981306050271567,
+      "overall_acc": 0.8188679245283019,
+      "overall_fpr95": 0.4482758620689655,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.9655172413793103,
+          "accuracy": 0.9375,
+          "fpr95": 0.06896551724137931,
+          "n_good": 29,
+          "n_bad": 3
+        },
+        "dist1": {
+          "auroc": 0.8904761904761905,
+          "accuracy": 0.7419354838709677,
+          "fpr95": 0.3333333333333333,
+          "n_good": 21,
+          "n_bad": 10
+        },
+        "dist6": {
+          "auroc": 0.9724137931034482,
+          "accuracy": 0.9411764705882353,
+          "fpr95": 0.13793103448275862,
+          "n_good": 29,
+          "n_bad": 5
+        },
+        "dots": {
+          "auroc": 0.9488636363636364,
+          "accuracy": 0.9302325581395349,
+          "fpr95": 0.5625,
+          "n_good": 32,
+          "n_bad": 11
+        },
+        "edge1": {
+          "auroc": 0.9396825396825397,
+          "accuracy": 0.8611111111111112,
+          "fpr95": 0.19047619047619047,
+          "n_good": 21,
+          "n_bad": 15
+        },
+        "edge2": {
+          "auroc": 0.8745098039215686,
+          "accuracy": 0.78125,
+          "fpr95": 0.8666666666666667,
+          "n_good": 15,
+          "n_bad": 17
+        },
+        "edge3": {
+          "auroc": 0.7236842105263158,
+          "accuracy": 0.6285714285714286,
+          "fpr95": 0.9375,
+          "n_good": 16,
+          "n_bad": 19
+        },
+        "edge4": {
+          "auroc": 0.7355371900826446,
+          "accuracy": 0.6363636363636364,
+          "fpr95": 0.7272727272727273,
+          "n_good": 11,
+          "n_bad": 11
+        }
+      }
+    },
+    {
+      "fold": 5,
+      "overall_auroc": 0.8733105974485285,
+      "overall_acc": 0.8226415094339623,
+      "overall_fpr95": 0.41379310344827586,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.625,
+          "accuracy": 0.9545454545454546,
+          "fpr95": 0.75,
+          "n_good": 20,
+          "n_bad": 2
+        },
+        "dist1": {
+          "auroc": 0.7714285714285715,
+          "accuracy": 0.6744186046511628,
+          "fpr95": 0.37142857142857144,
+          "n_good": 35,
+          "n_bad": 8
+        },
+        "dist6": {
+          "auroc": 0.9448275862068966,
+          "accuracy": 0.9411764705882353,
+          "fpr95": 0.27586206896551724,
+          "n_good": 29,
+          "n_bad": 5
+        },
+        "dots": {
+          "auroc": 1.0,
+          "accuracy": 1.0,
+          "fpr95": 0.0,
+          "n_good": 17,
+          "n_bad": 11
+        },
+        "edge1": {
+          "auroc": 0.8888888888888888,
+          "accuracy": 0.78125,
+          "fpr95": 0.4444444444444444,
+          "n_good": 18,
+          "n_bad": 14
+        },
+        "edge2": {
+          "auroc": 0.8635294117647059,
+          "accuracy": 0.8095238095238095,
+          "fpr95": 0.4117647058823529,
+          "n_good": 17,
+          "n_bad": 25
+        },
+        "edge3": {
+          "auroc": 0.7046783625730995,
+          "accuracy": 0.6756756756756757,
+          "fpr95": 0.7368421052631579,
+          "n_good": 19,
+          "n_bad": 18
+        },
+        "edge4": {
+          "auroc": 0.8223684210526315,
+          "accuracy": 0.8888888888888888,
+          "fpr95": 0.6842105263157895,
+          "n_good": 19,
+          "n_bad": 8
+        }
+      }
+    }
+  ]
+}

results/cv_lambda_ablation/cv_lambda0.02/cv_summary.json

@@ -0,0 +1,387 @@
+{
+  "n_folds": 5,
+  "mean_auroc": 0.8510010295303342,
+  "std_auroc": 0.03255581041902637,
+  "mean_acc": 0.800312101007235,
+  "std_acc": 0.024640175188649346,
+  "mean_fpr95": 0.6240116935751778,
+  "std_fpr95": 0.13342787329034592,
+  "fold_aurocs": [
+    0.8885557221389306,
+    0.8181534232883558,
+    0.808306107062076,
+    0.8815207780725023,
+    0.8584691170898068
+  ],
+  "fold_accs": [
+    0.8270676691729323,
+    0.7593984962406015,
+    0.7886792452830189,
+    0.8226415094339623,
+    0.8037735849056604
+  ],
+  "fold_fpr95s": [
+    0.5632183908045977,
+    0.5689655172413793,
+    0.8901734104046243,
+    0.5574712643678161,
+    0.5402298850574713
+  ],
+  "sep_loss_weight": 0.02,
+  "per_feature": {
+    "angle": {
+      "mean": 0.7681503147235906,
+      "std": 0.18356394878016324
+    },
+    "dist1": {
+      "mean": 0.8274083725132677,
+      "std": 0.14568751189354393
+    },
+    "dist6": {
+      "mean": 0.9393103448275862,
+      "std": 0.07496650064386862
+    },
+    "dots": {
+      "mean": 0.9473853410490239,
+      "std": 0.04000806848419659
+    },
+    "edge1": {
+      "mean": 0.7822622212095897,
+      "std": 0.16072058322924615
+    },
+    "edge2": {
+      "mean": 0.7992906776745786,
+      "std": 0.06020908424821382
+    },
+    "edge3": {
+      "mean": 0.7774228709027471,
+      "std": 0.06470652954234209
+    },
+    "edge4": {
+      "mean": 0.7642603869637362,
+      "std": 0.08982855073417187
+    }
+  },
+  "fold_details": [
+    {
+      "fold": 1,
+      "overall_auroc": 0.8885557221389306,
+      "overall_acc": 0.8270676691729323,
+      "overall_fpr95": 0.5632183908045977,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.984375,
+          "accuracy": 0.9705882352941176,
+          "fpr95": 0.03125,
+          "n_good": 32,
+          "n_bad": 2
+        },
+        "dist1": {
+          "auroc": 0.9834710743801653,
+          "accuracy": 0.8787878787878788,
+          "fpr95": 0.045454545454545456,
+          "n_good": 22,
+          "n_bad": 11
+        },
+        "dist6": {
+          "auroc": 1.0,
+          "accuracy": 1.0,
+          "fpr95": 0.0,
+          "n_good": 26,
+          "n_bad": 2
+        },
+        "dots": {
+          "auroc": 0.9907407407407407,
+          "accuracy": 0.9333333333333333,
+          "fpr95": 0.05555555555555555,
+          "n_good": 18,
+          "n_bad": 12
+        },
+        "edge1": {
+          "auroc": 0.9636363636363636,
+          "accuracy": 0.8571428571428571,
+          "fpr95": 0.18181818181818182,
+          "n_good": 11,
+          "n_bad": 10
+        },
+        "edge2": {
+          "auroc": 0.8784722222222222,
+          "accuracy": 0.7352941176470589,
+          "fpr95": 0.16666666666666666,
+          "n_good": 18,
+          "n_bad": 16
+        },
+        "edge3": {
+          "auroc": 0.7225,
+          "accuracy": 0.725,
+          "fpr95": 0.7,
+          "n_good": 20,
+          "n_bad": 20
+        },
+        "edge4": {
+          "auroc": 0.7017543859649124,
+          "accuracy": 0.6521739130434783,
+          "fpr95": 0.8148148148148148,
+          "n_good": 27,
+          "n_bad": 19
+        }
+      }
+    },
+    {
+      "fold": 2,
+      "overall_auroc": 0.8181534232883558,
+      "overall_acc": 0.7593984962406015,
+      "overall_fpr95": 0.5689655172413793,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.5972222222222222,
+          "accuracy": 0.875,
+          "fpr95": 0.75,
+          "n_good": 36,
+          "n_bad": 4
+        },
+        "dist1": {
+          "auroc": 0.5818181818181818,
+          "accuracy": 0.6666666666666666,
+          "fpr95": 0.7727272727272727,
+          "n_good": 22,
+          "n_bad": 5
+        },
+        "dist6": {
+          "auroc": 1.0,
+          "accuracy": 0.8571428571428571,
+          "fpr95": 0.0,
+          "n_good": 19,
+          "n_bad": 9
+        },
+        "dots": {
+          "auroc": 0.907928388746803,
+          "accuracy": 0.7,
+          "fpr95": 0.6521739130434783,
+          "n_good": 23,
+          "n_bad": 17
+        },
+        "edge1": {
+          "auroc": 0.6962962962962964,
+          "accuracy": 0.696969696969697,
+          "fpr95": 0.9333333333333333,
+          "n_good": 15,
+          "n_bad": 18
+        },
+        "edge2": {
+          "auroc": 0.7604166666666666,
+          "accuracy": 0.7058823529411765,
+          "fpr95": 0.7777777777777778,
+          "n_good": 18,
+          "n_bad": 16
+        },
+        "edge3": {
+          "auroc": 0.8151260504201681,
+          "accuracy": 0.7741935483870968,
+          "fpr95": 0.7647058823529411,
+          "n_good": 17,
+          "n_bad": 14
+        },
+        "edge4": {
+          "auroc": 0.7453703703703703,
+          "accuracy": 0.7878787878787878,
+          "fpr95": 0.7916666666666666,
+          "n_good": 24,
+          "n_bad": 9
+        }
+      }
+    },
+    {
+      "fold": 3,
+      "overall_auroc": 0.808306107062076,
+      "overall_acc": 0.7886792452830189,
+      "overall_fpr95": 0.8901734104046243,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.7321428571428571,
+          "accuracy": 0.875,
+          "fpr95": 0.7857142857142857,
+          "n_good": 28,
+          "n_bad": 4
+        },
+        "dist1": {
+          "auroc": 0.7455621301775148,
+          "accuracy": 0.6923076923076923,
+          "fpr95": 0.8076923076923077,
+          "n_good": 26,
+          "n_bad": 13
+        },
+        "dist6": {
+          "auroc": 0.8,
+          "accuracy": 0.9310344827586207,
+          "fpr95": 1.0,
+          "n_good": 24,
+          "n_bad": 5
+        },
+        "dots": {
+          "auroc": 0.9291666666666667,
+          "accuracy": 0.9411764705882353,
+          "fpr95": 0.5,
+          "n_good": 24,
+          "n_bad": 10
+        },
+        "edge1": {
+          "auroc": 0.5116959064327485,
+          "accuracy": 0.5405405405405406,
+          "fpr95": 0.8947368421052632,
+          "n_good": 19,
+          "n_bad": 18
+        },
+        "edge2": {
+          "auroc": 0.7473684210526316,
+          "accuracy": 0.7241379310344828,
+          "fpr95": 1.0,
+          "n_good": 10,
+          "n_bad": 19
+        },
+        "edge3": {
+          "auroc": 0.8875,
+          "accuracy": 0.75,
+          "fpr95": 0.4166666666666667,
+          "n_good": 12,
+          "n_bad": 20
+        },
+        "edge4": {
+          "auroc": 0.9333333333333333,
+          "accuracy": 0.9090909090909091,
+          "fpr95": 0.1,
+          "n_good": 30,
+          "n_bad": 3
+        }
+      }
+    },
+    {
+      "fold": 4,
+      "overall_auroc": 0.8815207780725023,
+      "overall_acc": 0.8226415094339623,
+      "overall_fpr95": 0.5574712643678161,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.9770114942528736,
+          "accuracy": 0.96875,
+          "fpr95": 0.06896551724137931,
+          "n_good": 29,
+          "n_bad": 3
+        },
+        "dist1": {
+          "auroc": 0.9190476190476191,
+          "accuracy": 0.8709677419354839,
+          "fpr95": 0.5238095238095238,
+          "n_good": 21,
+          "n_bad": 10
+        },
+        "dist6": {
+          "auroc": 0.9724137931034482,
+          "accuracy": 0.9117647058823529,
+          "fpr95": 0.13793103448275862,
+          "n_good": 29,
+          "n_bad": 5
+        },
+        "dots": {
+          "auroc": 0.9090909090909091,
+          "accuracy": 0.9302325581395349,
+          "fpr95": 0.96875,
+          "n_good": 32,
+          "n_bad": 11
+        },
+        "edge1": {
+          "auroc": 0.8825396825396826,
+          "accuracy": 0.8055555555555556,
+          "fpr95": 0.3333333333333333,
+          "n_good": 21,
+          "n_bad": 15
+        },
+        "edge2": {
+          "auroc": 0.8666666666666667,
+          "accuracy": 0.71875,
+          "fpr95": 0.8666666666666667,
+          "n_good": 15,
+          "n_bad": 17
+        },
+        "edge3": {
+          "auroc": 0.736842105263158,
+          "accuracy": 0.6857142857142857,
+          "fpr95": 0.9375,
+          "n_good": 16,
+          "n_bad": 19
+        },
+        "edge4": {
+          "auroc": 0.6776859504132232,
+          "accuracy": 0.5909090909090909,
+          "fpr95": 0.9090909090909091,
+          "n_good": 11,
+          "n_bad": 11
+        }
+      }
+    },
+    {
+      "fold": 5,
+      "overall_auroc": 0.8584691170898068,
+      "overall_acc": 0.8037735849056604,
+      "overall_fpr95": 0.5402298850574713,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.55,
+          "accuracy": 0.9090909090909091,
+          "fpr95": 0.9,
+          "n_good": 20,
+          "n_bad": 2
+        },
+        "dist1": {
+          "auroc": 0.9071428571428571,
+          "accuracy": 0.8604651162790697,
+          "fpr95": 0.14285714285714285,
+          "n_good": 35,
+          "n_bad": 8
+        },
+        "dist6": {
+          "auroc": 0.9241379310344828,
+          "accuracy": 0.9411764705882353,
+          "fpr95": 0.3793103448275862,
+          "n_good": 29,
+          "n_bad": 5
+        },
+        "dots": {
+          "auroc": 1.0,
+          "accuracy": 1.0,
+          "fpr95": 0.0,
+          "n_good": 17,
+          "n_bad": 11
+        },
+        "edge1": {
+          "auroc": 0.8571428571428572,
+          "accuracy": 0.75,
+          "fpr95": 0.7222222222222222,
+          "n_good": 18,
+          "n_bad": 14
+        },
+        "edge2": {
+          "auroc": 0.743529411764706,
+          "accuracy": 0.6904761904761905,
+          "fpr95": 0.8235294117647058,
+          "n_good": 17,
+          "n_bad": 25
+        },
+        "edge3": {
+          "auroc": 0.7251461988304094,
+          "accuracy": 0.5405405405405406,
+          "fpr95": 0.7894736842105263,
+          "n_good": 19,
+          "n_bad": 18
+        },
+        "edge4": {
+          "auroc": 0.7631578947368421,
+          "accuracy": 0.8518518518518519,
+          "fpr95": 0.8947368421052632,
+          "n_good": 19,
+          "n_bad": 8
+        }
+      }
+    }
+  ]
+}

results/cv_lambda_ablation/cv_lambda0.02/fold1/metrics_fold1.json

@@ -0,0 +1,63 @@
+{
+  "overall_auroc": 0.8885557221389306,
+  "overall_acc": 0.8270676691729323,
+  "overall_fpr95": 0.5632183908045977,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.984375,
+      "accuracy": 0.9705882352941176,
+      "fpr95": 0.03125,
+      "n_good": 32,
+      "n_bad": 2
+    },
+    "dist1": {
+      "auroc": 0.9834710743801653,
+      "accuracy": 0.8787878787878788,
+      "fpr95": 0.045454545454545456,
+      "n_good": 22,
+      "n_bad": 11
+    },
+    "dist6": {
+      "auroc": 1.0,
+      "accuracy": 1.0,
+      "fpr95": 0.0,
+      "n_good": 26,
+      "n_bad": 2
+    },
+    "dots": {
+      "auroc": 0.9907407407407407,
+      "accuracy": 0.9333333333333333,
+      "fpr95": 0.05555555555555555,
+      "n_good": 18,
+      "n_bad": 12
+    },
+    "edge1": {
+      "auroc": 0.9636363636363636,
+      "accuracy": 0.8571428571428571,
+      "fpr95": 0.18181818181818182,
+      "n_good": 11,
+      "n_bad": 10
+    },
+    "edge2": {
+      "auroc": 0.8784722222222222,
+      "accuracy": 0.7352941176470589,
+      "fpr95": 0.16666666666666666,
+      "n_good": 18,
+      "n_bad": 16
+    },
+    "edge3": {
+      "auroc": 0.7225,
+      "accuracy": 0.725,
+      "fpr95": 0.7,
+      "n_good": 20,
+      "n_bad": 20
+    },
+    "edge4": {
+      "auroc": 0.7017543859649124,
+      "accuracy": 0.6521739130434783,
+      "fpr95": 0.8148148148148148,
+      "n_good": 27,
+      "n_bad": 19
+    }
+  }
+}

results/cv_lambda_ablation/cv_lambda0.02/fold1/scores_fold1.csv

@@ -0,0 +1,267 @@
+score,true_label,pred_label,template_id,feature_id
+0.00021471557440236322,1,1,0,4
+-0.00011354086629580817,0,0,2,0
+-0.0002694206903106535,1,0,2,7
+-3.066491568461029e-05,1,0,0,1
+-6.782494747312705e-05,1,0,0,5
+-3.848026099149103e-05,1,0,0,5
+-0.0002441727375844492,0,0,1,6
+-0.0007058569096261638,0,0,2,0
+3.707729047165015e-06,1,1,0,5
+-2.154457906726817e-05,1,0,0,4
+0.00013711733976378912,1,1,1,6
+-0.0005580275630927643,0,0,0,1
+-0.0006452103910851298,0,0,1,6
+-0.0010358445547171864,0,0,2,0
+-0.00015990839194273895,0,0,0,4
+8.075866382569082e-05,0,1,0,4
+-0.00041449229873251163,0,0,0,5
+-0.0004795381290023243,0,0,1,6
+-0.0008910372757236478,0,0,2,2
+-0.0005779374367557467,0,0,0,1
+-7.087950682034723e-05,0,0,0,4
+-0.0007244291860843081,0,0,2,2
+-0.0007462030256283475,0,0,2,3
+-0.000907766147865913,0,0,2,2
+-0.0004882160012493822,0,0,0,4
+-0.0006591479870257898,0,0,0,1
+-0.00043581209145486174,0,0,0,5
+-0.0006213237272459093,0,0,0,4
+-0.0006007928692270056,0,0,0,5
+-0.00047160813439404634,0,0,2,7
+5.099983070976956e-05,1,1,1,6
+-0.000484182860527654,0,0,0,1
+-0.0005585153170977707,0,0,0,5
+-0.0010297913249814883,0,0,2,3
+-0.0008298600607668048,0,0,2,0
+-0.0004194477968849234,0,0,2,7
+-0.00032800326298456674,0,0,0,5
+-0.0009586626145755874,0,0,2,2
+-0.0002029211382614457,1,0,1,6
+-0.000915469390747603,0,0,2,2
+9.04750966583389e-05,1,1,0,4
+0.00010998361714882783,1,1,1,6
+-6.331656622933241e-05,1,0,2,7
+0.0002834758174140011,1,1,0,1
+0.00024235609278548502,0,1,1,6
+-0.00032750976533861813,0,0,2,0
+-0.00015665759157854932,0,0,2,7
+-7.360386633081372e-05,1,0,0,5
+-0.0001432795170694598,0,0,2,0
+-0.00010897047031903622,0,0,2,7
+0.00024290644621942185,1,1,0,1
+0.0003226894707768222,1,1,2,3
+0.000470017514307983,1,1,0,5
+0.000492557811376174,1,1,1,6
+0.0004549074333044696,1,1,2,7
+0.0006615361949661742,1,1,2,0
+6.035158527083716e-05,1,1,0,4
+-2.1045586327090096e-05,0,0,1,6
+-0.0003959972065058536,0,0,2,0
+-0.0003310195670928797,0,0,2,0
+0.00016522257530596045,1,1,2,3
+-7.783961395034684e-05,0,0,1,6
+0.00010072118107927897,1,1,2,3
+0.00011013559182174518,1,1,0,1
+-9.799359366297663e-05,1,0,2,7
+0.00010644042660714996,1,1,0,1
+0.00021743061050074307,1,1,0,4
+0.0004811431161942893,1,1,2,3
+0.00016424346162239103,1,1,2,7
+-0.00037458576436620074,0,0,2,0
+0.0003682851890334845,1,1,2,3
+-0.00021920398197835395,1,0,2,7
+0.0003702816599980008,1,1,2,2
+0.0002715003583580247,0,1,2,7
+0.0003618116237339569,1,1,2,2
+0.00043680372851667877,1,1,0,1
+1.3168824661988036e-05,1,1,2,0
+5.1263150526210596e-05,1,1,0,1
+1.4083159039727372e-05,1,1,1,6
+-0.0009054124454269188,0,0,2,0
+0.00024579077697126084,1,1,0,5
+-0.000131967694615015,1,0,0,1
+-0.0009471260814461844,0,0,2,3
+-0.0007336718254373417,0,0,2,0
+-9.970904910005379e-05,1,0,0,5
+-0.000579235200711993,0,0,2,2
+-0.00025663058302598067,0,0,0,5
+-0.00038987775769783184,0,0,2,7
+-6.211214640643344e-05,1,0,1,6
+-0.0003500521706882864,0,0,2,2
+-0.00046222601085901295,0,0,2,3
+-0.0003881488242768686,0,0,0,5
+-0.0010839016389218165,0,0,2,0
+-0.0004770292350440297,0,0,0,1
+-0.001204158005421051,0,0,2,0
+1.3924008817412306e-05,1,1,0,1
+0.0002497311736806302,1,1,0,4
+0.00028857576980954,1,1,2,3
+-8.230975508922642e-05,1,0,0,1
+-0.00016573584813158945,1,0,2,3
+0.00072867327311542,1,1,0,5
+0.0013028780138120075,0,1,1,6
+0.0008665945497341454,1,1,2,3
+-0.00010690451104892508,0,0,2,0
+0.000600304745021275,1,1,2,7
+0.0007249066064832707,1,1,2,7
+-0.0010843164534890086,0,0,2,2
+-0.0003588564737583514,1,0,2,7
+-0.0002809438630356452,0,0,0,1
+-0.00020903459168039248,1,0,1,6
+-0.0001423015940235934,1,0,2,7
+-0.0005845419107936324,0,0,2,2
+-0.0003272297469084154,0,0,2,7
+-0.0001622101085376939,0,0,0,5
+-0.0007126774222706439,0,0,2,0
+-0.00042416091891936923,0,0,2,0
+-0.00016459092264994868,1,0,2,7
+-0.0005031451710965484,0,0,2,2
+-0.00030690047249663614,0,0,2,7
+-0.0006571332135354166,0,0,2,0
+-0.0004354441311443221,0,0,2,3
+-0.00030275596480350796,1,0,2,7
+-0.0006711580470437177,0,0,2,0
+-0.0003859022812685003,0,0,2,7
+-0.0005449185095494618,0,0,0,1
+-0.0007785942481132214,0,0,0,1
+-0.00024395432323217302,0,0,0,5
+-0.0006685328230378214,0,0,2,3
+-0.0003332440106896678,0,0,2,7
+-0.0003027391928480949,0,0,0,1
+-0.000160078695334959,0,0,0,4
+-0.0001718942035222413,0,0,0,5
+-4.1095897904599304e-06,1,0,0,5
+-0.0001686685840832057,0,0,0,1
+-0.0001737954546115368,1,0,1,6
+-0.00021219441230641706,0,0,2,7
+-0.000659319536935073,0,0,2,0
+-0.0002428836136823525,1,0,2,7
+-0.00010676533827791088,1,0,0,5
+-5.614749534288421e-05,0,0,1,6
+3.779608116019455e-05,1,1,2,3
+-0.0002836734731681643,0,0,0,5
+-0.00015170497936196556,0,0,1,6
+0.00028705076983897085,1,1,1,6
+-0.0005536921083694329,0,0,2,7
+-0.0011226582404924561,0,0,2,2
+0.0002775792969623575,0,1,1,6
+-9.21816020854737e-05,0,0,0,4
+-0.0001292973756790161,1,0,1,6
+-0.0004216482237097809,0,0,2,3
+7.595604809466569e-05,1,1,0,5
+-9.186626586597454e-05,0,0,1,6
+-0.0009967229314497674,0,0,2,2
+-8.572616818128265e-05,1,0,0,4
+-0.00026427957811392747,0,0,0,5
+-0.00019143129320582306,0,0,0,5
+-9.241039195330945e-05,0,0,1,6
+-0.00020647317811381058,0,0,1,6
+-0.0012996882211882624,0,0,2,2
+-0.0010601833867258399,0,0,0,1
+-0.0010063169753993864,0,0,0,1
+-0.00019243186776293428,0,0,2,3
+-0.0007899328140774757,1,0,2,7
+-0.0002955698952428077,0,0,2,3
+0.0010875694436253969,1,1,0,5
+0.0006623354321345695,1,1,1,6
+-0.0005023667571367671,0,0,2,7
+-0.0008130403386894629,0,0,2,3
+-0.0009133738427772216,0,0,2,0
+-0.0002550603140844057,0,0,0,1
+0.0009349287539953365,1,1,1,6
+-0.0012210231390781708,0,0,2,2
+-0.0009278140118112784,0,0,2,0
+0.0007942147552967085,0,1,0,5
+-2.8229849413037578e-05,0,0,2,3
+-0.000674885817570612,0,0,0,1
+-0.0009439045708859346,0,0,0,1
+0.0011515146520105178,1,1,0,5
+-0.00031509560241829715,0,0,2,7
+-0.0008171359097468682,0,0,2,0
+0.0007383213052526115,1,1,1,6
+-0.0006546943561988889,0,0,2,3
+-0.0002783914111205363,0,0,0,1
+0.0008124496683012683,1,1,1,6
+-0.00045627666404470836,0,0,2,7
+-0.0026855319400783603,0,0,2,2
+0.0007648386899381868,1,1,1,6
+0.0011707719083642598,1,1,0,5
+-0.001615641413081903,0,0,2,2
+-0.00044940013991436045,0,0,0,1
+-0.0005059777232236246,0,0,0,1
+-0.0005122301983647038,0,0,2,7
+-0.0016235697403317317,0,0,2,2
+-0.001139986806374508,0,0,2,0
+-0.0014997699967352687,0,0,1,6
+-0.0036634362972108626,0,0,2,3
+-0.001986155563499778,0,0,2,0
+-0.0036129226486082203,0,0,2,3
+-0.0021795336366631085,0,0,2,2
+-0.004115652948094066,0,0,0,5
+-0.00386785711045377,0,0,2,2
+-0.002509900949080473,0,0,2,0
+-0.0024401275857235324,0,0,1,6
+-0.0019895213097333928,0,0,2,7
+-0.004418581172067209,0,0,2,3
+-0.0010664566504419784,0,0,2,2
+-0.0008112026940216305,0,0,2,3
+-0.0012870986753841848,0,0,2,0
+-0.0005066155394888479,0,0,2,7
+-0.0014446115854661913,0,0,2,2
+-0.0006316823806264433,1,0,2,7
+-0.0005017263648915103,0,0,0,4
+-0.0009960675734328097,0,0,0,1
+-0.0005584628623910241,1,0,2,7
+0.000749645258183591,1,1,1,6
+-0.0006630245663109234,0,0,2,7
+-0.0009777472223504446,0,0,2,3
+-0.0004898054318618963,0,0,2,7
+-0.000186136536940466,1,0,2,3
+-0.0006061850383412108,0,0,0,4
+0.0007168234113487416,1,1,0,5
+-0.00062228502356447,1,0,2,7
+0.00010125555476406603,1,1,0,1
+-0.00119431833474664,0,0,2,0
+-0.00024214602686697256,0,0,0,1
+0.000459874115185813,1,1,0,5
+4.2521378200037485e-05,0,1,0,1
+-0.0009626937276334471,0,0,2,7
+0.0004184886510483921,1,1,0,4
+-0.0010916902479948462,0,0,2,7
+-0.0022833318798802793,0,0,2,2
+-0.0014679261538549319,0,0,2,0
+0.0002920727676246314,1,1,0,4
+-0.001229447120858821,0,0,2,3
+0.0006342912928084844,0,1,0,5
+-0.00021914927841862678,0,0,0,1
+-0.001306210076727439,0,0,2,0
+1.253591821295981e-05,0,1,1,6
+-0.0010918723602662793,0,0,2,7
+-4.6203909441829005e-05,0,0,1,6
+-0.0002040433266665767,0,0,0,4
+0.0005818473629187783,0,1,0,5
+0.001553711185697467,1,1,1,6
+0.000723326824954712,1,1,2,3
+-0.0009860120472149,0,0,2,7
+-0.000856352890841662,1,0,2,7
+-0.002835975547495765,0,0,2,2
+0.0003061286266893143,1,1,1,6
+-0.00010701401391997878,0,0,1,6
+-0.0007561100833117958,0,0,0,4
+-0.0003822954077622835,0,0,0,5
+-0.0023954027987201677,0,0,2,2
+-0.0011788004159461705,0,0,2,0
+-0.0007868204219266747,0,0,2,7
+0.00023575851053465186,1,1,1,6
+-0.0007983035562210833,0,0,2,7
+0.0005242057901341524,0,1,2,0
+-0.0013539287965977503,0,0,2,2
+-0.0006931706552859435,1,0,2,7
+0.0010866438239463608,0,1,1,6
+-0.0003278948168735918,0,0,2,7
+0.0026618597569176977,1,1,2,3
+-0.0009659095841925702,0,0,2,2
+0.000543216509395282,0,1,1,6
+-0.0005873254608013689,0,0,2,0
+0.0017731657644617371,1,1,0,4

results/cv_lambda_ablation/cv_lambda0.02/fold2/metrics_fold2.json

@@ -0,0 +1,63 @@
+{
+  "overall_auroc": 0.8181534232883558,
+  "overall_acc": 0.7593984962406015,
+  "overall_fpr95": 0.5689655172413793,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.5972222222222222,
+      "accuracy": 0.875,
+      "fpr95": 0.75,
+      "n_good": 36,
+      "n_bad": 4
+    },
+    "dist1": {
+      "auroc": 0.5818181818181818,
+      "accuracy": 0.6666666666666666,
+      "fpr95": 0.7727272727272727,
+      "n_good": 22,
+      "n_bad": 5
+    },
+    "dist6": {
+      "auroc": 1.0,
+      "accuracy": 0.8571428571428571,
+      "fpr95": 0.0,
+      "n_good": 19,
+      "n_bad": 9
+    },
+    "dots": {
+      "auroc": 0.907928388746803,
+      "accuracy": 0.7,
+      "fpr95": 0.6521739130434783,
+      "n_good": 23,
+      "n_bad": 17
+    },
+    "edge1": {
+      "auroc": 0.6962962962962964,
+      "accuracy": 0.696969696969697,
+      "fpr95": 0.9333333333333333,
+      "n_good": 15,
+      "n_bad": 18
+    },
+    "edge2": {
+      "auroc": 0.7604166666666666,
+      "accuracy": 0.7058823529411765,
+      "fpr95": 0.7777777777777778,
+      "n_good": 18,
+      "n_bad": 16
+    },
+    "edge3": {
+      "auroc": 0.8151260504201681,
+      "accuracy": 0.7741935483870968,
+      "fpr95": 0.7647058823529411,
+      "n_good": 17,
+      "n_bad": 14
+    },
+    "edge4": {
+      "auroc": 0.7453703703703703,
+      "accuracy": 0.7878787878787878,
+      "fpr95": 0.7916666666666666,
+      "n_good": 24,
+      "n_bad": 9
+    }
+  }
+}

results/cv_lambda_ablation/cv_lambda0.02/fold2/scores_fold2.csv

@@ -0,0 +1,267 @@
+score,true_label,pred_label,template_id,feature_id
+0.0006909172591986132,1,1,2,3
+0.0004753396243904716,1,1,0,1
+0.0003415248423698364,1,1,0,4
+-0.0006012882658978924,0,0,1,6
+0.00048362469795392923,0,1,2,3
+-0.00017487243807409042,1,0,2,7
+9.39236531849047e-05,1,1,0,5
+-0.00011882508231792627,0,0,1,6
+5.90772251598537e-05,0,1,2,3
+9.78737662081161e-07,0,1,1,6
+0.0001494999256101432,1,1,0,5
+-6.388987530954113e-05,1,0,0,5
+-0.0005598135685431771,0,0,2,0
+-0.00020347029203549027,0,0,2,3
+-0.00021355216653319228,0,0,0,4
+-0.0001700347251608024,0,0,0,1
+-0.00023750743072014323,0,0,0,1
+-0.000502015773090534,0,0,2,7
+-9.179913176922058e-05,0,0,0,4
+-0.00029244537872727926,0,0,0,5
+-4.815023276023497e-05,0,0,2,3
+-0.0002753828334971335,0,0,2,7
+-0.0004992702306481078,0,0,0,1
+-9.077597642317276e-06,0,0,1,6
+-0.0005347936597536306,0,0,2,7
+-0.001226701815030538,0,0,2,2
+-0.0004627648918540204,0,0,2,7
+-0.0003067880336311661,0,0,0,1
+-0.00015039752237498705,0,0,1,6
+-0.0006738187477458267,0,0,2,0
+-0.00041924675955669864,0,0,2,7
+-0.00023108260327717352,0,0,1,6
+-0.00033033530053216964,0,0,0,1
+-0.0007823653044761163,0,0,2,2
+-0.0003919923212379224,1,0,0,4
+-0.0005922974349232384,0,0,2,0
+-0.000565776074072346,0,0,2,2
+-0.0001008499649469731,0,0,0,4
+-0.000778485343034846,0,0,2,0
+-0.0008812158936052587,0,0,2,2
+-4.389476787764607e-05,0,0,2,3
+-0.000637627443065867,0,0,2,2
+-0.0004485644752276127,0,0,2,7
+-0.00017718892922857776,0,0,0,4
+2.8157249325885467e-05,0,1,2,3
+-0.0001920506171882129,0,0,0,1
+-0.00038095636933576336,1,0,0,5
+7.286177395144505e-05,0,1,2,3
+0.0008696665996103543,1,1,2,3
+1.6941520734689228e-05,0,1,2,0
+0.0004653388579026795,1,1,2,3
+0.0006958176754415036,1,1,2,3
+-7.106809993274489e-05,0,0,0,1
+-0.00035813445720123127,0,0,1,6
+-0.000141858332499396,0,0,2,7
+-0.00011347548483172515,0,0,0,5
+-0.00023480667849071213,0,0,0,5
+-0.0003731859463732695,0,0,1,6
+-0.0001546963164582825,0,0,0,4
+-0.00015760249953018432,0,0,2,7
+-0.0004823662879061899,1,0,1,6
+0.00029442488594213535,1,1,2,3
+0.0004839557097875511,1,1,0,4
+0.0002280190907185892,1,1,0,4
+0.0007540572574362154,1,1,2,3
+-5.461940891109476e-05,1,0,2,0
+0.0006614492577500641,1,1,0,1
+0.0001743776837247425,1,1,2,2
+-8.985542663140188e-05,0,0,2,0
+0.00031592352665029376,0,1,0,5
+9.405345714185408e-05,1,1,2,7
+0.00010351682081818567,1,1,0,4
+-0.0005652408971218385,0,0,2,0
+9.425691678173476e-07,0,1,0,1
+-0.00014304948970675357,1,0,0,5
+3.3140015730168115e-05,0,1,2,3
+0.0003456920242751946,1,1,0,4
+9.666938189184168e-05,1,1,0,5
+-0.0007160591066349298,0,0,2,0
+-0.0004563435059390039,0,0,2,3
+0.00031044826202560223,1,1,0,4
+-0.000631312100740614,0,0,2,0
+0.0004611753852805135,1,1,0,5
+0.00024119233072269694,1,1,1,6
+0.00043839060235768487,1,1,0,5
+0.00022709703334839945,1,1,2,2
+0.0003803853440331288,1,1,2,7
+0.0003406811444438048,1,1,2,2
+0.000411044792854226,1,1,2,7
+-0.000521194959874266,1,0,2,0
+-0.00020199325517751207,0,0,0,5
+-0.0006151703782961704,0,0,2,0
+0.00012531246175058203,0,1,2,3
+0.0002686400761012925,0,1,2,3
+-0.00022902675584191446,1,0,2,7
+-0.0005963435804005705,0,0,2,2
+0.0002693078864831472,1,1,0,4
+0.00010819968360010514,1,1,0,5
+0.00027270296704955295,1,1,0,4
+-2.20631551928821e-05,1,0,1,6
+-0.00011185374576598399,1,0,2,2
+-0.00012145308224717141,1,0,2,2
+0.0006973103265045209,1,1,2,2
+0.00044106252433266484,0,1,2,7
+0.0009460843703709559,0,1,0,1
+0.0008894759032409636,1,1,2,3
+0.00044507788144983185,0,1,0,5
+0.001663477191468702,1,1,2,3
+0.0006983894319273522,1,1,2,2
+-1.557168463477837e-05,1,0,0,4
+0.0003339427072205584,1,1,1,6
+-0.0009417868897435254,0,0,2,0
+-0.000775639699713792,1,0,0,1
+-0.00021502590971067548,1,0,0,4
+-0.0007887659399420949,0,0,2,0
+-0.0005506026823422876,0,0,2,2
+2.921410603448811e-05,0,1,2,3
+-0.0004263564007123931,0,0,0,5
+-0.00025871630146866685,0,0,0,5
+-0.0001289924024604266,1,0,0,5
+-0.0006188370211748402,0,0,2,0
+-0.0002581815779558374,1,0,1,6
+-5.230691778706359e-05,0,0,0,4
+0.000422651407134253,1,1,2,3
+-7.290880806976908e-05,0,0,0,5
+0.00036037136538652655,1,1,2,3
+-0.00043646685400744803,0,0,2,0
+-0.00017936339805601186,1,0,0,5
+-1.2823362485506369e-05,1,0,0,5
+-0.0004830321020563126,0,0,2,7
+-0.0006150840196642096,0,0,2,2
+9.495627920841875e-05,0,1,2,3
+-0.00020564575446769574,0,0,2,7
+-0.0002561300882371158,0,0,0,4
+-0.0002916587947402148,1,0,0,5
+-0.00033199970610440013,0,0,2,3
+-0.00014833477878710422,0,0,0,1
+-0.00033092010853579337,1,0,0,1
+-0.00024122224567690864,1,0,0,4
+-3.0277616169768415e-05,0,0,2,3
+-0.0009159181432914932,0,0,2,0
+-0.0006935533590149145,0,0,2,2
+0.0001058459607884291,0,1,2,3
+-0.00036325375403976033,0,0,2,7
+-0.00017188817640999277,1,0,0,1
+-0.0002773558752960526,1,0,0,4
+-0.00027688669768394757,0,0,0,5
+-2.4733734899199528e-05,0,0,1,6
+4.349363793153607e-05,1,1,1,6
+-0.00030281014915090054,1,0,0,4
+-4.754675785079501e-05,0,0,1,6
+-0.0003721404660609554,1,0,2,2
+0.0005739897995954354,1,1,0,4
+-0.000470780737523456,1,0,2,7
+-0.00021368799323681804,0,0,1,6
+-0.0006541309232125057,0,0,2,0
+-0.0002926267904695117,0,0,0,5
+-0.0004892654946888793,0,0,2,2
+0.00022944072232348837,1,1,2,3
+-0.0002861024852609268,0,0,2,7
+-0.00019141781434882377,0,0,0,1
+-0.0003585714247310535,0,0,2,7
+-0.0003449420057586392,0,0,2,7
+8.91620037145914e-05,0,1,0,1
+0.0007730809313943616,1,1,2,3
+-0.0002891124461893921,1,0,2,7
+-0.0007871001158491703,0,0,2,0
+-0.0007402129980619062,1,0,2,7
+-0.0007353797779069285,0,0,2,0
+-0.0005438305396819488,0,0,2,2
+0.0007523561595007798,0,1,0,1
+0.0008640051734982966,1,1,0,4
+-0.001202018161711749,0,0,2,0
+-0.0005763365750317464,0,0,2,7
+-0.00012360002176137717,1,0,1,6
+-0.0009989921830128885,0,0,2,0
+-0.000348357190378009,0,0,0,5
+-5.3284485475159e-05,0,0,2,3
+0.0008540729733067555,1,1,1,6
+-0.0003423639247193931,1,0,2,7
+-0.00116043294343399,0,0,2,0
+-1.766864617820836e-05,0,0,0,4
+-0.0009447360859485311,0,0,2,7
+4.052497679367731e-05,0,1,0,4
+-0.0017858238980988962,0,0,0,1
+-0.0010222662735031897,0,0,2,0
+-0.0005637353984639069,0,0,0,1
+-0.0013489512010710306,0,0,2,2
+-0.0010235703981015878,0,0,2,0
+0.00016659484856063403,0,1,0,4
+-0.00046854544983943903,0,0,2,7
+-0.00016976838785922113,0,0,0,4
+0.00043966349679976785,1,1,0,5
+-0.00023269897967111326,1,0,2,3
+-0.0023036070706439205,0,0,2,2
+0.0007275434728944673,1,1,1,6
+-0.00025315799080999346,1,0,2,3
+2.1936634439043146e-05,1,1,0,4
+-0.0006648583905189288,0,0,2,7
+-0.0009116429503774268,0,0,2,0
+4.3835424585268734e-05,0,1,0,4
+-0.0028559754550224192,0,0,0,1
+-0.0017707974382210473,0,0,0,5
+-0.0040850647469051204,0,0,2,3
+-0.0021040224260650588,0,0,2,7
+-0.00924851963995025,0,0,0,1
+-0.003152282071532682,0,0,0,5
+-0.004094104057876392,0,0,0,5
+-0.004515144138422331,0,0,2,3
+-0.0025032984593417514,0,0,2,7
+-0.002873641332844269,0,0,0,5
+-0.0023586651950608974,0,0,1,6
+-0.0032626248142332767,0,0,2,7
+-0.005135958726168611,0,0,2,3
+-0.0037621348450193187,0,0,0,4
+-0.0025911281921435145,0,0,1,6
+-0.001454835814947729,0,0,2,0
+0.0006017553352285175,1,1,1,6
+-0.0011656224325997753,0,0,2,0
+-0.0013885436655255018,0,0,2,3
+-0.0019751896397792734,0,0,2,0
+0.0006515649432549259,1,1,1,6
+-0.00034161710180342225,0,0,0,4
+-0.002175323691626544,0,0,2,0
+-0.0009105942529276951,0,0,2,0
+-0.0020107173547148707,0,0,2,2
+-0.0023214085056679316,0,0,2,0
+0.00042529522557742915,0,1,1,6
+0.0001405899511883052,1,1,1,6
+-0.0024711074889637516,0,0,2,2
+0.00025349525152705554,0,1,0,1
+0.0010703347105300055,1,1,1,6
+-0.0021494994452223182,0,0,2,0
+-0.0011128864419879397,0,0,0,1
+-0.001961216334602795,0,0,2,3
+-0.0032519823103211824,0,0,2,2
+-0.0020507804150111057,0,0,2,0
+-0.0021172573429066675,0,0,2,3
+-0.00024764367379248066,1,0,2,2
+-7.063947909045909e-05,0,0,0,5
+-0.0019933519716141736,0,0,2,0
+-5.735947692301138e-05,0,0,0,1
+0.0005868403014028453,1,1,1,6
+-9.823850123210492e-06,0,0,1,6
+-0.0011329060350544762,0,0,2,7
+-0.00012565841840114604,0,0,0,5
+-0.0036823490256210796,0,0,2,2
+1.577396935317603e-05,0,1,0,1
+-0.0018457816983573141,1,0,2,0
+-0.0030334170651622125,0,0,2,0
+8.922415086999591e-05,1,1,0,4
+-0.0005643252027220935,1,0,0,5
+-0.0014035985094960832,0,0,1,6
+-0.0003800339606823411,0,0,2,7
+0.0008276672335341545,1,1,2,3
+0.0009974376234458751,1,1,2,3
+0.0002466128970263562,0,1,1,6
+-0.003060538595309484,0,0,2,2
+-0.002371904775500297,0,0,0,1
+0.0015983013686491215,1,1,2,3
+-0.0025178978149779097,0,0,2,2
+-0.00361624981247588,0,0,2,0
+-0.002716086746950161,0,0,2,0
+0.0003956643346464256,0,1,0,4
+0.0010666982783004639,1,1,0,5
+-0.001896035029785706,1,0,2,0

results/cv_lambda_ablation/cv_lambda0.02/fold3/metrics_fold3.json

@@ -0,0 +1,63 @@
+{
+  "overall_auroc": 0.808306107062076,
+  "overall_acc": 0.7886792452830189,
+  "overall_fpr95": 0.8901734104046243,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.7321428571428571,
+      "accuracy": 0.875,
+      "fpr95": 0.7857142857142857,
+      "n_good": 28,
+      "n_bad": 4
+    },
+    "dist1": {
+      "auroc": 0.7455621301775148,
+      "accuracy": 0.6923076923076923,
+      "fpr95": 0.8076923076923077,
+      "n_good": 26,
+      "n_bad": 13
+    },
+    "dist6": {
+      "auroc": 0.8,
+      "accuracy": 0.9310344827586207,
+      "fpr95": 1.0,
+      "n_good": 24,
+      "n_bad": 5
+    },
+    "dots": {
+      "auroc": 0.9291666666666667,
+      "accuracy": 0.9411764705882353,
+      "fpr95": 0.5,
+      "n_good": 24,
+      "n_bad": 10
+    },
+    "edge1": {
+      "auroc": 0.5116959064327485,
+      "accuracy": 0.5405405405405406,
+      "fpr95": 0.8947368421052632,
+      "n_good": 19,
+      "n_bad": 18
+    },
+    "edge2": {
+      "auroc": 0.7473684210526316,
+      "accuracy": 0.7241379310344828,
+      "fpr95": 1.0,
+      "n_good": 10,
+      "n_bad": 19
+    },
+    "edge3": {
+      "auroc": 0.8875,
+      "accuracy": 0.75,
+      "fpr95": 0.4166666666666667,
+      "n_good": 12,
+      "n_bad": 20
+    },
+    "edge4": {
+      "auroc": 0.9333333333333333,
+      "accuracy": 0.9090909090909091,
+      "fpr95": 0.1,
+      "n_good": 30,
+      "n_bad": 3
+    }
+  }
+}

results/cv_lambda_ablation/cv_lambda0.02/fold3/scores_fold3.csv

@@ -0,0 +1,266 @@
+score,true_label,pred_label,template_id,feature_id
+0.00036659958073869195,1,1,1,6
+-2.9212188965174052e-05,1,0,2,7
+-0.0006687426852295172,0,0,2,0
+-1.2147583765908555e-05,0,0,0,4
+-0.0008761788986157626,0,0,2,0
+-1.320447947364238e-05,1,0,1,6
+-0.0012091070637688956,0,0,2,0
+-0.0007201051153242592,0,0,2,2
+-0.000884056143113412,0,0,2,0
+-0.001151661759649869,0,0,2,2
+-0.00032330300076864604,1,0,0,4
+-0.0007894019308150747,0,0,2,2
+-0.0006891821144381534,0,0,2,2
+-0.0006278517443570305,0,0,2,2
+-0.0008397697270265785,0,0,2,3
+-0.0003643046741490241,1,0,1,6
+-0.001012736674165354,0,0,2,0
+-0.0005693377152783795,0,0,2,2
+-0.0005649862246355046,1,0,0,5
+-0.0002792192675406119,0,0,0,4
+-0.0005115342186763887,0,0,0,4
+-0.0004876851660083042,0,0,2,3
+-0.0007424567517591643,0,0,2,3
+-0.000403012328024489,0,0,1,6
+-0.0010130272983224133,0,0,2,3
+-0.000598364699981176,0,0,2,7
+-0.0005592242491547945,0,0,2,7
+-0.0011997218526084916,0,0,0,1
+-0.0005970883416011926,0,0,2,7
+-0.0003798580690636301,0,0,2,7
+8.371070463908826e-05,1,1,0,5
+0.0003512017123284746,1,1,0,4
+0.0001678956896648742,1,1,0,1
+0.0003867474274011325,1,1,2,3
+-7.715457060839936e-05,0,0,1,6
+0.00015473752224352127,1,1,0,1
+0.0002739030914381147,1,1,0,1
+0.0008911683841142798,1,1,2,3
+-1.7951307527254087e-05,0,0,0,1
+4.758897615829427e-05,0,1,0,4
+-0.00012936403189087262,0,0,2,7
+-2.588054165244061e-05,0,0,2,7
+-0.0005344229273032408,0,0,2,0
+2.4598554882685614e-05,0,1,0,4
+-7.716082182014292e-05,0,0,0,5
+-0.00012791548855602738,0,0,2,7
+0.0002480381893110468,1,1,0,1
+0.0002058000583201637,1,1,1,6
+0.00044914514757692786,1,1,2,3
+0.00022928037273231908,1,1,1,6
+-0.0004650581884197893,0,0,2,2
+-0.0002092545531922948,0,0,2,3
+-4.341561580077299e-06,1,0,0,1
+0.00019471018342301187,1,1,0,4
+-0.0006459503364749245,0,0,2,0
+-0.0006423433159943679,0,0,2,2
+-0.00021613136079395162,0,0,2,3
+-2.8743550356011976e-05,1,0,1,6
+-0.00026703215815359824,1,0,0,1
+-0.0002604066199273815,1,0,2,0
+0.0006611982354661448,1,1,2,3
+-0.00035534188908059247,1,0,2,0
+1.0663078573999035e-05,0,1,0,1
+-0.0006601634994149209,0,0,2,0
+-0.0001752452910295686,0,0,2,3
+-0.0006483320955885594,0,0,2,7
+0.00012165582447778468,0,1,0,1
+-0.0004288135841488839,0,0,2,2
+-0.0004934420605422936,0,0,2,7
+-6.154921109555352e-05,1,0,0,1
+-0.0003526908575440758,0,0,2,3
+-0.0006119250020128674,0,0,2,2
+-0.00017393444431945682,0,0,0,4
+-0.0005378498841309923,0,0,1,6
+-0.00033146626141388087,1,0,0,4
+-0.000135468398802914,0,0,0,5
+-0.0006384792996686883,0,0,2,2
+-0.0006864695475087505,1,0,0,1
+-0.00032144705910468543,1,0,0,4
+-0.00044437986769480485,0,0,2,3
+-0.00010768813168397263,1,0,2,7
+-0.00027383271313738064,1,0,0,1
+-0.0002315788983833049,1,0,0,4
+1.530599285615511e-05,0,1,0,5
+7.848018169170218e-05,1,1,1,6
+3.534862597007367e-05,1,1,2,3
+-2.6804948865900846e-05,1,0,1,6
+0.0005706465351977379,1,1,0,5
+0.0004260896405321544,1,1,2,2
+0.0004552833916386598,1,1,2,7
+0.00020466000743908815,1,1,2,2
+-0.00014029403711902054,1,0,2,2
+0.0005210290735703886,1,1,2,2
+0.0006568304993561438,0,1,0,1
+-0.0011431298471870823,0,0,2,0
+-0.0009723608678905288,0,0,2,2
+-0.0003684556722873822,0,0,2,7
+-0.001250494632404297,0,0,2,0
+-0.0004295690919389021,0,0,2,3
+-0.0009163712360896175,0,0,2,0
+-0.0003984306482016104,0,0,0,1
+-6.241490715183304e-05,0,0,0,4
+-0.0003086945257382463,0,0,0,4
+-0.0002608835487626481,1,0,0,4
+-0.000705004125193226,0,0,2,0
+2.0466251880861888e-05,1,1,2,3
+-0.0002332672508782701,0,0,2,7
+-0.0007112387224333366,0,0,2,2
+2.900548104662476e-05,1,1,0,4
+0.00030076449038460816,1,1,0,5
+0.00014490496978396446,1,1,1,6
+-0.0013571294449502593,0,0,2,0
+-0.0003977647234569299,0,0,0,1
+-0.00020361676288303127,0,0,0,4
+-0.00038369169778889115,0,0,2,7
+-0.00046963901957496924,0,0,2,2
+-0.00016647037933580598,0,0,1,6
+6.528115744004054e-05,1,1,0,5
+-0.0009431684194714761,1,0,0,4
+-0.0001302149941329843,1,0,1,6
+-0.0008266986257513056,0,0,2,2
+-0.0004464068735251202,1,0,0,1
+-0.0005504527018638326,0,0,2,3
+-6.923697306774473e-05,0,0,0,5
+-0.001274396988737863,0,0,2,0
+-0.0008509898747433922,0,0,2,3
+-0.0005866148899076502,0,0,0,1
+-0.0008453216194175186,1,0,0,1
+-0.00047624217229895287,1,0,0,4
+-0.00022619864234002356,1,0,0,5
+0.0006445610965602096,1,1,0,4
+-0.00015650370158255128,0,0,0,1
+-0.00026243215281283525,1,0,0,4
+2.394859620835632e-06,1,1,2,3
+-0.0002037637439207176,0,0,2,7
+-0.0006660635661683047,0,0,2,0
+-0.00032330247951904353,0,0,0,5
+-6.0561986174427485e-06,0,0,1,6
+-0.00022644690354354634,0,0,0,1
+-0.0005921275322907597,0,0,2,0
+-0.00021265321905957618,0,0,0,4
+-3.5349042445887574e-05,0,0,1,6
+-0.0006011052467511033,0,0,2,2
+-0.00013187134609324823,1,0,0,5
+-0.00030341307952767155,0,0,2,3
+-0.00024371452251216365,0,0,0,1
+-0.00010196612507570521,1,0,0,1
+-0.0002373904763953768,0,0,0,1
+-0.0011693671037210154,1,0,0,4
+0.000174299015197903,1,1,1,6
+-0.0006078767860890379,0,0,2,7
+-0.00012033086619339835,1,0,1,6
+-0.0008145480367238627,0,0,2,0
+-0.0004285074002109458,0,0,0,4
+0.0004037327948026373,1,1,0,5
+-0.0007004987276741301,0,0,2,7
+-0.0017843076982535424,0,0,2,0
+0.000587024443957489,1,1,0,5
+-0.0021505633523338508,0,0,0,1
+0.0006540818588109693,1,1,1,6
+-0.00041284276958322014,0,0,2,3
+0.0006612031115218985,1,1,0,5
+7.091417850460871e-05,1,1,0,4
+-0.0006986014070571381,0,0,2,3
+0.000673527337494307,1,1,1,6
+-0.00047308368608355543,0,0,2,7
+-0.00960259372368455,1,0,2,2
+-0.002630792329437099,0,0,0,1
+0.00026814477489097224,0,1,0,5
+-0.00013986980571644228,0,0,2,7
+-0.0006862025405280289,0,0,0,4
+-0.0019102604844374576,0,0,0,1
+-0.0006424442279967489,0,0,2,7
+-0.003139605329488402,0,0,2,0
+-0.0008677141313091843,0,0,2,7
+-0.0023805785190779694,0,0,2,2
+-0.0028220216892077578,0,0,2,0
+-0.0016292488915496499,0,0,0,1
+0.0005586019554175425,1,1,1,6
+-0.0007047936710296195,0,0,2,3
+-0.009780649953754612,1,0,0,4
+-0.0034029194744653046,0,0,2,7
+-0.0046347487755701855,0,0,2,0
+-0.003923306938668249,0,0,1,6
+-0.00508643246110296,0,0,2,2
+-0.014223563001432918,0,0,0,4
+-0.04608884881425183,0,0,0,1
+-0.0041563495551235966,0,0,2,7
+-0.005038102894905024,0,0,2,0
+-0.014532350964727811,0,0,0,4
+-0.00432265847688541,0,0,2,0
+-0.02358222060487606,0,0,0,4
+-0.005861823822779114,0,0,2,3
+-0.0021749994109268277,0,0,2,0
+-0.04916288293083198,0,0,0,1
+-0.0019281700521241868,0,0,2,7
+-0.004316077901748941,0,0,2,2
+-0.002426554242847486,0,0,2,0
+-0.014416005437087737,1,0,0,4
+-0.03697443962912075,0,0,0,1
+-0.007077767816372212,1,0,0,5
+-0.0022026728306082065,1,0,2,0
+-0.0025172688157181246,0,0,2,2
+-0.0006818216593819683,1,0,2,3
+0.0007872124202549478,1,1,0,5
+-0.0004165382281644276,0,0,2,3
+0.0005181758306571291,1,1,0,5
+0.00041665983851998933,1,1,1,6
+-0.0037316976857255206,0,0,0,1
+-0.0009846064963494397,0,0,0,4
+0.0008079074791748995,1,1,0,5
+-0.0004406624933471909,0,0,2,3
+-0.0015722322207875547,0,0,2,2
+-0.002967231110378634,0,0,0,1
+-0.002484014396031851,0,0,2,2
+-0.0026954473633668367,0,0,0,1
+-0.00253544307721313,0,0,0,1
+0.0006286387363797982,1,1,1,6
+-0.00036935572687070775,1,0,2,3
+-0.0019353371494798925,0,0,2,3
+0.000561930493277029,1,1,0,5
+0.00011630354681983757,1,1,1,6
+-0.0006182744330726561,0,0,0,1
+-0.001651702678063886,0,0,2,0
+-3.1712386698933004e-05,1,0,0,4
+-0.001398622854030692,0,0,0,1
+0.00038328074384480493,0,1,0,5
+-0.0006451359845232214,0,0,2,7
+-0.002853678548708558,0,0,2,2
+-0.001107267368352042,0,0,2,3
+-0.000225466053525452,1,0,1,6
+-0.00075301198696252,0,0,2,7
+-0.0017422584354062597,0,0,2,3
+-0.0001845013559795914,0,0,0,5
+-0.0008486381697002804,1,0,2,0
+-0.0002555897895945222,0,0,1,6
+-0.0008679410588229072,0,0,2,7
+-0.0036218542075948795,0,0,0,1
+-0.0002917873783735564,0,0,0,5
+-0.0005857947602635229,0,0,1,6
+-0.0017851137495017606,0,0,2,3
+-0.0016460392141016203,0,0,2,0
+-0.0002153751548030422,1,0,0,5
+-0.001062967267644127,0,0,2,7
+-0.00017224506475031418,0,0,0,5
+-0.0005129640005179653,0,0,1,6
+-0.0007548587615019663,0,0,0,4
+-0.0009906630753539498,0,0,2,7
+-0.0020553226422634924,0,0,2,0
+-0.00032451442762976637,0,0,1,6
+-0.0016568483301671218,0,0,2,3
+-0.0009726082705310557,1,0,0,4
+5.096887936815661e-05,1,1,0,5
+-0.002973685935430695,1,0,0,1
+-0.003613816963334099,0,0,2,2
+0.0035526694962754873,1,1,0,5
+-0.00015522299217991808,0,0,2,7
+0.0008390451921150149,0,1,0,4
+0.003386135151667989,1,1,0,5
+0.00010522462485823558,0,1,1,6
+7.766853814245717e-05,0,1,2,7
+0.0014028889909968728,0,1,0,4
+-8.046749280787857e-07,0,0,2,7
+0.0036071525374427385,1,1,2,3
+0.0007677021366544051,1,1,1,6

results/cv_lambda_ablation/cv_lambda0.02/fold4/metrics_fold4.json

@@ -0,0 +1,63 @@
+{
+  "overall_auroc": 0.8815207780725023,
+  "overall_acc": 0.8226415094339623,
+  "overall_fpr95": 0.5574712643678161,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.9770114942528736,
+      "accuracy": 0.96875,
+      "fpr95": 0.06896551724137931,
+      "n_good": 29,
+      "n_bad": 3
+    },
+    "dist1": {
+      "auroc": 0.9190476190476191,
+      "accuracy": 0.8709677419354839,
+      "fpr95": 0.5238095238095238,
+      "n_good": 21,
+      "n_bad": 10
+    },
+    "dist6": {
+      "auroc": 0.9724137931034482,
+      "accuracy": 0.9117647058823529,
+      "fpr95": 0.13793103448275862,
+      "n_good": 29,
+      "n_bad": 5
+    },
+    "dots": {
+      "auroc": 0.9090909090909091,
+      "accuracy": 0.9302325581395349,
+      "fpr95": 0.96875,
+      "n_good": 32,
+      "n_bad": 11
+    },
+    "edge1": {
+      "auroc": 0.8825396825396826,
+      "accuracy": 0.8055555555555556,
+      "fpr95": 0.3333333333333333,
+      "n_good": 21,
+      "n_bad": 15
+    },
+    "edge2": {
+      "auroc": 0.8666666666666667,
+      "accuracy": 0.71875,
+      "fpr95": 0.8666666666666667,
+      "n_good": 15,
+      "n_bad": 17
+    },
+    "edge3": {
+      "auroc": 0.736842105263158,
+      "accuracy": 0.6857142857142857,
+      "fpr95": 0.9375,
+      "n_good": 16,
+      "n_bad": 19
+    },
+    "edge4": {
+      "auroc": 0.6776859504132232,
+      "accuracy": 0.5909090909090909,
+      "fpr95": 0.9090909090909091,
+      "n_good": 11,
+      "n_bad": 11
+    }
+  }
+}

results/cv_lambda_ablation/cv_lambda0.02/fold4/scores_fold4.csv

@@ -0,0 +1,266 @@
+score,true_label,pred_label,template_id,feature_id
+-0.00044973695068620256,1,0,0,1
+0.0006969253343413577,1,1,0,5
+-0.0006318466988159358,0,0,2,0
+-0.00032592597155598965,0,0,2,3
+-0.00013495994295226361,0,0,0,5
+-0.0001204120725742542,1,0,1,6
+0.000183459427498746,1,1,2,3
+-0.000422913236834575,0,0,2,7
+-0.0010091834989725615,0,0,2,3
+1.2409320333979512e-06,0,1,0,1
+-0.0005944791308138513,0,0,2,3
+-0.0002943206127383746,0,0,0,5
+-0.0009029101187479679,0,0,2,0
+-0.0009926817764062436,0,0,2,3
+-0.00025513969070743714,0,0,0,4
+-0.0009991260312381192,0,0,2,0
+-0.0013465830151108103,0,0,2,3
+-0.0007845635400735764,0,0,2,0
+-0.0003285847578081303,0,0,2,3
+-0.0002640492244972842,0,0,2,7
+-0.00034006421192316234,0,0,0,1
+-0.0003023907798342419,0,0,0,5
+-0.0009573424881091346,0,0,2,0
+-0.0009074784541735423,0,0,2,2
+-0.0007619759376393635,0,0,2,3
+-0.00030443174473475734,0,0,0,4
+-0.0008548358455300331,0,0,2,0
+-0.0008708602227852687,0,0,2,2
+-0.00016724823974072905,0,0,0,5
+-0.00041362374235177565,0,0,0,1
+-0.0002950791284092695,0,0,1,6
+-0.0007212523801717902,0,0,2,0
+-0.0007385510901804088,0,0,2,7
+0.00017590913368621675,1,1,1,6
+-6.335608486550751e-06,1,0,1,6
+-0.0005563650492695169,0,0,0,1
+-0.00047300842736149214,0,0,0,4
+-0.0005309157638112069,0,0,2,3
+-0.00026270373607985634,0,0,0,4
+-0.000624278164759743,0,0,2,3
+0.00011502182867843651,1,1,0,1
+0.00014714610791998015,1,1,2,3
+7.308582920813843e-05,0,1,1,6
+9.427908516954596e-05,1,1,0,1
+0.0003317600255832078,0,1,1,6
+8.505042496835882e-05,1,1,2,0
+0.0008203195602982313,1,1,1,6
+0.00019989305525086772,1,1,0,5
+0.0005032855435274548,1,1,2,7
+-0.0006074839693610556,0,0,2,0
+-0.00032845096749952113,0,0,2,0
+-1.277810602914528e-05,1,0,0,4
+-0.0003623409540159635,0,0,2,0
+0.0001374103315174586,1,1,2,3
+-0.00018534379807533773,0,0,1,6
+1.9464004144538298e-05,1,1,2,3
+-0.0003640638123033578,0,0,0,1
+-0.00043077637179521865,0,0,2,2
+4.371733142761493e-05,0,1,0,5
+-0.0003980156124453061,0,0,2,0
+-0.00030251165386289275,0,0,2,2
+-8.0292230122723e-05,0,0,2,7
+-0.00017963995196623728,0,0,0,1
+-0.0005243219889234756,0,0,2,0
+5.920393014093896e-05,1,1,2,2
+3.5305514174978e-05,1,1,0,1
+0.001135318489978089,0,1,1,6
+-0.00011410020757466558,1,0,2,2
+-9.071001637494212e-05,0,0,2,7
+9.751732693985028e-05,0,1,0,5
+-0.00037538958626100703,1,0,2,0
+0.00015375314833363483,1,1,2,2
+8.505315374350217e-05,0,1,2,7
+0.0004178079354460358,0,1,1,6
+0.0004841902793850765,1,1,0,5
+-0.00032936362695181777,0,0,2,3
+-0.0005541719158645708,0,0,2,3
+-6.395918899215758e-05,1,0,2,7
+0.0005537280431599356,1,1,1,6
+6.874947866891343e-06,1,1,2,0
+0.0004629508868674737,1,1,2,3
+0.0001479372606263494,1,1,0,1
+-0.00017666665342403604,1,0,0,4
+-0.00030949826061260014,0,0,2,3
+-0.0002733759803231796,1,0,2,7
+-0.00010927487310255155,0,0,2,3
+-0.00012741537590045364,1,0,0,1
+-0.0001502432697452604,0,0,0,5
+-0.0006220144242979585,0,0,2,0
+-0.0003189657413167884,1,0,1,6
+-0.000716846318100579,0,0,2,0
+-0.00033637208369327685,0,0,2,7
+0.00013046780775766847,0,1,0,5
+5.074606218840802e-05,1,1,1,6
+0.00021954392548650253,1,1,1,6
+-0.0006086795325973074,0,0,2,2
+-0.0006648355536162855,0,0,2,3
+-0.0004832916334271435,0,0,2,0
+-0.0005341683051665325,1,0,2,2
+3.087525547016204e-06,1,1,2,3
+-0.00012159706006059472,1,0,2,7
+-0.00018322088290005992,1,0,2,2
+9.277079981984473e-05,1,1,0,1
+0.00023806612356565976,1,1,1,6
+1.2501025339589633e-05,1,1,0,1
+0.00027524357486981635,1,1,0,5
+-0.0009804451587842773,0,0,2,3
+0.0002742810512427248,1,1,0,4
+-0.0007116724018123924,0,0,2,3
+-7.080739393131724e-05,1,0,0,1
+0.0003263052858528681,0,1,1,6
+-5.456823942949843e-05,1,0,0,4
+-2.2466919035650165e-05,0,0,0,1
+-0.0005626132720499295,0,0,2,7
+-7.197916827863127e-05,0,0,0,1
+-5.407279008068193e-05,0,0,0,4
+-6.661535357125117e-05,0,0,1,6
+-8.130062691634465e-05,0,0,0,1
+-0.00021177970309508957,0,0,1,6
+-0.0005011403519893064,0,0,2,2
+-0.00012332005659118368,1,0,2,3
+-0.00018841689918190282,1,0,1,6
+-0.0003002820684923791,0,0,0,4
+3.6361567617859344e-05,1,1,0,5
+-1.1025160492863367e-05,0,0,0,4
+-6.879254069645097e-05,0,0,0,5
+-0.000617220221320167,0,0,2,2
+-4.6665523259435335e-05,0,0,0,4
+-0.00030752253282116844,0,0,0,1
+-0.00014290069753769737,0,0,0,4
+6.214123452082283e-05,0,1,0,5
+-0.0007052774788462559,0,0,2,0
+-0.0009906351074459967,0,0,2,2
+-0.0005776898964541038,0,0,2,2
+0.00013864242384442962,1,1,0,5
+-7.049153849948177e-05,1,0,2,7
+-3.067895217100103e-05,1,0,0,5
+2.7397582307457646e-05,1,1,2,7
+-0.0008536346530308957,0,0,2,0
+8.739372395211858e-05,1,1,0,1
+0.00013507965340977396,1,1,0,4
+0.00015650890243705355,1,1,1,6
+-0.00012795337650459322,1,0,0,4
+-0.00041704745526658336,0,0,2,0
+-0.0001904272596584631,0,0,1,6
+-0.0008178477099863805,0,0,2,0
+-0.0005487076941062697,0,0,2,3
+-0.0006678433899651275,0,0,2,3
+-0.00014171229791827382,1,0,2,7
+0.0003116791302454659,1,1,0,5
+0.001322397772164547,1,1,1,6
+8.428049477515742e-05,1,1,0,4
+-5.364810640458295e-06,1,0,2,3
+0.0001229000161401938,0,1,0,5
+-0.0005902732693357383,0,0,2,2
+-0.0009004205375094899,0,0,2,0
+-0.00017329677677480407,0,0,0,1
+-0.0004047539349994623,0,0,2,2
+-0.00013440495851682437,0,0,0,1
+-0.0013773361395578842,0,0,2,0
+-0.0013046133957686848,0,0,2,2
+-4.430051252711509e-05,1,0,0,5
+0.0008716649177949853,1,1,1,6
+-8.362271968508084e-05,1,0,0,5
+-0.0007270970527315512,0,0,2,0
+-0.0007981412616209129,0,0,2,0
+-0.0009524185754708012,0,0,2,2
+-0.0005809117943863379,0,0,2,2
+-0.0002581537229707463,0,0,2,7
+-0.0001080092729534958,0,0,0,5
+-0.0011288109666202206,0,0,2,2
+-0.00040901253873016664,0,0,2,3
+-0.00040269485674798364,0,0,2,7
+-0.0002712405880447473,0,0,0,4
+-0.0014933439207379696,0,0,2,3
+-0.00046139438927639276,0,0,0,4
+-0.0009342979599023232,0,0,2,2
+0.0009141584666213023,1,1,0,5
+-0.0010172398353461148,0,0,2,0
+-0.0012493494368391103,0,0,0,4
+0.0007721092365682112,1,1,1,6
+-0.0018827766593312832,0,0,2,3
+0.00017163786571472786,0,1,0,4
+-0.003131182263896335,0,0,2,3
+0.0005345272729755379,1,1,1,6
+-0.00090948071941966,0,0,2,2
+-0.0015526871581096197,0,0,2,2
+9.421193244634266e-05,1,1,0,4
+0.0008766647049924352,1,1,0,5
+-0.0035731201179441986,0,0,2,3
+-0.0013875645451480508,0,0,2,0
+0.0002378980239154775,0,1,0,4
+-0.0049423335836036135,1,0,2,3
+0.0008881007006857518,1,1,1,6
+-0.0026002689785673307,0,0,2,2
+-0.012460661184741179,0,0,0,1
+-0.004706009033834557,0,0,0,5
+-0.0027771078111254663,0,0,0,4
+-0.0027885060437256443,0,0,2,0
+-0.004613941179704855,0,0,2,2
+-0.010629068434063811,0,0,0,1
+-0.011092488862341272,0,0,1,6
+-0.013901153339538723,0,0,0,1
+-0.0031338778760982705,0,0,0,4
+-0.004140499850618658,0,0,0,5
+-0.008260852711682677,1,0,1,6
+-0.003971935122972354,0,0,2,3
+-0.0007604234397877024,0,0,0,4
+-0.0013227475262829122,0,0,0,1
+0.0010247383126989022,1,1,0,5
+0.0005886721989372731,1,1,1,6
+0.0008139113424113012,1,1,0,5
+-0.0009584893996361643,0,0,2,2
+-0.0014449276216328168,0,0,2,0
+-0.0010772015567636108,0,0,2,0
+0.001687743769434747,1,1,0,4
+0.00030306836706586217,1,1,0,5
+0.0001528641805634834,0,1,1,6
+-0.0010233334134682089,0,0,2,7
+0.00040688681008759636,1,1,0,4
+-0.002746695804235061,0,0,2,2
+0.0002872416813625017,1,1,0,5
+-0.004000741430500056,0,0,2,3
+-0.00047575249802321237,0,0,0,1
+-0.0026478346416843115,0,0,2,2
+-0.006342036399291831,0,0,2,3
+0.0006859125205664895,1,1,0,4
+-0.0006868589593796055,1,0,2,7
+6.604851572774434e-05,1,1,0,4
+-0.0006356443429831431,0,0,1,6
+-0.00026269415451679254,0,0,0,4
+-0.0013788923653191877,0,0,0,1
+-0.003139175171090757,0,0,2,2
+-0.0018041383501258695,0,0,0,1
+0.0008547293028095728,1,1,2,7
+-0.0006405509749311047,0,0,0,4
+-0.0036166731489356597,0,0,2,3
+-0.0026160131921642467,0,0,2,2
+-0.003533033308922316,0,0,2,3
+-0.0024189950336585744,0,0,2,3
+-0.0024143212661147107,0,0,2,2
+-0.0006822343205567447,1,0,0,5
+-0.0016973156578023937,0,0,2,3
+-0.0036996920322417276,0,0,2,3
+-0.002418009543616789,0,0,0,1
+-0.0005141189851565302,0,0,1,6
+-0.002237304831505754,0,0,2,3
+-0.002480703969486058,0,0,2,2
+-0.00041906838800059507,0,0,0,4
+-0.0015601675372454334,0,0,0,1
+-0.00015500856388826062,0,0,0,5
+-0.0009387729535228553,1,0,2,7
+0.002687621204822785,1,1,0,4
+0.00023260007321369153,1,1,1,6
+0.001254848236567338,1,1,0,4
+-0.001496213414939121,0,0,1,6
+0.0016278433750267218,1,1,2,3
+-0.0021215634935651817,0,0,2,2
+0.001421660510532094,1,1,0,4
+-0.0013307899906067136,0,0,1,6
+0.003708788450749123,1,1,2,3
+-0.0005539563606725972,0,0,2,0
+0.0004808614347712112,0,1,0,4
+-0.0012294958258280558,0,0,2,2
+-0.0003873167472193059,1,0,2,7

results/cv_lambda_ablation/cv_lambda0.02/fold5/metrics_fold5.json

@@ -0,0 +1,63 @@
+{
+  "overall_auroc": 0.8584691170898068,
+  "overall_acc": 0.8037735849056604,
+  "overall_fpr95": 0.5402298850574713,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.55,
+      "accuracy": 0.9090909090909091,
+      "fpr95": 0.9,
+      "n_good": 20,
+      "n_bad": 2
+    },
+    "dist1": {
+      "auroc": 0.9071428571428571,
+      "accuracy": 0.8604651162790697,
+      "fpr95": 0.14285714285714285,
+      "n_good": 35,
+      "n_bad": 8
+    },
+    "dist6": {
+      "auroc": 0.9241379310344828,
+      "accuracy": 0.9411764705882353,
+      "fpr95": 0.3793103448275862,
+      "n_good": 29,
+      "n_bad": 5
+    },
+    "dots": {
+      "auroc": 1.0,
+      "accuracy": 1.0,
+      "fpr95": 0.0,
+      "n_good": 17,
+      "n_bad": 11
+    },
+    "edge1": {
+      "auroc": 0.8571428571428572,
+      "accuracy": 0.75,
+      "fpr95": 0.7222222222222222,
+      "n_good": 18,
+      "n_bad": 14
+    },
+    "edge2": {
+      "auroc": 0.743529411764706,
+      "accuracy": 0.6904761904761905,
+      "fpr95": 0.8235294117647058,
+      "n_good": 17,
+      "n_bad": 25
+    },
+    "edge3": {
+      "auroc": 0.7251461988304094,
+      "accuracy": 0.5405405405405406,
+      "fpr95": 0.7894736842105263,
+      "n_good": 19,
+      "n_bad": 18
+    },
+    "edge4": {
+      "auroc": 0.7631578947368421,
+      "accuracy": 0.8518518518518519,
+      "fpr95": 0.8947368421052632,
+      "n_good": 19,
+      "n_bad": 8
+    }
+  }
+}

results/cv_lambda_ablation/cv_lambda0.05/cv_summary.json

@@ -0,0 +1,387 @@
+{
+  "n_folds": 5,
+  "mean_auroc": 0.8669904558931865,
+  "std_auroc": 0.02558649173467779,
+  "mean_acc": 0.8070932047098879,
+  "std_acc": 0.024065474709131417,
+  "mean_fpr95": 0.5700418576838748,
+  "std_fpr95": 0.1948045274986299,
+  "fold_aurocs": [
+    0.8666291854072963,
+    0.8549475262368815,
+    0.8269665745162101,
+    0.8844890741442466,
+    0.9019199191612984
+  ],
+  "fold_accs": [
+    0.793233082706767,
+    0.8082706766917294,
+    0.769811320754717,
+    0.8377358490566038,
+    0.8264150943396227
+  ],
+  "fold_fpr95s": [
+    0.5229885057471264,
+    0.5287356321839081,
+    0.9364161849710982,
+    0.5114942528735632,
+    0.3505747126436782
+  ],
+  "sep_loss_weight": 0.05,
+  "per_feature": {
+    "angle": {
+      "mean": 0.737798645320197,
+      "std": 0.1593432787093559
+    },
+    "dist1": {
+      "mean": 0.8827829513144199,
+      "std": 0.05807908808620675
+    },
+    "dist6": {
+      "mean": 0.9473331316797742,
+      "std": 0.07095646929948354
+    },
+    "dots": {
+      "mean": 0.9412753171097161,
+      "std": 0.039430915524717856
+    },
+    "edge1": {
+      "mean": 0.8574053821422243,
+      "std": 0.12832503531098052
+    },
+    "edge2": {
+      "mean": 0.8402235122119024,
+      "std": 0.028288127851722036
+    },
+    "edge3": {
+      "mean": 0.7348686176224876,
+      "std": 0.059556970843241165
+    },
+    "edge4": {
+      "mean": 0.7774653230873326,
+      "std": 0.11248936013207073
+    }
+  },
+  "fold_details": [
+    {
+      "fold": 1,
+      "overall_auroc": 0.8666291854072963,
+      "overall_acc": 0.793233082706767,
+      "overall_fpr95": 0.5229885057471264,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.859375,
+          "accuracy": 0.9411764705882353,
+          "fpr95": 0.25,
+          "n_good": 32,
+          "n_bad": 2
+        },
+        "dist1": {
+          "auroc": 0.9132231404958677,
+          "accuracy": 0.8181818181818182,
+          "fpr95": 0.5454545454545454,
+          "n_good": 22,
+          "n_bad": 11
+        },
+        "dist6": {
+          "auroc": 1.0,
+          "accuracy": 0.9285714285714286,
+          "fpr95": 0.0,
+          "n_good": 26,
+          "n_bad": 2
+        },
+        "dots": {
+          "auroc": 0.9722222222222223,
+          "accuracy": 0.9,
+          "fpr95": 0.3333333333333333,
+          "n_good": 18,
+          "n_bad": 12
+        },
+        "edge1": {
+          "auroc": 0.9727272727272727,
+          "accuracy": 0.8571428571428571,
+          "fpr95": 0.18181818181818182,
+          "n_good": 11,
+          "n_bad": 10
+        },
+        "edge2": {
+          "auroc": 0.8506944444444445,
+          "accuracy": 0.7352941176470589,
+          "fpr95": 0.3888888888888889,
+          "n_good": 18,
+          "n_bad": 16
+        },
+        "edge3": {
+          "auroc": 0.6925,
+          "accuracy": 0.675,
+          "fpr95": 0.9,
+          "n_good": 20,
+          "n_bad": 20
+        },
+        "edge4": {
+          "auroc": 0.6686159844054581,
+          "accuracy": 0.6304347826086957,
+          "fpr95": 0.9259259259259259,
+          "n_good": 27,
+          "n_bad": 19
+        }
+      }
+    },
+    {
+      "fold": 2,
+      "overall_auroc": 0.8549475262368815,
+      "overall_acc": 0.8082706766917294,
+      "overall_fpr95": 0.5287356321839081,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.5416666666666666,
+          "accuracy": 0.9,
+          "fpr95": 0.9722222222222222,
+          "n_good": 36,
+          "n_bad": 4
+        },
+        "dist1": {
+          "auroc": 0.8363636363636364,
+          "accuracy": 0.8518518518518519,
+          "fpr95": 0.3181818181818182,
+          "n_good": 22,
+          "n_bad": 5
+        },
+        "dist6": {
+          "auroc": 0.9766081871345029,
+          "accuracy": 0.7857142857142857,
+          "fpr95": 0.15789473684210525,
+          "n_good": 19,
+          "n_bad": 9
+        },
+        "dots": {
+          "auroc": 0.8900255754475703,
+          "accuracy": 0.875,
+          "fpr95": 0.7391304347826086,
+          "n_good": 23,
+          "n_bad": 17
+        },
+        "edge1": {
+          "auroc": 0.8629629629629629,
+          "accuracy": 0.8181818181818182,
+          "fpr95": 0.8666666666666667,
+          "n_good": 15,
+          "n_bad": 18
+        },
+        "edge2": {
+          "auroc": 0.8333333333333333,
+          "accuracy": 0.7352941176470589,
+          "fpr95": 0.4444444444444444,
+          "n_good": 18,
+          "n_bad": 16
+        },
+        "edge3": {
+          "auroc": 0.7773109243697478,
+          "accuracy": 0.6774193548387096,
+          "fpr95": 0.7058823529411765,
+          "n_good": 17,
+          "n_bad": 14
+        },
+        "edge4": {
+          "auroc": 0.7685185185185186,
+          "accuracy": 0.7878787878787878,
+          "fpr95": 0.7083333333333334,
+          "n_good": 24,
+          "n_bad": 9
+        }
+      }
+    },
+    {
+      "fold": 3,
+      "overall_auroc": 0.8269665745162101,
+      "overall_acc": 0.769811320754717,
+      "overall_fpr95": 0.9364161849710982,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.7589285714285714,
+          "accuracy": 0.875,
+          "fpr95": 0.8928571428571429,
+          "n_good": 28,
+          "n_bad": 4
+        },
+        "dist1": {
+          "auroc": 0.7928994082840237,
+          "accuracy": 0.6666666666666666,
+          "fpr95": 0.8846153846153846,
+          "n_good": 26,
+          "n_bad": 13
+        },
+        "dist6": {
+          "auroc": 0.8083333333333333,
+          "accuracy": 0.896551724137931,
+          "fpr95": 0.9583333333333334,
+          "n_good": 24,
+          "n_bad": 5
+        },
+        "dots": {
+          "auroc": 0.9208333333333334,
+          "accuracy": 0.8529411764705882,
+          "fpr95": 0.4583333333333333,
+          "n_good": 24,
+          "n_bad": 10
+        },
+        "edge1": {
+          "auroc": 0.6140350877192983,
+          "accuracy": 0.5405405405405406,
+          "fpr95": 0.8421052631578947,
+          "n_good": 19,
+          "n_bad": 18
+        },
+        "edge2": {
+          "auroc": 0.7947368421052632,
+          "accuracy": 0.7586206896551724,
+          "fpr95": 1.0,
+          "n_good": 10,
+          "n_bad": 19
+        },
+        "edge3": {
+          "auroc": 0.8291666666666667,
+          "accuracy": 0.71875,
+          "fpr95": 0.6666666666666666,
+          "n_good": 12,
+          "n_bad": 20
+        },
+        "edge4": {
+          "auroc": 0.9666666666666667,
+          "accuracy": 0.9090909090909091,
+          "fpr95": 0.1,
+          "n_good": 30,
+          "n_bad": 3
+        }
+      }
+    },
+    {
+      "fold": 4,
+      "overall_auroc": 0.8844890741442466,
+      "overall_acc": 0.8377358490566038,
+      "overall_fpr95": 0.5114942528735632,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.9540229885057472,
+          "accuracy": 0.90625,
+          "fpr95": 0.10344827586206896,
+          "n_good": 29,
+          "n_bad": 3
+        },
+        "dist1": {
+          "auroc": 0.9428571428571428,
+          "accuracy": 0.8064516129032258,
+          "fpr95": 0.19047619047619047,
+          "n_good": 21,
+          "n_bad": 10
+        },
+        "dist6": {
+          "auroc": 0.9586206896551723,
+          "accuracy": 0.9411764705882353,
+          "fpr95": 0.1724137931034483,
+          "n_good": 29,
+          "n_bad": 5
+        },
+        "dots": {
+          "auroc": 0.9232954545454545,
+          "accuracy": 0.9302325581395349,
+          "fpr95": 0.8125,
+          "n_good": 32,
+          "n_bad": 11
+        },
+        "edge1": {
+          "auroc": 0.9523809523809524,
+          "accuracy": 0.9166666666666666,
+          "fpr95": 0.3333333333333333,
+          "n_good": 21,
+          "n_bad": 15
+        },
+        "edge2": {
+          "auroc": 0.8823529411764706,
+          "accuracy": 0.84375,
+          "fpr95": 0.8,
+          "n_good": 15,
+          "n_bad": 17
+        },
+        "edge3": {
+          "auroc": 0.6677631578947368,
+          "accuracy": 0.6571428571428571,
+          "fpr95": 0.9375,
+          "n_good": 16,
+          "n_bad": 19
+        },
+        "edge4": {
+          "auroc": 0.6611570247933884,
+          "accuracy": 0.5909090909090909,
+          "fpr95": 0.9090909090909091,
+          "n_good": 11,
+          "n_bad": 11
+        }
+      }
+    },
+    {
+      "fold": 5,
+      "overall_auroc": 0.9019199191612984,
+      "overall_acc": 0.8264150943396227,
+      "overall_fpr95": 0.3505747126436782,
+      "per_feature": {
+        "angle": {
+          "auroc": 0.575,
+          "accuracy": 0.8636363636363636,
+          "fpr95": 0.8,
+          "n_good": 20,
+          "n_bad": 2
+        },
+        "dist1": {
+          "auroc": 0.9285714285714286,
+          "accuracy": 0.8837209302325582,
+          "fpr95": 0.2571428571428571,
+          "n_good": 35,
+          "n_bad": 8
+        },
+        "dist6": {
+          "auroc": 0.993103448275862,
+          "accuracy": 0.9117647058823529,
+          "fpr95": 0.034482758620689655,
+          "n_good": 29,
+          "n_bad": 5
+        },
+        "dots": {
+          "auroc": 1.0,
+          "accuracy": 1.0,
+          "fpr95": 0.0,
+          "n_good": 17,
+          "n_bad": 11
+        },
+        "edge1": {
+          "auroc": 0.8849206349206349,
+          "accuracy": 0.75,
+          "fpr95": 0.6111111111111112,
+          "n_good": 18,
+          "n_bad": 14
+        },
+        "edge2": {
+          "auroc": 0.8400000000000001,
+          "accuracy": 0.7857142857142857,
+          "fpr95": 0.47058823529411764,
+          "n_good": 17,
+          "n_bad": 25
+        },
+        "edge3": {
+          "auroc": 0.7076023391812866,
+          "accuracy": 0.6486486486486487,
+          "fpr95": 0.8421052631578947,
+          "n_good": 19,
+          "n_bad": 18
+        },
+        "edge4": {
+          "auroc": 0.8223684210526315,
+          "accuracy": 0.8148148148148148,
+          "fpr95": 0.9473684210526315,
+          "n_good": 19,
+          "n_bad": 8
+        }
+      }
+    }
+  ]
+}

results/cv_lambda_ablation/cv_lambda0.05/fold1/metrics_fold1.json

@@ -0,0 +1,63 @@
+{
+  "overall_auroc": 0.8666291854072963,
+  "overall_acc": 0.793233082706767,
+  "overall_fpr95": 0.5229885057471264,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.859375,
+      "accuracy": 0.9411764705882353,
+      "fpr95": 0.25,
+      "n_good": 32,
+      "n_bad": 2
+    },
+    "dist1": {
+      "auroc": 0.9132231404958677,
+      "accuracy": 0.8181818181818182,
+      "fpr95": 0.5454545454545454,
+      "n_good": 22,
+      "n_bad": 11
+    },
+    "dist6": {
+      "auroc": 1.0,
+      "accuracy": 0.9285714285714286,
+      "fpr95": 0.0,
+      "n_good": 26,
+      "n_bad": 2
+    },
+    "dots": {
+      "auroc": 0.9722222222222223,
+      "accuracy": 0.9,
+      "fpr95": 0.3333333333333333,
+      "n_good": 18,
+      "n_bad": 12
+    },
+    "edge1": {
+      "auroc": 0.9727272727272727,
+      "accuracy": 0.8571428571428571,
+      "fpr95": 0.18181818181818182,
+      "n_good": 11,
+      "n_bad": 10
+    },
+    "edge2": {
+      "auroc": 0.8506944444444445,
+      "accuracy": 0.7352941176470589,
+      "fpr95": 0.3888888888888889,
+      "n_good": 18,
+      "n_bad": 16
+    },
+    "edge3": {
+      "auroc": 0.6925,
+      "accuracy": 0.675,
+      "fpr95": 0.9,
+      "n_good": 20,
+      "n_bad": 20
+    },
+    "edge4": {
+      "auroc": 0.6686159844054581,
+      "accuracy": 0.6304347826086957,
+      "fpr95": 0.9259259259259259,
+      "n_good": 27,
+      "n_bad": 19
+    }
+  }
+}

results/cv_lambda_ablation/cv_lambda0.05/fold1/scores_fold1.csv

@@ -0,0 +1,267 @@
+score,true_label,pred_label,template_id,feature_id
+0.0006390826008282613,1,1,0,4
+-0.00044002641225233605,0,0,2,0
+-0.0003330795781221231,1,0,2,7
+-0.00024947713944129674,1,0,0,1
+-0.0003164168243529261,1,0,0,5
+-0.00021180767711484819,1,0,0,5
+-0.0002760209469124669,0,0,1,6
+-0.0014407114585628725,0,0,2,0
+-5.884027341380646e-05,1,0,0,5
+-0.00011017597571481158,1,0,0,4
+3.3198305754919555e-06,1,1,1,6
+-0.0009195119392825281,0,0,0,1
+-0.00030202404013834894,0,0,1,6
+-0.0013335938687669118,0,0,2,0
+-0.00028458335640607277,0,0,0,4
+-0.00012313672777963802,0,0,0,4
+-0.0005247805663384508,0,0,0,5
+-0.0003851075257989573,0,0,1,6
+-0.0009427560915355572,0,0,2,2
+-0.0007443050332949491,0,0,0,1
+-0.0001716703322017562,0,0,0,4
+-0.0008398805523756898,0,0,2,2
+-0.0006011728174053132,0,0,2,3
+-0.0007678987726103511,0,0,2,2
+-0.0005092012518434774,0,0,0,4
+-0.0021971325573394984,0,0,0,1
+-0.0006415407100575973,0,0,0,5
+-0.0012359290593303741,0,0,0,4
+-0.0007343467383179817,0,0,0,5
+-0.00045424243900924876,0,0,2,7
+-0.0004074866464361532,1,0,1,6
+-0.0007836871515610258,0,0,0,1
+-0.0003822612966177981,0,0,0,5
+-0.0011539258377160876,0,0,2,3
+-0.0012850115762557835,0,0,2,0
+-0.0005388831245363677,0,0,2,7
+-0.0008414555212948489,0,0,0,5
+-0.0014058796482277099,0,0,2,2
+-0.0007633358499151664,1,0,1,6
+-0.0006363140800385737,0,0,2,2
+0.00030853329692035955,1,1,0,4
+-3.561127610737444e-05,1,0,1,6
+-0.00010223738267086443,1,0,2,7
+-0.00031951049808412804,1,0,0,1
+5.147959687747117e-05,0,1,1,6
+-0.0007413147820625457,0,0,2,0
+-0.00036254604230634944,0,0,2,7
+-0.00031118918355787194,1,0,0,5
+-0.00038629112415947044,0,0,2,0
+-8.233200176618969e-05,0,0,2,7
+0.00024651033279951724,1,1,0,1
+0.0005066039852681566,1,1,2,3
+0.0006440523389028372,1,1,0,5
+0.0007821594475535664,1,1,1,6
+0.0004473076850990765,1,1,2,7
+0.00012116914993384829,1,1,2,0
+0.00022727611096343137,1,1,0,4
+-0.00012091801792848793,0,0,1,6
+-0.0008198335592169313,0,0,2,0
+-0.0010775742100668142,0,0,2,0
+0.0001143509009853003,1,1,2,3
+-5.5091188114600453e-05,0,0,1,6
+5.5902717576827854e-05,1,1,2,3
+0.0001863404366304177,1,1,0,1
+-0.00023262716073077158,1,0,2,7
+-9.705946431496151e-06,1,0,0,1
+0.0002628479761187918,1,1,0,4
+0.00022283606493147135,1,1,2,3
+0.00018308437312953077,1,1,2,7
+-0.0007612105883890762,0,0,2,0
+0.00029748731321888044,1,1,2,3
+-7.964225835166861e-05,1,0,2,7
+-0.00020583542238455238,1,0,2,2
+8.714013820281235e-05,0,1,2,7
+-6.562446942552907e-05,1,0,2,2
+0.0004374569343053719,1,1,0,1
+-0.0008831538620870556,1,0,2,0
+0.0002104708255501464,1,1,0,1
+-1.8377040687484608e-05,1,0,1,6
+-0.0013674413724220364,0,0,2,0
+0.00010975999030051788,1,1,0,5
+5.596355418674773e-05,1,1,0,1
+-0.0010243583843112004,0,0,2,3
+-0.0013615255491458839,0,0,2,0
+-0.00021068340021884215,1,0,0,5
+-0.0008522363251540818,0,0,2,2
+-0.0005733988570864323,0,0,0,5
+-0.0005683537293225526,0,0,2,7
+-0.0001345026717171991,1,0,1,6
+-0.00046684436762006905,0,0,2,2
+-0.00035526098014088386,0,0,2,3
+-0.0005299008268048067,0,0,0,5
+-0.0012571980469510899,0,0,2,0
+-2.4813156633171976e-05,0,0,0,1
+-0.0016907593453652225,0,0,2,0
+-0.0004679922433570016,1,0,0,1
+0.00027237885893555483,1,1,0,4
+0.000214186448138207,1,1,2,3
+-0.00013721456809435116,1,0,0,1
+-0.00010594252467853969,1,0,2,3
+0.0013023297087056546,1,1,0,5
+0.0020969598469673667,0,1,1,6
+0.0012580813109525479,1,1,2,3
+-0.0004502134968060994,0,0,2,0
+0.0008065964971319759,1,1,2,7
+0.0014283048611832792,1,1,2,7
+-0.0007861175769357936,0,0,2,2
+-0.0002504477716865957,1,0,2,7
+-0.0003747277884394856,0,0,0,1
+-0.00033106460556155126,1,0,1,6
+-0.00022910815197974346,1,0,2,7
+-0.0009876644817995838,0,0,2,2
+-0.0003538719264906832,0,0,2,7
+-0.0003841325399116611,0,0,0,5
+-0.0009677812588051891,0,0,2,0
+-0.0009121975989546628,0,0,2,0
+-0.00028102319047320495,1,0,2,7
+-0.000387378631276079,0,0,2,2
+-0.00030708730861078986,0,0,2,7
+-0.0011860289340256715,0,0,2,0
+-0.0005324924216256466,0,0,2,3
+-0.0005000421724980688,1,0,2,7
+-0.0012046753859613086,0,0,2,0
+-0.00027754589100368286,0,0,2,7
+-0.0011198317463276906,0,0,0,1
+-0.002588665836374276,0,0,0,1
+-0.0002652649828814909,0,0,0,5
+-0.0010298950754804546,0,0,2,3
+-0.00035884674784028917,0,0,2,7
+-0.00038338873157044845,0,0,0,1
+-9.602070203982346e-05,0,0,0,4
+-0.00020909611077513622,0,0,0,5
+-0.00021334405930247165,1,0,0,5
+-0.000148490069550462,0,0,0,1
+-7.54175597103314e-05,1,0,1,6
+-0.0001734860081342056,0,0,2,7
+-0.001067855587461964,0,0,2,0
+-0.00030525242647854584,1,0,2,7
+-0.0003541836646036231,1,0,0,5
+-0.00015187804645393037,0,0,1,6
+-6.767399900127133e-05,1,0,2,3
+-0.00042379023652756517,0,0,0,5
+-8.124260843033089e-05,0,0,1,6
+-8.213058375986307e-05,1,0,1,6
+-0.0005456837569363408,0,0,2,7
+-0.0009155377236311317,0,0,2,2
+5.915293091675322e-05,0,1,1,6
+0.00012919221480842732,0,1,0,4
+0.00014470199414063321,1,1,1,6
+-0.0004941877402598041,0,0,2,3
+0.0004460257536266013,1,1,0,5
+-0.0003897807150497097,0,0,1,6
+-0.001294816600566264,0,0,2,2
+-1.2613562867046235e-05,1,0,0,4
+-0.00029439972073305384,0,0,0,5
+-0.00011774132290156593,0,0,0,5
+-8.244898810516946e-05,0,0,1,6
+-0.0002039852988673374,0,0,1,6
+-0.0012264347649761486,0,0,2,2
+-0.005567185155523475,0,0,0,1
+-0.005940530906082132,0,0,0,1
+-0.0009611240142839948,0,0,2,3
+-0.0008214879274601114,1,0,2,7
+-0.0008433960084221308,0,0,2,3
+0.0007795636990340429,1,1,0,5
+0.0008495357737410801,1,1,1,6
+-0.0005297652143053718,0,0,2,7
+-0.001571227554522922,0,0,2,3
+-0.001965844005462713,0,0,2,0
+-0.003942087038303725,0,0,0,1
+0.0012316974916029705,1,1,1,6
+-0.0014323622468509697,0,0,2,2
+-0.002129913369426504,0,0,2,0
+-0.0002261513529811035,0,0,0,5
+-0.000498748092213645,0,0,2,3
+-0.006183094783336854,0,0,0,1
+-0.007699823032307901,0,0,0,1
+0.0009159848777926527,1,1,0,5
+-0.00037524226674577263,0,0,2,7
+-0.002058194498822559,0,0,2,0
+0.0011087304004467995,1,1,1,6
+-0.0014858417678624385,0,0,2,3
+-0.005702678468369412,0,0,0,1
+0.0010445749724749473,1,1,1,6
+-0.0007062995238811716,0,0,2,7
+-0.0017428531320183571,0,0,2,2
+0.000977495640981943,1,1,1,6
+0.0011114924016874285,1,1,0,5
+-0.0019774674368090925,0,0,2,2
+-0.006188091080985031,0,0,0,1
+-0.005799398684757762,0,0,0,1
+-0.0004847421104204816,0,0,2,7
+-0.0015348068243474684,0,0,2,2
+-0.002031265709374567,0,0,2,0
+-0.0038343406375497546,0,0,1,6
+-0.014438533524516968,0,0,2,3
+-0.004344149234821088,0,0,2,0
+-0.015587117671384477,0,0,2,3
+-0.008096875422634184,0,0,2,2
+-0.00732908623642288,0,0,0,5
+-0.011643355572305156,0,0,2,2
+-0.005631522535695693,0,0,2,0
+-0.005080208069412041,0,0,1,6
+-0.006107644619769412,0,0,2,7
+-0.019997057246509936,0,0,2,3
+-0.0033105995081132294,0,0,2,2
+-0.009168444273236674,0,0,2,3
+-0.0026958976656897005,0,0,2,0
+-0.0006597026795498104,0,0,2,7
+-0.0014589926972985272,0,0,2,2
+-0.0006860832998063417,1,0,2,7
+-0.0010741558857262148,0,0,0,4
+-0.005696439630992246,0,0,0,1
+-0.0007221443444723266,1,0,2,7
+0.0007234308900660839,1,1,1,6
+-0.001022414586332161,0,0,2,7
+-0.009544336893886795,0,0,2,3
+-0.0004591770959086715,0,0,2,7
+-0.0008631183250690788,1,0,2,3
+-0.0005211025613243679,0,0,0,4
+0.0009198654626379737,1,1,0,5
+-0.0008973048359621309,1,0,2,7
+-0.004306351178383922,1,0,0,1
+-0.0028136661814642146,0,0,2,0
+-0.0049611718097003175,0,0,0,1
+0.000258596264757216,1,1,0,5
+-0.003677721818385181,0,0,0,1
+-0.001199019025661982,0,0,2,7
+0.0006360087447683342,1,1,0,4
+-0.0010863685212098052,0,0,2,7
+-0.0029789017644361593,0,0,2,2
+-0.002112871935241855,0,0,2,0
+0.0005563400706159877,1,1,0,4
+-0.008254892533586827,0,0,2,3
+0.0008499947562813776,0,1,0,5
+-0.004495451553957537,0,0,0,1
+-0.0016987460752716285,0,0,2,0
+-0.00014359992579556988,0,0,1,6
+-0.0013777067625778718,0,0,2,7
+-0.0002170079489587813,0,0,1,6
+-0.00012864155491115574,0,0,0,4
+0.0005922206805553283,0,1,0,5
+0.0007244157209061082,1,1,1,6
+0.0009679432219127197,1,1,2,3
+-0.001749074467224998,0,0,2,7
+-0.0017463790922192836,1,0,2,7
+-0.004203430122579448,0,0,2,2
+0.0002414073050022135,1,1,1,6
+-0.00010259518341626875,0,0,1,6
+-0.0009294597007101398,0,0,0,4
+-0.0004400025148061093,0,0,0,5
+-0.0047648263053270065,0,0,2,2
+-0.0023456317154341363,0,0,2,0
+-0.0013146060833241803,0,0,2,7
+0.00042178427916951533,1,1,1,6
+-0.0006716428042273029,0,0,2,7
+0.0003093846773845055,0,1,2,0
+-0.0009554602947901003,0,0,2,2
+-0.0008500714419642508,1,0,2,7
+0.002535006673424503,0,1,1,6
+8.060552936513254e-06,0,1,2,7
+0.028782270190422427,1,1,2,3
+-0.0006472347333328771,0,0,2,2
+0.0023121894983341856,0,1,1,6
+-0.0007999811976333038,0,0,2,0
+0.0016659570459160028,1,1,0,4

results/cv_lambda_ablation/cv_lambda0.05/fold2/metrics_fold2.json

@@ -0,0 +1,63 @@
+{
+  "overall_auroc": 0.8549475262368815,
+  "overall_acc": 0.8082706766917294,
+  "overall_fpr95": 0.5287356321839081,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.5416666666666666,
+      "accuracy": 0.9,
+      "fpr95": 0.9722222222222222,
+      "n_good": 36,
+      "n_bad": 4
+    },
+    "dist1": {
+      "auroc": 0.8363636363636364,
+      "accuracy": 0.8518518518518519,
+      "fpr95": 0.3181818181818182,
+      "n_good": 22,
+      "n_bad": 5
+    },
+    "dist6": {
+      "auroc": 0.9766081871345029,
+      "accuracy": 0.7857142857142857,
+      "fpr95": 0.15789473684210525,
+      "n_good": 19,
+      "n_bad": 9
+    },
+    "dots": {
+      "auroc": 0.8900255754475703,
+      "accuracy": 0.875,
+      "fpr95": 0.7391304347826086,
+      "n_good": 23,
+      "n_bad": 17
+    },
+    "edge1": {
+      "auroc": 0.8629629629629629,
+      "accuracy": 0.8181818181818182,
+      "fpr95": 0.8666666666666667,
+      "n_good": 15,
+      "n_bad": 18
+    },
+    "edge2": {
+      "auroc": 0.8333333333333333,
+      "accuracy": 0.7352941176470589,
+      "fpr95": 0.4444444444444444,
+      "n_good": 18,
+      "n_bad": 16
+    },
+    "edge3": {
+      "auroc": 0.7773109243697478,
+      "accuracy": 0.6774193548387096,
+      "fpr95": 0.7058823529411765,
+      "n_good": 17,
+      "n_bad": 14
+    },
+    "edge4": {
+      "auroc": 0.7685185185185186,
+      "accuracy": 0.7878787878787878,
+      "fpr95": 0.7083333333333334,
+      "n_good": 24,
+      "n_bad": 9
+    }
+  }
+}

results/cv_lambda_ablation/cv_lambda0.05/fold2/scores_fold2.csv

@@ -0,0 +1,267 @@
+score,true_label,pred_label,template_id,feature_id
+0.0002858774273772728,1,1,2,3
+0.00025205227051628733,1,1,0,1
+0.000505572582478634,1,1,0,4
+-0.00028419564652722336,0,0,1,6
+-0.00019048673973884496,0,0,2,3
+-0.00017113556677941268,1,0,2,7
+0.000242993286228739,1,1,0,5
+1.4801546349189898e-05,0,1,1,6
+-0.0007262397310114459,0,0,2,3
+-5.646534118568594e-05,0,0,1,6
+0.00013025251857470659,1,1,0,5
+3.0945977778173636e-05,1,1,0,5
+-0.0016174358138232505,0,0,2,0
+-0.001238270547182764,0,0,2,3
+-0.0003688364112167629,0,0,0,4
+-0.0014724027918418866,0,0,0,1
+-0.0012700274097733194,0,0,0,1
+-0.0005970131955109533,0,0,2,7
+-0.00030067469459026976,0,0,0,4
+-0.0004922993687796404,0,0,0,5
+-0.000685895123460796,0,0,2,3
+-0.0003115013727801848,0,0,2,7
+-0.0016908470520866116,0,0,0,1
+0.00023734558955766302,0,1,1,6
+-0.00058970574173145,0,0,2,7
+-0.0014066845984780233,0,0,2,2
+-0.00040028718562098217,0,0,2,7
+-0.0006315223465207971,0,0,0,1
+5.660740687744685e-05,0,1,1,6
+-0.0017189145003794698,0,0,2,0
+-0.0010297131293918928,0,0,2,7
+-0.0004927293589571495,0,0,1,6
+-0.0011693307850509883,0,0,0,1
+-0.002175305494747589,0,0,2,2
+-0.0006867330236127595,1,0,0,4
+-0.003159510334080552,0,0,2,0
+-0.001870863382355311,0,0,2,2
+-0.0005086768139153722,0,0,0,4
+-0.0020181259268429123,0,0,2,0
+-0.003099774434813296,0,0,2,2
+-0.0011426471901359037,0,0,2,3
+-0.0021166467122384347,0,0,2,2
+-0.0006446229162975214,0,0,2,7
+-0.00037045168719487126,0,0,0,4
+-0.0010200991696910945,0,0,2,3
+-0.00151714674051618,0,0,0,1
+-0.0001821767221554197,1,0,0,5
+-0.0007445064501371236,0,0,2,3
+0.0005536808111355641,1,1,2,3
+-0.00026488162402529304,0,0,2,0
+0.00026294629409676345,1,1,2,3
+0.00039005821308819594,1,1,2,3
+-4.5350339787546615e-05,0,0,0,1
+-0.0004058351853745996,0,0,1,6
+-8.542195078916834e-05,0,0,2,7
+0.00010496080416487648,0,1,0,5
+-5.217358702793752e-05,0,0,0,5
+-0.00022836429969174885,0,0,1,6
+-0.00027015014464268523,0,0,0,4
+-0.00029676828795345556,0,0,2,7
+0.0008213070756755772,1,1,1,6
+3.185868030413947e-05,1,1,2,3
+0.0004344872577348724,1,1,0,4
+0.00026149483426706876,1,1,0,4
+0.000730810162494891,1,1,2,3
+-0.0006147581670666114,1,0,2,0
+0.0004735089678433724,1,1,0,1
+-0.0003318170018610546,1,0,2,2
+-0.000665436116978526,0,0,2,0
+0.0004709869163343674,0,1,0,5
+0.00018140581552870613,1,1,2,7
+0.00021722115547163565,1,1,0,4
+-0.0010226055438397452,0,0,2,0
+7.304514292628961e-07,0,1,0,1
+-8.256916131358684e-05,1,0,0,5
+-0.00041893858491675977,0,0,2,3
+0.00032023196312365963,1,1,0,4
+0.0003998288256116221,1,1,0,5
+-0.003276990769954864,0,0,2,0
+-0.0012183977122185753,0,0,2,3
+0.00014223335776478163,1,1,0,4
+-0.002146658649144229,0,0,2,0
+0.0008000407909275961,1,1,0,5
+0.0005168871578644027,1,1,1,6
+0.0005813977692741903,1,1,0,5
+-0.0003238960556336684,1,0,2,2
+0.0004293924852390779,1,1,2,7
+-0.00014428420283366106,1,0,2,2
+0.0004916042304830628,1,1,2,7
+-0.001698499468620867,1,0,2,0
+-0.0001108434054185644,0,0,0,5
+-0.0015226122108288123,0,0,2,0
+-0.00026349108316935533,0,0,2,3
+-0.0002689156038104563,0,0,2,3
+-0.00033801844198023956,1,0,2,7
+-0.00129347850830527,0,0,2,2
+0.00013639574986882598,1,1,0,4
+2.7712491282727222e-05,1,1,0,5
+0.000248964352358598,1,1,0,4
+0.0001308288366999484,1,1,1,6
+-0.0009091840431210592,1,0,2,2
+-0.0008678578006220058,1,0,2,2
+0.0006533888779813434,1,1,2,2
+0.0006225360650569198,0,1,2,7
+-0.00011295742529910183,0,0,0,1
+0.0003655047985375859,1,1,2,3
+0.0010870866282493806,0,1,0,5
+0.0011600516369799137,1,1,2,3
+0.00024164157919585698,1,1,2,2
+-0.0002270376004162248,1,0,0,4
+0.000774616033886559,1,1,1,6
+-0.0016651601076591756,0,0,2,0
+-0.0008162561670178557,1,0,0,1
+-0.00021496567176654963,1,0,0,4
+-0.0017483882664237173,0,0,2,0
+-0.0017825659163645466,0,0,2,2
+-0.0005052297224756343,0,0,2,3
+-0.00048529941443120964,0,0,0,5
+-0.00014143979991786126,0,0,0,5
+-0.0002569993314682512,1,0,0,5
+-0.0012655915686627852,0,0,2,0
+-0.00025237483670935076,1,0,1,6
+-0.00016143763496074756,0,0,0,4
+-7.415189058519907e-05,1,0,2,3
+-0.00014459225436439654,0,0,0,5
+4.481296637095372e-05,1,1,2,3
+-0.000951901196676771,0,0,2,0
+-0.00014006806071847552,1,0,0,5
+-1.7918349476531495e-05,1,0,0,5
+-0.00047083140612812734,0,0,2,7
+-0.0011488119533169076,0,0,2,2
+-0.0005120001087198043,0,0,2,3
+-0.0004664090546430081,0,0,2,7
+-0.00025988490204326836,0,0,0,4
+-9.535845223581502e-05,1,0,0,5
+-0.0007367596004041847,0,0,2,3
+-0.000423447670473252,0,0,0,1
+-0.0003923220571596178,1,0,0,1
+-0.0002822200846276249,1,0,0,4
+-0.0006864840316120543,0,0,2,3
+-0.0014941369948792268,0,0,2,0
+-0.0008948561319266444,0,0,2,2
+-0.00040234190499177126,0,0,2,3
+-0.0002297031899797736,0,0,2,7
+-0.0005199506660574105,1,0,0,1
+-0.00024866308085620334,1,0,0,4
+-0.00029687442322028773,0,0,0,5
+-3.777793084736719e-05,0,0,1,6
+0.00021158299961825805,1,1,1,6
+-0.000566405250574463,1,0,0,4
+-0.00012616486113984152,0,0,1,6
+-0.0011897482236963696,1,0,2,2
+0.0008126211230410248,1,1,0,4
+-0.0003247578424634412,1,0,2,7
+1.3555079931393266e-05,0,1,1,6
+-0.0015117874686256996,0,0,2,0
+-0.000288866134942509,0,0,0,5
+-0.0009708863971172833,0,0,2,2
+-0.00018852854991564545,1,0,2,3
+-0.000275995220290497,0,0,2,7
+-0.0003591042413609094,0,0,0,1
+-0.0005556102134869432,0,0,2,7
+-0.0005360437167109908,0,0,2,7
+-0.0002596640694537203,0,0,0,1
+0.00016187272180104642,1,1,2,3
+-0.00019436541100731168,1,0,2,7
+-0.0021109111612895495,0,0,2,0
+-0.0008783161352039319,1,0,2,7
+-0.0021384402934927493,0,0,2,0
+-0.0021320403780555348,0,0,2,2
+-0.0009808520606020485,0,0,0,1
+4.6692071191501566e-05,1,1,0,4
+-0.0020945353212300684,0,0,2,0
+-0.0005916289627202788,0,0,2,7
+-8.39605473447599e-05,1,0,1,6
+-0.003227644796425011,0,0,2,0
+-0.00042767671082401607,0,0,0,5
+-0.0013154941087123,0,0,2,3
+0.0007729136178386414,1,1,1,6
+-0.0006114134515519277,1,0,2,7
+-0.002236079045105725,0,0,2,0
+-0.0005891028788755654,0,0,0,4
+-0.0007861893021618022,0,0,2,7
+-0.0004504859514418043,0,0,0,4
+-0.005071818614378573,0,0,0,1
+-0.002534992754226551,0,0,2,0
+-0.0023952942105825055,0,0,0,1
+-0.002503658420173452,0,0,2,2
+-0.0034173008459038104,0,0,2,0
+-0.0006704460774199089,0,0,0,4
+-0.0006119966355618081,0,0,2,7
+-0.0006811161487712512,0,0,0,4
+0.0006661051564151414,1,1,0,5
+-0.0012027562598814247,1,0,2,3
+-0.002566324942163193,0,0,2,2
+0.0008600495391874567,1,1,1,6
+-0.0016606195038184543,1,0,2,3
+1.1176541738676382e-05,1,1,0,4
+-0.0009752826177282271,0,0,2,7
+-0.004015706017089542,0,0,2,0
+-0.00016972814657492674,0,0,0,4
+-0.02349567758908961,0,0,0,1
+-0.0027306199865415673,0,0,0,5
+-0.005907094153226353,0,0,2,3
+-0.003520188639231491,0,0,2,7
+-0.041419980940409,0,0,0,1
+-0.004709196958283431,0,0,0,5
+-0.005332385158399115,0,0,0,5
+-0.006461830211628692,0,0,2,3
+-0.002507303401071111,0,0,2,7
+-0.005193703619879669,0,0,0,5
+-0.004018185397144401,0,0,1,6
+-0.002994578304351306,0,0,2,7
+-0.008894678211072461,0,0,2,3
+-0.00882064116944093,0,0,0,4
+-0.004527570436766833,0,0,1,6
+-0.002136917603202164,0,0,2,0
+0.0007429039152339091,1,1,1,6
+-0.0027566861032391893,0,0,2,0
+-0.002655063771235292,0,0,2,3
+-0.004025859042885714,0,0,2,0
+0.0007199369711452155,1,1,1,6
+-0.0015740561668644674,0,0,0,4
+-0.0039557636436074965,0,0,2,0
+-0.0033572955496492807,0,0,2,0
+-0.005275035949016456,0,0,2,2
+-0.00401130453683436,0,0,2,0
+0.00022741690219845595,0,1,1,6
+6.877037230876987e-07,1,1,1,6
+-0.006680000452033711,0,0,2,2
+-0.0011800479498924668,0,0,0,1
+0.0009034001908730695,1,1,1,6
+-0.005139119378000032,0,0,2,0
+-0.006389884612872269,0,0,0,1
+-0.002858506251941435,0,0,2,3
+-0.0059908103483030575,0,0,2,2
+-0.0032046487511252046,0,0,2,0
+-0.003835251008858903,0,0,2,3
+0.00020664341456722544,1,1,2,2
+-0.00013345870800549227,0,0,0,5
+-0.003958193633006885,0,0,2,0
+-0.0020943595579592487,0,0,0,1
+-7.698314788285622e-05,1,0,1,6
+-0.00023954986914759453,0,0,1,6
+-0.0018282667032326581,0,0,2,7
+-0.0004968874301994226,0,0,0,5
+-0.006879585503484123,0,0,2,2
+-0.004486457638267893,0,0,0,1
+-0.0026195378368720403,1,0,2,0
+-0.017050932667334553,0,0,2,0
+0.0006160895281936973,1,1,0,4
+0.0016244963504141174,1,1,0,5
+0.00023748633917421158,0,1,1,6
+-0.0013758226498612192,0,0,2,7
+-0.0009796174618531897,1,0,2,3
+0.0008064443321200016,1,1,2,3
+0.0014563254214590426,0,1,1,6
+-0.017098862704297058,0,0,2,2
+-0.01198575390852056,0,0,0,1
+0.0012725707108620504,1,1,2,3
+-0.0149393224119558,0,0,2,2
+-0.0114587046491215,0,0,2,0
+-0.011830310032528366,0,0,2,0
+-5.379310867283105e-05,0,0,0,4
+0.002180868609575559,1,1,0,5
+-0.012397436225437555,1,0,2,0

results/cv_lambda_ablation/cv_lambda0.05/fold3/metrics_fold3.json

@@ -0,0 +1,63 @@
+{
+  "overall_auroc": 0.8269665745162101,
+  "overall_acc": 0.769811320754717,
+  "overall_fpr95": 0.9364161849710982,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.7589285714285714,
+      "accuracy": 0.875,
+      "fpr95": 0.8928571428571429,
+      "n_good": 28,
+      "n_bad": 4
+    },
+    "dist1": {
+      "auroc": 0.7928994082840237,
+      "accuracy": 0.6666666666666666,
+      "fpr95": 0.8846153846153846,
+      "n_good": 26,
+      "n_bad": 13
+    },
+    "dist6": {
+      "auroc": 0.8083333333333333,
+      "accuracy": 0.896551724137931,
+      "fpr95": 0.9583333333333334,
+      "n_good": 24,
+      "n_bad": 5
+    },
+    "dots": {
+      "auroc": 0.9208333333333334,
+      "accuracy": 0.8529411764705882,
+      "fpr95": 0.4583333333333333,
+      "n_good": 24,
+      "n_bad": 10
+    },
+    "edge1": {
+      "auroc": 0.6140350877192983,
+      "accuracy": 0.5405405405405406,
+      "fpr95": 0.8421052631578947,
+      "n_good": 19,
+      "n_bad": 18
+    },
+    "edge2": {
+      "auroc": 0.7947368421052632,
+      "accuracy": 0.7586206896551724,
+      "fpr95": 1.0,
+      "n_good": 10,
+      "n_bad": 19
+    },
+    "edge3": {
+      "auroc": 0.8291666666666667,
+      "accuracy": 0.71875,
+      "fpr95": 0.6666666666666666,
+      "n_good": 12,
+      "n_bad": 20
+    },
+    "edge4": {
+      "auroc": 0.9666666666666667,
+      "accuracy": 0.9090909090909091,
+      "fpr95": 0.1,
+      "n_good": 30,
+      "n_bad": 3
+    }
+  }
+}

results/cv_lambda_ablation/cv_lambda0.05/fold3/scores_fold3.csv

@@ -0,0 +1,266 @@
+score,true_label,pred_label,template_id,feature_id
+0.0009043245087377721,1,1,1,6
+6.0010402230545844e-05,1,1,2,7
+-0.0010380183256347673,0,0,2,0
+0.00015304316650144832,0,1,0,4
+-0.0018175912980223068,0,0,2,0
+-0.00016759593156166416,1,0,1,6
+-0.0028032707769307316,0,0,2,0
+-0.0022080845569144025,0,0,2,2
+-0.0019354526486131361,0,0,2,0
+-0.0020633441291283815,0,0,2,2
+-8.943869150243762e-05,1,0,0,4
+-0.0017349624074995528,0,0,2,2
+-0.0017542890767799687,0,0,2,2
+-0.001352895937743597,0,0,2,2
+-0.001112143505888525,0,0,2,3
+-0.00042932059441227327,1,0,1,6
+-0.0012487586241331887,0,0,2,0
+-0.001432318809092976,0,0,2,2
+-0.00028268346271943107,1,0,0,5
+-0.00031976982776541277,0,0,0,4
+-0.00036160621675662696,0,0,0,4
+-0.0008235296461498367,0,0,2,3
+-0.0007592706507421106,0,0,2,3
+-0.00032492850325070363,0,0,1,6
+-0.0009791529446374625,0,0,2,3
+-0.00045868182380217996,0,0,2,7
+-0.0004150593251688404,0,0,2,7
+-0.0014040866505820298,0,0,0,1
+-0.0005005936918314538,0,0,2,7
+-0.00028412952873622943,0,0,2,7
+0.00019059291109442718,1,1,0,5
+0.0005534555879421529,1,1,0,4
+-0.00022799427271820612,1,0,0,1
+0.00040351532807108,1,1,2,3
+-0.00021874952421058018,0,0,1,6
+-0.0008495488873450086,1,0,0,1
+-0.00014983894478064023,1,0,0,1
+0.0010056022863136616,1,1,2,3
+2.2752399090678055e-05,0,1,0,1
+0.0001137167256092652,0,1,0,4
+2.8532699216158447e-05,0,1,2,7
+-0.00015848310984438452,0,0,2,7
+-0.0006443061470054103,0,0,2,0
+0.000245238948846236,0,1,0,4
+-0.0005058754474157462,0,0,0,5
+-0.00015498466498684114,0,0,2,7
+2.311187097802682e-05,1,1,0,1
+6.719356053508721e-05,1,1,1,6
+0.000568540272652171,1,1,2,3
+0.0001505953224841504,1,1,1,6
+-0.00137618312961422,0,0,2,2
+-0.00048344015318434533,0,0,2,3
+-0.0002403382770717142,1,0,0,1
+0.00011927999672479636,1,1,0,4
+-0.0016039203212130804,0,0,2,0
+-0.0012482479395112024,0,0,2,2
+-0.0004400042138877312,0,0,2,3
+-4.315176571253673e-05,1,0,1,6
+-0.00018055866414215353,1,0,0,1
+-0.0009169749548891548,1,0,2,0
+0.0004524600377771998,1,1,2,3
+-9.043686382938058e-05,1,0,2,0
+-0.00031906012562103736,0,0,0,1
+-0.001001813095936087,0,0,2,0
+-0.0004544963722582906,0,0,2,3
+-0.0003356474015163251,0,0,2,7
+-0.0003197878779610634,0,0,0,1
+-0.0014022815704811363,0,0,2,2
+-0.000294510285020806,0,0,2,7
+-0.0004179890191880996,1,0,0,1
+-0.00044830449478467857,0,0,2,3
+-0.0014144404523540287,0,0,2,2
+1.5749854501336674e-05,0,1,0,4
+-0.00020331451145466466,0,0,1,6
+-0.000294137395685538,1,0,0,4
+-0.00011102158750873198,0,0,0,5
+-0.0010327822319231936,0,0,2,2
+-0.0009645949662080988,1,0,0,1
+-7.779403065797004e-05,1,0,0,4
+-0.0007516805277555261,0,0,2,3
+-2.9691900708712568e-05,1,0,2,7
+-0.00025390040420461384,1,0,0,1
+6.91709067905319e-05,1,1,0,4
+0.0003792073595104764,0,1,0,5
+-2.4498772691004464e-05,1,0,1,6
+-0.0001441696588881311,1,0,2,3
+-9.531438292469802e-05,1,0,1,6
+0.0010116974316770212,1,1,0,5
+0.00014360182627569905,1,1,2,2
+0.0007159903342835588,1,1,2,7
+0.00015661206736694924,1,1,2,2
+-0.0009257304528728118,1,0,2,2
+-3.3972678356804925e-05,1,0,2,2
+-0.007782466386561285,0,0,0,1
+-0.0011925833730492743,0,0,2,0
+-0.0009260194690432393,0,0,2,2
+-0.00021595859347144086,0,0,2,7
+-0.0017865136434556915,0,0,2,0
+-0.000562641695432832,0,0,2,3
+-0.0015615348401479424,0,0,2,0
+-0.0004910559538984678,0,0,0,1
+-0.00016438892285805182,0,0,0,4
+-0.0002318013930926099,0,0,0,4
+-1.8714789766817452e-05,1,0,0,4
+-0.0011108607240021221,0,0,2,0
+-0.00014663963287602985,1,0,2,3
+-0.0002020504701067691,0,0,2,7
+-0.0009850645752158011,0,0,2,2
+0.00011777658364735613,1,1,0,4
+0.0004364312882535158,1,1,0,5
+0.0001793696457752965,1,1,1,6
+-0.0015450656137545593,0,0,2,0
+-0.0006395775673445313,0,0,0,1
+-0.0001661899912869559,0,0,0,4
+-0.00015584704553475628,0,0,2,7
+-0.0009904156255652209,0,0,2,2
+-0.00019922537030652235,0,0,1,6
+6.422317237592817e-06,1,1,0,5
+-0.0008710670459549895,1,0,0,4
+-9.703091927804053e-05,1,0,1,6
+-0.0019813950976822516,0,0,2,2
+-0.0007373889337759475,1,0,0,1
+-0.0006542277312837543,0,0,2,3
+-0.00026458904787432513,0,0,0,5
+-0.0011063553547137426,0,0,2,0
+-0.0011126715363934635,0,0,2,3
+-0.00045639346237294487,0,0,0,1
+-0.0009096846979809933,1,0,0,1
+-0.00028877594973891925,1,0,0,4
+-0.0002944076206767947,1,0,0,5
+0.0004178792232414699,1,1,0,4
+-0.0004787614021915954,0,0,0,1
+-2.1410864428616713e-05,1,0,0,4
+-6.544737552758272e-05,1,0,2,3
+-0.0002660028886748477,0,0,2,7
+-0.0012265470105921848,0,0,2,0
+-0.00030045930470805685,0,0,0,5
+8.268975710962016e-05,0,1,1,6
+-0.0007712221750989547,0,0,0,1
+-0.000890471382881515,0,0,2,0
+-0.0002053588512353592,0,0,0,4
+-0.00013153542357031197,0,0,1,6
+-0.0015614039794309073,0,0,2,2
+0.00011978495458606596,1,1,0,5
+-0.0008106300374493003,0,0,2,3
+-0.0011104726657504213,0,0,0,1
+-0.0010326514113694445,1,0,0,1
+-0.001670107030076905,0,0,0,1
+-0.0003029922128189363,1,0,0,4
+0.00011752975929994226,1,1,1,6
+-0.0004707502175006085,0,0,2,7
+-0.0002506886713672437,1,0,1,6
+-0.0015340215055039165,0,0,2,0
+-0.000407141193281859,0,0,0,4
+0.000874669352779165,1,1,0,5
+-0.0015981436672154806,0,0,2,7
+-0.0038572265068069107,0,0,2,0
+0.0008187528344569713,1,1,0,5
+-0.007530791168101134,0,0,0,1
+0.0011458353104535461,1,1,1,6
+-0.0009925521502736952,0,0,2,3
+0.0010559289070079109,1,1,0,5
+0.00010737347009125847,1,1,0,4
+-0.002252668473520316,0,0,2,3
+0.0012152654270175846,1,1,1,6
+-0.0009600756253348684,0,0,2,7
+-0.025359427636140026,1,0,2,2
+-0.006289657179731876,0,0,0,1
+0.0002279454399831595,0,1,0,5
+-0.00025876688363496436,0,0,2,7
+-0.0006739897909574209,0,0,0,4
+-0.005803382802987473,0,0,0,1
+-0.001616176392417401,0,0,2,7
+-0.00361384257732425,0,0,2,0
+-0.0024248655658448103,0,0,2,7
+-0.00507755488913972,0,0,2,2
+-0.003083301149890758,0,0,2,0
+-0.006832074269186705,0,0,0,1
+0.0009677678614389146,1,1,1,6
+-0.0028988665586803103,0,0,2,3
+-0.015307916146120995,1,0,0,4
+-0.010257453349186107,0,0,2,7
+-0.012867651184205898,0,0,2,0
+-0.007044544313685043,0,0,1,6
+-0.0211847812513588,0,0,2,2
+-0.040212329854257406,0,0,0,4
+-0.08736759047373197,0,0,0,1
+-0.013538229113328266,0,0,2,7
+-0.020508698055054994,0,0,2,0
+-0.039326601448119614,0,0,0,4
+-0.016371022171806544,0,0,2,0
+-0.052726131200324745,0,0,0,4
+-0.014004108197987083,0,0,2,3
+-0.005171574045089073,0,0,2,0
+-0.09676741635426878,0,0,0,1
+-0.004057700725097674,0,0,2,7
+-0.02247767376538832,0,0,2,2
+-0.006075580813339913,0,0,2,0
+-0.03861521292536054,1,0,0,4
+-0.07262391690688673,0,0,0,1
+-0.0221713494771393,1,0,0,5
+-0.007809590724064036,1,0,2,0
+-0.012839769955608064,0,0,2,2
+-0.0009630319254938513,1,0,2,3
+0.001100802720175124,1,1,0,5
+-0.0010746141726849616,0,0,2,3
+0.0005369872087612748,1,1,0,5
+0.0009300849959254284,1,1,1,6
+-0.008093840163783169,0,0,0,1
+-0.0005441200203495089,0,0,0,4
+0.0013698645203839982,1,1,0,5
+-0.0009418392600491661,0,0,2,3
+-0.0045051476935623214,0,0,2,2
+-0.008723567663109864,0,0,0,1
+-0.013060941012809052,0,0,2,2
+-0.009939225036068821,0,0,0,1
+-0.008142758496105674,0,0,0,1
+0.0010083944204961857,1,1,1,6
+-0.0007675913057755661,1,0,2,3
+-0.007630521769169719,0,0,2,3
+0.0005868926434777662,1,1,0,5
+9.109900158364367e-05,1,1,1,6
+-0.005047154233325274,0,0,0,1
+-0.003413755583460444,0,0,2,0
+0.00016593653126619617,1,1,0,4
+-0.006428414165857248,0,0,0,1
+0.00024859055760316495,0,1,0,5
+-0.0023954820737708375,0,0,2,7
+-0.011075233469600789,0,0,2,2
+-0.00806931888393592,0,0,2,3
+0.0002673040586523706,1,1,1,6
+-0.0015135666262358453,0,0,2,7
+-0.00692284129210748,0,0,2,3
+-0.00034287053800653375,0,0,0,5
+-0.0004757706410600804,1,0,2,0
+-0.0002610715088667351,0,0,1,6
+-0.002108277945662849,0,0,2,7
+-0.009691069633117878,0,0,0,1
+-0.00013382073142565593,0,0,0,5
+-0.00023481929791159886,0,0,1,6
+-0.007659070478985086,0,0,2,3
+-0.002778772329329512,0,0,2,0
+-0.00017996569105889482,1,0,0,5
+-0.0017305041005602102,0,0,2,7
+-0.0003348660183837629,0,0,0,5
+-0.0004380778729682795,0,0,1,6
+-0.0009140989993466041,0,0,0,4
+-0.0024547721917042506,0,0,2,7
+-0.003893333763116971,0,0,2,0
+-0.00018273042631335515,0,0,1,6
+-0.006852644738391971,0,0,2,3
+-0.00013791893958114282,1,0,0,4
+0.00021729516563936957,1,1,0,5
+-0.015021401857957244,1,0,0,1
+-0.037133874238934376,0,0,2,2
+0.0009041214588796712,1,1,0,5
+-0.00012221461103763265,0,0,2,7
+0.0012198452837765272,0,1,0,4
+0.0020492811960866678,1,1,0,5
+0.001161086819483896,0,1,1,6
+5.424634262453376e-05,0,1,2,7
+0.002264821233111433,0,1,0,4
+-4.708849010062743e-06,0,0,2,7
+0.0018641521339304717,1,1,2,3
+0.0020381817565066768,1,1,1,6

results/cv_lambda_ablation/cv_lambda0.05/fold4/metrics_fold4.json

@@ -0,0 +1,63 @@
+{
+  "overall_auroc": 0.8844890741442466,
+  "overall_acc": 0.8377358490566038,
+  "overall_fpr95": 0.5114942528735632,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.9540229885057472,
+      "accuracy": 0.90625,
+      "fpr95": 0.10344827586206896,
+      "n_good": 29,
+      "n_bad": 3
+    },
+    "dist1": {
+      "auroc": 0.9428571428571428,
+      "accuracy": 0.8064516129032258,
+      "fpr95": 0.19047619047619047,
+      "n_good": 21,
+      "n_bad": 10
+    },
+    "dist6": {
+      "auroc": 0.9586206896551723,
+      "accuracy": 0.9411764705882353,
+      "fpr95": 0.1724137931034483,
+      "n_good": 29,
+      "n_bad": 5
+    },
+    "dots": {
+      "auroc": 0.9232954545454545,
+      "accuracy": 0.9302325581395349,
+      "fpr95": 0.8125,
+      "n_good": 32,
+      "n_bad": 11
+    },
+    "edge1": {
+      "auroc": 0.9523809523809524,
+      "accuracy": 0.9166666666666666,
+      "fpr95": 0.3333333333333333,
+      "n_good": 21,
+      "n_bad": 15
+    },
+    "edge2": {
+      "auroc": 0.8823529411764706,
+      "accuracy": 0.84375,
+      "fpr95": 0.8,
+      "n_good": 15,
+      "n_bad": 17
+    },
+    "edge3": {
+      "auroc": 0.6677631578947368,
+      "accuracy": 0.6571428571428571,
+      "fpr95": 0.9375,
+      "n_good": 16,
+      "n_bad": 19
+    },
+    "edge4": {
+      "auroc": 0.6611570247933884,
+      "accuracy": 0.5909090909090909,
+      "fpr95": 0.9090909090909091,
+      "n_good": 11,
+      "n_bad": 11
+    }
+  }
+}

results/cv_lambda_ablation/cv_lambda0.05/fold4/scores_fold4.csv

@@ -0,0 +1,266 @@
+score,true_label,pred_label,template_id,feature_id
+-0.0001241526560625053,1,0,0,1
+0.0006979003112064681,1,1,0,5
+-0.001510755410417914,0,0,2,0
+-0.0003914629449718633,0,0,2,3
+-4.047130903927533e-05,0,0,0,5
+-0.00012341579771600782,1,0,1,6
+0.00039856291085015906,1,1,2,3
+-0.0005687012246926321,0,0,2,7
+-0.0010730925391544584,0,0,2,3
+-0.00018531693232944242,0,0,0,1
+-0.0008391714360914213,0,0,2,3
+-0.0005126046348595988,0,0,0,5
+-0.0013871684917830859,0,0,2,0
+-0.001166199621220585,0,0,2,3
+-0.0002709650789620346,0,0,0,4
+-0.0016300066548865285,0,0,2,0
+-0.0015294135257136084,0,0,2,3
+-0.0015427742310566827,0,0,2,0
+-0.0004373808327363801,0,0,2,3
+-0.000585379409894813,0,0,2,7
+-0.00025345235189888626,0,0,0,1
+-0.0002452003088546928,0,0,0,5
+-0.0014995300219743518,0,0,2,0
+-0.0012684129647095692,0,0,2,2
+-0.0014796434869640517,0,0,2,3
+-0.0005611860554199689,0,0,0,4
+-0.001002647365094162,0,0,2,0
+-0.0007132951568928556,0,0,2,2
+-0.00010572974395472653,0,0,0,5
+-0.001282316862198058,0,0,0,1
+-0.00045740739442408015,0,0,1,6
+-0.001690604579634965,0,0,2,0
+-0.0006634236456011435,0,0,2,7
+-0.00048823376826476195,1,0,1,6
+-0.00034224881615955473,1,0,1,6
+-0.001238398708810564,0,0,0,1
+-0.0006142172607360406,0,0,0,4
+-0.0007726463992730713,0,0,2,3
+-0.00024716318526770906,0,0,0,4
+-0.0006321554255555384,0,0,2,3
+7.777151127811525e-05,1,1,0,1
+0.00024355941743124318,1,1,2,3
+-0.00016418578394222956,0,0,1,6
+0.00024546532513341004,1,1,0,1
+0.00012146187189500737,0,1,1,6
+-2.8728442848660087e-05,1,0,2,0
+0.0006274453131482009,1,1,1,6
+0.0005540964455576607,1,1,0,5
+0.0005862516837078155,1,1,2,7
+-0.001048234429617878,0,0,2,0
+-0.00104086271574488,0,0,2,0
+5.3873059514444877e-05,1,1,0,4
+-0.0007940081617562105,0,0,2,0
+0.00024243404594017238,1,1,2,3
+-0.0003157018148340282,0,0,1,6
+0.00033150614443002227,1,1,2,3
+-0.0005845629010582335,0,0,0,1
+-0.00013058843120234039,0,0,2,2
+-4.569445154629578e-06,0,0,0,5
+-0.0013001405121758573,0,0,2,0
+-5.109045014250677e-06,0,0,2,2
+-7.359109062235816e-05,0,0,2,7
+-0.00029753197974059786,0,0,0,1
+-0.0012634268612600856,0,0,2,0
+0.00035460198996588657,1,1,2,2
+0.0001517431045067498,1,1,0,1
+0.0007473041670164101,0,1,1,6
+-3.395759646082283e-05,1,0,2,2
+-4.162988974712803e-05,0,0,2,7
+0.000396050244453364,0,1,0,5
+-0.0009092673717532307,1,0,2,0
+0.0003430631215451275,1,1,2,2
+5.992749443976212e-05,0,1,2,7
+0.0003745807154336943,0,1,1,6
+0.00027081523934611995,1,1,0,5
+-0.0004442312251194385,0,0,2,3
+-0.00045865716936532505,0,0,2,3
+-0.00011288947775028868,1,0,2,7
+7.312217087019311e-05,1,1,1,6
+-0.0003010761845507661,1,0,2,0
+0.0007434381564962698,1,1,2,3
+-0.00018031468585832065,1,0,0,1
+4.3803963344545005e-05,1,1,0,4
+-0.00039315924921538674,0,0,2,3
+-0.00026047265797387795,1,0,2,7
+-0.00022932292398763782,0,0,2,3
+-0.0004040857614018029,1,0,0,1
+-2.1899364073761293e-05,0,0,0,5
+-0.0012614812297397282,0,0,2,0
+-0.0004412712459452451,1,0,1,6
+-0.0009155550674768169,0,0,2,0
+-0.0005055328705930139,0,0,2,7
+0.00023926298163132707,0,1,0,5
+0.0001961775065865371,1,1,1,6
+0.00017712084925733396,1,1,1,6
+-0.0006887570235994643,0,0,2,2
+-0.0008639179068268277,0,0,2,3
+-0.0011743089088122362,0,0,2,0
+-0.00042856783140450723,1,0,2,2
+-1.332894549705213e-05,1,0,2,3
+-0.00015246322320308474,1,0,2,7
+4.781118128448659e-05,1,1,2,2
+0.00013178261142456765,1,1,0,1
+8.42081499285996e-05,1,1,1,6
+-0.00036394299560925034,1,0,0,1
+0.0003417403242201543,1,1,0,5
+-0.0009671052329940713,0,0,2,3
+0.0005620523914694787,1,1,0,4
+-0.0008683349555940363,0,0,2,3
+-0.00026666240009944747,1,0,0,1
+-0.00011654298868961702,0,0,1,6
+-5.052791268099141e-05,1,0,0,4
+-0.0004086681004264389,0,0,0,1
+-0.00040041910076979523,0,0,2,7
+-0.00047328174434369506,0,0,0,1
+-0.00015126827725907808,0,0,0,4
+-0.0001936484628822649,0,0,1,6
+-0.00024400389287620831,0,0,0,1
+-0.0002170723804738369,0,0,1,6
+-0.00040632691467180904,0,0,2,2
+-0.0002926898683654148,1,0,2,3
+-0.00013095682137645795,1,0,1,6
+-0.00015849845949560473,0,0,0,4
+2.3326856025960782e-05,1,1,0,5
+-3.364805743331055e-05,0,0,0,4
+-4.6501494944095265e-05,0,0,0,5
+-0.0005994424183154478,0,0,2,2
+-5.353060725610606e-05,0,0,0,4
+-0.0004628825659165163,0,0,0,1
+-0.00016416182363172913,0,0,0,4
+1.2401718413455887e-05,0,1,0,5
+-0.0008070929718087426,0,0,2,0
+-0.0004557263597962432,0,0,2,2
+-0.0005331595643656324,0,0,2,2
+0.00016598419169895455,1,1,0,5
+4.8542089643888175e-05,1,1,2,7
+6.562104681506406e-06,1,1,0,5
+-2.311419433681343e-05,1,0,2,7
+-0.0017050004369230012,0,0,2,0
+-0.00024086581834126264,1,0,0,1
+6.229871418327038e-05,1,1,0,4
+-7.031424844171873e-05,1,0,1,6
+-0.0001867131641483874,1,0,0,4
+-0.0012677301911753602,0,0,2,0
+-0.0003911751002306108,0,0,1,6
+-0.0019352303599589511,0,0,2,0
+-0.000781409824849106,0,0,2,3
+-0.0006009387591620905,0,0,2,3
+-0.0002529886108823126,1,0,2,7
+0.0004278110273298816,1,1,0,5
+0.0007115491668810113,1,1,1,6
+0.00019233255414292244,1,1,0,4
+0.00014764171442948303,1,1,2,3
+-3.457334823906474e-05,0,0,0,5
+-0.00024046985083259675,0,0,2,2
+-0.00134943872311851,0,0,2,0
+-0.0006216245939140198,0,0,0,1
+-0.00029042854439467226,0,0,2,2
+-0.0010225098073715338,0,0,0,1
+-0.0016550225601531573,0,0,2,0
+-0.0011640409589745108,0,0,2,2
+4.951875249389545e-05,1,1,0,5
+0.0008706673578126357,1,1,1,6
+-4.220256203552736e-05,1,0,0,5
+-0.0012829062252421863,0,0,2,0
+-0.0018399000787758246,0,0,2,0
+-0.0008351438725367182,0,0,2,2
+-0.0004734604639816103,0,0,2,2
+-0.0002567445524618954,0,0,2,7
+-0.00018595163652207654,0,0,0,5
+-0.0011383439681958412,0,0,2,2
+-0.0008293920784490181,0,0,2,3
+-0.0006094211593153896,0,0,2,7
+-0.0005258018989115956,0,0,0,4
+-0.0037209526402875778,0,0,2,3
+-0.0011078453541267658,0,0,0,4
+-0.001945319802907762,0,0,2,2
+0.0007232695585116748,1,1,0,5
+-0.0017770322452997783,0,0,2,0
+-0.00164270433422644,0,0,0,4
+0.0008070782740833247,1,1,1,6
+-0.003644230457430238,0,0,2,3
+-0.00015598298690747484,0,0,0,4
+-0.009355056303902529,0,0,2,3
+0.000885770574968775,1,1,1,6
+-0.0012022705090930681,0,0,2,2
+-0.0018328633703640664,0,0,2,2
+-0.00020849384833127385,1,0,0,4
+0.0010666820878395787,1,1,0,5
+-0.0078022379268077174,0,0,2,3
+-0.0024226626101881248,0,0,2,0
+-0.00012553558743092982,0,0,0,4
+-0.007566598901757971,1,0,2,3
+0.0007349778595380479,1,1,1,6
+-0.0037162502115825197,0,0,2,2
+-0.03481162868061801,0,0,0,1
+-0.005433259845012799,0,0,0,5
+-0.0024051036260789235,0,0,0,4
+-0.003207403291307854,0,0,2,0
+-0.006561173995723948,0,0,2,2
+-0.03639969390816987,0,0,0,1
+-0.017638327094027774,0,0,1,6
+-0.04415706462459638,0,0,0,1
+-0.003361888235667722,0,0,0,4
+-0.005228867884143253,0,0,0,5
+-0.016095563597045844,1,0,1,6
+-0.009292330047755967,0,0,2,3
+-0.0013085329334717247,0,0,0,4
+-0.00552870318701025,0,0,0,1
+0.0008560478314757376,1,1,0,5
+0.0007284386787796381,1,1,1,6
+0.0007888155162800101,1,1,0,5
+-0.0011885927195544356,0,0,2,2
+-0.0017839998082490646,0,0,2,0
+-0.0016104306565830478,0,0,2,0
+0.001660000926640353,1,1,0,4
+0.00047903399448841893,1,1,0,5
+-1.9638460362330495e-05,0,0,1,6
+-0.0017999998590676122,0,0,2,7
+1.5010946954135962e-05,1,1,0,4
+-0.0046382284973515205,0,0,2,2
+0.0005061372381169361,1,1,0,5
+-0.011543990187346934,0,0,2,3
+-0.002075695011299103,0,0,0,1
+-0.003557550809928216,0,0,2,2
+-0.011224806676036677,0,0,2,3
+0.000687075409805402,1,1,0,4
+-0.0014207852116669521,1,0,2,7
+0.00027619569445960085,1,1,0,4
+-0.0009454989689402296,0,0,1,6
+-0.0005776990443700919,0,0,0,4
+-0.003899555483658331,0,0,0,1
+-0.003327923564647792,0,0,2,2
+-0.0013347141721169464,0,0,0,1
+0.000904344420705458,1,1,2,7
+-0.0015481066488428064,0,0,0,4
+-0.005440172598755453,0,0,2,3
+-0.00378474827914033,0,0,2,2
+-0.004827971785271074,0,0,2,3
+-0.010177299140486866,0,0,2,3
+-0.004301767990691589,0,0,2,2
+-0.00043714325816836264,1,0,0,5
+-0.004842867673723956,0,0,2,3
+-0.0066174036625307065,0,0,2,3
+-0.003984504274558275,0,0,0,1
+-0.00038162210723385187,0,0,1,6
+-0.006348670520819721,0,0,2,3
+-0.0034238003625068816,0,0,2,2
+-0.0012860309387906464,0,0,0,4
+-0.005398210945422761,0,0,0,1
+-0.0003139895142521727,0,0,0,5
+-0.0009964637359371406,1,0,2,7
+0.0017777056270278993,1,1,0,4
+-0.00043611460714601535,1,0,1,6
+0.0004510503669735086,1,1,0,4
+0.00023071054136380997,0,1,1,6
+0.0036327595383045252,1,1,2,3
+-0.001784825828799519,0,0,2,2
+0.00037157376820687316,1,1,0,4
+-0.00026842708117329195,0,0,1,6
+0.006552044290292547,1,1,2,3
+-0.00020533778733806268,0,0,2,0
+-0.00023623437213245863,0,0,0,4
+-0.0011948843451682484,0,0,2,2
+-0.0005436195217771445,1,0,2,7

results/cv_lambda_ablation/cv_lambda0.05/fold5/metrics_fold5.json

@@ -0,0 +1,63 @@
+{
+  "overall_auroc": 0.9019199191612984,
+  "overall_acc": 0.8264150943396227,
+  "overall_fpr95": 0.3505747126436782,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.575,
+      "accuracy": 0.8636363636363636,
+      "fpr95": 0.8,
+      "n_good": 20,
+      "n_bad": 2
+    },
+    "dist1": {
+      "auroc": 0.9285714285714286,
+      "accuracy": 0.8837209302325582,
+      "fpr95": 0.2571428571428571,
+      "n_good": 35,
+      "n_bad": 8
+    },
+    "dist6": {
+      "auroc": 0.993103448275862,
+      "accuracy": 0.9117647058823529,
+      "fpr95": 0.034482758620689655,
+      "n_good": 29,
+      "n_bad": 5
+    },
+    "dots": {
+      "auroc": 1.0,
+      "accuracy": 1.0,
+      "fpr95": 0.0,
+      "n_good": 17,
+      "n_bad": 11
+    },
+    "edge1": {
+      "auroc": 0.8849206349206349,
+      "accuracy": 0.75,
+      "fpr95": 0.6111111111111112,
+      "n_good": 18,
+      "n_bad": 14
+    },
+    "edge2": {
+      "auroc": 0.8400000000000001,
+      "accuracy": 0.7857142857142857,
+      "fpr95": 0.47058823529411764,
+      "n_good": 17,
+      "n_bad": 25
+    },
+    "edge3": {
+      "auroc": 0.7076023391812866,
+      "accuracy": 0.6486486486486487,
+      "fpr95": 0.8421052631578947,
+      "n_good": 19,
+      "n_bad": 18
+    },
+    "edge4": {
+      "auroc": 0.8223684210526315,
+      "accuracy": 0.8148148148148148,
+      "fpr95": 0.9473684210526315,
+      "n_good": 19,
+      "n_bad": 8
+    }
+  }
+}

results/cv_lambda_ablation/cv_lambda0.05/fold5/scores_fold5.csv

@@ -0,0 +1,266 @@
+score,true_label,pred_label,template_id,feature_id
+-0.00041950248589273445,0,0,2,2
+0.00015970892622135628,1,1,1,6
+-0.0013255381741328152,0,0,2,2
+0.0011280259335762835,1,1,2,7
+-0.0001582621279521846,0,0,0,1
+-0.00018030481121968523,1,0,2,2
+1.221072656335323e-05,1,1,2,7
+-0.0014746982822543934,0,0,2,2
+4.714935232186593e-05,0,1,1,6
+0.000200680567068047,1,1,0,5
+7.584757055155954e-05,0,1,1,6
+-0.0007882523370790293,0,0,2,7
+0.00036897235928336214,1,1,0,5
+-0.0012187845903099525,0,0,2,0
+-0.000285572854627391,0,0,1,6
+-0.0009672336824587558,0,0,2,3
+-0.0011736439334345036,0,0,2,2
+-0.0005815565332886775,0,0,0,1
+-0.00025525811011902974,1,0,1,6
+-0.0004513834885437971,0,0,0,1
+8.270979742517476e-06,0,1,1,6
+-0.00018281208613188903,0,0,0,5
+-0.0009192345352494156,0,0,2,3
+-0.0007601342361886042,0,0,2,7
+-0.0023170395137276506,0,0,0,1
+-3.437858656980125e-05,1,0,0,5
+-0.0006054313571075918,0,0,2,7
+-0.0012290609182673516,0,0,0,1
+-0.00022406440548366005,0,0,1,6
+-0.00033908606681507143,0,0,0,4
+-0.001570038789650426,0,0,2,0
+-2.0280941680540093e-05,1,0,0,5
+-0.0024433771558688006,0,0,2,3
+-0.00114795048168162,0,0,0,1
+-0.0018720476186717856,0,0,2,3
+-0.00033169681963045143,0,0,0,4
+-6.524044205434565e-05,0,0,1,6
+-0.0011339959103497683,0,0,0,1
+-0.0014040085472515774,0,0,2,2
+-0.0005268782720668219,0,0,0,5
+-0.0012943900018581173,0,0,0,1
+-0.00032172109611565067,0,0,0,4
+0.0002692796688643275,1,1,0,5
+0.0001584096171427518,1,1,0,4
+-0.0003595804324140776,0,0,2,0
+0.00024860830366378595,1,1,2,3
+0.00024775646219495767,1,1,0,4
+-1.42216356471183e-05,1,0,0,1
+-2.200257644290049e-05,1,0,0,4
+0.0007918336903094316,1,1,2,3
+0.0005532286578090843,1,1,0,5
+0.0003644538967637343,1,1,1,6
+0.0007332382260938162,1,1,2,2
+-0.00019262463989434646,0,0,0,5
+0.00016232612135354436,1,1,0,4
+-0.00019013303652172867,0,0,2,7
+-0.00018474141863407574,0,0,0,1
+-0.0009730989619856694,0,0,2,0
+-0.001008390330825934,0,0,0,1
+-0.0001989196208887732,0,0,1,6
+0.00018725659378105754,1,1,2,3
+-0.00017877389473142102,0,0,0,1
+3.780281054787272e-05,1,1,2,3
+-0.0002805515826912585,1,0,2,0
+0.000365456232102589,1,1,2,3
+0.00040039503597654423,0,1,1,6
+0.0003674617290380412,1,1,0,4
+0.0007202931144274782,1,1,1,6
+0.0002473173951148058,1,1,0,1
+-5.545797845115871e-05,1,0,2,2
+-0.00031524805672233924,0,0,2,7
+0.0005038030660944066,1,1,1,6
+5.2135852456558496e-05,1,1,2,3
+-6.655747740296619e-05,1,0,2,7
+4.530836187768752e-05,0,1,2,0
+0.00033239867916563456,1,1,0,4
+6.987386383116266e-05,1,1,2,2
+-5.1473603525665004e-05,1,0,0,4
+-0.0018320056449738324,0,0,2,0
+5.831087764818221e-05,1,1,0,1
+-0.0014690455090021707,0,0,2,2
+0.00024936894798884175,1,1,0,5
+-0.001571870639745612,0,0,2,2
+-0.0005906077954568895,1,0,2,2
+0.00040674772899365,1,1,2,7
+4.630079609341947e-05,1,1,1,6
+-4.586395487422151e-05,0,0,0,4
+-0.0016836885662632993,0,0,2,0
+-0.0010509307790198368,0,0,2,2
+8.684841508511433e-05,0,1,0,4
+-0.0003895400304463701,0,0,0,1
+-9.412129817064872e-05,1,0,0,4
+-0.001028469625452999,0,0,2,2
+-0.0003071554790949442,0,0,0,1
+-8.025761140743274e-05,0,0,0,5
+-0.00045163563132518975,0,0,2,3
+-0.00010939716768916703,0,0,0,5
+-0.0007136465105577368,0,0,2,3
+-0.000985459303192329,0,0,2,0
+-0.0008452770381700263,1,0,2,7
+-0.00034880048187915097,1,0,0,1
+-0.00011850632989080477,1,0,2,7
+0.000655121615272947,1,1,2,3
+0.0009798013186082255,1,1,2,3
+0.0019168297498254102,1,1,0,5
+0.0007183018920477475,0,1,1,6
+0.0006254490310675462,0,1,0,5
+0.0008956241788109758,0,1,1,6
+0.0019239988477784235,1,1,2,3
+0.00035324513737577945,1,1,1,6
+-0.0001925733490497801,0,0,0,5
+-3.197593032381804e-06,0,0,0,5
+5.9739635034930516e-05,0,1,1,6
+-0.0010728424080298279,0,0,2,2
+-0.0010667789177387019,0,0,2,2
+-0.0004183846016530884,0,0,2,7
+-0.00062786689522909,0,0,0,1
+-0.00029457147174980457,0,0,0,5
+-0.00010318422253476414,1,0,0,5
+-0.00011451580619905276,1,0,0,5
+1.3007413945161778e-05,1,1,1,6
+-0.0002674428201862606,1,0,0,1
+-0.00023593248479301075,1,0,1,6
+-0.0012607750546885649,0,0,2,0
+-0.0004592795681674037,0,0,0,1
+0.0004163200219045385,0,1,0,5
+0.0004828176373848688,0,1,1,6
+-0.0015041111747268587,0,0,2,0
+-0.0004895660141482957,0,0,0,1
+-8.247531572123934e-05,0,0,0,4
+-2.8040813049301505e-05,1,0,2,7
+-8.177791518392159e-05,1,0,0,4
+-0.00015109217230929034,1,0,1,6
+-0.0009331834444310516,0,0,2,2
+0.00026620544173056385,1,1,0,5
+-0.00022876399132656065,0,0,0,4
+-3.833269292954217e-05,0,0,1,6
+-5.601472832495423e-05,0,0,0,5
+-0.001041860415716656,0,0,2,3
+-0.0014562442270107563,0,0,2,3
+-0.0003026077730464774,0,0,0,5
+-0.001281002323667053,0,0,2,2
+-0.00018883364187786272,1,0,0,1
+0.0006058304541511454,1,1,0,5
+0.0005752318381564703,1,1,1,6
+0.0002265878918115069,1,1,0,1
+-0.001389493259193842,0,0,2,0
+-0.0011904660661821269,0,0,2,2
+4.1746836504898827e-05,1,1,0,5
+-0.0009278285680920824,0,0,2,2
+-0.0002242343855323268,0,0,1,6
+-0.00031647000985685764,0,0,0,1
+-0.00019331948889885105,0,0,1,6
+-0.00022855945280753076,0,0,0,4
+-0.0003080791662796404,0,0,0,4
+0.00028212861856445615,0,1,0,5
+-0.0005190604098606853,1,0,0,1
+0.0002802273811539635,1,1,0,4
+-0.0011456348709180022,0,0,0,1
+-0.0007211609982186928,0,0,2,3
+-0.0005747986989445061,0,0,2,7
+9.095465822611063e-05,0,1,0,5
+0.00010913230071309989,1,1,1,6
+-0.0003953580319648615,1,0,0,4
+-0.0001945940699079069,1,0,0,4
+-0.000530417550471611,0,0,0,4
+-0.0007917413781979108,0,0,2,3
+-0.002278213289100676,0,0,2,0
+-0.0007375028793467206,0,0,2,7
+0.00048472835798747844,1,1,1,6
+-0.004047664047102446,0,0,2,2
+-0.0010333255527075393,0,0,0,4
+-0.0076519367427681576,0,0,0,1
+-0.0017696429564966823,0,0,2,2
+-0.001929847217688803,0,0,2,0
+-0.008991777881165036,0,0,0,1
+-0.0013628168706782146,0,0,2,3
+-0.0031755107146454985,0,0,2,2
+-0.0018211895597050904,0,0,2,2
+0.001269294559606348,1,1,0,5
+-0.0010934122890466809,0,0,2,7
+0.0009765322302700934,1,1,0,5
+-0.00653984758682782,0,0,0,1
+0.00036745602032169783,0,1,1,6
+-0.0030867701576789836,0,0,2,7
+-0.0021103556256275614,0,0,2,0
+0.00043452512007206845,1,1,1,6
+-0.0014719055066234436,0,0,2,0
+-0.0005078123440034697,0,0,0,4
+0.0010189760226057831,1,1,0,5
+-0.002239274970488623,0,0,2,0
+0.0006524087203433736,1,1,0,5
+-0.001259699998190627,0,0,2,3
+-0.0003598757472354913,0,0,0,4
+-0.006683137588843237,0,0,0,1
+0.0007896170316962527,1,1,1,6
+-0.0010560658268514087,0,0,2,7
+-0.0006466823825030579,0,0,2,3
+0.0009568877294077564,1,1,0,5
+-0.0029154678224585957,0,0,2,2
+0.0014071087428601456,1,1,0,5
+-0.004538531231519302,0,0,0,4
+-0.0033793591969879314,0,0,1,6
+-0.01071385970921256,0,0,2,7
+-0.0037101914186496235,0,0,1,6
+-0.00987406466942048,0,0,2,3
+-0.006106002524611541,0,0,2,2
+-0.05325004490208812,0,0,0,1
+-0.006524437388870867,0,0,2,2
+-0.004538361185113894,0,0,0,5
+-0.003502382127626335,1,0,2,7
+-0.009062938849092459,0,0,0,1
+0.0010719847807195045,1,1,0,5
+0.0007447029877221227,1,1,1,6
+-0.0015212199132656687,0,0,2,7
+-0.0006562651565764095,0,0,0,4
+-0.006176825945440213,0,0,0,1
+-0.0038558264944003884,0,0,2,2
+-0.007374552435940131,0,0,0,1
+-0.0008268437260994695,0,0,0,4
+0.000817130692303181,1,1,0,5
+0.0006432280951412378,1,1,1,6
+-0.0020653504488291237,0,0,2,2
+-0.002294275544991252,0,0,2,7
+-0.00031517050694674156,1,0,0,4
+-0.004840641242626589,0,0,2,0
+0.0005115904117701572,1,1,1,6
+-0.0029605096526211135,0,0,2,7
+0.00025993862247560354,0,1,1,6
+-0.0023040832270635313,0,0,2,3
+-0.00290219310496468,0,0,2,2
+-0.003453229871229268,0,0,2,0
+-0.00022265480802161758,0,0,0,4
+0.0007431213022209698,1,1,0,5
+-0.0026620410097530113,1,0,2,0
+-0.004633781988522968,0,0,2,2
+-0.002066941908560691,0,0,2,7
+-0.004578240239061417,0,0,2,3
+-0.0003753670217702173,0,0,0,5
+-0.002805805610842071,0,0,2,7
+-0.003540281674941071,0,0,2,0
+-0.007894300133048093,0,0,0,1
+-0.0041155838756822055,0,0,2,2
+-0.0009654705325374373,0,0,0,4
+0.0002252351772040126,1,1,0,4
+-0.006619302624021658,0,0,0,1
+-0.0043603966612136,0,0,2,3
+-0.00880383268580772,0,0,0,1
+-0.0023776361122145317,0,0,2,7
+-0.0028297300328267733,0,0,2,0
+-0.004053613696305549,0,0,2,2
+0.006667081986670383,1,1,0,5
+0.003313993127085274,1,1,2,3
+-0.008238332238397567,0,0,0,1
+-0.006689885305531787,0,0,0,1
+-0.001704637731891126,0,0,2,2
+-0.006737597041646948,0,0,0,1
+0.0048031875578453745,0,1,0,5
+-0.00026135149208129926,0,0,2,7
+-0.015555279692052865,0,0,0,1
+0.005817981477594006,1,1,0,5
+-0.003939208899682849,0,0,0,1
+0.004272736936109141,1,1,0,5
+-0.010624796673073436,0,0,0,1
+0.004137129498412842,1,1,2,3

results/single_split_lambda_ablation/inkjet_lambda0.01/metrics_final.json

@@ -0,0 +1,54 @@
+{
+  "overall_auroc": 0.8602573713143427,
+  "overall_acc": 0.8157894736842105,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.5679012345679011,
+      "accuracy": 0.8666666666666667,
+      "n_good": 27,
+      "n_bad": 3
+    },
+    "dist1": {
+      "auroc": 0.8571428571428571,
+      "accuracy": 0.7419354838709677,
+      "n_good": 21,
+      "n_bad": 10
+    },
+    "dist6": {
+      "auroc": 0.8111111111111111,
+      "accuracy": 0.9444444444444444,
+      "n_good": 30,
+      "n_bad": 6
+    },
+    "dots": {
+      "auroc": 0.9265734265734266,
+      "accuracy": 0.9459459459459459,
+      "n_good": 26,
+      "n_bad": 11
+    },
+    "edge1": {
+      "auroc": 0.8177083333333333,
+      "accuracy": 0.75,
+      "n_good": 12,
+      "n_bad": 16
+    },
+    "edge2": {
+      "auroc": 0.7242063492063492,
+      "accuracy": 0.6521739130434783,
+      "n_good": 18,
+      "n_bad": 28
+    },
+    "edge3": {
+      "auroc": 0.875,
+      "accuracy": 0.8214285714285714,
+      "n_good": 16,
+      "n_bad": 12
+    },
+    "edge4": {
+      "auroc": 0.7361111111111112,
+      "accuracy": 0.8333333333333334,
+      "n_good": 24,
+      "n_bad": 6
+    }
+  }
+}

results/single_split_lambda_ablation/inkjet_lambda0.01/scores_final.csv

@@ -0,0 +1,267 @@
+score,true_label,pred_label,template_id,feature_id
+-0.0001786431111395351,1,0,0,1
+-0.0011423978552920731,0,0,0,1
+-0.00018911421939265052,0,0,0,5
+-0.0011020839790580883,0,0,2,7
+0.0007452807115623725,1,1,0,5
+4.291905846912428e-05,1,1,0,1
+-0.0009933844004990644,0,0,1,6
+-0.0013027262146351852,0,0,0,1
+-0.00032546747941523783,0,0,2,2
+-4.512651939876308e-05,0,0,1,6
+7.99692235887127e-06,1,1,0,4
+2.3589637712576397e-05,1,1,0,5
+-0.0012639777740696446,0,0,2,0
+-0.0018777770627639231,0,0,2,0
+-0.0007442053506383668,0,0,2,7
+6.0173769015816475e-05,1,1,1,6
+0.00014766942127607784,1,1,2,3
+-0.0002706340968143192,0,0,2,3
+-0.0003212497825734323,0,0,2,3
+-0.0002709608321310973,0,0,1,6
+-0.00017797390406485564,0,0,0,4
+0.0007627808453980825,0,1,0,5
+-0.0024317535356385665,0,0,2,2
+-0.0019420934654772286,0,0,2,2
+0.0002815723035018876,0,1,0,5
+-0.00015616844815667663,0,0,0,5
+-0.0001376926360535432,0,0,2,3
+0.0004564658878371125,0,1,0,5
+0.00023729510605335208,1,1,0,5
+-0.0004955654533114282,0,0,0,4
+-9.095911111216964e-05,0,0,1,6
+-0.00031292872794438237,0,0,0,1
+-0.00010894230334088269,1,0,1,6
+-0.0002868476795265442,0,0,0,4
+-0.0013655663607642053,0,0,2,0
+-0.002636892183800227,1,0,2,2
+0.00040531001228373553,1,1,0,4
+-0.0017637870903126907,0,0,2,3
+0.0011414490436436608,1,1,0,5
+-0.0015931331494357416,0,0,2,0
+-8.474531641695753e-05,1,0,0,5
+-0.00017851277312729442,0,0,2,7
+0.00048465125670190887,1,1,1,6
+0.00030213974474463737,1,1,2,2
+0.0003216412084293555,1,1,2,3
+3.819449455477308e-05,1,1,1,6
+-0.00025991545757278727,0,0,2,2
+-0.0002109976136125645,0,0,0,4
+0.00021903581335209338,0,1,0,5
+-0.00017956512339878904,0,0,0,5
+0.00028556961566209797,1,1,2,3
+-0.00027972022129688426,0,0,0,4
+-0.0002063119644299151,0,0,2,7
+-0.000115501753170974,0,0,0,1
+0.0008957871340680868,1,1,0,5
+-0.00014380375854670946,0,0,2,7
+-0.000237207731115633,1,0,0,5
+-0.004023515069857236,0,0,0,1
+-0.0003130392817547543,0,0,2,3
+-4.154522612225334e-05,1,0,0,5
+-0.0006892500841058791,0,0,2,3
+-0.0006890911603113625,0,0,2,7
+0.00017765469208825407,1,1,1,6
+-0.00370945459115319,0,0,2,7
+0.0005358120257733384,0,1,0,4
+-0.0007017386367078864,0,0,2,2
+-0.0015851770772133018,0,0,2,2
+0.00030901036458089864,1,1,0,4
+-0.00013943348836619254,0,0,2,3
+-0.0008027271676110097,0,0,2,2
+-0.0004406928530079311,0,0,2,3
+-0.00043977456894936026,1,0,2,0
+0.0002004516433225939,0,1,1,6
+-0.0006395371357211833,0,0,2,0
+-0.000998407593870071,0,0,2,3
+0.003494524622801684,1,1,0,5
+-0.0006043709657387805,0,0,2,0
+6.805680983234131e-05,1,1,2,3
+-0.0004164628969738262,0,0,2,2
+4.1301571764035955e-06,0,1,1,6
+-0.00011027077387552701,1,0,2,7
+-0.00040702820988372003,0,0,2,3
+-1.039743947331244e-06,0,0,2,0
+0.0017971275333548004,1,1,2,3
+9.800606523640426e-05,1,1,0,5
+-0.003521966327680273,1,0,2,0
+2.9645394533876074e-06,1,1,2,7
+0.00010978177655488267,1,1,1,6
+-0.002464917622273785,0,0,2,2
+-7.276151853147922e-05,0,0,2,7
+0.00015708790393546487,1,1,0,5
+-1.2279795482754777e-05,1,0,0,4
+-0.00024264837964438,0,0,1,6
+8.357385930139738e-05,1,1,1,6
+-0.0006225871091010051,1,0,0,4
+-0.0002448904037009921,0,0,2,2
+-0.0003792373859323548,0,0,2,7
+0.0011532299761893225,1,1,2,3
+-0.00017199567635543637,0,0,1,6
+-3.422013425733936e-05,1,0,0,1
+-0.0008466154447523879,0,0,2,2
+-0.0004957614454906417,0,0,2,2
+-0.00032346886058803696,0,0,0,4
+-4.579597734847077e-06,1,0,1,6
+-0.00037248163658659943,0,0,2,0
+0.005952488492475823,1,1,0,5
+-0.0004225823469460009,1,0,0,1
+2.135436167009265e-05,0,1,2,0
+0.0018315446807537251,1,1,0,5
+-0.0009118631371529771,0,0,2,3
+0.005331811638898218,1,1,2,3
+-0.0003709839156363151,1,0,2,2
+-0.0005739190196618425,0,0,0,1
+-0.00011293835821561508,0,0,2,3
+-0.00024790809082332957,0,0,1,6
+-0.0008578305295668526,0,0,2,3
+-0.0008449894352816033,0,0,0,1
+0.00016571268672123385,1,1,0,4
+-0.0004954496829304832,1,0,0,1
+-0.0003310038056224592,0,0,2,2
+0.00012645312701351982,1,1,0,4
+-0.0001874936470994716,1,0,2,7
+-0.00013217756059021035,1,0,0,5
+-4.259471141267613e-05,1,0,0,5
+2.26193014532328e-05,1,1,2,2
+0.00018399767577648163,1,1,0,4
+0.00023071483068633908,0,1,1,6
+-0.00017588301561772754,1,0,0,4
+0.00015343460021540606,1,1,0,4
+-0.008160336468135937,0,0,0,1
+-0.000251806619344281,1,0,2,7
+-0.0006340810871915924,0,0,2,3
+-0.0001702815870521599,0,0,2,3
+-3.0017084791326976e-05,1,0,0,4
+9.867216402199147e-05,1,1,0,5
+-0.004822461662697605,0,0,2,3
+-5.650528706610189e-05,0,0,2,0
+-0.00019088474800810218,0,0,0,5
+-0.0003445531631587062,0,0,0,4
+-0.0003687758525484242,0,0,2,3
+-0.0012713429646100845,1,0,2,3
+5.3166089928707166e-05,0,1,0,5
+-0.005447448567720128,0,0,2,2
+-0.00017304670414887424,0,0,2,7
+-0.00042878599429968696,0,0,2,2
+-0.0005450340127572412,0,0,2,2
+-0.0014653650962281987,0,0,2,0
+-0.00021784117474453468,0,0,2,3
+-0.0005658121081069112,0,0,2,0
+-0.001286818954977207,0,0,2,0
+-0.00025792685744818224,0,0,1,6
+-0.0004949270794168108,0,0,2,0
+-0.0014176848460920165,0,0,2,0
+0.0012448254314949732,1,1,0,5
+0.00014057967113330957,0,1,2,7
+0.0010216882318491109,1,1,0,5
+-0.0029955839051399383,0,0,1,6
+-0.0003501816163770863,0,0,0,1
+-0.0006729643093422045,0,0,2,0
+-0.00023714460723567744,0,0,2,2
+-3.987605450674753e-05,1,0,0,4
+0.0002717716502957032,1,1,0,1
+0.0002982462238287544,1,1,1,6
+-0.00023357348924037038,0,0,2,7
+-0.0004474732058588416,0,0,2,0
+6.032410310581257e-05,1,1,0,4
+-0.0010004460188793012,0,0,0,1
+7.299024728128073e-06,1,1,0,5
+-0.0031357015960384177,0,0,2,2
+-0.00018916442058980465,0,0,0,1
+-9.002810460515334e-05,1,0,2,3
+-0.0006767850479809569,0,0,2,0
+-0.0013761657691793508,1,0,0,4
+-0.0002590807108208519,1,0,0,5
+0.0007839220372261452,1,1,1,6
+-0.0020254048507194963,0,0,2,7
+-0.0016379465942736697,0,0,2,2
+-0.0010763889295049012,0,0,0,1
+0.000150192244909704,0,1,0,5
+-0.00660158627899364,0,0,0,1
+1.5059081488293102e-06,0,1,0,1
+-0.0001068122929427772,0,0,1,6
+-0.0023233246558811496,0,0,2,2
+0.0007618861482478687,0,1,0,5
+-0.0005215886910445982,0,0,2,3
+0.00021843143738806137,1,1,0,4
+-0.0002676492359023532,1,0,2,7
+0.0005281878833193338,0,1,0,5
+-0.0004165107564767824,0,0,0,5
+-0.00019065725093241753,0,0,0,1
+-0.00041620338743086904,0,0,0,4
+-0.0016201763565186385,0,0,2,0
+-1.5313619514926644e-05,0,0,1,6
+0.0011571451678173625,1,1,0,5
+0.0005124370270641505,1,1,2,2
+-0.00031304346281103845,0,0,0,1
+-0.0005729913263348863,0,0,2,0
+3.829641558695564e-05,0,1,0,5
+-0.0007270062284078435,0,0,2,7
+-6.290412158705284e-05,0,0,2,7
+-0.00027767844032496394,0,0,2,7
+-0.000819586373399947,0,0,0,1
+8.353232755325263e-05,0,1,0,1
+-0.00015112197434064,0,0,2,3
+-0.0005613690149039036,0,0,0,1
+9.788283670786845e-05,1,1,2,2
+-0.00047200454282574425,0,0,2,2
+5.5704757687635026e-05,1,1,0,1
+0.0003039405914023515,1,1,2,7
+-0.0005837837466970107,0,0,0,4
+-0.003904075921745971,0,0,1,6
+-0.00042226517602102914,0,0,2,3
+0.0003544122545281426,1,1,2,3
+-0.0006937096075853334,0,0,2,2
+-0.0015447541006142276,0,0,2,2
+0.000927986806491388,1,1,0,5
+-0.0005757630232255916,0,0,2,2
+-0.002235631266958079,0,0,2,3
+-0.0019829648867016637,0,0,2,7
+-0.0004951069582602944,0,0,2,3
+-0.001616387821850366,0,0,2,3
+9.372618689667406e-05,1,1,0,4
+-0.001118795192451217,0,0,2,0
+8.413481060415458e-05,1,1,1,6
+-0.0011666807724395774,0,0,2,2
+0.0002366543209063824,1,1,2,3
+-0.000674554677680135,0,0,0,4
+-3.809993155300638e-05,0,0,2,7
+-0.00046905880852136754,0,0,2,3
+0.00014039753295946866,1,1,0,5
+-0.00025640829931944734,0,0,0,5
+-8.504142460878921e-05,0,0,2,7
+-0.0005703360837651418,0,0,2,0
+0.0001383387768873947,1,1,0,5
+-0.00012049005483277114,0,0,0,5
+-0.0019333142216783027,0,0,2,7
+-0.0005746701406314971,0,0,0,1
+6.352418567985227e-06,0,1,0,1
+-0.00027963069325778744,0,0,1,6
+-0.002362565661314875,0,0,2,2
+-0.0004002170381136247,0,0,2,7
+0.00013217921543400707,1,1,0,1
+-0.00046758937474805987,0,0,2,7
+-0.0037433369079371906,0,0,0,4
+-0.0006427784572588281,0,0,2,2
+0.0002746511180885137,1,1,0,5
+0.0011256911815144127,1,1,0,5
+-0.0017365477554267261,0,0,2,0
+-0.000515735471853988,0,0,2,0
+-4.250828642398535e-06,1,0,0,5
+-0.00045283995161298635,0,0,2,2
+-0.00026217658247332937,0,0,0,5
+3.473962889984257e-05,1,1,1,6
+0.0014970530831487847,1,1,0,5
+-7.238035323094832e-08,0,0,2,7
+-0.0007135019963607197,0,0,2,0
+-0.001066119471797718,0,0,2,2
+-0.0007605915743624792,0,0,2,7
+-0.00440240031923167,0,0,2,0
+-0.0001403582975035532,0,0,2,2
+0.00031154396303463734,1,1,0,1
+-4.1858991608023643e-05,1,0,0,1
+-0.00011327049694955394,0,0,2,0
+-0.0020311303896596647,0,0,2,3
+-0.0003049442032352096,0,0,0,5
+-0.00020666327909566427,1,0,2,0

results/single_split_lambda_ablation/inkjet_lambda0.02/metrics_final.json

@@ -0,0 +1,54 @@
+{
+  "overall_auroc": 0.8581334332833583,
+  "overall_acc": 0.8045112781954887,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.6419753086419753,
+      "accuracy": 0.8666666666666667,
+      "n_good": 27,
+      "n_bad": 3
+    },
+    "dist1": {
+      "auroc": 0.9333333333333333,
+      "accuracy": 0.8387096774193549,
+      "n_good": 21,
+      "n_bad": 10
+    },
+    "dist6": {
+      "auroc": 0.8444444444444446,
+      "accuracy": 0.9444444444444444,
+      "n_good": 30,
+      "n_bad": 6
+    },
+    "dots": {
+      "auroc": 0.9125874125874126,
+      "accuracy": 0.972972972972973,
+      "n_good": 26,
+      "n_bad": 11
+    },
+    "edge1": {
+      "auroc": 0.84375,
+      "accuracy": 0.8214285714285714,
+      "n_good": 12,
+      "n_bad": 16
+    },
+    "edge2": {
+      "auroc": 0.7103174603174603,
+      "accuracy": 0.6304347826086957,
+      "n_good": 18,
+      "n_bad": 28
+    },
+    "edge3": {
+      "auroc": 0.7916666666666667,
+      "accuracy": 0.75,
+      "n_good": 16,
+      "n_bad": 12
+    },
+    "edge4": {
+      "auroc": 0.6875,
+      "accuracy": 0.6333333333333333,
+      "n_good": 24,
+      "n_bad": 6
+    }
+  }
+}

results/single_split_lambda_ablation/inkjet_lambda0.02/scores_final.csv

@@ -0,0 +1,267 @@
+score,true_label,pred_label,template_id,feature_id
+3.420361957978506e-05,1,1,0,1
+-0.0015551711886655564,0,0,0,1
+-0.0001874831551685937,0,0,0,5
+-0.0005519780982285727,0,0,2,7
+0.00046471880923490937,1,1,0,5
+0.00017536605417262704,1,1,0,1
+-0.0028975329449167503,0,0,1,6
+-0.0016836180491372962,0,0,0,1
+-0.0008804314531153067,0,0,2,2
+2.525086980313153e-05,0,1,1,6
+0.0001238063088385387,1,1,0,4
+-4.1906529222614924e-05,1,0,0,5
+-0.0029702313372399654,0,0,2,0
+-0.002463026478653773,0,0,2,0
+-0.0006483464560005821,0,0,2,7
+3.1915508443489116e-05,1,1,1,6
+9.89122735336424e-05,1,1,2,3
+-0.0001341487740864969,0,0,2,3
+-0.0001773246732773258,0,0,2,3
+2.9550415929405216e-05,0,1,1,6
+-8.616624225396725e-05,0,0,0,4
+9.499417385086528e-05,0,1,0,5
+-0.0034622522210702306,0,0,2,2
+-0.0018006217019865284,0,0,2,2
+0.0002682547539006909,0,1,0,5
+-0.00013105876219924456,0,0,0,5
+-0.00016587077989242954,0,0,2,3
+0.0006401620618998981,0,1,0,5
+6.978767749387868e-05,1,1,0,5
+-0.0008932393934810538,0,0,0,4
+-6.951711198780602e-05,0,0,1,6
+-0.000618590924423186,0,0,0,1
+-7.853146584238871e-05,1,0,1,6
+-0.00022910833882633548,0,0,0,4
+-0.001718352780444547,0,0,2,0
+-0.00299468344950583,1,0,2,2
+-3.199756145477045e-05,1,0,0,4
+-0.0025133678776910542,0,0,2,3
+0.0010372741613537094,1,1,0,5
+-0.0021751966571900982,0,0,2,0
+-0.0004621173744089914,1,0,0,5
+-1.8174375873052442e-06,0,0,2,7
+0.0006927162985084605,1,1,1,6
+0.0002581719285808491,1,1,2,2
+0.0004919798986520616,1,1,2,3
+6.218343565706289e-05,1,1,1,6
+-0.00038350472284946584,0,0,2,2
+-0.0004069686488946886,0,0,0,4
+-0.00013882802973967136,0,0,0,5
+-0.0005956476699793718,0,0,0,5
+0.00031851114123128323,1,1,2,3
+-0.00043589450477156746,0,0,0,4
+-0.00022938030306249822,0,0,2,7
+0.00010038219508714911,0,1,0,1
+0.0009297595301177349,1,1,0,5
+3.923380281775982e-06,0,1,2,7
+-0.00011775016202591332,1,0,0,5
+-0.006487712832167744,0,0,0,1
+-0.0005909173155669127,0,0,2,3
+-0.0002849767089355727,1,0,0,5
+-0.0009603449841961258,0,0,2,3
+-0.0007819026662036765,0,0,2,7
+0.0003553710115374968,1,1,1,6
+-0.004746730904444118,0,0,2,7
+0.0005268363945651819,0,1,0,4
+-0.0008538275730097661,0,0,2,2
+-0.0013591507956152785,0,0,2,2
+0.0011735835665604086,1,1,0,4
+-0.0001333186030387859,0,0,2,3
+-0.0011581966764060772,0,0,2,2
+-0.0006360386754386133,0,0,2,3
+-0.0002692485193256291,1,0,2,0
+8.160913537722143e-05,0,1,1,6
+-0.001783788754837588,0,0,2,0
+-0.0010322256374638503,0,0,2,3
+-0.0003816488396841955,1,0,0,5
+-0.0015433891798602422,0,0,2,0
+0.0002802588825579731,1,1,2,3
+-0.0006550165906082841,0,0,2,2
+-3.518760495353594e-05,0,0,1,6
+-2.955169766210089e-05,1,0,2,7
+-0.0004666429309872905,0,0,2,3
+-3.799358266405768e-05,0,0,2,0
+0.00015328726090956674,1,1,2,3
+2.630513627082179e-05,1,1,0,5
+-0.002132065327605223,1,0,2,0
+0.00012515481386799442,1,1,2,7
+0.00040358580183237625,1,1,1,6
+-0.004338176201563328,0,0,2,2
+4.7830851981416414e-05,0,1,2,7
+0.00013751871301792562,1,1,0,5
+0.00017572648241184676,1,1,0,4
+-1.4297903981060067e-05,0,0,1,6
+6.531377555802492e-05,1,1,1,6
+-0.0004124616784974937,1,0,0,4
+-0.001136903853039259,0,0,2,2
+-0.0005899174697697167,0,0,2,7
+0.0010196903441101295,1,1,2,3
+-7.34548136824744e-05,0,0,1,6
+-6.784782279282975e-05,1,0,0,1
+-0.001517223155824466,0,0,2,2
+-0.0007857757329475142,0,0,2,2
+-0.0006817787565523738,0,0,0,4
+0.00012320770532824217,1,1,1,6
+-0.0016692984604742382,0,0,2,0
+0.004226966118440034,1,1,0,5
+-9.864690422545924e-05,1,0,0,1
+7.247017230838527e-05,0,1,2,0
+0.0012265775399282544,1,1,0,5
+-0.0005595983040984718,0,0,2,3
+0.000420496768201703,1,1,2,3
+-0.0002363786270143456,1,0,2,2
+-0.0008889292553067203,0,0,0,1
+-0.00029210925917141223,0,0,2,3
+-0.00031156458659097573,0,0,1,6
+-0.0008936105662724,0,0,2,3
+-0.0009037707163952278,0,0,0,1
+0.00026200937747489564,1,1,0,4
+-0.00027239568415097885,1,0,0,1
+-0.0004970164154656231,0,0,2,2
+0.000381679526180961,1,1,0,4
+-6.461560027673854e-05,1,0,2,7
+-0.00027235870657022973,1,0,0,5
+-5.370566213969141e-05,1,0,0,5
+0.00010153132432606099,1,1,2,2
+0.00025818892230745384,1,1,0,4
+0.00042120250756852266,0,1,1,6
+-3.502877079881832e-05,1,0,0,4
+0.00018917199573479856,1,1,0,4
+-0.017538419051561505,0,0,0,1
+-0.00021367875859141378,1,0,2,7
+-0.001072484184987843,0,0,2,3
+-7.760745647828957e-05,0,0,2,3
+0.00015866569650825092,1,1,0,4
+4.436152521520957e-05,1,1,0,5
+-0.00997243157122284,0,0,2,3
+-0.00022326518548652485,0,0,2,0
+-0.00017915545962750908,0,0,0,5
+-0.00026036686147563187,0,0,0,4
+-0.0006570375815499576,0,0,2,3
+-0.007198627145844509,1,0,2,3
+0.00017901580024044968,0,1,0,5
+-0.0026232768059708256,0,0,2,2
+-0.00010624105867464064,0,0,2,7
+-0.000935908165993168,0,0,2,2
+-0.0006284332415089008,0,0,2,2
+-0.0018533086619572717,0,0,2,0
+-0.00039386882330291056,0,0,2,3
+-0.0009568328730529176,0,0,2,0
+-0.0021667256625369207,0,0,2,0
+-0.00010059749125503059,0,0,1,6
+-0.0009713068016571919,0,0,2,0
+-0.002598403482115829,0,0,2,0
+0.0009086013864725816,1,1,0,5
+0.00034043813589960253,0,1,2,7
+0.0011064173572231072,1,1,0,5
+-0.002463454605313019,0,0,1,6
+-0.000573810278438034,0,0,0,1
+-0.0011134578357450669,0,0,2,0
+-0.00044704609375912675,0,0,2,2
+7.942782831378212e-06,1,1,0,4
+0.00021264803188387332,1,1,0,1
+0.0003600692725740373,1,1,1,6
+-0.00013692270789761058,0,0,2,7
+-0.0009495638974476601,0,0,2,0
+0.0002612645283807079,1,1,0,4
+-0.0012408290070015943,0,0,0,1
+-0.0003765211190329895,1,0,0,5
+-0.004299751271028069,0,0,2,2
+-0.00020609727944247494,0,0,0,1
+0.0001705509249586637,1,1,2,3
+-0.0005723699356894934,0,0,2,0
+-0.0013138712401268987,1,0,0,4
+-0.00017869883915409487,1,0,0,5
+0.0007314979261718695,1,1,1,6
+-0.002182700855191795,0,0,2,7
+-0.0008094697864726198,0,0,2,2
+-0.0004701931413728759,0,0,0,1
+-2.9124511638655218e-05,0,0,0,5
+-0.013499154907767666,0,0,0,1
+7.564261555671747e-05,0,1,0,1
+-2.485780278220842e-05,0,0,1,6
+-0.002816255256184376,0,0,2,2
+0.0005543602199759336,0,1,0,5
+-0.0006699835549807175,0,0,2,3
+0.00043287291540764286,1,1,0,4
+-0.00021457272116094835,1,0,2,7
+0.00012164893909357571,0,1,0,5
+-0.0006337621889542788,0,0,0,5
+-0.00048122192558366916,0,0,0,1
+-0.0002442347141914068,0,0,0,4
+-0.002439312256756237,0,0,2,0
+0.0005063526309095311,0,1,1,6
+0.0007446827262174334,1,1,0,5
+0.0005989278818015007,1,1,2,2
+-0.0005078946531284602,0,0,0,1
+-0.0008183021459262823,0,0,2,0
+-0.00018154101562686126,0,0,0,5
+-0.00018108295684214626,0,0,2,7
+4.7125426353886515e-05,0,1,2,7
+-1.5695367474109767e-05,0,0,2,7
+-0.00045757666171994046,0,0,0,1
+-0.0009445576026337194,0,0,0,1
+-0.00019119636388495627,0,0,2,3
+-0.0008654313650913528,0,0,0,1
+3.04047716781497e-07,1,1,2,2
+-0.0008258982084225867,0,0,2,2
+0.00026802576787304105,1,1,0,1
+0.0005997005949029688,1,1,2,7
+-0.0004894310596864661,0,0,0,4
+-0.004605297364760189,0,0,1,6
+-0.00044561698567122283,0,0,2,3
+0.0007901215372839946,1,1,2,3
+-0.0013467607821803541,0,0,2,2
+-0.0027415649354225052,0,0,2,2
+0.0004950050066690904,1,1,0,5
+-0.0009838352311635386,0,0,2,2
+-0.0029993951309006655,0,0,2,3
+-0.0025031281344126896,0,0,2,7
+-0.0006985236174659789,0,0,2,3
+-0.00260520452284254,0,0,2,3
+0.00015159630740527175,1,1,0,4
+-0.0012322817090898754,0,0,2,0
+0.00018184974032919854,1,1,1,6
+-0.0009273777616908771,0,0,2,2
+0.00038909079681616245,1,1,2,3
+-0.0004578343767207109,0,0,0,4
+0.0001470301911467688,0,1,2,7
+-0.0005090859218034888,0,0,2,3
+0.00017054600466508314,1,1,0,5
+-0.0005400692793773518,0,0,0,5
+2.1429273183458386e-05,0,1,2,7
+-0.0011512681527528906,0,0,2,0
+0.00019125158491078813,1,1,0,5
+-0.0003199785336619257,0,0,0,5
+-0.0023700072034262103,0,0,2,7
+-0.0005632489809067921,0,0,0,1
+-2.173040236812078e-05,0,0,0,1
+-0.00018383337475825004,0,0,1,6
+-0.0019065328635042532,0,0,2,2
+-0.0005745157040655605,0,0,2,7
+0.000255835970165208,1,1,0,1
+-0.0002846892655361446,0,0,2,7
+-0.007179187081055714,0,0,0,4
+-0.000985596048994921,0,0,2,2
+-7.022283796686776e-05,1,0,0,5
+0.0007542400714009995,1,1,0,5
+-0.0018061821290757535,0,0,2,0
+-0.0009763492573983956,0,0,2,0
+-7.011136447545186e-05,1,0,0,5
+-0.0007961676199920486,0,0,2,2
+-0.0008154413907323008,0,0,0,5
+-0.00015910852933302515,1,0,1,6
+0.001703459935961292,1,1,0,5
+0.00020843101374339295,0,1,2,7
+-0.0011626783170504487,0,0,2,0
+-0.0012319139746250589,0,0,2,2
+-0.000544595592073165,0,0,2,7
+-0.0035893498186487655,0,0,2,0
+-9.914148598909496e-05,0,0,2,2
+0.00021676130185369416,1,1,0,1
+1.1285521904937437e-05,1,1,0,1
+-0.00022904326906427776,0,0,2,0
+-0.002173371654935181,0,0,2,3
+-0.0001755801925901329,0,0,0,5
+-0.000806734816287645,1,0,2,0

results/single_split_lambda_ablation/inkjet_lambda0.02_eval/metrics.json

@@ -0,0 +1,54 @@
+{
+  "overall_auroc": 0.8473888055972014,
+  "overall_acc": 0.8045112781954887,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.5802469135802469,
+      "accuracy": 0.8666666666666667,
+      "n_good": 27,
+      "n_bad": 3
+    },
+    "dist1": {
+      "auroc": 0.9190476190476191,
+      "accuracy": 0.8387096774193549,
+      "n_good": 21,
+      "n_bad": 10
+    },
+    "dist6": {
+      "auroc": 0.8277777777777777,
+      "accuracy": 0.9166666666666666,
+      "n_good": 30,
+      "n_bad": 6
+    },
+    "dots": {
+      "auroc": 0.9090909090909092,
+      "accuracy": 0.9459459459459459,
+      "n_good": 26,
+      "n_bad": 11
+    },
+    "edge1": {
+      "auroc": 0.8385416666666666,
+      "accuracy": 0.8571428571428571,
+      "n_good": 12,
+      "n_bad": 16
+    },
+    "edge2": {
+      "auroc": 0.7500000000000001,
+      "accuracy": 0.6304347826086957,
+      "n_good": 18,
+      "n_bad": 28
+    },
+    "edge3": {
+      "auroc": 0.7656249999999999,
+      "accuracy": 0.7142857142857143,
+      "n_good": 16,
+      "n_bad": 12
+    },
+    "edge4": {
+      "auroc": 0.6041666666666667,
+      "accuracy": 0.7,
+      "n_good": 24,
+      "n_bad": 6
+    }
+  }
+}

results/single_split_lambda_ablation/inkjet_lambda0.02_eval/scores.csv

@@ -0,0 +1,267 @@
+score,true_label,pred_label,template_id,feature_id
+4.348672460764615e-05,1,1,0,1
+-0.0014595093071693571,0,0,0,1
+-0.00026559086865745457,0,0,0,5
+-0.0005600108846556401,0,0,2,7
+0.0003388300578808408,1,1,0,5
+7.086210825946182e-05,1,1,0,1
+-0.0019314171071164316,0,0,1,6
+-0.001429445231333374,0,0,0,1
+-0.0008635538263479271,0,0,2,2
+-4.2575912084421436e-06,0,0,1,6
+0.00019650664587970816,1,1,0,4
+-2.6363634970036354e-06,1,0,0,5
+-0.002764843151089734,0,0,2,0
+-0.002278395841130987,0,0,2,0
+-0.0007575568417087196,0,0,2,7
+3.242940001655335e-05,1,1,1,6
+5.952833569608691e-05,1,1,2,3
+-0.00012357714760582918,0,0,2,3
+-0.0001113106351112933,0,0,2,3
+3.7772402865812044e-05,0,1,1,6
+-3.4172009909525305e-05,0,0,0,4
+-7.719874498434559e-05,0,0,0,5
+-0.0024920600035693487,0,0,2,2
+-0.002849308805307371,0,0,2,2
+7.952156302053464e-05,0,1,0,5
+-0.00013472983031533568,0,0,0,5
+-0.00021280332235619395,0,0,2,3
+0.0008190248161554364,0,1,0,5
+0.0004377650999231285,1,1,0,5
+-0.00043751981982495525,0,0,0,4
+2.442201948725173e-06,0,1,1,6
+-0.0008371427963720646,0,0,0,1
+-9.268308058381532e-06,1,0,1,6
+-0.00032356052834075014,0,0,0,4
+-0.0028789544996106996,0,0,2,0
+-0.004558184102643283,1,0,2,2
+0.00011573750816751271,1,1,0,4
+-0.0037909564771689483,0,0,2,3
+0.0007624698337167501,1,1,0,5
+-0.0021763142460258685,0,0,2,0
+-0.000357374367304146,1,0,0,5
+-3.9645449724047543e-05,0,0,2,7
+0.00046295710606500395,1,1,1,6
+0.00032495103310793694,1,1,2,2
+0.00044132871727924766,1,1,2,3
+7.272280519828207e-05,1,1,1,6
+-0.0005879335774807253,0,0,2,2
+-0.0004312143463175745,0,0,0,4
+-0.0002461832080734895,0,0,0,5
+-0.0008396622119471418,0,0,0,5
+0.0004190866881981486,1,1,2,3
+-0.00041606607905123397,0,0,0,4
+-0.00023144344682805244,0,0,2,7
+2.800137735903263e-05,0,1,0,1
+0.0011101722350576881,1,1,0,5
+-3.75086779240523e-05,0,0,2,7
+-0.00015811004559509317,1,0,0,5
+-0.007936015177983792,0,0,0,1
+-0.0004052979673724614,0,0,2,3
+-0.00028069264197256444,1,0,0,5
+-0.0009184427675791081,0,0,2,3
+-0.000735560056637042,0,0,2,7
+0.00028820759500376714,1,1,1,6
+-0.002217254753340968,0,0,2,7
+0.0003872495383257027,0,1,0,4
+-0.00047095729678403586,0,0,2,2
+-0.0009377600240986807,0,0,2,2
+0.0008005128812510476,1,1,0,4
+-0.0001676350604975594,0,0,2,3
+-0.0008950596029171712,0,0,2,2
+-0.0003670572658302264,0,0,2,3
+-0.0008131077582947885,1,0,2,0
+0.00016881614690646835,0,1,1,6
+-0.0013335806725081056,0,0,2,0
+-0.0006787443521898235,0,0,2,3
+0.0002464986080303738,1,1,0,5
+-0.0018269884766777995,0,0,2,0
+0.0001991331967292351,1,1,2,3
+-0.0006182308110874151,0,0,2,2
+-4.356350924353963e-05,0,0,1,6
+-0.0002193037094548357,1,0,2,7
+-0.0006315829191589732,0,0,2,3
+-5.9969426947645635e-05,0,0,2,0
+0.0004124536493327474,1,1,2,3
+-7.124155759811318e-05,1,0,0,5
+-0.0020893272006651392,1,0,2,0
+0.00017037858488038252,1,1,2,7
+0.000610019592568279,1,1,1,6
+-0.0032928490126505495,0,0,2,2
+4.986115382052923e-05,0,1,2,7
+0.00010769179032649786,1,1,0,5
+0.00043657335336319894,1,1,0,4
+-4.3653160682879596e-05,0,0,1,6
+-0.0006362399540375888,1,0,1,6
+-0.0005220451351488006,1,0,0,4
+-0.0008380525524262338,0,0,2,2
+-0.00022442596557084495,0,0,2,7
+0.0007852229132549841,1,1,2,3
+-8.03776725661013e-05,0,0,1,6
+-8.81085055880193e-06,1,0,0,1
+-0.0010198558820411555,0,0,2,2
+-0.0006231393030611794,0,0,2,2
+-0.00032014432537835064,0,0,0,4
+0.00037867959355935427,1,1,1,6
+-0.0007131924986606463,0,0,2,0
+0.003543454341124741,1,1,0,5
+-0.00043842915329150856,1,0,0,1
+9.262432227842479e-05,0,1,2,0
+0.0009399560012388966,1,1,0,5
+-0.0006144636956742049,0,0,2,3
+0.0008483832946512837,1,1,2,3
+-0.00030775733001064606,1,0,2,2
+-0.001109839776763693,0,0,0,1
+-0.0001550889585632837,0,0,2,3
+-0.0003018033504486074,0,0,1,6
+-0.0006445696408627563,0,0,2,3
+-0.000798560787807219,0,0,0,1
+0.0001859147212235259,1,1,0,4
+-0.0004299826151691385,1,0,0,1
+-0.000770610470208339,0,0,2,2
+0.0005338080169167356,1,1,0,4
+-0.00032522321969736395,1,0,2,7
+-0.00038295160338748177,1,0,0,5
+-9.795450023375278e-06,1,0,0,5
+0.00011731942591723145,1,1,2,2
+0.00038712044246494784,1,1,0,4
+0.0005718573159538207,0,1,1,6
+-3.3809897722676674e-05,1,0,0,4
+0.00019337748410180183,1,1,0,4
+-0.017307846415787934,0,0,0,1
+-0.0002962720772484318,1,0,2,7
+-0.0014296687906607992,0,0,2,3
+-0.0002547142043476929,0,0,2,3
+0.00011334464885294493,1,1,0,4
+-9.496214333921602e-06,1,0,0,5
+-0.00988650084764231,0,0,2,3
+-0.0001071197749115526,0,0,2,0
+-0.00037261006655171444,0,0,0,5
+-0.00014053569815587232,0,0,0,4
+-0.0011779019189998505,0,0,2,3
+-0.002052289739367552,1,0,2,3
+8.254726708400995e-05,0,1,0,5
+-0.0024934692593524266,0,0,2,2
+-9.27806075196715e-05,0,0,2,7
+-0.0006061864225193857,0,0,2,2
+-0.0011669617576990296,0,0,2,2
+-0.0017233946931082753,0,0,2,0
+-0.0004923164861975228,0,0,2,3
+-0.0016619247698690734,0,0,2,0
+-0.002687537284800784,0,0,2,0
+-7.92442355304953e-05,0,0,1,6
+-0.0011137490748660654,0,0,2,0
+-0.0018419640528736653,0,0,2,0
+0.0010494905780069515,1,1,0,5
+0.0004917085688794023,0,1,2,7
+0.0010176322248298675,1,1,0,5
+-0.0018255392170976811,0,0,1,6
+-0.0006910552771296347,0,0,0,1
+-0.0006757713091792537,0,0,2,0
+-0.00027672704018186743,0,0,2,2
+7.109788130037382e-05,1,1,0,4
+0.0002513312833616506,1,1,0,1
+0.00034904146159533446,1,1,1,6
+-0.00015039470337796946,0,0,2,7
+-0.0018309434404363834,0,0,2,0
+0.0002319883432937795,1,1,0,4
+-0.001425915307481776,0,0,0,1
+-0.0004973617364885281,1,0,0,5
+-0.005016787371132522,0,0,2,2
+-0.0001138113660272208,0,0,0,1
+-0.00020031407824717268,1,0,2,3
+-0.0013222461170516887,0,0,2,0
+-0.002320195607608183,1,0,0,4
+8.520414121448647e-06,1,1,0,5
+0.0006307283334899688,1,1,1,6
+-0.002001439941232093,0,0,2,7
+-0.002059332074713899,0,0,2,2
+-0.0008754911960568278,0,0,0,1
+1.760402403306119e-05,0,1,0,5
+-0.01369144289579708,0,0,0,1
+5.3729093051515034e-05,0,1,0,1
+-6.766348553355686e-05,0,0,1,6
+-0.0025831409503007316,0,0,2,2
+0.0006450194667559118,0,1,0,5
+-0.0007565918524051092,0,0,2,3
+0.000450533038238065,1,1,0,4
+-0.0005707339436048647,1,0,2,7
+0.00014470510999672198,0,1,0,5
+-0.0003465599112678318,0,0,0,5
+-0.00033232005836907746,0,0,0,1
+-0.0006405031611211622,0,0,0,4
+-0.002044587665004656,0,0,2,0
+0.00014621990150772042,0,1,1,6
+0.000661071828217244,1,1,0,5
+0.0006380645226454363,1,1,2,2
+-0.00048502753081265844,0,0,0,1
+-0.0011695452791172997,0,0,2,0
+-0.0001637732470408087,0,0,0,5
+-0.00030820649117231473,0,0,2,7
+0.00010050916869658918,0,1,2,7
+-7.884975639171898e-05,0,0,2,7
+-0.0008562991523649553,0,0,0,1
+-0.0024420633597765154,0,0,0,1
+-0.00017120554926805184,0,0,2,3
+-0.000986730859731326,0,0,0,1
+-6.513940985314673e-05,1,0,2,2
+-0.0011483597574988395,0,0,2,2
+0.00022516651544719941,1,1,0,1
+0.0005378629697952424,1,1,2,7
+-0.00035888294689357184,0,0,0,4
+-0.003292822272633205,0,0,1,6
+-0.000477685718215071,0,0,2,3
+0.0003693028812995179,1,1,2,3
+-0.0009149601537501448,0,0,2,2
+-0.0024202558415709063,0,0,2,2
+0.0005403689958620814,1,1,0,5
+-0.0009762097755447018,0,0,2,2
+-0.004686072804033759,0,0,2,3
+-0.0016548697417601947,0,0,2,7
+-0.0006813665869412934,0,0,2,3
+-0.00234492618124932,0,0,2,3
+0.00013759101391769887,1,1,0,4
+-0.0014643730525858703,0,0,2,0
+0.00011206679511815369,1,1,1,6
+-0.000657233154634014,0,0,2,2
+0.0002355738944606854,1,1,2,3
+-0.0002042176527902468,0,0,0,4
+-8.148325141519447e-05,0,0,2,7
+-0.000889072772697546,0,0,2,3
+0.0001789627398829912,1,1,0,5
+-0.0006366569158853981,0,0,0,5
+0.0001246047636959697,0,1,2,7
+-0.0007173636526567858,0,0,2,0
+0.00015733252977952368,1,1,0,5
+-0.00012873380444943915,0,0,0,5
+-0.0027370838064234675,0,0,2,7
+-0.00041308835905510925,0,0,0,1
+-0.0001368265185737981,0,0,0,1
+-0.00015780505200382183,0,0,1,6
+-0.002228247933671809,0,0,2,2
+-0.0003494197566760709,0,0,2,7
+0.000268371559213847,1,1,0,1
+-0.0006737924419576331,0,0,2,7
+-0.006129581280983988,0,0,0,4
+-0.0008311191422399142,0,0,2,2
+-3.683806746266824e-05,1,0,0,5
+0.0008759256976190958,1,1,0,5
+-0.002160666911513544,0,0,2,0
+-0.0011655948159750556,0,0,2,0
+-8.710586640518075e-05,1,0,0,5
+-0.0009324602724518627,0,0,2,2
+-0.000493401298881509,0,0,0,5
+-0.00018312434607651099,1,0,1,6
+0.0012805658305296702,1,1,0,5
+0.0002124564285622913,0,1,2,7
+-0.0014664563391124825,0,0,2,0
+-0.001068552659125999,0,0,2,2
+-0.0005326424539089207,0,0,2,7
+-0.0037988398550078267,0,0,2,0
+-7.83926720032465e-05,0,0,2,2
+0.0005783538543619217,1,1,0,1
+2.1720206714235385e-05,1,1,0,1
+-8.654431381728469e-05,0,0,2,0
+-0.00272757826372981,0,0,2,3
+-0.0003443170455284406,0,0,0,5
+-0.0008726302155992009,1,0,2,0

results/single_split_lambda_ablation/inkjet_lambda0.02_k100/metrics.json

@@ -0,0 +1,54 @@
+{
+  "overall_auroc": 0.854135432283858,
+  "overall_acc": 0.8007518796992481,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.617283950617284,
+      "accuracy": 0.8666666666666667,
+      "n_good": 27,
+      "n_bad": 3
+    },
+    "dist1": {
+      "auroc": 0.9428571428571428,
+      "accuracy": 0.8387096774193549,
+      "n_good": 21,
+      "n_bad": 10
+    },
+    "dist6": {
+      "auroc": 0.8388888888888889,
+      "accuracy": 0.9444444444444444,
+      "n_good": 30,
+      "n_bad": 6
+    },
+    "dots": {
+      "auroc": 0.9125874125874126,
+      "accuracy": 0.972972972972973,
+      "n_good": 26,
+      "n_bad": 11
+    },
+    "edge1": {
+      "auroc": 0.8593750000000001,
+      "accuracy": 0.8214285714285714,
+      "n_good": 12,
+      "n_bad": 16
+    },
+    "edge2": {
+      "auroc": 0.6785714285714287,
+      "accuracy": 0.5869565217391305,
+      "n_good": 18,
+      "n_bad": 28
+    },
+    "edge3": {
+      "auroc": 0.7708333333333334,
+      "accuracy": 0.75,
+      "n_good": 16,
+      "n_bad": 12
+    },
+    "edge4": {
+      "auroc": 0.6805555555555556,
+      "accuracy": 0.6666666666666666,
+      "n_good": 24,
+      "n_bad": 6
+    }
+  }
+}

results/single_split_lambda_ablation/inkjet_lambda0.02_k100/scores.csv

@@ -0,0 +1,267 @@
+score,true_label,pred_label,template_id,feature_id
+-9.409602498635337e-06,1,0,0,1
+-0.00117481164750643,0,0,0,1
+-0.00022798100893851522,0,0,0,5
+-0.0005808233455172757,0,0,2,7
+0.0004220506217097863,1,1,0,5
+0.00018639477289980264,1,1,0,1
+-0.0018734975243569346,0,0,1,6
+-0.001459128575515934,0,0,0,1
+-0.0006774348620092498,0,0,2,2
+-5.2412613877094935e-05,0,0,1,6
+0.0001878282026154931,1,1,0,4
+0.00019293967139674337,1,1,0,5
+-0.0022439529679832045,0,0,2,0
+-0.0017748926172498607,0,0,2,0
+-0.0006807471084175638,0,0,2,7
+6.6681855241772225e-06,1,1,1,6
+0.00010350940836360671,1,1,2,3
+-0.00014084467897191623,0,0,2,3
+-0.0001961025202763273,0,0,2,3
+0.00015818774700164864,0,1,1,6
+7.274432864505347e-06,0,1,0,4
+0.00041356145404279274,0,1,0,5
+-0.0030110454082023334,0,0,2,2
+-0.00271790709462948,0,0,2,2
+0.00045981565228430527,0,1,0,5
+-8.895871083950714e-05,0,0,0,5
+-0.0002150561517919418,0,0,2,3
+0.0007854697131551812,0,1,0,5
+0.00014090996352024432,1,1,0,5
+-0.0006900632884935497,0,0,0,4
+-2.7652573189698368e-05,0,0,1,6
+-0.0006215485438588082,0,0,0,1
+-7.441656052833458e-05,1,0,1,6
+-0.0004548868245910845,0,0,0,4
+-0.0017442890361417072,0,0,2,0
+-0.0031411312098498477,1,0,2,2
+4.9374426889698536e-05,1,1,0,4
+-0.004045859584584834,0,0,2,3
+0.0007816695712972409,1,1,0,5
+-0.002090899780159817,0,0,2,0
+-0.000322174370230641,1,0,0,5
+-1.2327956210355165e-05,0,0,2,7
+0.0006719089185935438,1,1,1,6
+0.0002568885943037471,1,1,2,2
+0.0005909879814134916,1,1,2,3
+8.605906768934836e-05,1,1,1,6
+-0.00038294804748147636,0,0,2,2
+-0.0003691386504215202,0,0,0,4
+-0.0001597450149711209,0,0,0,5
+-0.00023777413181960607,0,0,0,5
+0.0003189288685098296,1,1,2,3
+-0.00034425275283865575,0,0,0,4
+-0.00023167756269685917,0,0,2,7
+5.95485456869934e-05,0,1,0,1
+0.0012346327214618237,1,1,0,5
+-4.847217089263764e-05,0,0,2,7
+-0.00022389175923308302,1,0,0,5
+-0.006565188036765902,0,0,0,1
+-0.00038607150985626483,0,0,2,3
+-0.0002580241442774415,1,0,0,5
+-0.0007718314195517446,0,0,2,3
+-0.0007698093503131533,0,0,2,7
+0.0001667204694240365,1,1,1,6
+-0.004753650213242509,0,0,2,7
+0.0001766444969689418,0,1,0,4
+-0.0008442663567257112,0,0,2,2
+-0.001417475390480833,0,0,2,2
+0.0010506788795464628,1,1,0,4
+-0.0001880023119156249,0,0,2,3
+-0.0011762381863081837,0,0,2,2
+-0.00042147800209932015,0,0,2,3
+-0.00025896465609548565,1,0,2,0
+0.00026006589119788315,0,1,1,6
+-0.0010853517838404518,0,0,2,0
+-0.000680754274653736,0,0,2,3
+-2.9522441618606843e-05,1,0,0,5
+-0.0009780643461272118,0,0,2,0
+0.00016178193502128062,1,1,2,3
+-0.0008494415949098777,0,0,2,2
+-5.667003046255434e-05,0,0,1,6
+-0.00010125425586011289,1,0,2,7
+-0.0007333122967975204,0,0,2,3
+-0.0002534689186722977,0,0,2,0
+0.0005221006777719611,1,1,2,3
+-0.00013482655398547663,1,0,0,5
+-0.0020934763306286186,1,0,2,0
+0.00014052509330213063,1,1,2,7
+0.00048009503661887616,1,1,1,6
+-0.003997093138750643,0,0,2,2
+4.6795420639683324e-05,0,1,2,7
+0.0001184559636749332,1,1,0,5
+0.00031109087314689504,1,1,0,4
+-5.7416908675805844e-05,0,0,1,6
+7.388050318695577e-05,1,1,1,6
+-0.0007044936795136889,1,0,0,4
+-0.000664675322768744,0,0,2,2
+-0.00038248254888458184,0,0,2,7
+0.0009309151029447092,1,1,2,3
+-5.685075564542709e-05,0,0,1,6
+1.8042207811959712e-05,1,1,0,1
+-0.0009777118990314197,0,0,2,2
+-0.0004448722422239369,0,0,2,2
+-0.00043544354673940786,0,0,0,4
+0.00021590855205431622,1,1,1,6
+-0.0008474377068341714,0,0,2,0
+0.0042248036514502055,1,1,0,5
+-0.00010951914882753253,1,0,0,1
+4.7736860869918366e-05,0,1,2,0
+0.0009828324412228546,1,1,0,5
+-0.0005770141605171385,0,0,2,3
+0.0008323897182708623,1,1,2,3
+-0.0002733880846062676,1,0,2,2
+-0.0008264382692868832,0,0,0,1
+-0.0001781467258115295,0,0,2,3
+-0.0002372728384216309,0,0,1,6
+-0.0007454971733386623,0,0,2,3
+-0.001534268908435485,0,0,0,1
+0.000204295644070953,1,1,0,4
+-0.00045281607628567047,1,0,0,1
+-0.00038731777429347816,0,0,2,2
+0.0003178475724416787,1,1,0,4
+-0.00020157701801508388,1,0,2,7
+-0.0003067120141349716,1,0,0,5
+-6.0573941736946635e-05,1,0,0,5
+7.32017686823393e-05,1,1,2,2
+0.00032787767122499645,1,1,0,4
+0.000412460618827026,0,1,1,6
+-4.257042310199313e-06,1,0,0,4
+0.00023441929690306944,1,1,0,4
+-0.018741075064754108,0,0,0,1
+-0.0003189279185608039,1,0,2,7
+-0.0013724985803128219,0,0,2,3
+-0.00023794946813723156,0,0,2,3
+0.00012262817821465447,1,1,0,4
+-0.0001718965341569855,1,0,0,5
+-0.009818871463940015,0,0,2,3
+-0.0001851948798866944,0,0,2,0
+-0.00019382925704121597,0,0,0,5
+-0.00014259787829359882,0,0,0,4
+-0.0008165735387592587,0,0,2,3
+-0.00523449002939742,1,0,2,3
+5.609203362837156e-06,0,1,0,5
+-0.002739124959334732,0,0,2,2
+-0.00012900298024760577,0,0,2,7
+-0.0009120800704113217,0,0,2,2
+-0.0008465686140698373,0,0,2,2
+-0.0015607980254571888,0,0,2,0
+-0.0002741311216959714,0,0,2,3
+-0.001201413181552197,0,0,2,0
+-0.002708442774310243,0,0,2,0
+-8.573520375648562e-05,0,0,1,6
+-0.0011419284113799218,0,0,2,0
+-0.002174701869953423,0,0,2,0
+0.0009037935163360086,1,1,0,5
+0.00033693941688397924,0,1,2,7
+0.0010376423061825325,1,1,0,5
+-0.0022373640275327525,0,0,1,6
+-0.0009089981776196528,0,0,0,1
+-0.001244257584330626,0,0,2,0
+-0.0005000539388856856,0,0,2,2
+5.672487284755397e-05,1,1,0,4
+0.0002330103004351252,1,1,0,1
+0.00041992836835561334,1,1,1,6
+-0.00028866022999864127,0,0,2,7
+-0.0008067962329369044,0,0,2,0
+0.00019234630075515985,1,1,0,4
+-0.0015892996010370558,0,0,0,1
+-0.0002116079168627029,1,0,0,5
+-0.005105913230800074,0,0,2,2
+-0.00019543848815374056,0,0,0,1
+9.751491626957404e-05,1,1,2,3
+-0.0010506303847068934,0,0,2,0
+-0.0020941358513664465,1,0,0,4
+-0.00023213605047203711,1,0,0,5
+0.0007746418617898612,1,1,1,6
+-0.002051635537063702,0,0,2,7
+-0.0011961307949968601,0,0,2,2
+-0.0005310384038602935,0,0,0,1
+7.390501559711646e-06,0,1,0,5
+-0.015318541259330229,0,0,0,1
+3.5036357294302226e-05,0,1,0,1
+3.3842779521365066e-05,0,1,1,6
+-0.002272178364219144,0,0,2,2
+0.0005673012800980343,0,1,0,5
+-0.0007501407954259767,0,0,2,3
+0.0004088994377525537,1,1,0,4
+-0.00019879601051798167,1,0,2,7
+0.00015038753452244896,0,1,0,5
+-0.0003837984491838146,0,0,0,5
+-0.00029368352319579516,0,0,0,1
+-0.0004237902013119292,0,0,0,4
+-0.0025109179891296685,0,0,2,0
+0.00029941150452941767,0,1,1,6
+0.0006280265559325919,1,1,0,5
+0.0007041820566519169,1,1,2,2
+-0.0005988202223670674,0,0,0,1
+-0.001027280251437333,0,0,2,0
+-7.044610392767901e-05,0,0,0,5
+-0.0002582021095440725,0,0,2,7
+8.633133489638518e-05,0,1,2,7
+-3.139298991300041e-05,0,0,2,7
+-0.0005265348541433935,0,0,0,1
+-0.0021822211469407216,0,0,0,1
+-0.0002677277132170277,0,0,2,3
+-0.0007598402365692897,0,0,0,1
+6.387119006831185e-05,1,1,2,2
+-0.0009458841109881173,0,0,2,2
+8.76888906350369e-05,1,1,0,1
+0.0005650942985084839,1,1,2,7
+-0.00044037683954229606,0,0,0,4
+-0.003918938872520809,0,0,1,6
+-0.0006189954144065271,0,0,2,3
+0.0005184506188379602,1,1,2,3
+-0.000982458372891416,0,0,2,2
+-0.002066419335023967,0,0,2,2
+0.0008370199604542002,1,1,0,5
+-0.0007687420502770692,0,0,2,2
+-0.0032794535896391586,0,0,2,3
+-0.001870386942755431,0,0,2,7
+-0.0006192012119572605,0,0,2,3
+-0.0026285178793477824,0,0,2,3
+0.00024492861091857977,1,1,0,4
+-0.001743820024130401,0,0,2,0
+0.00010299961752025487,1,1,1,6
+-0.0011439309926936404,0,0,2,2
+0.00030325500410981546,1,1,2,3
+-0.00046452833339571953,0,0,0,4
+9.829398622969183e-05,0,1,2,7
+-0.0005926837396691559,0,0,2,3
+0.00017673311143880793,1,1,0,5
+-0.0005226827939623022,0,0,0,5
+2.1766932331955882e-05,0,1,2,7
+-0.0014421960074105303,0,0,2,0
+7.08097591996179e-05,1,1,0,5
+-0.00027456082258140585,0,0,0,5
+-0.0031860506502562197,0,0,2,7
+-0.000665246715070681,0,0,0,1
+-5.633491440676201e-05,0,0,0,1
+-0.00012886010168585912,0,0,1,6
+-0.0021101105023990374,0,0,2,2
+-0.00030634833005024117,0,0,2,7
+0.00027251318591879625,1,1,0,1
+-0.0003095682049752205,0,0,2,7
+-0.006074961612757758,0,0,0,4
+-0.000940793145564384,0,0,2,2
+-7.537035678979037e-05,1,0,0,5
+0.0009754837749642309,1,1,0,5
+-0.0022951831217505984,0,0,2,0
+-0.0007769780035596328,0,0,2,0
+-0.00011979268747381883,1,0,0,5
+-0.0011278446685173554,0,0,2,2
+-0.00024119969806634008,0,0,0,5
+-0.00012292734492803242,1,0,1,6
+0.0017210749533842287,1,1,0,5
+0.00016896016401005912,0,1,2,7
+-0.001170039420539979,0,0,2,0
+-0.0009065983843174752,0,0,2,2
+-0.0005659142107469952,0,0,2,7
+-0.0031841721417731617,0,0,2,0
+-0.0001217962341615933,0,0,2,2
+0.0004133794779772864,1,1,0,1
+7.473795383702853e-05,1,1,0,1
+-0.00036375037918333006,0,0,2,0
+-0.0037449285059119562,0,0,2,3
+-0.0001847984967753291,0,0,0,5
+-0.0009314114382141282,1,0,2,0

results/single_split_lambda_ablation/inkjet_lambda0.05/metrics_final.json

@@ -0,0 +1,54 @@
+{
+  "overall_auroc": 0.8552598700649676,
+  "overall_acc": 0.8233082706766918,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.5555555555555556,
+      "accuracy": 0.9,
+      "n_good": 27,
+      "n_bad": 3
+    },
+    "dist1": {
+      "auroc": 0.9142857142857143,
+      "accuracy": 0.8709677419354839,
+      "n_good": 21,
+      "n_bad": 10
+    },
+    "dist6": {
+      "auroc": 0.8277777777777778,
+      "accuracy": 0.8333333333333334,
+      "n_good": 30,
+      "n_bad": 6
+    },
+    "dots": {
+      "auroc": 0.8881118881118881,
+      "accuracy": 0.8918918918918919,
+      "n_good": 26,
+      "n_bad": 11
+    },
+    "edge1": {
+      "auroc": 0.8541666666666667,
+      "accuracy": 0.8571428571428571,
+      "n_good": 12,
+      "n_bad": 16
+    },
+    "edge2": {
+      "auroc": 0.7857142857142858,
+      "accuracy": 0.717391304347826,
+      "n_good": 18,
+      "n_bad": 28
+    },
+    "edge3": {
+      "auroc": 0.8229166666666666,
+      "accuracy": 0.7857142857142857,
+      "n_good": 16,
+      "n_bad": 12
+    },
+    "edge4": {
+      "auroc": 0.6597222222222222,
+      "accuracy": 0.7666666666666667,
+      "n_good": 24,
+      "n_bad": 6
+    }
+  }
+}

results/single_split_lambda_ablation/inkjet_lambda0.05/scores_final.csv

@@ -0,0 +1,267 @@
+score,true_label,pred_label,template_id,feature_id
+-0.0001592397649073981,1,0,0,1
+-0.0018998265138361614,0,0,0,1
+-0.0002784400479868051,0,0,0,5
+-0.0007066390666295858,0,0,2,7
+0.00048563592688878957,1,1,0,5
+0.00020659955509472628,1,1,0,1
+-0.0035053329743095635,0,0,1,6
+-0.003374322981690058,0,0,0,1
+-0.00268009888852248,0,0,2,2
+-8.532976760761753e-05,0,0,1,6
+0.00014313155756099163,1,1,0,4
+6.0744073707610924e-05,1,1,0,5
+-0.004466575622500385,0,0,2,0
+-0.003937088764505459,0,0,2,0
+-0.001513860742561518,0,0,2,7
+5.219213140662815e-05,1,1,1,6
+-0.00011175725056091455,1,0,2,3
+-0.0005380858617718334,0,0,2,3
+-0.0018141952133737503,0,0,2,3
+-2.6692662504504416e-05,0,0,1,6
+1.799858961021479e-05,0,1,0,4
+0.0009286602429347118,0,1,0,5
+-0.01546992276853416,0,0,2,2
+-0.013023520717106295,0,0,2,2
+9.622131852665868e-05,0,1,0,5
+-0.00015838213381357512,0,0,0,5
+-0.0005901245071436283,0,0,2,3
+0.0010358809394529107,0,1,0,5
+0.0002546311466721813,1,1,0,5
+-0.0007159646088257428,0,0,0,4
+9.480852691922358e-05,0,1,1,6
+-0.00044594048231374334,0,0,0,1
+-3.425280796364068e-05,1,0,1,6
+-0.00034932065318571404,0,0,0,4
+-0.0030425050901249067,0,0,2,0
+-0.030971354083449112,1,0,2,2
+0.0004946988346637218,1,1,0,4
+-0.011626278418989389,0,0,2,3
+0.0015207907551666733,1,1,0,5
+-0.004999385040719062,0,0,2,0
+-0.00011667024984490099,1,0,0,5
+-0.00016121646185638348,0,0,2,7
+0.0006497135455720126,1,1,1,6
+-0.0003620844634133393,1,0,2,2
+0.0004387764111743308,1,1,2,3
+0.000366573915525803,1,1,1,6
+-0.001063287304132245,0,0,2,2
+-0.0003029073693323857,0,0,0,4
+-6.946734181838103e-05,0,0,0,5
+-0.00043388415040681097,0,0,0,5
+0.0002842554391827429,1,1,2,3
+-0.0003732068609679125,0,0,0,4
+-0.00022557009477168315,0,0,2,7
+-3.884106408804603e-05,0,0,0,1
+0.0014397003530757513,1,1,0,5
+-0.00046674477460328413,0,0,2,7
+-0.00010173634917009719,1,0,0,5
+-0.03554955237021204,0,0,0,1
+-0.0005229029938345776,0,0,2,3
+-0.0001989761556615132,1,0,0,5
+-0.0006772644913871761,0,0,2,3
+-0.0016069551184773444,0,0,2,7
+0.00027722686354536563,1,1,1,6
+-0.010205767490842847,0,0,2,7
+0.00017538661893922586,0,1,0,4
+-0.002203817003755831,0,0,2,2
+-0.008100647752871736,0,0,2,2
+0.0010325290999026043,1,1,0,4
+-0.0021524250152288003,0,0,2,3
+-0.004133386196044739,0,0,2,2
+-0.0013327481117448772,0,0,2,3
+-0.001224387570109682,1,0,2,0
+0.000162049058708362,0,1,1,6
+-0.0017088479094672943,0,0,2,0
+-0.0018889533120091083,0,0,2,3
+0.003798747159889898,1,1,0,5
+-0.0020378913168678982,0,0,2,0
+-0.00012112790980609196,1,0,2,3
+-0.0016415474607492792,0,0,2,2
+-3.5050910373684435e-05,0,0,1,6
+-7.745263603283099e-05,1,0,2,7
+-0.0007111057350994094,0,0,2,3
+-0.00041670033475384085,0,0,2,0
+0.05031887321238172,1,1,2,3
+5.346000602003166e-05,1,1,0,5
+-0.051926554813690025,1,0,2,0
+6.861413276055932e-05,1,1,2,7
+0.0007458278397098166,1,1,1,6
+-0.04154831114632543,0,0,2,2
+-6.42334666918036e-05,0,0,2,7
+0.0001045058190356947,1,1,0,5
+6.946220149984562e-05,1,1,0,4
+-2.8929181280546584e-05,0,0,1,6
+0.0004688128340058044,1,1,1,6
+-0.0009484835847979392,1,0,0,4
+-0.0017840396010433326,0,0,2,2
+-0.0005015942428144632,0,0,2,7
+0.0006938104334403769,1,1,2,3
+-5.7403736573176725e-05,0,0,1,6
+-0.00011474238883238397,1,0,0,1
+-0.008191466318967288,0,0,2,2
+-0.0019034759551868775,0,0,2,2
+-0.0006205726732150633,0,0,0,4
+0.0001812668240745545,1,1,1,6
+-0.0007735640494502152,0,0,2,0
+0.004523413673741743,1,1,0,5
+-0.0004503506000037294,1,0,0,1
+-0.0002636740400339477,0,0,2,0
+0.0009884831073577512,1,1,0,5
+-0.0011141543628764346,0,0,2,3
+0.04321976592909778,1,1,2,3
+-0.0006606482053757648,1,0,2,2
+-0.0023509266757173464,0,0,0,1
+-0.0003237314356374539,0,0,2,3
+-0.0003630366700235754,0,0,1,6
+-0.0012124813452828662,0,0,2,3
+-0.0025423468489316273,0,0,0,1
+0.00026109381695278017,1,1,0,4
+-0.0005034181289374825,1,0,0,1
+-0.0008445911516901106,0,0,2,2
+0.0003671474498696623,1,1,0,4
+-0.0008968400658341119,1,0,2,7
+-6.198889488587267e-05,1,0,0,5
+-8.610711636720253e-05,1,0,0,5
+-0.000252794926927891,1,0,2,2
+0.00017472650884883464,1,1,0,4
+0.0005897508992347873,0,1,1,6
+1.0314508690498561e-05,1,1,0,4
+0.00034222515183500966,1,1,0,4
+-0.059699533089878964,0,0,0,1
+-0.0007718798358109773,1,0,2,7
+-0.0029309808593825423,0,0,2,3
+-0.0006766853833687497,0,0,2,3
+0.00017796780652133692,1,1,0,4
+2.1939461585133296e-05,1,1,0,5
+-0.02088331920764176,0,0,2,3
+-0.0003410174913005908,0,0,2,0
+-0.00024081414507236276,0,0,0,5
+-0.00043110596976475915,0,0,0,4
+-0.001225666251848452,0,0,2,3
+-0.005743525353609585,1,0,2,3
+-3.9582401514054455e-05,0,0,0,5
+-0.032900327951647344,0,0,2,2
+-0.00015041175996884787,0,0,2,7
+-0.0017172095392015765,0,0,2,2
+-0.0018793530418770386,0,0,2,2
+-0.004344977112777999,0,0,2,0
+-0.0005913388615590535,0,0,2,3
+-0.0013199239858658982,0,0,2,0
+-0.004773552495171314,0,0,2,0
+-5.376122426241706e-05,0,0,1,6
+-0.0017978135636076333,0,0,2,0
+-0.004740275621879847,0,0,2,0
+0.0014213312312494966,1,1,0,5
+0.00010947335627861307,0,1,2,7
+0.0014731313847005378,1,1,0,5
+-0.0031374064419651404,0,0,1,6
+-0.0017707363274530527,0,0,0,1
+-0.0014371239888714624,0,0,2,0
+-0.001008949419192504,0,0,2,2
+0.00010075298283481957,1,1,0,4
+0.00032754887855844517,1,1,0,1
+0.0004637636587722215,1,1,1,6
+-0.0002540946225053631,0,0,2,7
+-0.0010553229629294949,0,0,2,0
+0.00038524777104612454,1,1,0,4
+-0.0032326077061588902,0,0,0,1
+-0.00016973479912849206,1,0,0,5
+-0.03715985778515459,0,0,2,2
+-0.0002484075695974759,0,0,0,1
+-0.000905716785055119,1,0,2,3
+-0.000922550451068674,0,0,2,0
+-0.004098330081324084,1,0,0,4
+-0.00044287547381827684,1,0,0,5
+0.0009919781604548934,1,1,1,6
+-0.005609944316674956,0,0,2,7
+-0.0031222088891081513,0,0,2,2
+-0.0012427354877581828,0,0,0,1
+-0.00010936257982393713,0,0,0,5
+-0.05718700129946228,0,0,0,1
+-7.69747918820951e-05,0,0,0,1
+-0.00013091978820739363,0,0,1,6
+-0.02552066030271817,0,0,2,2
+0.0009542101851548069,0,1,0,5
+-0.0012819435619167048,0,0,2,3
+0.0004793260517180903,1,1,0,4
+-0.00029288038669619647,1,0,2,7
+6.420824327505945e-05,0,1,0,5
+-0.00023278327164007358,0,0,0,5
+-0.0010831142010283672,0,0,0,1
+-0.0006697806378360843,0,0,0,4
+-0.005897079715214207,0,0,2,0
+0.0004236544255400085,0,1,1,6
+0.0014080935815582035,1,1,0,5
+-0.00036015745834447506,1,0,2,2
+-0.000896892244636547,0,0,0,1
+-0.0024460361825185822,0,0,2,0
+-0.00010687950445571921,0,0,0,5
+-0.0011168620878015643,0,0,2,7
+-1.600649557073966e-07,0,0,2,7
+-0.0006386051402660076,0,0,2,7
+-0.0013563706379500218,0,0,0,1
+-0.007567480840953059,0,0,0,1
+-0.0004488295881310486,0,0,2,3
+-0.001868236176960636,0,0,0,1
+-0.0012129628445836718,1,0,2,2
+-0.002281356577877888,0,0,2,2
+0.00013088090927340125,1,1,0,1
+0.0005980666217510582,1,1,2,7
+-0.0005754827309283426,0,0,0,4
+-0.005528701829607602,0,0,1,6
+-0.00037439808977069323,0,0,2,3
+0.00043141661997651674,1,1,2,3
+-0.0039058838947676113,0,0,2,2
+-0.014886964271427133,0,0,2,2
+0.0011466915340861313,1,1,0,5
+-0.0015492478746455176,0,0,2,2
+-0.011473436948435847,0,0,2,3
+-0.005219232577946969,0,0,2,7
+-0.0007049470918718721,0,0,2,3
+-0.010998086407780647,0,0,2,3
+0.00025549185433192123,1,1,0,4
+-0.002945264034788125,0,0,2,0
+0.00014561857911758042,1,1,1,6
+-0.006551974124158734,0,0,2,2
+0.00021512517763767457,1,1,2,3
+-0.00074104428931605,0,0,0,4
+1.800585218006759e-05,0,1,2,7
+-0.0010249490026035344,0,0,2,3
+0.0005387677229009569,1,1,0,5
+-0.0003940664767287678,0,0,0,5
+2.8139707865193486e-05,0,1,2,7
+-0.0017383605486247684,0,0,2,0
+0.00018160823761718264,1,1,0,5
+-0.00021446981932967887,0,0,0,5
+-0.0064907226560171685,0,0,2,7
+-0.00043418902001576484,0,0,0,1
+-5.6197475641965935e-05,0,0,0,1
+-0.00027397392434068066,0,0,1,6
+-0.017608689418993892,0,0,2,2
+-0.0005386151443235575,0,0,2,7
+0.00021203364070970614,1,1,0,1
+-0.0007072332303505388,0,0,2,7
+-0.006485925308661537,0,0,0,4
+-0.0022838486765976995,0,0,2,2
+7.790885400027047e-05,1,1,0,5
+0.0012017442286014549,1,1,0,5
+-0.0035869024074054335,0,0,2,0
+-0.001515544249559752,0,0,2,0
+-1.557416544528678e-05,1,0,0,5
+-0.001641261474578641,0,0,2,2
+-0.0002390708003076735,0,0,0,5
+-0.00042762498720548894,1,0,1,6
+0.0015993185143452146,1,1,0,5
+-5.585133010754372e-05,0,0,2,7
+-0.002366304442693947,0,0,2,0
+-0.003271399521327111,0,0,2,2
+-0.001333614136674443,0,0,2,7
+-0.08892439779010602,0,0,2,0
+-0.0007031988486414769,0,0,2,2
+0.00041802095685852716,1,1,0,1
+3.4572534495964924e-05,1,1,0,1
+-0.0009463326106197208,0,0,2,0
+-0.009987564272014426,0,0,2,3
+-0.00018211308051831968,0,0,0,5
+-0.00046862048271577826,1,0,2,0

results/single_split_lambda_ablation/inkjet_lambda0/metrics_final.json

@@ -0,0 +1,54 @@
+{
+  "overall_auroc": 0.8358945527236381,
+  "overall_acc": 0.7857142857142857,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.4197530864197531,
+      "accuracy": 0.9,
+      "n_good": 27,
+      "n_bad": 3
+    },
+    "dist1": {
+      "auroc": 0.9428571428571428,
+      "accuracy": 0.8709677419354839,
+      "n_good": 21,
+      "n_bad": 10
+    },
+    "dist6": {
+      "auroc": 0.8444444444444446,
+      "accuracy": 0.8611111111111112,
+      "n_good": 30,
+      "n_bad": 6
+    },
+    "dots": {
+      "auroc": 0.9090909090909091,
+      "accuracy": 0.8918918918918919,
+      "n_good": 26,
+      "n_bad": 11
+    },
+    "edge1": {
+      "auroc": 0.7916666666666666,
+      "accuracy": 0.75,
+      "n_good": 12,
+      "n_bad": 16
+    },
+    "edge2": {
+      "auroc": 0.7142857142857143,
+      "accuracy": 0.6086956521739131,
+      "n_good": 18,
+      "n_bad": 28
+    },
+    "edge3": {
+      "auroc": 0.8020833333333334,
+      "accuracy": 0.6428571428571429,
+      "n_good": 16,
+      "n_bad": 12
+    },
+    "edge4": {
+      "auroc": 0.6527777777777778,
+      "accuracy": 0.8,
+      "n_good": 24,
+      "n_bad": 6
+    }
+  }
+}

results/single_split_lambda_ablation/inkjet_lambda0/scores_final.csv

@@ -0,0 +1,267 @@
+score,true_label,pred_label,template_id,feature_id
+5.7551218196750034e-05,1,1,0,1
+-0.001919698463752867,0,0,0,1
+-0.0002801612019538869,0,0,0,5
+-0.0012604985851794592,0,0,2,7
+0.0007614469318650667,1,1,0,5
+0.000207797228358687,1,1,0,1
+-0.0021943268808536226,0,0,1,6
+-0.00221378812333569,0,0,0,1
+-0.0005044720554724328,0,0,2,2
+0.0001155180623754852,0,1,1,6
+5.3603281266988156e-05,1,1,0,4
+-6.772173568606217e-05,1,0,0,5
+-0.001275857021100819,0,0,2,0
+-0.0022521049645729332,0,0,2,0
+-0.0020283927163109172,0,0,2,7
+7.072885287925587e-05,1,1,1,6
+-0.00011883872561156694,1,0,2,3
+-0.00026168195065110877,0,0,2,3
+-8.310351753607406e-05,0,0,2,3
+7.695412030443542e-05,0,1,1,6
+0.0002913657692261047,0,1,0,4
+0.00026998332468792727,0,1,0,5
+-0.0025803715060465048,0,0,2,2
+-0.0021698269876651473,0,0,2,2
+0.0002811172651126983,0,1,0,5
+-0.00022117533488199073,0,0,0,5
+-0.0003338733222335577,0,0,2,3
+0.00031824513804167703,0,1,0,5
+0.0002705285768024625,1,1,0,5
+-0.00040855797007679877,0,0,0,4
+0.00010407901369035265,0,1,1,6
+-0.0004897593986242996,0,0,0,1
+3.3445800654589505e-05,1,1,1,6
+-0.00038305133115500156,0,0,0,4
+-0.002018807441927492,0,0,2,0
+-0.002244205535389482,1,0,2,2
+-0.00047680834075436093,1,0,0,4
+-0.0018371775723062464,0,0,2,3
+0.0015131336590275164,1,1,0,5
+-0.0010531063424423347,0,0,2,0
+-0.0006723039876669647,1,0,0,5
+-4.3399613350629355e-05,0,0,2,7
+0.0009233176056295637,1,1,1,6
+-4.292860627174225e-05,1,0,2,2
+0.0003003005008213218,1,1,2,3
+0.0001923259254544979,1,1,1,6
+-0.0010218048072420036,0,0,2,2
+-0.000691127344034613,0,0,0,4
+1.0447362437845595e-05,0,1,0,5
+-0.00033588680671527843,0,0,0,5
+0.0004220502637326713,1,1,2,3
+-0.0001668356521986429,0,0,0,4
+-0.0002698402316309509,0,0,2,7
+1.3837032020090623e-05,0,1,0,1
+0.0009527966962195947,1,1,0,5
+-0.00037494786083698203,0,0,2,7
+-0.0005327281379140895,1,0,0,5
+-0.005231531669851394,0,0,0,1
+-0.00048153606709092946,0,0,2,3
+-0.00019087775610387298,1,0,0,5
+-0.0007633859477937221,0,0,2,3
+-0.0008225443889386952,0,0,2,7
+0.00026314974296838024,1,1,1,6
+-0.0045633310033008495,0,0,2,7
+4.9973139539361e-05,0,1,0,4
+-0.0009676067624241128,0,0,2,2
+-0.0013978558103553973,0,0,2,2
+0.0011666525760665597,1,1,0,4
+-0.0002819814509712158,0,0,2,3
+-0.0008924561692401772,0,0,2,2
+-0.0009028474311344337,0,0,2,3
+-0.0009021366713568551,1,0,2,0
+0.000697369400877508,0,1,1,6
+-0.0009422192908823479,0,0,2,0
+-0.001150707476772369,0,0,2,3
+0.0019762378744781067,1,1,0,5
+-0.0016549192671664083,0,0,2,0
+-6.255040643736703e-05,1,0,2,3
+-0.0004580228147096927,0,0,2,2
+2.9036027844995052e-05,0,1,1,6
+-0.00040300558088347295,1,0,2,7
+-0.00045378102222457493,0,0,2,3
+-0.0003248706669546665,0,0,2,0
+0.0009797868691384798,1,1,2,3
+-3.0779920052737e-05,1,0,0,5
+-0.0035714322701096493,1,0,2,0
+5.81896770745511e-05,1,1,2,7
+0.0007284647016786022,1,1,1,6
+-0.0030200561555102437,0,0,2,2
+-0.00014041771413758446,0,0,2,7
+0.00024106626864522744,1,1,0,5
+0.00039492739830166226,1,1,0,4
+0.0002605610340833675,0,1,1,6
+0.00035567470360547276,1,1,1,6
+-0.0007817478803917796,1,0,0,4
+-0.001123844378162174,0,0,2,2
+-0.0005022743530571448,0,0,2,7
+0.0008627474354580052,1,1,2,3
+0.00011481111869215965,0,1,1,6
+4.342064727097701e-05,1,1,0,1
+-0.0009610124723985793,0,0,2,2
+-0.0007681142399087546,0,0,2,2
+-0.00028180816210806453,0,0,0,4
+0.00019122073892504157,1,1,1,6
+-0.002844501654617488,0,0,2,0
+0.004425540096126494,1,1,0,5
+-0.00024070929735899027,1,0,0,1
+-0.0003072323417291044,0,0,2,0
+0.001507644192315638,1,1,0,5
+-0.0018058339576236884,0,0,2,3
+0.0030552077037282466,1,1,2,3
+-0.0003315883432514974,1,0,2,2
+-0.0018858680361881833,0,0,0,1
+-0.0003919961559586214,0,0,2,3
+-0.00025156890042126075,0,0,1,6
+-0.0008207243029028175,0,0,2,3
+-0.0011734424717724304,0,0,0,1
+0.00027147047920152503,1,1,0,4
+-0.0003729116311296833,1,0,0,1
+-0.0005696904100477695,0,0,2,2
+0.0003396153100766238,1,1,0,4
+-0.0005323479254730028,1,0,2,7
+-0.0003580393223091959,1,0,0,5
+-4.264130722731255e-05,1,0,0,5
+-8.99090874008834e-05,1,0,2,2
+0.000549056252930313,1,1,0,4
+0.0006836400623433311,0,1,1,6
+7.782299304381034e-05,1,1,0,4
+0.0006734072114340951,1,1,0,4
+-0.012588279345072811,0,0,0,1
+-0.0003539299406111248,1,0,2,7
+-0.0005336239770986143,0,0,2,3
+-0.00025908891111612285,0,0,2,3
+-0.0002730195270851249,1,0,0,4
+6.578353699296727e-05,1,1,0,5
+-0.0037926488439552486,0,0,2,3
+-0.0005042949831113221,0,0,2,0
+-0.00019963656086474657,0,0,0,5
+-0.0005096668959595242,0,0,0,4
+-0.0008968078834004703,0,0,2,3
+-0.0024554694723337885,1,0,2,3
+0.00034681995399296617,0,1,0,5
+-0.0016980258934199827,0,0,2,2
+-0.0002697434695437548,0,0,2,7
+-0.0006097170547582215,0,0,2,2
+-0.0005175603134557599,0,0,2,2
+-0.0011435144301503893,0,0,2,0
+-0.0005956711224280303,0,0,2,3
+-0.001454344436060637,0,0,2,0
+-0.0019154289248399445,0,0,2,0
+-0.0002411364880390461,0,0,1,6
+-0.0012545736250467628,0,0,2,0
+-0.00164834396680817,0,0,2,0
+0.001068552900105716,1,1,0,5
+0.00016929756384342735,0,1,2,7
+0.0008943808428011832,1,1,0,5
+-0.002703342495951798,0,0,1,6
+-0.0012914460175670675,0,0,0,1
+-0.0020439513004384957,0,0,2,0
+-0.0006755184940993777,0,0,2,2
+0.00011476113228127329,1,1,0,4
+0.00028829645598307113,1,1,0,1
+0.000847709323279558,1,1,1,6
+-0.00038774271262809624,0,0,2,7
+-0.0008357822219841188,0,0,2,0
+0.0004526701779104768,1,1,0,4
+-0.0009252812876366073,0,0,0,1
+-0.00017984001664444853,1,0,0,5
+-0.002740789807867256,0,0,2,2
+-0.0008788172272033989,0,0,0,1
+-7.364853518083622e-05,1,0,2,3
+-0.0006125023425556722,0,0,2,0
+-0.0052259769663214745,1,0,0,4
+-0.00012254758970811977,1,0,0,5
+0.001246348149143161,1,1,1,6
+-0.003201695261523127,0,0,2,7
+-0.0010202326136641224,0,0,2,2
+-0.0007020124513655891,0,0,0,1
+-8.311982266604942e-05,0,0,0,5
+-0.013541883085854342,0,0,0,1
+8.417701581492874e-05,0,1,0,1
+5.2902970928699056e-05,0,1,1,6
+-0.0024327353364788015,0,0,2,2
+0.0010331293079070755,0,1,0,5
+-0.0008619601419195544,0,0,2,3
+-0.0002532164868898658,1,0,0,4
+-0.00030328659107908655,1,0,2,7
+0.0004164764890447273,0,1,0,5
+-0.0013670562114566562,0,0,0,5
+-0.0007202120125293747,0,0,0,1
+-0.0004258457827381792,0,0,0,4
+-0.0017371207266114637,0,0,2,0
+0.0008769840025343,0,1,1,6
+0.0010499154590070253,1,1,0,5
+0.00014877928653731984,1,1,2,2
+-0.00036323433043435223,0,0,0,1
+-0.0012456344836391502,0,0,2,0
+2.1784892305731426e-05,0,1,0,5
+-0.0007162772025912992,0,0,2,7
+6.0278952587394796e-05,0,1,2,7
+-0.00010348152602091334,0,0,2,7
+-0.0007500602002255621,0,0,0,1
+-0.001303730178624389,0,0,0,1
+-0.0003134692204184823,0,0,2,3
+-0.0014434849866665905,0,0,0,1
+-0.00044789650011807514,1,0,2,2
+-0.001103008831851186,0,0,2,2
+0.00022822466678917505,1,1,0,1
+0.00013400108553469122,1,1,2,7
+-0.0003032380808144792,0,0,0,4
+-0.003178386441431938,0,0,1,6
+-0.0003803872782737021,0,0,2,3
+0.00031395958503708295,1,1,2,3
+-0.0007607128680683674,0,0,2,2
+-0.0016037785355001687,0,0,2,2
+0.0005092357122339318,1,1,0,5
+-0.0012627666420303284,0,0,2,2
+-0.0018859333777800184,0,0,2,3
+-0.0023347207298502343,0,0,2,7
+-0.0005768683017231529,0,0,2,3
+-0.0014518209430389103,0,0,2,3
+0.00022024588659405597,1,1,0,4
+-0.0016641114884987478,0,0,2,0
+0.00023283683694898943,1,1,1,6
+-0.0012312231049872932,0,0,2,2
+1.2613816652446916e-05,1,1,2,3
+-0.0014317945996299396,0,0,0,4
+-4.435830516740652e-05,0,0,2,7
+-0.00040758794406428947,0,0,2,3
+0.0004171272832900301,1,1,0,5
+-0.0011735020554624505,0,0,0,5
+-2.9301736503839493e-06,0,0,2,7
+-0.0013486324553377924,0,0,2,0
+-7.152120117098065e-05,1,0,0,5
+-0.00018136285943910757,0,0,0,5
+-0.002478140434250233,0,0,2,7
+-0.0003403043607249842,0,0,0,1
+-2.1855528466403762e-05,0,0,0,1
+-0.0001978051685728125,0,0,1,6
+-0.0013766569015569979,0,0,2,2
+-0.0005541136697866031,0,0,2,7
+0.00046380348037928197,1,1,0,1
+-0.0003806104813702405,0,0,2,7
+-0.008536051595583558,0,0,0,4
+-0.0015688156546093505,0,0,2,2
+0.0001423335145227609,1,1,0,5
+0.001601479330565779,1,1,0,5
+-0.001863211586605757,0,0,2,0
+-0.0007314471667632461,0,0,2,0
+2.128806663677027e-05,1,1,0,5
+-0.0008965810504741982,0,0,2,2
+-0.00045400859788060147,0,0,0,5
+0.0007567387539893396,1,1,1,6
+0.001706107356585565,1,1,0,5
+-0.00023807478137314285,0,0,2,7
+-0.0009848419972695397,0,0,2,0
+-0.0009943539835512646,0,0,2,2
+-0.0005721209547482429,0,0,2,7
+-0.005606842981651428,0,0,2,0
+-0.00017783199902624003,0,0,2,2
+0.00024599506286904216,1,1,0,1
+1.3204421848059256e-05,1,1,0,1
+-0.0004024257231503726,0,0,2,0
+-0.001962742090690882,0,0,2,3
+-0.00017950041685253423,0,0,0,5
+-0.0013426412781700515,1,0,2,0

results/single_split_lambda_ablation/inkjet_lambda0_k100/metrics.json

@@ -0,0 +1,54 @@
+{
+  "overall_auroc": 0.8324587706146925,
+  "overall_acc": 0.7894736842105263,
+  "per_feature": {
+    "angle": {
+      "auroc": 0.5555555555555556,
+      "accuracy": 0.9,
+      "n_good": 27,
+      "n_bad": 3
+    },
+    "dist1": {
+      "auroc": 0.8999999999999999,
+      "accuracy": 0.8064516129032258,
+      "n_good": 21,
+      "n_bad": 10
+    },
+    "dist6": {
+      "auroc": 0.8277777777777778,
+      "accuracy": 0.8888888888888888,
+      "n_good": 30,
+      "n_bad": 6
+    },
+    "dots": {
+      "auroc": 0.9125874125874126,
+      "accuracy": 0.9459459459459459,
+      "n_good": 26,
+      "n_bad": 11
+    },
+    "edge1": {
+      "auroc": 0.7760416666666667,
+      "accuracy": 0.7142857142857143,
+      "n_good": 12,
+      "n_bad": 16
+    },
+    "edge2": {
+      "auroc": 0.7301587301587302,
+      "accuracy": 0.6086956521739131,
+      "n_good": 18,
+      "n_bad": 28
+    },
+    "edge3": {
+      "auroc": 0.71875,
+      "accuracy": 0.6785714285714286,
+      "n_good": 16,
+      "n_bad": 12
+    },
+    "edge4": {
+      "auroc": 0.6666666666666666,
+      "accuracy": 0.8,
+      "n_good": 24,
+      "n_bad": 6
+    }
+  }
+}

results/single_split_lambda_ablation/inkjet_lambda0_k100/scores.csv

@@ -0,0 +1,267 @@
+score,true_label,pred_label,template_id,feature_id
+-9.554469725116602e-06,1,0,0,1
+-0.0012993827799800788,0,0,0,1
+-0.00031760759535245583,0,0,0,5
+-0.0014764362864661956,0,0,2,7
+0.0006982657709158954,1,1,0,5
+0.00019466285826638327,1,1,0,1
+-0.002177701763575901,0,0,1,6
+-0.0012889409542549413,0,0,0,1
+-0.0006267773837316786,0,0,2,2
+0.00026392589439638153,0,1,1,6
+8.767178864218378e-05,1,1,0,4
+-0.00011407743906602028,1,0,0,5
+-0.0020173431478906455,0,0,2,0
+-0.001997406579321251,0,0,2,0
+-0.0011879051814321431,0,0,2,7
+4.280564025975833e-05,1,1,1,6
+-0.0001015696907415986,1,0,2,3
+-0.0003334035619627692,0,0,2,3
+-0.00040042195934802266,0,0,2,3
+0.0002352772117592411,0,1,1,6
+0.00011320319841615772,0,1,0,4
+0.00013735381187871334,0,1,0,5
+-0.0016083758708555253,0,0,2,2
+-0.0016111620387528093,0,0,2,2
+0.0002653341391123838,0,1,0,5
+-0.0001949038181919601,0,0,0,5
+-0.00027075876016169776,0,0,2,3
+0.00015685443649999728,0,1,0,5
+0.0002027627371717234,1,1,0,5
+-0.0005633674480486663,0,0,0,4
+0.00020397357176989324,0,1,1,6
+-0.0003214834339451047,0,0,0,1
+6.842531729488524e-06,1,1,1,6
+-0.00047089416882954704,0,0,0,4
+-0.0014128335739951575,0,0,2,0
+-0.0030663867527618988,1,0,2,2
+-0.0015892998245544687,1,0,0,4
+-0.0016502094909083102,0,0,2,3
+0.0010748815035913142,1,1,0,5
+-0.0019556569284759455,0,0,2,0
+-0.0003951793676242252,1,0,0,5
+-0.00011744282674044405,0,0,2,7
+0.0005750714102759954,1,1,1,6
+7.560102036222732e-05,1,1,2,2
+0.00020077065913937965,1,1,2,3
+9.982273331843317e-05,1,1,1,6
+-0.0005190350557677439,0,0,2,2
+-0.00037250868394039446,0,0,0,4
+-2.1383754210546266e-05,0,0,0,5
+-0.0003194294206332413,0,0,0,5
+0.00011329071363434199,1,1,2,3
+-0.00029568448313511964,0,0,0,4
+-0.00036130552529357526,0,0,2,7
+3.807565430179183e-05,0,1,0,1
+0.0013776364398654546,1,1,0,5
+-0.0003261471306905163,0,0,2,7
+-0.0005681195610668516,1,0,0,5
+-0.008138068311382092,0,0,0,1
+-0.0005904886883217834,0,0,2,3
+-0.0001554479228798311,1,0,0,5
+-0.00048753574374131754,0,0,2,3
+-0.0007977662456687545,0,0,2,7
+0.000216378872282803,1,1,1,6
+-0.003494018237106499,0,0,2,7
+0.00015310689108446324,0,1,0,4
+-0.0011027383932378145,0,0,2,2
+-0.0012286882731132216,0,0,2,2
+0.0011276600661221867,1,1,0,4
+-0.00013196345418691607,0,0,2,3
+-0.0007274905149824915,0,0,2,2
+-0.0009690443589352067,0,0,2,3
+-0.0006732322566676899,1,0,2,0
+0.00047142821014858893,0,1,1,6
+-0.0013611601211596276,0,0,2,0
+-0.0011260448477696632,0,0,2,3
+0.0012104483589064213,1,1,0,5
+-0.0015638124244287607,0,0,2,0
+6.536406348459436e-05,1,1,2,3
+-0.0007571127288974831,0,0,2,2
+5.92968065757319e-05,0,1,1,6
+-0.00022075019660405815,1,0,2,7
+-0.0006754555238876504,0,0,2,3
+-0.00043702442781068385,0,0,2,0
+0.0012676615733653307,1,1,2,3
+-0.00012001270428299918,1,0,0,5
+-0.00352738471585326,1,0,2,0
+1.1346534593030186e-05,1,1,2,7
+0.0006084602826740594,1,1,1,6
+-0.0035352304461412146,0,0,2,2
+1.524558058008628e-05,0,1,2,7
+0.00027124770916998366,1,1,0,5
+0.00025253814761527077,1,1,0,4
+0.00031118349405005394,0,1,1,6
+0.0005401251185685393,1,1,1,6
+-0.0006467057322151956,1,0,0,4
+-0.0007382922479882842,0,0,2,2
+-0.0007112001767382026,0,0,2,7
+0.0009762516664341084,1,1,2,3
+0.00011157669126987506,0,1,1,6
+9.118812158703804e-05,1,1,0,1
+-0.0008327848382759847,0,0,2,2
+-0.0009524829545989634,0,0,2,2
+-0.0004234138666652128,0,0,0,4
+0.00018001646734774196,1,1,1,6
+-0.0011354775342624635,0,0,2,0
+0.004023292696801947,1,1,0,5
+-0.0003289441624656324,1,0,0,1
+-0.0003397269814740863,0,0,2,0
+0.0018104972841683775,1,1,0,5
+-0.00065723892650567,0,0,2,3
+0.002066263627493764,1,1,2,3
+-0.0004971291241236027,1,0,2,2
+-0.0007472013903316102,0,0,0,1
+-0.00032380418502725786,0,0,2,3
+-0.00044004537630826235,0,0,1,6
+-0.0012180237506981932,0,0,2,3
+-0.0011612377699930232,0,0,0,1
+0.0002713688055519005,1,1,0,4
+-0.00034822755726054283,1,0,0,1
+-0.0005622158071491856,0,0,2,2
+0.0003649536543525768,1,1,0,4
+-0.000595303790178147,1,0,2,7
+-0.0005520438158418996,1,0,0,5
+-1.7095005605370292e-05,1,0,0,5
+-0.00018313196604140096,1,0,2,2
+0.00039938199101015835,1,1,0,4
+0.0008583359688054785,0,1,1,6
+-6.857370608486305e-05,1,0,0,4
+0.0003628907282836741,1,1,0,4
+-0.013646851294906823,0,0,0,1
+-0.00038364461972378167,1,0,2,7
+-0.0005484789668116705,0,0,2,3
+-0.00020950375241227462,0,0,2,3
+0.00017629979178309524,1,1,0,4
+-5.345375160686894e-05,1,0,0,5
+-0.003889127766015011,0,0,2,3
+-0.0007791739224921913,0,0,2,0
+-0.00019429067964665467,0,0,0,5
+-0.0004266017267946162,0,0,0,4
+-0.0007856910722330197,0,0,2,3
+-0.002841489015845581,1,0,2,3
+0.0003650653059594326,0,1,0,5
+-0.0018055455374997031,0,0,2,2
+-0.00033807559055276225,0,0,2,7
+-0.0009467872022651137,0,0,2,2
+-0.000932800532318652,0,0,2,2
+-0.0012578624172601867,0,0,2,0
+-0.00043052772409282736,0,0,2,3
+-0.0016056844848208116,0,0,2,0
+-0.0015366222325246797,0,0,2,0
+-0.00026337323011830366,0,0,1,6
+-0.001072680031647906,0,0,2,0
+-0.0022774999553803327,0,0,2,0
+0.0009616456215735519,1,1,0,5
+0.00019347749184817076,0,1,2,7
+0.0009879572957288488,1,1,0,5
+-0.0028031592059414856,0,0,1,6
+-0.0012566636188421423,0,0,0,1
+-0.001344811615999789,0,0,2,0
+-0.0008035916509106754,0,0,2,2
+0.00015061535872518995,1,1,0,4
+0.0003005571628455078,1,1,0,1
+0.000775633471785114,1,1,1,6
+-0.0004929053399246198,0,0,2,7
+-0.0013577966240700332,0,0,2,0
+-5.2403565496204896e-05,1,0,0,4
+-0.0018729215522762382,0,0,0,1
+-0.000203391204122454,1,0,0,5
+-0.0028961684543173764,0,0,2,2
+-0.00017878195969387757,0,0,0,1
+0.0003235730866435922,1,1,2,3
+-0.000894318593200296,0,0,2,0
+-0.003917646837653589,1,0,0,4
+-0.0002033112652134149,1,0,0,5
+0.0015865554218180457,1,1,1,6
+-0.0017169059440493573,0,0,2,7
+-0.0010308104404248286,0,0,2,2
+-0.0011319727741647481,0,0,0,1
+-0.0002167755935806783,0,0,0,5
+-0.010463011359097434,0,0,0,1
+0.0001856530271470546,0,1,0,1
+-1.4063059352337204e-05,0,0,1,6
+-0.0020209528238046906,0,0,2,2
+0.0007552384363953037,0,1,0,5
+-0.0006083755695726733,0,0,2,3
+-0.0001933931163512169,1,0,0,4
+-0.0004890145536046459,1,0,2,7
+0.0003852236678358184,0,1,0,5
+-0.0008304657542612393,0,0,0,5
+-0.0007987114845309402,0,0,0,1
+-0.0004449901275802408,0,0,0,4
+-0.0019321736355777837,0,0,2,0
+0.0004404012020677323,0,1,1,6
+0.0010624042223207654,1,1,0,5
+0.0001164552883710715,1,1,2,2
+-0.0003928722580894828,0,0,0,1
+-0.001158735032659024,0,0,2,0
+0.00013700664625503414,0,1,0,5
+-0.0009254113503266125,0,0,2,7
+-4.818064393475631e-05,0,0,2,7
+-0.00020191999967210036,0,0,2,7
+-0.0009835302259307357,0,0,0,1
+-0.0014964910817798216,0,0,0,1
+-0.00028650919673964444,0,0,2,3
+-0.0005916058691218494,0,0,0,1
+-0.00023216477828100213,1,0,2,2
+-0.0012802854424808184,0,0,2,2
+3.0155578861011667e-05,1,1,0,1
+0.00015847264090552982,1,1,2,7
+-0.0004923553531989457,0,0,0,4
+-0.004585270243696871,0,0,1,6
+-0.000792034799233076,0,0,2,3
+0.00041176963131874834,1,1,2,3
+-0.00100953286048025,0,0,2,2
+-0.0018384375108871608,0,0,2,2
+0.0008299424161668876,1,1,0,5
+-0.0009825329307932423,0,0,2,2
+-0.0018490493646822863,0,0,2,3
+-0.002845470937900243,0,0,2,7
+-0.0007159416389185946,0,0,2,3
+-0.001477111697895453,0,0,2,3
+8.174191811121875e-05,1,1,0,4
+-0.0013155037141405045,0,0,2,0
+0.0001578804710879924,1,1,1,6
+-0.0006749476352706559,0,0,2,2
+1.726198592223293e-05,1,1,2,3
+-0.0008134388795588178,0,0,0,4
+-1.2640547938643637e-05,0,0,2,7
+-0.0006703788426239039,0,0,2,3
+0.0001843718602322058,1,1,0,5
+-0.0006610541266854845,0,0,0,5
+-3.6587612703437716e-06,0,0,2,7
+-0.0014425987040158367,0,0,2,0
+-4.9577774479985653e-05,1,0,0,5
+-0.0002124248852487659,0,0,0,5
+-0.0025788940349593774,0,0,2,7
+-0.00043636561837047394,0,0,0,1
+0.00011145593482069643,0,1,0,1
+-0.0002739321708213546,0,0,1,6
+-0.0017730902798939498,0,0,2,2
+-0.0006563830957748013,0,0,2,7
+0.0003827086591627442,1,1,0,1
+-0.00048738889978267204,0,0,2,7
+-0.006632461363915355,0,0,0,4
+-0.001265106133650988,0,0,2,2
+0.0003015411365777255,1,1,0,5
+0.0010881104040890953,1,1,0,5
+-0.0017792192776687442,0,0,2,0
+-0.0012446534191258244,0,0,2,0
+4.798424197361567e-06,1,1,0,5
+-0.0009694832947570835,0,0,2,2
+-0.00045949047664180405,0,0,0,5
+0.0002762691688258205,1,1,1,6
+0.0024066535115707666,1,1,0,5
+-0.00019797537126578328,0,0,2,7
+-0.0012433118396438653,0,0,2,0
+-0.0012114240624941895,0,0,2,2
+-0.0005912172643002117,0,0,2,7
+-0.00541591746732592,0,0,2,0
+-0.00020413821213878698,0,0,2,2
+0.0002638021076563756,1,1,0,1
+1.089783618226682e-05,1,1,0,1
+-0.0003716060472652317,0,0,2,0
+-0.0017504926538094885,0,0,2,3
+-0.00032126413541845936,0,0,0,5
+-0.0011584743019193398,1,0,2,0

results/tables/inkjet_ablation_table.tex

@@ -0,0 +1,15 @@
+\begin{table}[h]
+\centering
+\caption{Separation loss ablation on the Inkjet QC dataset (5-fold stratified CV, K=100 MC trials). Results reported as mean~$\pm$~std across folds. No $\lambda$ value significantly outperforms the baseline.}
+\label{tab:inkjet_ablation}
+\begin{tabular}{lccc}
+\toprule
+$\lambda$ & AUROC $\uparrow$ & Accuracy $\uparrow$ & FPR@95TPR $\downarrow$ \\
+\midrule
+\textbf{0.0 (baseline)} & 0.8673 $\pm$ 0.0230 & 0.8094 $\pm$ 0.0151 & 0.5631 $\pm$ 0.1697 \\
+0.01 & 0.8628 $\pm$ 0.0286 & 0.7928 $\pm$ 0.0291 & 0.5516 $\pm$ 0.1841 \\
+0.02 & 0.8510 $\pm$ 0.0326 & 0.8003 $\pm$ 0.0246 & 0.6240 $\pm$ 0.1334 \\
+0.05 & 0.8670 $\pm$ 0.0256 & 0.8071 $\pm$ 0.0241 & 0.5700 $\pm$ 0.1948 \\
+\bottomrule
+\end{tabular}
+\end{table}

results/tables/inkjet_cross_domain_table.tex

@@ -0,0 +1,15 @@
+\begin{table}[h]
+\centering
+\caption{Cross-domain comparison of the separation loss effect. CIFAR-10 results use a single train/test split (seed=42); Inkjet results use 5-fold stratified CV (seed=42, mean~$\pm$~std).}
+\label{tab:cross_domain}
+\begin{tabular}{lccc}
+\toprule
+Dataset & Baseline ($\lambda=0$) & Best sep.\ loss & $\Delta$ AUROC \\
+\midrule
+CIFAR-10 & 0.8025 & \textbf{0.9911} ($\lambda=0.02$) & \textbf{+18.9\%} \\
+Inkjet QC & \textbf{0.8673}~$\pm$~0.0230 & 0.8670~$\pm$~0.0256 ($\lambda=0.05$) & $\approx$0.0\% (n.s.) \\
+\midrule
+\multicolumn{4}{l}{\small n.s.\ = not statistically significant (all $\Delta \leq 0.016$, within cross-fold std)} \\
+\bottomrule
+\end{tabular}
+\end{table}

results/tables/inkjet_per_feature_table.tex

@@ -0,0 +1,19 @@
+\begin{table}[h]
+\centering
+\caption{Per-feature AUROC on the Inkjet QC dataset (5-fold CV, mean~$\pm$~std). $\dagger$~angle has fewer than 5 BAD samples per fold; AUROC is unreliable.}
+\label{tab:inkjet_per_feature}
+\begin{tabular}{lcccc}
+\toprule
+Feature & $\lambda=0.0$ & $\lambda=0.01$ & $\lambda=0.02$ & $\lambda=0.05$ \\
+\midrule
+Angle ($\dagger$) & \textbf{0.8166}$\pm$0.1381 & 0.7727$\pm$0.1580 & 0.7682$\pm$0.1836 & 0.7378$\pm$0.1593 \\
+Distance 1 & \textbf{0.8866}$\pm$0.0729 & 0.8508$\pm$0.0768 & 0.8274$\pm$0.1457 & 0.8828$\pm$0.0581 \\
+Distance 6 & 0.9362$\pm$0.0666 & 0.9423$\pm$0.0737 & 0.9393$\pm$0.0750 & \textbf{0.9473}$\pm$0.0710 \\
+Dots & 0.9557$\pm$0.0353 & \textbf{0.9655}$\pm$0.0247 & 0.9474$\pm$0.0400 & 0.9413$\pm$0.0394 \\
+Edge 1 & 0.7958$\pm$0.1376 & 0.8410$\pm$0.1379 & 0.7823$\pm$0.1607 & \textbf{0.8574}$\pm$0.1283 \\
+Edge 2 & 0.8129$\pm$0.0307 & \textbf{0.8426}$\pm$0.0443 & 0.7993$\pm$0.0602 & 0.8402$\pm$0.0283 \\
+Edge 3 & 0.7441$\pm$0.0757 & 0.7008$\pm$0.0390 & \textbf{0.7774}$\pm$0.0647 & 0.7349$\pm$0.0596 \\
+Edge 4 & 0.7617$\pm$0.0985 & 0.7529$\pm$0.0543 & 0.7643$\pm$0.0898 & \textbf{0.7775}$\pm$0.1125 \\
+\bottomrule
+\end{tabular}
+\end{table}